Search icon CANCEL
Subscription
0
Cart icon
Your Cart (0 item)
Close icon
You have no products in your basket yet
Save more on your purchases! discount-offer-chevron-icon
Savings automatically calculated. No voucher code required.
Arrow left icon
Explore Products
Best Sellers
New Releases
Books
Events
Videos
Audiobooks
Packt Hub
Free Learning
Arrow right icon
timer SALE ENDS IN
0 Days
:
00 Hours
:
00 Minutes
:
00 Seconds

How-To Tutorials

7018 Articles
article-image-working-tooltips
Packt
23 Dec 2013
6 min read
Save for later

Working with Tooltips

Packt
23 Dec 2013
6 min read
(For more resources related to this topic, see here.) The jQuery team introduced their version of the tooltip as part of changes to Version 1.9 of the library; it was designed to act as a direct replacement for the standard tooltip used in all browsers. The difference here, though, was that whilst you can't style the standard tooltip, jQuery UI's replacement is intended to be accessible, themeable, and completely customizable. It has been set to display not only when a control receives focus, but also when you hover over that control, which makes it easier to use for keyboard users. Implementing a default tooltip Tooltips were built to act as direct replacements for the browser's native tooltips. They will recognize the default markup of the title attribute in a tag, and use it to automatically add the additional markup required for the widget. The target selector can be customized though using tooltip's items and content options. Let's first have a look at the basic structure required for implementing tooltips. In a new file in your text editor, create the following page: <!DOCTYPE HTML> <html> <head> <meta charset="utf-8"> <title>Tooltip</title> <link rel="stylesheet" href="development- bundle/themes/redmond/jquery.ui.all.css"> <style> p { font-family: Verdana, sans-serif; } </style> <script src = "js/jquery-2.0.3.js"></script> <script src = "development- bundle/ui/jquery.ui.core.js"></script> <script src = "development-bundle/ui/jquery.ui.widget.js"> </script> <script src = "development-bundle/ui/jquery.ui.position.js"> </script> <script src = "development-bundle/ui/jquery.ui.tooltip.js"> </script> <script> $(document).ready(function($){ $(document).tooltip(); }); </script> </head> <body> <p>Lorem ipsum dolor sit amet, consectetur adipiscing elit. Nulla blandit mi quis imperdiet semper. Fusce vulputate venenatis fringilla. Donec vitae facilisis tortor. Mauris dignissim nibh ac justo ultricies, nec vehicula ipsum ultricies. Mauris molestie felis ligula, id tincidunt urna consectetur at. Praesent <a href="http://www. ipsum.com" title="This was generated from www.ipsum.com">blandit</a> faucibus ante ut semper. Pellentesque non tristique nisi. Ut hendrerit tempus nulla, sit amet venenatis felis lobortis feugiat. Nam ac facilisis magna. Praesent consequat, risus in semper imperdiet, nulla lorem aliquet nisi, a laoreet nisl leo rutrum mauris.</p> </body> </html> Save the code as tooltip1.html in your jqueryui working folder. Let's review what was used. The following script and CSS resources are needed for the default tooltip widget configuration: jquery.ui.all.css jquery-2.0.3.js jquery.ui.core.js jquery.ui.widget.js jquery.ui.tooltip.js The script required to create a tooltip, when using the title element in the underlying HTML can be as simple as this, which should be added after the last <script> element in your code, as shown in the previous example: <script> $(document).ready(function($){ $(document).tooltip(); }); </script> In this example, when hovering over the link, the library adds in the requisite aria described by the code for screen readers into the HTML link. The widget then dynamically generates the markup for the tooltip, and appends it to the document, just before the closing </body> tag. This is automatically removed as soon as the target element loses focus. ARIA, or Accessible Rich Internet Applications, provides a way to make content more accessible to people with disabilities. You can learn more about this initiative at https://developer.mozilla.org/en-US/docs/Accessibility/ARIA. It is not necessary to only use the $(document) element when adding tooltips. Tooltips will work equally well with classes or selector IDs; using a selector ID, will give a finer degree of control. Overriding the default styles When styling the Tooltip widget, we are not limited to merely using the prebuilt themes on offer, we can always elect to override existing styles with our own. In our next example, we’ll see how easy this is to accomplish, by making some minor changes to the example from tooltip1.html. In a new document, add the following styles, and save it as tooltipOverride.css, within the css folder: p { font-family: Verdana, sans-serif; } .ui-tooltip { background: #637887; color: #fff; } Don't forget to link to the new style sheet from the <head> of your document: <link rel="stylesheet" href="css/tooltipOverride.css"> Before we continue, it is worth explaining a great trick for styling tooltips before committing the results to code. If you are using Firefox, you can download and install the Toggle JS add-on for Firefox, which is available from https://addons.mozilla.org/en-US/firefox/addon/toggle-js/. This allows us to switch off JavaScript on a per-page basis; we can then hover over the link to create the tooltip, before expanding the markup in Firebug and styling it at our leisure. Save your HTML document as tooltip2.html. When we run the page in a browser, you should see the modified tooltip appear when hovering over the link in the text: Using prebuilt themes If creating completely new styles by hand is overkill for your needs, you can always elect to use one of the prebuilt themes that are available for download from the jQuery UI site. This is a really easy change to make. We first need to download a copy of the replacement theme; in our example, we’re going to use one called Excite Bike. Let’s start by browsing to http://jqueryui.com/download/, then deselecting the Toggle All option. We don’t need to download the whole library, just the theme at the bottom, change the theme option to display Excite Bike then select Download. Next, open a copy of tooltip2.html then look for this line: <link rel="stylesheet" href="development-bundle/themes/redmond /jquery.ui.all.css"> You will notice the highlighted word in the above line. This is the name of the existing theme. Change this to excite-bike then save the document as tooltip3.html, then remove the tooltipOverride.css link, and you’re all set. The following is our replacement theme in action: With a single change of word, we can switch between any of the prebuilt themes available for use with jQuery UI (or indeed even any of the custom ones that others have made available online), as long as you have downloaded and copied the theme into the appropriate folder. There may be occasions though, were we need to tweak the settings. This gives us the best of both worlds, where we only need to concentrate on making the required changes. Let’s take a look at how we can alter an existing theme, using ThemeRoller.
Read more
  • 0
  • 0
  • 12279

article-image-getting-started-glsl
Packt
20 Dec 2013
15 min read
Save for later

Getting Started with GLSL

Packt
20 Dec 2013
15 min read
(For more resources related to this topic, see here.) In this article, we will cover the following recipes: Using a function loader to access the latest OpenGL functionality Using GLM for mathematics Determining the GLSL and OpenGL version Compiling a shader Linking a shader program The OpenGL Shading Language(GLSL) Version 4 brings unprecedented power and flexibility to programmers interested in creating modern, interactive, and graphical programs. It allows us to harness the power of modern Graphics Processing Units(GPUs) in a straightforward way by providing a simple yet powerful language and API. Of course, the first step towards using GLSL is to create a program that utilizes the latest version of the OpenGL API. GLSL programs don't stand on their own; they must be a part of a larger OpenGL program. In this article, we will provide some tips and techniques for getting a basic program up and running. First, let's start with some background. The OpenGL Shading Language The GLSL is now a fundamental and integral part of the OpenGL API. Going forward, every program written using the OpenGL API will internally utilize one or several GLSL programs. These "mini-programs" are often referred to as shader programs. A shader program usually consists of several components called shaders. Each shader executes within a different section of the OpenGL pipeline. Each shader runs on the GPU, and as the name implies, (typically) implement the algorithms related to the lighting and shading effects of an image. However, shaders are capable of doing much more than just implementing a shading algorithm. They are also capable of performing animation, tessellation, or even generalized computation. The field of study dubbed GPGPU(General Purpose Computing on Graphics Processing Units) is concerned with utilization of GPUs (often using specialized APIs such as CUDA or OpenCL) to perform general purpose computations such as fluid dynamics, molecular dynamics, cryptography, and so on. With compute shaders, introduced in OpenGL 4.3, we can now do GPGPU within OpenGL. Shader programs are designed for direct execution on the GPU and are executed in parallel. For example, a fragment shader might be executed once for every pixel, with each execution running simultaneously on a separate GPU thread. The number of processors on the graphics card determines how many can be executed at one time. This makes shader programs incredibly efficient, and provides the programmer with a simple API for implementing highly parallel computation. The computing power available in modern graphics cards is impressive. The following table shows the number of shader processors available for several models in the NVIDIA GeForce series cards (source: http://en.wikipedia.org/wiki/Comparison_of_Nvidia_graphics_processing_units). Model Unified Shader Processors GeForce GTS 450 192 GeForce GTX 480 480 GeForce GTX 780 2304 Shader programs are intended to replace parts of the OpenGL architecture referred to as the fixed-function pipeline. Prior to OpenGL Version 2.0, the shading algorithm was "hard-coded" into the pipeline and had only limited configurability. This default lighting/shading algorithm was a core part of the fixed-function pipeline. When we, as programmers, wanted to implement more advanced or realistic effects, we used various tricks to force the fixed-function pipeline into being more flexible than it really was. The advent of GLSL will help by providing us with the ability to replace this "hard-coded" functionality with our own programs written in GLSL, thus giving us a great deal of additional flexibility and power. In fact, recent (core) versions of OpenGL not only provide this capability, but they require shader programs as part of every OpenGL program. The old fixed-function pipeline has been deprecated in favor of a new programmable pipeline, a key part of which is the shader program written in GLSL. Profiles – Core vs. Compatibility OpenGL Version 3.0 introduced a deprecation model, which allowed for the gradual removal of functions from the OpenGL specification. Functions or features can be marked as deprecated, meaning that they are expected to be removed from a future version of OpenGL. For example, immediate mode rendering using glBegin/glEndwas marked deprecated in version 3.0 and removed in version 3.1. In order to maintain backwards compatibility, the concept of compatibility profiles was introduced with OpenGL 3.2. A programmer that is writing code intended to be used with a particular version of OpenGL (with older features removed) would use the so-called core profile. Someone who also wanted to maintain compatibility with older functionality could use the compatibility profile. It may be somewhat confusing that there is also the concept of a forward compatible context, which is distinguished slightly from the concept of a core/compatibility profile. A context that is considered forward compatible basically indicates that all deprecated functionality has been removed. In other words, if a context is forward compatible, it only includes functions that are in the core, but not those that were marked as deprecated. Some window APIs provide the ability to select forward compatible status along with the profile. The steps for selecting a core or compatibility profile are window system API dependent. For example, in GLFW, one can select a forward compatible, 4.3 core profile using the following code: glfwWindowHint( GLFW_CONTEXT_VERSION_MAJOR, 4 ); glfwWindowHint( GLFW_CONTEXT_VERSION_MINOR, 3 ); glfwWindowHint(GLFW_OPENGL_FORWARD_COMPAT, GL_TRUE); glfwWindowHint(GLFW_OPENGL_PROFILE, GLFW_OPENGL_CORE_PROFILE); GLFWwindow *window = glfwCreateWindow(640, 480, "Title", NULL, NULL); All programs in this article are designed to be compatible with a forward compatible OpenGL 4.3 core profile. Using a function loader to access the latest OpenGL functionality The OpenGL ABI (application binary interface) is frozen to OpenGL version 1.1 on Windows. Unfortunately for Windows developers, that means that it is not possible to link directly to functions that are provided in newer versions of OpenGL. Instead, one must get access to these functions by acquiring a function pointer at runtime. Getting access to the function pointers is not difficult, but requires somewhat tedious work, and has a tendency to clutter your code. Additionally, Windows typically comes with a standard OpenGL gl.h file that also conforms to OpenGL 1.1. The OpenGL wiki states that Microsoft has no plans to ever update the gl.hand opengl32.lib that come with their compilers. Thankfully, others have provided libraries that manage all of this for us by transparently providing the needed function pointers, while also exposing the needed functionality in header files. There are several libraries available that provide this kind of support. One of the oldest and most common is GLEW (OpenGL Extension Wrangler). However, there are a few serious issues with GLEW that might make it less desirable, and insufficient for my purposes when writing this article. First, at time of writing, it doesn't yet support core profiles properly, and for this article, I want to focus only on the latest non-deprecated functionality. Second, it provides one large header file that includes everything from all versions of OpenGL. It might be preferable to have a more streamlined header file that only includes functions that we might use. Finally, GLEW is distributed as a library that needs to be compiled separately and linked into our project. It is often preferable to have a loader that can be included into a project simply by adding the source files and compiling them directly into our executable, avoiding the need to support another link-time dependency. In this recipe, we'll use the OpenGL Loader Generator (GLLoadGen), available from https://bitbucket.org/alfonse/glloadgen/wiki/Home. This very flexible and efficient library solves all three of the issues described in the previous paragraph. It supports core profiles and it can generate a header that includes only the needed functionality, and also generates just a couple of files (a source file and a header) that we can add directly into our project. Getting ready To use GLLoadGen, you'll need Lua. Lua is a lightweight embeddable scripting language that is available for nearly all platforms. Binaries are available at http://luabinaries.sourceforge.net, and a fully packaged install for Windows (LuaForWindows) is available at: https://code.google.com/p/luaforwindows Download the GLLoadGen distribution from: https://bitbucket.org/alfonse/glloadgen/downloads. The distribution is compressed using 7zip, which is not widely installed, so you may need to install a 7zip utility, available at http://7-zip.org/. Extract the distribution to a convenient location on your hard drive. Since GLLoadGen is written in Lua, there's nothing to compile, once the distribution is uncompressed, you're ready to go. How to do it... The first step is to generate the header and source files for the OpenGL version and profile of choice. For this example, we'll generate files for an OpenGL 4.3 core profile. We can then copy the files into our project and compile them directly alongside our code: To generate the header and source files, navigate to the GLLoadGen distribution directory, and run GLLoadGen with the following arguments: lua LoadGen.lua -style=pointer_c -spec=gl -version=4.3 -profile=core core_4_3 The previous step should generate two files: gl_core_4_3.c and gl_core_4_3.h. Move these files into your project and include gl_core_4_3.c in your build. Within your program code, you can include the gl_core_4_3.h file whenever you need access to the OpenGL functions. However, in order to initialize the function pointers, you need to make sure to call a function to do so. The needed function is called ogl_LoadFunctions. Somewhere just after the GL context is created (typically in an initialization function), and before any OpenGL functions are called, use the following code: int loaded = ogl_LoadFunctions(); if(loaded == ogl_LOAD_FAILED) { //Destroy the context and abort return; } int num_failed = loaded - ogl_LOAD_SUCCEEDED; printf("Number of functions that failed to load: %i.n", num_failed); That's all there is to it! How it works... The lua command in step 1 generates a pair of files, that is; a header and a source file. The header provides prototypes for all of the selected OpenGL functions and redefines them as function pointers, and defines all of the OpenGL constants as well. The source file provides initialization code for the function pointers as well as some other utility functions. We can include the gl_core_4_3.h header file wherever we need prototypes for OpenGL functions, so all function entry points are available at compile time. At run time, the ogl_LoadFunctions()function will initialize all available function pointers. If some functions fail to load, the number of failures can be determined by the subtraction operation shown in step 2. If a function is not available in the selected OpenGL version, the code may not compile, because only function prototypes for the selected OpenGL version and profile are available in the header (depending on how it was generated). The command line arguments available to GLLoadGen are fully documented here: https://bitbucket.org/alfonse/glloadgen/wiki/Command_Line_Options. The previous example shows the most commonly used setup, but there's a good amount of flexibility built into this tool. Now that we have generated this source/header pair, we no longer have any dependency on GLLoadGen and our program can be compiled without it. This is a significant advantage over tools such as GLEW. There's more... GLLoadGen includes a few additional features that are quite useful. We can generate more C++ friendly code, manage extensions, and generate files that work without the need to call an initialization function. Generating a C++ loader GLLoadGen supports generation of C++ header/source files as well. This can be selected via the -style parameter. For example, to generate C++ files, use -style=pointer_cpp as in the following example: lua LoadGen.lua -style=pointer_cpp -spec=gl -version=4.3 -profile=core core_4_3 This will generate gl_core_4_3.cpp and gl_core_4_3.hpp. This places all OpenGL functions and constants within the gl::namespace, and removes their gl(or GL) prefix. For example, to call the function glBufferData, you might use the following syntax. gl::BufferData(gl::ARRAY_BUFFER, size, data, gl::STATIC_DRAW); Loading the function pointers is also slightly different. The return value is an object rather than just a simple integer and LoadFunctions is in the gl::sys namespace. gl::exts::LoadTest didLoad = gl::sys::LoadFunctions(); if(!didLoad) { // Clean up (destroy the context) and abort. return; } printf("Number of functions that failed to load: %i.n", didLoad.GetNumMissing()); -load styles GLLoadGen supports the automatic initialization of function pointers. This can be selected using the noload_c or noload_cpp options for the style parameter. With these styles, there is no need to call the initialization function ogl_LoadFunctions. The pointers are loaded automatically, the first time a function is called. This can be convenient, but there's very little overhead to loading them all at initialization. Using Extensions GLLoadGen does not automatically support extensions. Instead, you need to ask for them with command line parameters. For example, to request ARB_texture_view and ARB_vertex_attrib_binding extensions, you might use the following command. lua LoadGen.lua -style=pointer_c -spec=gl -version=3.3 -profile=core core_3_3 -exts ARB_texture_view ARB_vertex_attrib_binding The -exts parameter is a space-separated list of extensions. GLLoadGen also provides the ability to load a list of extensions from a file (via the -extfile parameter) and provides some common extension files on the website. You can also use GLLoadGen to check for the existence of an extension at run-time. For details, see the GLLoadGen wiki. See also GLEW, an older, and more common loader and extension manager, available from glew.sourceforge.net. Using GLM for mathematics Mathematics is core to all of computer graphics. In earlier versions, OpenGL provided support for managing coordinate transformations and projections using the standard matrix stacks (GL_MODELVIEW and GL_PROJECTION). In recent versions of core OpenGL however, all of the functionality supporting the matrix stacks has been removed. Therefore, it is up to us to provide our own support for the usual transformation and projection matrices, and then to pass them into our shaders. Of course, we could write our own matrix and vector classes to manage this, but some might prefer to use a ready-made, robust library. One such library is GLM(OpenGL Mathematics) written by Christophe Riccio. Its design is based on the GLSL specification, so the syntax is very similar to the mathematical support in GLSL. For experienced GLSL programmers, this makes GLM very easy to use and familiar. Additionally, it provides extensions that include functionality similar to some of the much missed OpenGL functions such as glOrtho, glRotate, or gluLookAt. Getting ready Since GLM is a header-only library, installation is simple. Download the latest GLM distribution from http://glm.g-truc.net. Then, unzip the archive file, and copy the glm directory contained inside to anywhere in your compiler's include path. How to do it... To use the GLM libraries, it is simply a matter of including the core header file, and headers for any extensions. For this example, we'll include the matrix transform extension as follows: #include <glm/glm.hpp> #include <glm/gtc/matrix_transform.hpp> Then the GLM classes are available in the glm namespace. The following is an example of how you might go about making use of some of them: glm::vec4 position = glm::vec4( 1.0f, 0.0f, 0.0f, 1.0f ); glm::mat4 view = glm::lookAt( glm::vec3(0.0,0.0,5.0), glm::vec3(0.0,0.0,0.0), glm::vec3(0.0,1.0,0.0) ); glm::mat4 model(1.0f); // The identity matrix model = glm::rotate( model, 90.0f, glm::vec3(0.0f,1.0f,0.0) ); glm::mat4 mv = view * model; glm::vec4 transformed = mv * position; How it works... The GLM library is a header-only library. All of the implementation is included within the header files. It doesn't require separate compilation and you don't need to link your program to it. Just placing the header files in your include path is all that's required! The previous example first creates a vec4(four coordinate vector) representing a position. Then it creates a 4 x 4 view matrix by using the glm::lookAt function. This works in a similar fashion to the old gluLookAt function. Here, we set the camera's location at (0, 0, 5), looking towards the origin, with the "up" direction in the direction of the y-axis. We then go on to create the model matrix by first storing the identity matrix in the variable model(via the single argument constructor), and multiplying by a rotation matrix using the glm::rotate function. The multiplication here is implicitly done by the glm::rotate function. It multiplies its first parameter by the rotation matrix (on the right) that is generated by the function. The second parameter is the angle of rotation (in degrees), and the third parameter is the axis of rotation. Since before this statement, model is the identity matrix, the net result is that model becomes a rotation matrix of 90 degrees around the y-axis. Finally, we create our modelview matrix (mv) by multiplying the view and model variables, and then using the combined matrix to transform the position. Note that the multiplication operator has been overloaded to behave in the expected way. There's more... It is not recommended to import all of the GLM namespace by using the following command: using namespace glm; This will most likely cause a number of namespace clashes. Instead, it is preferable to import symbols one at a time, as needed. For example: #include <glm/glm.hpp> using glm::vec3; using glm::mat4; Using the GLM types as input to OpenGL GLM supports directly passing a GLM type to OpenGL using one of the OpenGL vector functions (with the suffix v). For example, to pass a mat4 named proj to OpenGL we can use the following code: glm::mat4 proj = glm::perspective( viewAngle, aspect, nearDist, farDist ); glUniformMatrix4fv(location, 1, GL_FALSE, &proj[0][0]); See also The Qt SDK includes many classes for vector/matrix mathematics, and is another good option if you're already using Qt The GLM website http://glm.g-truc.net has additional documentation and examples
Read more
  • 0
  • 0
  • 4455

article-image-learning-option-pricing
Packt
20 Dec 2013
19 min read
Save for later

Learning Option Pricing

Packt
20 Dec 2013
19 min read
(for more resources related to this topic, see here.) Introduction to options Options come in two variants, puts and calls. The call option gives the owner of the option the right, but not the obligation, to buy the underlying asset at the strike price. The put gives the holder of the contract, the right but not the obligation to sell the underlying asset. The Black-Scholes formula describes the European option, which can only be exercised on the maturity date, in contrast to for example American options. The buyer of the option pays a premium for this, to cover the risk taken from the counterpart side. Options have become very popular and they are traded on the major exchanges throughout the world, covering most asset-classes. The theory behind options can become complex pretty quick. In this article we'll look at the basics of options and how to explore them using code written in F#. Looking into contract specifications Options comes in a wide number of variations, some of them will be covered briefly below. The contract specifications for options will also depend on its type. Generally there are some properties that are more or less general to all of them. The general specifications are as follows: Side Quantity Strike price Expiration date Settlement terms The contract specifications, or know variables, are used then we valuate options. European options European options are the basic form of options that the other variants derive, American options and exotic options are some examples. We'll stick to European options in this article. American options American options are options that may be exercised on any trading day on or before expiry. Exotic options Exotic options are any of the broad category of options that may include complex financial structures and may be combinations of other instruments as well. Learning about Wiener processes Wiener processes are closely related to stochastic differential equations and volatility. Wiener processes or geometric Brownian motion, is defined as this: The formula describes the change in the stock price, or underlying, with a drift, μ, and a volatility, σ, and the Wiener process, Wt. This process is used to model the prices in Black-Scholes. We'll simulate market data using a Brownian motion, or Wiener process implemented in F# as a sequence. Sequences can be infinite and only the values used are evaluated, which suites or needs. We'll implement a generator function, to generate the Wiener process as a sequence as follows: // A normally distributed random generator let normd = new Normal(0.0, 1.0) let T = 1.0 let N = 500.0 let dt:float = T / N /// Sequences represent infinite number of elements // p -> probability mean // s -> scaling factor let W s = let rec loop x = seq { yield x; yield! loop (x + sqrt(dt)*normd.Sample()*s)} loop s;; Here we use the random function in normd.Sample(). Let's explain the parameters and the theory behind Brownian motion before looking at the implementation. The parameter T is the time used to create a discrete time increment dt. Notice that dt will assume there is 500 N:s, 500 items in the sequence, this is of course not always the case but will do fine in here. Next, we use recursion to create the sequence, where we add an increment to the previous value (x+...), where x c] xt-1. We can easily generate an arbitrary length of the sequence: > Seq.take 50 (W 55.00);; val it : seq<float> = seq [55.0; 56.72907873; 56.96071054;58.72850048; ...] Here we create a sequence of length 50. Let's plot the sequence to get a better understanding about the process. A Wiener process generated from the sequence generator above. Next we'll look at the code to generate the graph in the figure above. open System open System.Net open System.Windows.Forms open System.Windows.Forms.DataVisualization.Charting open Microsoft.FSharp.Control.WebExtensions open MathNet.Numerics.Distributions; // A normally distributed random generator let normd = new Normal(0.0, 1.0) // Create chart and form let chart = new Chart(Dock = DockStyle.Fill) let area = new ChartArea("Main") chart.ChartAreas.Add(area) let mainForm = new Form(Visible = true, TopMost = true, Width = 700, Height = 500) do mainForm.Text <- "Wiener process in F#" mainForm.Controls.Add(chart) // Create series for stock price let wienerProcess = new Series("process") do wienerProcess.ChartType <- SeriesChartType.Line do wienerProcess.BorderWidth <- 2 do wienerProcess.Color <- Drawing.Color.Red chart.Series.Add(wienerProcess) let random = new System.Random() let rnd() = random.NextDouble() let T = 1.0 let N = 500.0 let dt:float = T / N /// Sequences represent infinite number of elements let W s = let rec loop x = seq { yield x; yield! loop (x +/ sqrt(dt)*normd.Sample()*s)} loop s;; do (Seq.take 100 (W 55.00)) |> Seq.iter (wienerProcess.Points.Add>> ignore) Most of the code will be familiar to you at this stage, but the interesting part is the last line, where we can simply feed a chosen number of elements from the sequence into the Seq.iter which will plot the values, elegant and efficient. Learning the Black-Scholes formula The Black-Scholes formula was developed by Fischer Black and Myron Scholes in the 1970s. The Black-Scholes formula is a stochastic partial differential equation, which estimates the price an the option. The main idea behind the formula is the delta neutral portfolio. They created the theoretical delta neutral portfolio, to reduce the uncertainty involved. This was a necessary step to be able to come to the analytical formula which we’ll cover in this section. Below is the assumptions made under Black-Scholes: No arbitrage Possible to borrow money at a constant risk-free interest rate (throughout the holding of the option) Possible to buy, sell and short fractional amounts of underlying asset No transaction costs Price of underlying follows a Brownian Motion, constant drift and volatility No dividends paid from underlying security The simplest of the two variants is the one for call options. First the stock price is scaled using the cumulative distribution function with d1 as a parameter. Then the stock price is reduced by the discounted strike price scaled by the cumulative distribution function of d2. In other words, it’s the difference between the stock price and the strike using probability scaling of each and discounting the strike price. The formula for the put is a little more involved, but follows the same principles. The Black-Scholes formula are often separated into parts, where d1, d2 are the probability factors, describing the probability of the stock price being related to the strike price. The parameters used in the formula above can be summarized as follows: N – The cumulative distribution function T - Time to maturity, expressed in years S – The stock price, or other underlying K – The strike price r – The risk free interest rate σ – The volatility of the underlying Implementing Black-Scholes in F# Now that we've looked at the basics behind the Black-Scholes formula, and the parameters involved, we can implement it ourselves. The cumulative distribution function is implemented here to avoid dependencies and to illustrate that it's quite simple to implement it yourself too. Below is the Black-Scholes implemented in F#. It takes six arguments; the first is a call-put-flag that determines if it's a call or put option. The constants a1 to a5 are the Taylor series coefficients used in the approximation for the numerical implementation. let pow x n = exp(n * log(x)) type PutCallFlag = Put | Call /// Cumulative distribution function let cnd x = let a1 = 0.31938153 let a2 = -0.356563782 let a3 = 1.781477937 let a4 = -1.821255978 let a5 = 1.330274429 let pi = 3.141592654 let l = abs(x) let k = 1.0 / (1.0 + 0.2316419 * l) let w = (1.0-1.0/sqrt(2.0*pi)*exp(-l*l/2.0)*(a1*k+a2*k*k+a3*(pow k 3.0)+a4*(pow k 4.0)+a5*(pow k 5.0))) if x < 0.0 then 1.0 - w else w /// Black-Scholes // call_put_flag: Put | Call // s: stock price // x: strike price of option // t: time to expiration in years // r: risk free interest rate // v: volatility let black_scholes call_put_flag s x t r v = let d1=(log(s / x) + (r+v*v*0.5)*t)/(v*sqrt(t)) let d2=d1-v*sqrt(t) //let res = ref 0.0 match call_put_flag with | Put -> x*exp(-r*t)*cnd(-d2)-s*cnd(-d1) | Call -> s*cnd(d1)-x*exp(-r*t)*cnd(d2) Let's use the black_scholes function using some various numbers for call and put options. Suppose we want to know the price of an option, where the underlying is a stock traded at $58.60 with an annual volatility of 30%. The risk free interest rate is, let's say, 1%. Then we can use our formula, we defined previously to get the theoretical price according the Black-Scholes formula of a call option with 6 month to maturity (0.5 years): > black_scholes Call 58.60 60.0 0.5 0.01 0.3;; val it : float = 4.465202269 And the value for the put option, just by changing the flag to the function: > black_scholes Put 58.60 60.0 0.5 0.01 0.3;; val it : float = 5.565951021 Sometimes it's more convenient to express the time to maturity in number of days, instead of years. Let's introduce a helper function for that purpose. /// Convert the nr of days to years let days_to_years d = (float d) / 365.25 Note the number 365.25 which includes the factor for leap years. This is not necessary in our examples, but used for correctness. We can now use this function instead, when we know the time in days. > days_to_years 30;; val it : float = 0.08213552361 Let's use the same example above, but now with 20 days to maturity. > black_scholes Call 58.60 60.0 (days_to_years 20) 0.01 0.3;; val it : float = 1.065115482 > black_scholes Put 58.60 60.0 (days_to_years 20) 0.01 0.3;; val it : float = 2.432270266 Using Black-Scholes together with Charts Sometimes it's useful to be able to plot the price of an option until expiration. We can use our previously defined functions and vary the time left and plot the values coming out. In this example we'll make a program that outputs the graph seen below. Chart showing prices for call and put option as function of time /// Plot price of option as function of time left to maturity #r "System.Windows.Forms.DataVisualization.dll" open System open System.Net open System.Windows.Forms open System.Windows.Forms.DataVisualization.Charting open Microsoft.FSharp.Control.WebExtensions /// Create chart and form let chart = new Chart(Dock = DockStyle.Fill) let area = new ChartArea("Main") chart.ChartAreas.Add(area) chart.Legends.Add(new Legend()) let mainForm = new Form(Visible = true, TopMost = true, Width = 700, Height = 500) do mainForm.Text <- "Option price as a function of time" mainForm.Controls.Add(chart) /// Create series for call option price let optionPriceCall = new Series("Call option price") do optionPriceCall.ChartType <- SeriesChartType.Line do optionPriceCall.BorderWidth <- 2 do optionPriceCall.Color <- Drawing.Color.Red chart.Series.Add(optionPriceCall) /// Create series for put option price let optionPricePut = new Series("Put option price") do optionPricePut.ChartType <- SeriesChartType.Line do optionPricePut.BorderWidth <- 2 do optionPricePut.Color <- Drawing.Color.Blue chart.Series.Add(optionPricePut) /// Calculate and plot call option prices let opc = [for x in [(days_to_years 20)..(-(days_to_years 1))..0.0]do yield black_scholes Call 58.60 60.0 x 0.01 0.3] do opc |> Seq.iter (optionPriceCall.Points.Add >> ignore) /// Calculate and plot put option prices let opp = [for x in [(days_to_years 20)..(-(days_to_years 1))..0.0]do yield black_scholes Put 58.60 60.0 x 0.01 0.3] do opp |> Seq.iter (optionPricePut.Points.Add >> ignore) The code is just a modified version of the code seen in the previous article, with the options parts added. We have two series in this chart, one for call options and one for put options. We also add a legend for each of the series. The last part is the calculation of the prices and the actual plotting. List comprehensions are used for compact code, and the Black-Scholes formula is called for everyday until expiration, where the days are counted down by one day at each step. It's up to you as a reader to modify the code to plot various aspects of the option, such as the option price as a function of an increase in the underlying stock price etc. Introducing the greeks The greeks are partial derivatives of the Black-Scholes formula, with respect to a particular parameter such as time, rate, volatility or stock price. The greeks can be divided into two or more categories, with respect to the order of the derivatives. Below we'll look at the first and second order greeks. First order greeks In this section we'll present the first order greeks using the table below. Name Symbol Description Delta Δ Rate of change of option value with respect to change in the price of the underlying asset. Vega ν Rate of change of option value with respect to change in the volatility of the underlying asset. Referred to as the volatility sensitivity. Theta Θ Rate of change of option value with respect to time. The sensitivity with respect to time will decay as time elapses, phenomenon referred to as the "time decay." Rho ρ Rate of change of option value with respect to the interest rate. Second order greeks In this section we'll present the second order greeks using the table below. Name Symbol Description Gamma Γ Rate of change of delta with respect to change in the price of the underlying asset. Veta - Rate of change in Vega with respect to time. Vera - Rate of change in Rho with respect to volatility. Some of the second order greeks are omitted for clarity, we'll not cover these in this book. Implementing the greeks in F# Let's implement the greeks; Delta, Gamma, Vega, Theta and Rho. First we look at the formulas for each greek. In some of the cases they vary for calls and puts respectively. We need the derivative of the cumulative distribution function, which in fact is the normal distribution with zero mean and standard deviation of one: /// Normal distribution open MathNet.Numerics.Distributions; let normd = new Normal(0.0, 1.0) Delta Delta is the rate of change of option price with respect to change in the price of the underlying asset. /// Black-Scholes Delta // call_put_flag: Put | Call // s: stock price // x: strike price of option // t: time to expiration in years // r: risk free interest rate // v: volatility let black_scholes_delta call_put_flag s x t r v = let d1=(log(s / x) + (r+v*v*0.5)*t)/(v*sqrt(t)) match call_put_flag with | Put -> cnd(d1) - 1.0 | Call -> cnd(d1) Gamma Gamma is the rate of change of delta with respect to change in the price of the underlying asset. This is the 2nd derivative, with respect to price of the underlying asset. It measures the acceleration of the price of the option with respect to the underlying price. /// Black-Scholes Gamma // s: stock price // x: strike price of option // t: time to expiration in years // r: risk free interest rate // v: volatility let black_scholes_gamma s x t r v = let d1=(log(s / x) + (r+v*v*0.5)*t)/(v*sqrt(t)) normd.Density(d1) / (s*v*sqrt(t) Vega Vega is the rate of change of option value with respect to change in the volatility of the underlying asset. It is referred to as the volatility sensitivity. /// Black-Scholes Vega // s: stock price // x: strike price of option // t: time to expiration in years // r: risk free interest rate // v: volatility let black_scholes_vega s x t r v = let d1=(log(s / x) + (r+v*v*0.5)*t)/(v*sqrt(t)) s*normd.Density(d1)*sqrt(t) Theta Theta is the rate of change of option value with respect to time. The sensitivity with respect to time will decay as time elapses, phenomenon referred to as the “time decay.” /// Black-Scholes Theta // call_put_flag: Put | Call // s: stock price // x: strike price of option // t: time to expiration in years // r: risk free interest rate // v: volatility let black_scholes_theta call_put_flag s x t r v = let d1=(log(s / x) + (r+v*v*0.5)*t)/(v*sqrt(t)) let d2=d1-v*sqrt(t) let res = ref 0.0 match call_put_flag with | Put -> -(s*normd.Density(d1)*v)/(2.0*sqrt(t))+r*x*exp(-r*t)*cnd(-d2) | Call -> -(s*normd.Density(d1)*v)/(2.0*sqrt(t))-r*x*exp(-r*t)*cnd(d2) Rho Rho is rate of change of option value with respect to the interest rate. /// Black-Scholes Rho // call_put_flag: Put | Call // s: stock price // x: strike price of option // t: time to expiration in years // r: risk free interest rate // v: volatility let black_scholes_rho call_put_flag s x t r v = let d1=(log(s / x) + (r+v*v*0.5)*t)/(v*sqrt(t)) let d2=d1-v*sqrt(t) let res = ref 0.0 match call_put_flag with | Put -> -x*t*exp(-r*t)*cnd(-d2) | Call -> x*t*exp(-r*t)*cnd(d2) Investigating the sensitivity of the of the greeks Now that we have all the greeks implemented we'll investigate the sensitivity of some of them and see how they vary when the underlying stock price changes. The figure below is a surface plot with four of the greeks where time and underlying price is changing. The figure below is generated in MATLAB, and will not be generated in F#. We’ll use a 2D version of the graph to study the greeks below. Surface plot of Delta, Gamma, Theta and Rho of a call option. In this section we'll start by plotting the value of Delta for a call option where we vary the price of the underlying. This will result in the following 2D plot: A plot of call option delta versus price of underlying The result in the plot seen in figure above will be generated by the code presented next. We'll reuse most of the code from the example where we looked at the option prices for calls and puts. A slightly modified version is presented here, where the price of the underlying varies from $10.0 to $70.0. /// Plot delta of call option as function of underlying price #r "System.Windows.Forms.DataVisualization.dll" open System open System.Net open System.Windows.Forms open System.Windows.Forms.DataVisualization.Charting open Microsoft.FSharp.Control.WebExtensions /// Create chart and form let chart = new Chart(Dock = DockStyle.Fill) let area = new ChartArea("Main") chart.ChartAreas.Add(area) chart.Legends.Add(new Legend()) let mainForm = new Form(Visible = true, TopMost = true, Width = 700, Height = 500) do mainForm.Text <- "Option delta as a function of underlying price" mainForm.Controls.Add(chart) /// Create series for call option delta let optionDeltaCall = new Series("Call option delta") do optionDeltaCall.ChartType <- SeriesChartType.Line do optionDeltaCall.BorderWidth <- 2 do optionDeltaCall.Color <- Drawing.Color.Red chart.Series.Add(optionDeltaCall) /// Calculate and plot call delta let opc = [for x in [10.0..1.0..70.0] do yield black_scholes_delta Call x 60.0 0.5 0.01 0.3] do opc |> Seq.iter (optionDeltaCall.Points.Add >> ignore) We can extend the code to plot all four greeks, as in the figure with the surface plots, but here in 2D. The result will be a graph like seen in the figure below. Graph showing the for Greeks for a call option with respect to price change (x-axis). Code listing for visualizing the four greeks Below is the code listing for the entire program used to create the graph above. #r "System.Windows.Forms.DataVisualization.dll" open System open System.Net open System.Windows.Forms open System.Windows.Forms.DataVisualization.Charting open Microsoft.FSharp.Control.WebExtensions /// Create chart and form let chart = new Chart(Dock = DockStyle.Fill) let area = new ChartArea("Main") chart.ChartAreas.Add(area) chart.Legends.Add(new Legend()) let mainForm = new Form(Visible = true, TopMost = true, Width = 700, Height = 500) do mainForm.Text <- "Option delta as a function of underlying price" mainForm.Controls.Add(chart) We’ll create one series for each greek: /// Create series for call option delta let optionDeltaCall = new Series("Call option delta") do optionDeltaCall.ChartType <- SeriesChartType.Line do optionDeltaCall.BorderWidth <- 2 do optionDeltaCall.Color <- Drawing.Color.Red chart.Series.Add(optionDeltaCall) /// Create series for call option gamma let optionGammaCall = new Series("Call option gamma") do optionGammaCall.ChartType <- SeriesChartType.Line do optionGammaCall.BorderWidth <- 2 do optionGammaCall.Color <- Drawing.Color.Blue chart.Series.Add(optionGammaCall) /// Create series for call option theta let optionThetaCall = new Series("Call option theta") do optionThetaCall.ChartType <- SeriesChartType.Line do optionThetaCall.BorderWidth <- 2 do optionThetaCall.Color <- Drawing.Color.Green chart.Series.Add(optionThetaCall) /// Create series for call option vega let optionVegaCall = new Series("Call option vega") do optionVegaCall.ChartType <- SeriesChartType.Line do optionVegaCall.BorderWidth <- 2 do optionVegaCall.Color <- Drawing.Color.Purple chart.Series.Add(optionVegaCall) Next, we’ll calculate the values to plot for each greek: /// Calculate and plot call delta let opd = [for x in [10.0..1.0..70.0] do yield black_scholes_delta Call x 60.0 0.5 0.01 0.3] do opd |> Seq.iter (optionDeltaCall.Points.Add >> ignore) /// Calculate and plot call gamma let opg = [for x in [10.0..1.0..70.0] do yield black_scholes_gamma x 60.0 0.5 0.01 0.3] do opg |> Seq.iter (optionGammaCall.Points.Add >> ignore) /// Calculate and plot call theta let opt = [for x in [10.0..1.0..70.0] do yield black_scholes_theta Call x 60.0 0.5 0.01 0.3] do opt |> Seq.iter (optionThetaCall.Points.Add >> ignore) /// Calculate and plot call vega let opv = [for x in [10.0..1.0..70.0] do yield black_scholes_vega x 60.0 0.1 0.01 0.3] do opv |> Seq.iter (optionVegaCall.Points.Add >> ignore) Summary In this article, we looked into using F# for investigating different aspects of volatility. Volatility is an interesting dimension of finance where you quickly dive into complex theories and models. Here it's very much helpful to have a powerful tool such as F# and F# Interactive. We've just scratched the surface of options and volatility in this article. There is a lot more to cover, but that's outside the scope of this book. Most of the content here will be used in the trading system. resources for article: further resources on this subject: Working with Windows Phone Controls [article] Simplifying Parallelism Complexity in C# [article] Watching Multiple Threads in C# [article]
Read more
  • 0
  • 0
  • 3148

article-image-making-unit-very-mobile-controlling-legged-movement
Packt
20 Dec 2013
10 min read
Save for later

Making the Unit Very Mobile - Controlling Legged Movement

Packt
20 Dec 2013
10 min read
(for more resources related to this topic, see here.) Mission briefing We've covered creating robots using a wheeled/track base. In this article, you will be introduced to some of the basics of servo motors and using the BeagleBone Black to control the speed and direction of your legged platform. Here is an image of a finished project: Why is it awesome? Even though you've learned to make your robot mobile by adding wheels or tracks, this mobile platform will only work well on smooth, flat surfaces. Often, you'll want your robot to work in environments where it is not smooth or flat; perhaps, you'll even want your robot to go upstairs or over curbs. In this article, you'll learn how to attach your board, both mechanically and electrically, to a platform with legs, so your projects can be mobile in many more environments. Robots that can walk: what could be more amazing than that? Your objectives In this article, you will learn: Connecting the BeagleBone Black to a mobile platform using a servo controller Creating a program in Linux to control the movement of the mobile platform Making your mobile platform truly mobile by issuing voice commands   Mission checklist In this article, you'll need to add a legged platform to make your project mobile. So, here is your parts' list: A legged robot: There are a lot of choices. As before, some are completely assembled, others have some assembly required, and you may even choose to buy the components and construct your own custom mobile platform. Also, as before, I'm going to assume that you don't want to do any soldering or mechanical machining yourself, so let's look at a several choices that are available completely assembled or can be assembled by simple tools (screwdriver and/or pliers). One of the easiest legged mobile platforms is one that has two legs and four servo motors. Here is an image of this type of platform: You'll use this platform in this article because it is the simplest to program and because it is the least expensive, requiring only four servos. To construct this platform, you must purchase the parts and then assemble it yourself. Find the instructions and parts list at http://www.lynxmotion.com/images/html/build112.htm. Another easy way to get all the mechanical parts (except servos) is to purchase a biped robot kit with six degrees of freedom (DOF). This will contain the parts needed to construct your four-servo biped. These six DOF bipeds can be purchased by searching eBay or by going to http://www.robotshop.com/2-wheeled-development-platforms-1.html. You'll also need to purchase the servo motors. For this type of robot, you can use standard size servos. I like the Hitec HS-311 or HS-322 for this robot. They are inexpensive but powerful enough. You can get those on Amazon or eBay. Here is an image of an HS-311: You'll need a mobile power supply for the BeagleBone Black. Again, I personally like the 5V cell phone rechargeable batteries that are available almost anywhere that supplies cell phones. Choose one that comes with two USB connectors, just in case you want to also use the powered USB hub. This one mounts well on the biped HW platform: You'll also need a USB cable to connect your battery to the BeagleBone Black, but you can just use the cable supplied with the BeagleBone Black. If you want to connect your powered USB hub, you'll need a USB to DC jack adapter for that as well. You'll also need a way to connect your batteries to the servo motor controller. Here is an image of a four AA battery holder, available at most electronics parts stores or from Amazon: Now that you have the mechanical parts for your legged mobile platform, you'll need some HW that will take the control signals from your BeagleBone Black and turn them into a voltage that can control the servo motors. Servo motors are controlled using a control signal called PWM. For a good overview of this type of control, see http://pcbheaven.com/wikipages/How_RC_Servos_Works/ or https://www.ghielectronics.com/docs/18/pwm. You can find tutorials that show you how to control servos directly using the BeagleBone Black's GPIO pins, for example, here at http://learn.adafruit.com/controlling-a-servowith-a-beaglebone-black/overview and http://www.youtube.com/watch?v=6gv3gWtoBWQ. For ease of use I chose to purchase a motor controller that can talk over USB and control the servo motor. These protect my board and make controlling many servos easy. My personal favorite for this application is a simple servo motor controller utilizing USB from Pololu that can control 18 servo motors. Here is an image of the unit: Again, make sure you order the assembled version. This piece of HW will turn USB commands into voltage that control your servo motors. Pololu makes a number of different versions of this controller, each able to control a certain number of servos. Once you've chosen your legged platform, simply count the number of servos you need to control, and chose the controller that can control that number of servos. One advantage of the 18 servo controller is the ease of connecting power to the unit via screw type connectors. Since you are going to connect this controller to your BeagleBone Black via USB, you'll also need a USB A to mini-B cable. Now that you have all the HW, let's walk through a quick tutorial on how a two-legged system with servos works and then some step-by-step instructions to make your project walk. Connecting the BeagleBone Black to the mobile platform using a servo controller Now that you have a legged platform and a servo motor controller, you are ready to make your project walk! Prepare for lift off Before you begin, you'll need some background on servo motors. Servo motors are somewhat similar to DC motors; however, there is an important difference. While DC motors are generally designed to move in a continuous way—rotating 360 degrees at a given speed—servos are generally designed to move within a limited set of angles. In other words, in the DC motor world, you generally want your motors to spin with continuous rotation speed that you control. In the servo world, you want your motor to move to a specific position that you control. Engage thrusters To make your project walk, you first need to connect the servo motor controller to the servos. There are two connections you need to make: the first to the servo motors, the second to the battery holder. In this section, you'll connect your servo controller to your PC to check to see if everything is working. First, connect the servos to the controller. Here is an image of your two-legged robot, and the four different servo connections: In order to be consistent, let's connect your four servos to the connections marked 0 through 3 on the controller using this configuration: 0 – left foot, 1 – left hip, 2 – right foot, and 3 – right hip. Here is an image of the back of the controller; it will tell you where to connect your servos: Connect these to the servo motor controller like this: the left foot to the top O connector, black cable to the outside (–), the left hip to the 1 connector, black cable out, right foot to the 2 connector, black cable out, and right hip to the 3 connector, black cable out. See the following image for a clearer description: Now you need to connect the servo motor controller to your battery. If you are using a standard 4 AA battery holder, connect it to the two green screw connectors, the black cable to the outside, and the red cable to the inside, as shown in the following image: Now you can connect the motor controller to your PC to see if you can talk with it.   Objective complete – mini debriefing Now that the HW is connected, you can use some SW provided by Polulu to control the servos. It is easiest to do this using your personal computer. First, download the Polulu SW from http://www.pololu.com/docs/0J40/3.a and install it based on the instructions on the website. Once it is installed, run the SW, and you should see the following screen: You first will need to change the configuration on Serial Settings, so select the Serial Settings tab, and you should see a screen as shown in the following screenshot: Make sure that the USB Chained option is selected; this will allow you to connect and control the motor controller over USB. Now go back to the main screen by selecting the Status tab, and now you can turn on the four servos. The screen should look like the following screenshot: Now you can use the sliders to control the servos. Make sure that the servo 0 moves the left foot, 1 the left hip, 2 the right foot, and 3 the right hip. You've checked the motor controllers and the servos, and you'll now connect the motor controller up to the BeagleBone Black control the servos from it. Remove the USB cable from the PC and connect it into the powered USB hub. The entire system will look like the following image: Let's now talk to the motor controller by downloading the Linux code from Pololu at http://www.pololu.com/docs/0J40/3.b. Perhaps, the best way is to log in to your Beagle Bone Black by using vncserver and a vncviewer window on your PC. To do this, log in to your BeagleBone Black using PuTTY, then type vncserver at the prompt to make sure vncserver is running. On your PC open the VNC Viewer application, enter your IP address, then press connect. Then enter your password that you created for the vncserver, and you should see the BeagleBone Black Viewer screen, which should look like this: Open a Firefox browser window and go to http://www.pololu.com/docs/0J40/3.b. Click on the Maestro Servo Controller Linux Software link. You will download the file maestro_linux_100507.tar.gz to the Download directory. Go to your download directory, move this file to your home directory by typing mv maestro_linux_100507.tar.gz .. and then you can go back to your home directory. Unpack the file by typing tar –xzfv maestro_linux_011507.tar.gz. This will create a directory called maestro_linux. Go to that directory by typing cd maestro_linux and then type ls. You should see something like this: The document README.txt will give you explicit instructions on how to install the SW. Unfortunately you can't run MaestroControlCenter on your BeagleBone Black. Our version of windowing doesn't support the graphics, but you can control your servos using the UscCmd command-line application. First type ./UscCmd --list and you should see the following: The unit sees your servo controller. If you just type ./UscCmd you can see all the commands you could send to your controller: Notice you can send a servo a specific target angle, although the target is not in angle values, so it makes it a bit difficult to know where you are sending your servo. Try typing ./UscCmd --servo 0, 10. The servo will most likely move to its full angle position. Type ./UscCmd --servo 0, 0 and it will stop the servo from trying to move. In the next section, you'll write some SW that will translate your angles to the commands that the servo controller will want to see. If you didn't run the Windows version of Maestro Controller and set the serial settings to USB Chained, your motor controller may not respond.
Read more
  • 0
  • 0
  • 4060

article-image-what-lumion
Packt
20 Dec 2013
9 min read
Save for later

What is Lumion?

Packt
20 Dec 2013
9 min read
(For more resources related to this topic, see here.) Why use Lumion? The short answer is that Lumion is easy to use and the final product is of a good quality. The long answer is that every construction project needs technical drawings and documents. Although this technical information is fine for a construction crew, usually the client has no idea what a CAD plan means. They can have an idea where the kitchen or the living room will be, but translating that 2D information to 3D is not always easy in the client's mind. This can be an issue if we need to give a presentation or if we are trying to sell something that is not built yet. And truth be told, an image sells more than words. That's where Lumion comes in. Lumion is the fastest way to render high quality still pictures and videos, and it makes it so easy to import our 3D models from any 3D modeling software, such as SketchUp, AutoCAD, Revit, ArchiCAD, and 3ds Max, and create a scene in minutes. So, Lumion 3D is a distinct architectural visualization software not only because it is faster to render, but also because it is very user friendly and intuitive. Another reason why we can use Lumion to create architectural visualizations is because we can have a great idea of how our project will look in natural surroundings at any time of the day or season, and this in just a few minutes. Now if you are an architect, it is doubtless that you want to enhance your project characteristics in the best way possible. Lumion can help you achieve this in hours instead of the inevitable days and weeks of rendering time. The following screenshot is an example of what you can get with Lumion in just a few minutes: However, this tool is not exclusively meant for architects. For example, if you are an interior designer, you may want to present how the textures, furniture, and colors would look at different angles, places, moods, and light conditions. Lumion provides nice interior visualization with furniture, good lighting, and realistic textures. In conclusion, Lumion is a great tool that improves the process of creating a building, or an art, or an architectural project. The time we need to get those results is less in comparison to other solutions such as using 3ds Max and V-Ray. What can we get from Lumion? Asking what Lumion can us is a double-edged question. Looking at the previous screenshots, we can get an idea of the final result. The final quality depends only on your creativity and time. I have seen amazing videos created with Lumion, but then you may need a touch of other software to create eye-catching compositions. Now, the package that we get with Lumion is another thing. You already know that we can easily create beautiful still images and videos, but we need to bear in mind that Lumion is not a tool designed to create photo-realistic renders. Nevertheless, we can get so much from this application that you will forget photo-realistic renders. Lumion is a powerful and advanced 3D engine that allows us to work with heavier models, and we can make our scene come alive with the click-and-drag technique. To do this, Lumion comes with a massive library where we can find: 2409 models of trees, transports, exterior and interior models, and people 103 ambient, people, and machine sounds 518 types of materials and textures 28 types of landscape In addition to this extensive collection, there are more features that we can add; we can include realistic water in our scene (oceans, rivers, pools, waterfalls, and fountains), we can sculpt the landscape to adapt to our needs, and we can add rain, snow, fog, wind, and animate objects, and we can add camera effects. You just need a blank 3D model; import and start working because, as you can see, Lumion is well equipped with almost everything we need to create architectural visualisations. Lumion's 3D interface Now that we know what we can do with Lumion and the content available, we will take some time to explore Lumion and get our hands dirty. In my opinion and experience, it is much easier to learn something if at the same time we apply what we are learning. So, in the next section we are going to explore the Lumion interface with the menus and different settings. But to do that we will use a small tutorial as a quick start. By doing this, we will explore Lumion and at the same time see how easy it is to produce great results. We will see that Lumion is easier to learn and more accessible than other software. So go ahead and fire up Lumion and let's have a quick tour before we start working with it. I am going to explain to you what each tab does, to help you see how you can do simple tasks, such as saving and loading a scene, changing the settings, and creating a new scene. Let's start with the first tab that Lumion shows us. A look into the New tab On startup, Lumion goes straight to the New tab. The New tab, as the name indicates, is a good place to start when you want to create a new scene. We can create a new scene based on a few presets or just create an empty scene. We can choose from Night, Sunset, Sunny Day, Flatlands, Island, Lake, Desert, Cold Climate, and an Empty scene. I found these presets as a quick help to cut some time, because in the end everything we get from these presets, we can create in a few minutes. So, there is nothing special about them. When you start Lumion, this will be the first thing you will see: The nine presets you can find on the New tab Now, we will finally see the Objects menu and the following is what this menu looks like: The Objects menu Here is where the fun starts. We have at our disposal eight categories of objects and more, such as Nature, Transport, Sound, Effects, Indoor, People and Animals, Outdoor, and Lights and special objects. Each of these menus has subcategories. If you are working with the Lumion PRO version, you can choose from more than 2000 models. Even if you don't have this version, cheer up! You can still import your own models and textures. It is really simple to add a model. First, we need to select the category we want to use. So in this case, click on the Nature button. Now that we have this category selected, click on the thumbnail above the Place button and a new window with the Nature library will appear. We don't have just trees, we have grass, plants, flowers, rocks, and clusters. Now let me show you one trick. Click on the Grass tab and select Rough_Grass1_RT. Now that we are back to the Build mode, press the Ctrl key and click on the ground. We are randomly adding 10 copies of the object, which in this case is really handy. So, after playing a little with Lumion, we can get something like the following screenshot: Our scene after adding some trees, grass, and animals Just think, it took me about 30 minutes to create something like this. Now imagine what you can do. Let's save our scene and turn our attention to the right-hand side of the Lumion 3D interface, where we can find the menus as shown in the following screenshot: The Build mode button Starting from the top of the preceding screenshot we can see the blue rectangle with a question mark. If we put our mouse cursor over this rectangle, we can see a quick help guideline for our interface. The next button informs us that we are in the Build mode and if, for example, you are working in the Photo or Video mode, this button lets you go back to the scene. Lumion materials Lumion helps us with this important step by offering many types of materials: 518 materials that are ready to use. You may need to do some adjustments, but the major hard work was already done for you. The materials that we can assign to our model are as follows: Wood: 45 materials Wood floor : 67 materials Brick: 32 materials Tiles: 99 materials Ground: 39 materials Concrete: 43 materials Carpet: 20 materials Misc: 109 materials Asphalt: 12 materials Metal: 47 materials This is one of the reasons why it is so easy to create still images and videos with Lumion. Everything is set up for you, including parameters, details, and textures. However, we may need to do some minor adjustments, and for that, it is important to understand or at least have a basic notion of what each setting does. So, let's have a quick look at how we can configure materials in Lumion. The Landscape material The best way to explain this material is by showing you an example. So, let's say that along with the model, you also created a terrain like the one you can see in the following screenshot: The house along with a terrain model Import the terrain if needed, and add a new material to this terrain. Go to the Custom menu and click on the Landscape material. The Landscape material allows you to seamlessly blend or merge parts of the model with the landscape. Make sure that the terrain intersects with the ground so that it can be perfectly blended. The following screenshot shows this Landscape material applied to my 3D terrain: The terrain model merged with the landscape Adding this material not only allows you to use this cool feature, but as you can see in the picture, we can also start painting soil types of the landscape in the imported terrain. The other two materials that I want to introduce you to are the Standard and Water materials. The Standard material is a simple material without any textures or settings, and we can use this material to start something from scratch. The Water material can have several applications, but perhaps, the most common one is, for example, pools. Summary This article helped you in starting with Lumion, and gave you a taste of how easily and quickly you can get great images and videos. In particular, you have learned the basic steps to save and load scenes, import models, add materials, change the terrain and weather, and create photos and videos. You also learned how to use and configure the prebuilt materials in Lumion and found out how to use the Landscape material to create a terrain. Resources for Article: Further resources on this subject: The Spotfire Architecture Overview [Article] augmentedTi: The application architecture [Article] The architecture of JavaScriptMVC [Article]
Read more
  • 0
  • 0
  • 5474

article-image-welcome-paintnet
Packt
20 Dec 2013
3 min read
Save for later

Welcome to Paint.NET

Packt
20 Dec 2013
3 min read
(For more resources related to this topic, see here.) System requirements The minimum system requirements needed to run Paint.NET are as follows: Windows 7 (recommended) or Windows XP .NET Framework 3.5 SP1 A 800 MHz processor 512 MB of RAM More than 200 MB of hard drive space A 64-bit CPU and a 64-bit edition of Windows for 64-bit support (this is optional) A 1024 x 768 screen resolution At present, there is no Mac version of this product. Downloading and installing Paint.NET To download Paint.NET, go to www.getpaint.net/download.html. In order for Paint.NET to pay a few bills and keep the servers going, the people who maintain Paint.NET have placed a few ads on the site that sometimes lead to other programs. You will most likely not want to download these programs. If you want to download only the Paint.NET program, make sure you click on the link that looks like the following: It can be rather confusing, so choose wisely. Once you hit the correct link, it will take you to one of the Paint.NET mirror download sites. Hit the download link that looks like the following: A ZIP file should begin downloading onto your computer. Once this is done, open the ZIP file and you will find a file named Paint.NET.3.5.11.Install.exe. Run this file and the installation will begin. The installer will take you through a series of steps. For simplicity, choose Quick Installation and then hit Next. Read the terms and conditions, and if you agree to them, choose I Agree and hit Next. As the program installs, it will give you an opportunity to donate to the program if you wish. Because Paint.NET is a freeware, it only survives on donations and the time people put into it. So, if you like the program, donating here is one way to help out. When the software has finished installing, click on Finish and Paint.NET will open automatically. Once the program launches, you will see a screen similar to the following screenshot: This is the entire Paint.NET work area. If you have used photo editing software like Photoshop, some of what you see in the preceding screenshot may look a bit familiar to you. If you are not used to an image editing program, all of these windows may look a little overwhelming. Work area windows Let's take a look at the Paint.NET work area: Paint.NET is comprised of a series of windows, each with their own function to help you with your workflow. The following is a list of the various windows available on Paint.NET; the numbers correspond to the labels given in the previous screenshot: The title bar The menu bar The toolbar The image canvas The Colors window The status bar The Layers window The History window The image list Summary In this article, we learned about Paint.NET and went over the minimum requirements of the software and how to install it. We also learned about the Paint.NET work area and enumerated the various windows associated with it. Resources for Article: Further resources on this subject: Painting – Multi-finger Paint [Article] ExpressionEngine: Creating a Photo Gallery [Article] Photo Stream with iCloud [Article]
Read more
  • 0
  • 0
  • 2274
Unlock access to the largest independent learning library in Tech for FREE!
Get unlimited access to 7500+ expert-authored eBooks and video courses covering every tech area you can think of.
Renews at €18.99/month. Cancel anytime
article-image-key-components-and-inner-working-impala
Packt
20 Dec 2013
7 min read
Save for later

Key components and inner working of Impala

Packt
20 Dec 2013
7 min read
(For more resources related to this topic, see here.) Impala Core Components: Here we will discuss following three important components: Impala Daemon Impala Statestore Impala Metadata and Metastore Putting together above components with Hadoop and application or command line interface, we can conceptualize them as below: Impala Execution Architecture: Essentially Impala daemons receives queries from variety of sources and distribute query load to other Impala daemons running on other nodes and while doing so interact with Statestore for node specific update and access Metastore, either stored in centralized database or in local cache. Now to complete the Impala execution we will discuss how Impala interacts with other components i.e. Hive, HDFS and HBase.  Impala working with Apache Hive: We have already discussed earlier about Impala Metastore using the centralized database as Metastore and Hive also uses the same MySQL or PostgreSQL database for same kind of data. Impala provides same SQL like queries interface use in Apache Hive. Because both Impala and Hive share same database as Metastore, Impala can access Hive specific tables definitions if Hive table definition use the same file format, compression codecs and Impala-supported data types in their column values. Apache Hive provides various kinds of file type processing support to Impala. When using other then text file format i.e. RCFile, Avro, SequenceFile the data must be loaded through Hive first and then Impala can query the data from these file formats. Impala can perform read operation on more types of data using SELECT statement than it can perform write operation using INSERT statement. The ANALYZE TABLE statement in Hive generates useful table and column statistics and Impala use these valuable statistics to optimize the queries. Impala working with HDFS: Impala table data is actually regular data files stored in HDFS (Hadoop Distributed File System) and Impala uses HDFS as its primary data storage medium.  As soon as a data file or a collection of files is available in specific folder of new table, Impala reads all of the files regardless of their name and new data is included in files with the name controlled by Impala. HDFS provides data redundancy through replication factor and Impala relies on such redundancy to access data on other datanodes in case it is not available on a specific datanode. We have already learnt earlier that Impala also maintains the information about physical location of the blocks about data files in HDFS,which helps data access in case of node failure. Impala working with HBase: HBase is a distributed, scalable, big data storage system, provides random, real-time read and write access to data stored on HDFS. HBase is a database storage system, sits on top of HDFS however like other traditional database storage system, HBase does not provide built-in SQL support however 3party applications can provide such functionality. To use HBase, first user defines tables in Impala and then maps them to the equivalent HBase tables. Once table relationship is established, users can submit queries into HBase table through Impala. Not only that join operations can be formed including HBase and Impala tables. Impala Security: Impala is designed & developed on run on top of Hadoop. So you must understand the Hadoop security model as well as the security provided in OS where Hadoop is running. If Hadoop is running on Linux then as Linux administrator and Hadoop administrator user can harden and tighten the security, which definitely can be taken in account with the security provided by Impala. Impala 1.1 or later uses Sentry Open Source Project to provide detailed authorization framework for Hadoop. Impala 1.1.1 supports auditing capabilities in cluster by creating auditing data, which can be collected from all nodes and then processing for further analysis and insight. Data Visualization using Impala: Visualizing data is as important as processing the data. Human brain perceives pictures fast then reading data in tables and because of it data visualization provides super fast understanding to large amount of data in split seconds. Reports, charts, interactive dashboards and any form of info-graphics are all part of data visualization and provide deeper understanding of results. To connect with 3rd party applications, Cloudera provides ODBC and JDBC connectors. These connectors are installed on machines where 3rd party applications are running and by configuring correct Impala server and port details on those connectors, 3rd party applications connect with Impala and submit those queries and then take results back to application. The result then displayed on 3rd party application, where it is rendered on graphics device for visualization or displayed in table format or processed further depending on application requirement. In this section we will cover few notable 3rd party applications, which can take advantage of Impala super fast query processing and than display amazing graphical results. Tableau and Impala: Tableau Software supporting Impala by providing access to tables on Impala using Impala ODBC connector provided by Tableau. Tableau is one of the most prominent data visualization software technologies in recent days and used by thousands of enterprises daily to get intelligence out of their data. Tableau software is available on Windows OS and an ODBC connector is provided by Cloudera to make this connection a reality. You can visit the link below to download Impala connector for Tableau: http://go.cloudera.com/tableau_connector_download Once Impala connector is installed on a machine where Tableau software is running, and configured correctly, Tableau software is ready to work with Impala. In this image below Tableau is connected to Impala server at port 21000, and then selected a table located at Impala: Once table is selected, particular fields are select and data is displayed in graphical format in various mind-blowing visualizations. The screenshot below displays one example of showing such visualization:   Microsoft Excel and Impala Microsoft Excel is one of the widely adopted data processing application used by business professional worldwide. You can connect Microsoft Excel with Impala using another ODBC connector provided by Simba Technology. Microstrategy and Impala Microstrategy is another big player in data analysis and visualization software and uses ODBC drive to connect with Impala to render amazing looking visualizations. The connectivity model between Microstrategy software and Cloudera Impala is shown as below:    Zoomdata and Impala: Zoomdata is considered to new generation of data user interface by addressing streams of data instead of sets of data. Zoomdata processing engine performs continuous mathematical operations across data streams in real-time to create visualization on multitude of devices. The visualization updates itself as the new data arrives and re-computed by Zoomdata. As shown in in the image below, you can see Zoomdata application uses Impala as a source of data, which is configured underneath to use of one the available connectors to connect with Impala: Once connection are made user can see amazing data visualization as shown below: Real-time Query with Impala on Hadoop: Impala is marketed as a product, which can do “Real-time queries on Hadoop” by its developer Cloudera. Impala is open source implementation based on above-mentioned Google Dremel technology, available free for anyone of use. Impala is available as package product, free to use or can be compiled from its source, which can run queries in memory to make them real-time and in some cases depending on type of data, if Parquet file format is used as input data source, it can expedite the query processing to multifold speed.  Real-time query subscription with Impala: Cloudera provides Real-time Query (RTQ) Subscription as an add-on to Cloudera Enterprise subscription. You can still use Impala as free open source product however taking RTQ subscription makes you take advantage of Cloudera paid service to extend its usability and resilience. By accepting RTQ subscription you cannot only have access to Cloudera Technical support, but also you can work with Impala development team to provide ample feedback to shape up the product design and implementation. Summary Thus concludes the discussion on the key components of Impala and their inner working. Resources for Article: Further resources on this subject: Securing the Hadoop Ecosystem [Article] Cloudera Hadoop and HP Vertica [Article] Hadoop and HDInsight in a Heartbeat [Article]
Read more
  • 0
  • 0
  • 2651

article-image-transformation
Packt
20 Dec 2013
23 min read
Save for later

Apache Camel: Transformation

Packt
20 Dec 2013
23 min read
(For more resources related to this topic, see here.) The latest version of the example code for this article can be found at http://github.com/CamelCookbook/camel-cookbook-examples. You can also download the example code files for all Packt books you have purchased from your account at https://www.packtpub.com. If you purchased this book elsewhere, you can visit https://www.packtpub.com/books/content/support and register to have the files e-mailed directly to you. In this article we will explore a number of ways in which Camel performs message content transformation: Let us first look at some important concepts regarding transformation of messages in Camel: Using the transform statement. This allows you to reference Camel Expression Language code within the route to do message transformations. Calling a templating component, such as Camel's XSLT or Velocity template style components. This will typically reference an external template resource that is used in transforming your message. Calling a Java method (for example, beanref), defined by you, within a Camel route to perform the transformation. This is a special case processor that can invoke any referenced Java object method. Camel's Type Converter capability that can automatically cast data from one type to another transparently within your Camel route. This capability is extensible, so you can add your own Type Converters. Camel's Data Format capability that allows us to use built-in, or add our own, higher order message format converters. A Camel Data Format goes beyond simple data type converters, which handle simple data type translations such as String to int, or File to String. Data Formats are used to translate between a low-level representation (XML) and a high-level one (Java objects). Other examples include encrypting/decrypting data, and compressing/decompressing data. For more, see http://camel.apache.org/data-format.html. A number of Camel architectural concepts are used throughout this article. Full details can be found at the Apache Camel website at http://camel.apache.org. The code for this article is contained within the camel-cookbook-transformation module of the examples. Transforming using a Simple Expression When you want to transform a message in a relatively straightforward way, you use Camel's transform statement along with one of the Expression Languages provided by the framework. For example, Camel's Simple Expression Language provides you with a quick, inline mechanism for straightforward transformations. This recipe will show you how to use Camel's Simple Expression Language to transform the message body. Getting ready The Java code for this recipe is located in the org.camelcookbook.transformation.simple package. Spring XML files are located under src/main/resources/META-INF/spring and are prefixed with simple. How to do it... In a Camel route, use a transform DSL statement containing the Expression Language code to do your transformation. In the XML DSL, this is written as follows: <route> <from uri="direct:start"/> <transform> <simple>Hello ${body}</simple> </transform> </route> In the Java DSL, the same route is expressed as: from("direct:start") .transform(simple("Hello ${body}")); In this example, the message transformation prefixes the incoming message with the phrase Hello using the Simple Expression Language. The processing step after the transform statement will see the transformed message content in the body of the exchange. How it works... Camel's Simple Expression Language is quite good at manipulating the String content through its access to all aspects of the message being processed, through its rich String and logical operators. The result of your Simple Expression becomes the new message body after the transform step. This includes predicates such as using Simple's logical operators, to evaluate a true or false condition; the results of that Boolean operation become the new message body containing a String: "true" or "false". The advantage of using a distinct transform step within a route, as opposed to embedding it within a processor, is that the logic is clearly visible to other programmers. Ensure that the expression embedded within your route is kept simple so as to not distract the next developer from the overall purpose of the integration. It is best to move more complex (or just lengthy) transformation logic into its own subroute, and invoke it using direct: or seda:. There's more... The transform statement will work with any Expression Language available in Camel, so if you need more powerful message processing capabilities you can leverage scripting languages such as Groovy or JavaScript (among many others) as well. The Transforming inline with XQuery recipe will show you how to use the XQuery Expression Language to do transformations on XML messages. See also Message Translator: http://camel.apache.org/message-translator.html Camel Expression capabilities: http://camel.apache.org/expression.html Camel Simple Expression Language: http://camel.apache.org/simple.html Languages supported by Camel: http://camel.apache.org/languages.html The Transforming inline with XQuery recipe   Transforming inline with XQuery Camel supports the use of Camel's XQuery Expression Language along with the transform statement as a quick and easy way to transform an XML message within a route. This recipe will show you how to use an XQuery Expression to do in-route XML transformation. Getting ready The Java code for this recipe is located in the org.camelcookbook.transformation.xquery package. Spring XML files are located under src/main/resources/META-INF/spring and prefixed with xquery. To use the XQuery Expression Language, you need to add a dependency element for the camel-saxon library, which provides the implementation for the XQuery Expression Language. Add the following to the dependencies section of your Maven POM: <dependency> <groupId>org.apache.camel</groupId> <artifactId>camel-saxon</artifactId> <version>${camel-version}</version> </dependency>   How to do it... In the Camel route, specify a transform statement followed by the XQuery Expression Language code to do your transformation. In the XML DSL, this is written as: <route> <from uri="direct:start"/> <transform> <xquery> <books>{ for $x in /bookstore/book where $x/price>30 order by $x/title return $x/title }</books> </xquery> </transform> </route> When using the XML DSL, remember to XML encode the XQuery embedded XML elements. Therefore, < becomes &lt; and > becomes &gt;. In the Java DSL, the same route is expressed as: from("direct:start") .transform(xquery("<books>{ for $x in /bookstore/book " + "where $x/price>30 order by $x/title " + "return $x/title }</books>")); Feed the following input XML message through the transformation: <bookstore> <book category="COOKING"> <title lang="en">Everyday Italian</title> <author>Giada De Laurentiis</author> <year>2005</year> <price>30.00</price> </book> <book category="CHILDREN"> <title lang="en">Harry Potter</title> <author>J K. Rowling</author> <year>2005</year> <price>29.99</price> </book> <book category="PROGRAMMING"> <title lang="en">Apache Camel Developer's Cookbook</title> <author>Scott Cranton</author> <author>Jakub Korab</author> <year>2013</year> <price>49.99</price> </book> <book category="WEB"> <title lang="en">Learning XML</title> <author>Erik T. Ray</author> <year>2003</year> <price>39.95</price> </book> </bookstore> The resulting message will be: <books> <title lang="en">Apache Camel Developer's Cookbook</title> <title lang="en">Learning XML</title> </books> The processing step after the transform statement will see the transformed message content in the body of the exchange. How it works... Camel's XQuery Expression Language is a good way to inline XML transformation code within your route. The result of the XQuery Expression becomes the new message body after the transform step. All of the message's body, headers, and properties are made available to the XQuery Processor, so you can reference them directly within your XQuery statement. This provides you with a powerful mechanism for transforming XML messages. If you are more comfortable with XSLT, take a look at the Transforming with XSLT recipe. In-lining the transformation within your integration route can sometimes be an advantage as you can clearly see what is being changed. However, when the transformation expression becomes so complex that it starts to overwhelm the integration route, you may want to consider moving the transformation expression outside of the route. See the Transforming using a Simple Expression recipe for another inline transformation example, and see the Transforming with XSLT recipe for an example of externalizing your transformation. You can fetch the XQuery Expression from an external file using Camel's resource reference syntax. To reference an XQuery file on the classpath you can specify: <transform> <xquery>resource:classpath:/path/to/myxquery.xml</xquery> </transform> This is equivalent to using XQuery as an endpoint: <to uri="xquery:classpath:/path/to/myxquery.xml"/>   There's more... The XQuery Expression Language allows you to pass in headers associated with the message. These will show up as XQuery variables that can be referenced within your XQuery statements. Consider, from the previous example, to allow the value of the books that are filtered to be passed in with the message body, that is, parameterize the XQuery, you can modify the XQuery statement as follows: <transform> <xquery> declare variable $in.headers.myParamValue as xs:integer external; <books value='{$in.headers.myParamValue}'&gt;{ for $x in /bookstore/book where $x/price>$in.headers.myParamValue order by $x/title return $x/title }&lt;/books&gt; </xquery> </transform> Message headers will be associated with an XQuery variable called in.headers.<name of header>. To use this in your XQuery, you need to explicitly declare an external variable of the same name and XML Schema (xs:) type as the value of the message header. The transform statement will work with any Expression Language enabled within Camel, so if you need more powerful message processing capabilities you can leverage scripting languages such as Groovy or JavaScript (among many others) as well. The Transforming using a Simple Expression recipe will show you how to use the Simple Expression Language to do transformations on String messages. See also Message Translator: http://camel.apache.org/message-translator.html Camel Expression capabilities: http://camel.apache.org/expression.html Camel XQuery Expression Language: http://camel.apache.org/xquery.html XQuery language: http://www.w3.org/XML/Query/ Languages supported by Camel: http://camel.apache.org/languages.html The Transforming with XSLT recipe The Transforming using a Simple Expression recipe   Transforming with XSLT When you want to transform an XML message using XSLT, use Camel's XSLT Component. This is similar to the Transforming inline with XQuery recipe except that there is no XSLT Expression Language, so it can only be used as an endpoint. This recipe will show you how to transform a message using an external XSLT resource. Getting ready The Java code for this recipe is located in the org.camelcookbook.transformation.xslt package. Spring XML files are located under src/main/resources/META-INF/spring and prefixed with xslt. How to do it... In a Camel route, add the xslt processor step into the route at the point where you want the XSLT transformation to occur. The XSLT file must be referenced as an external resource, and depending on where the file is located, prefixed with either classpath:(default if not using a prefix), file:, or http:. In the XML DSL, this is written as: <route> <from uri="direct:start"/> <to uri="xslt:book.xslt"/> </route> In the Java DSL, the same route is expressed as: from("direct:start") .to("xslt:book.xslt"); The next processing step in the route will see the transformed message content in the body of the exchange. How it works... The following example shows how the preceding steps will process an XML file. Consider the following input XML message: <bookstore> <book category="COOKING"> <title lang="en">Everyday Italian</title> <author>Giada De Laurentiis</author> <year>2005</year> <price>30.00</price> </book> <book category="CHILDREN"> <title lang="en">Harry Potter</title> <author>J K. Rowling</author> <year>2005</year> <price>29.99</price> </book> <book category="PROGRAMMING"> <title lang="en">Apache Camel Developer's Cookbook</title> <author>Scott Cranton</author> <author>Jakub Korab</author> <year>2013</year> <price>49.99</price> </book> <book category="WEB"> <title lang="en">Learning XML</title> <author>Erik T. Ray</author> <year>2003</year> <price>39.95</price> </book> </bookstore> Process this with the following XSLT contained in books.xslt: <?xml version="1.0" encoding="UTF-8"?> <xsl:stylesheet version="1.0" > <xsl:output omit-xml-declaration="yes"/> <xsl:template match="/"> <books> <xsl:apply-templates select="/bookstore/book/title[../price>30]"> <xsl:sort select="."/> </xsl:apply-templates> </books> </xsl:template> <xsl:template match="node()|@*"> <xsl:copy> <xsl:apply-templates select="node()|@*"/> </xsl:copy> </xsl:template> </xsl:stylesheet> The result will appear as follows: <books> <title lang="en">Apache Camel Developer's Cookbook</title> <title lang="en">Learning XML</title> </books> The Camel XSLT Processor internally runs the message body through a registered Java XML transformer using the XSLT file referenced by the endpoint. This processor uses Camel's Type Converter capabilities to convert the input message body type to one of the supported XML source models in the following order of priority: StAXSource (off by default; this can be enabled by setting allowStAX=true on the endpoint URI) SAXSource StreamSource DOMSource Camel's Type Converter can convert from most input types (String, File, byte[], and so on) to one of the XML source types for most XML content loaded through other Camel endpoints with no extra work on your part. The output data type for the message is, by default, a String, and is configurable using the output parameter on the xslt endpoint URI. There's more... The XSLT Processor passes in headers, properties, and parameters associated with the message. These will show up as XSLT parameters that can be referenced within your XSLT statements. You can pass in the names of the books as parameters to the XSLT template; to do so, modify the previous XLST as follows: <xsl:param name="myParamValue"/> <xsl:template match="/"> <books> <xsl:attribute name="value"> <xsl:value-of select="$myParamValue"/> </xsl:attribute> <xsl:apply-templates select="/bookstore/book/title[../price>$myParamValue]"> <xsl:sort select="."/> </xsl:apply-templates> </books> </xsl:template> The Exchange instance will be associated with a parameter called exchange; the IN message with a parameter called in; and the message headers, properties, and parameters will be associated XSLT parameters with the same name. To use these in your XSLT, you need to explicitly declare a parameter of the same name in your XSLT file. In the previous example, it is possible to use either a message header or exchange property called myParamValue. See also Message Translator: http://camel.apache.org/message-translator.html Camel XSLT Component: http://camel.apache.org/xslt Camel Type Converter: http://camel.apache.org/type-converter.html XSL working group: http://www.w3.org/Style/XSL/ The Transforming inline with XQuery recipe   Transforming from Java to XML with JAXB Camel's JAXB Component is one of a number of components that can be used to convert your XML data back and forth from Java objects. It provides a Camel Data Format that allows you to use JAXB annotated Java classes, and then marshal (Java to XML) or unmarshal (XML to Java) your data. JAXB is a Java standard for translating between XML data and Java that is used by creating annotated Java classes that bind, or map, to your XML data schema. The framework takes care of the rest. This recipe will show you how to use the JAXB Camel Data Format to convert back and forth from Java to XML. Getting ready The Java code for this recipe is located in the org.camelcookbook.transformation.jaxb package. The Spring XML files are located under src/main/resources/META-INF/spring and prefixed with jaxb. To use Camel's JAXB Component, you need to add a dependency element for the camel-jaxb library, which provides the implementation for the JAXB Data Format. Add the following to the dependencies section of your Maven POM: <dependency> lt;groupId>org.apache.camel</groupId> <artifactId>camel-jaxb</artifactId> <version>${camel-version}</version> lt;/dependency>   How to do it... The main steps for converting between Java and XML are as follows: Given a JAXB annotated model, reference that model within a named Camel Data Format. Use that named Data Format within your Camel route using the marshal and unmarshal DSL statements. Create an annotated Java model using standard JAXB annotations. There are a number of external tools that can automate this creation from existing XML or XSD (XML Schema) files: @XmlAccessorType(XmlAccessType.FIELD) @XmlType(name = "", propOrder = { "title", "author", "year", "price" } ) @XmlRootElement(name = "book") public class Book { @XmlElement(required = true) protected Book.Title title; @XmlElement(required = true) protected List<String> author; protected int year; protected double price; // getters and setters } Instantiate a JAXB Data Format within your Camel route that refers to the Java package(s) containing your JAXB annotated classes. In the XML DSL, this is written as: <camelContext > <dataFormats> <jaxb id="myJaxb" contextPath="org.camelcookbook .transformation.myschema"/> </dataFormats> <!-- route definitions here --> </camelContext> In the Java DSL, the Data Format is defined as: public class JaxbRouteBuilder extends RouteBuilder { @Override public void configure() throws Exception { DataFormat myJaxb= new JaxbDataFormat( "org.camelcookbook.transformation.myschema"); // route definitions here } } Reference the Data Format within your route, choosing marshal(Java to XML) or unmarshal(XML to Java) as appropriate. In the XML DSL, this routing logic is written as: <route> <from uri="direct:unmarshal"/> <unmarshal ref="myJaxb"/> </route> In the Java DSL, this is expressed as: from("direct:unmarshal").unmarshal(myJaxb);   How it works... Using Camel JAXB to translate your XML data back and forth to Java makes it much easier for the Java processors defined later on in your route to do custom message processing. This is useful when the built-in XML translators (for example, XSLT or XQuery) are not enough, or you just want to call existing Java code. Camel JAXB eliminates the boilerplate code from your integration flows by providing a wrapper around the standard JAXB mechanisms for instantiating the Java binding for the XML data. There's more... Camel JAXB works just fine with existing JAXB tooling like the maven-jaxb2-plugin plugin, which can automatically create JAXB-annotated Java classes from an XML Schema (XSD). See also Camel JAXB: http://camel.apache.org/jaxb.html Available Data Formats: http://camel.apache.org/data-format.html JAXB Specification: http://jcp.org/en/jsr/detail?id=222   Transforming from Java to JSON Camel's JSON Component is used when you need to convert your JSON data back and forth from Java. It provides a Camel Data Format that, without any requirement for an annotated Java class, allows you to marshal (Java to JSON) or unmarshal (JSON to Java) your data. There is only one step to using Camel JSON to marshal and unmarshal XML data. Within your Camel route, insert the marshal(Java to JSON), or unmarshal(JSON to Java) statement, and configure it to use the JSON Data Format. This recipe will show you how to use the camel-xstream library to convert from Java to JSON, and back. Getting ready The Java code for this recipe is located in the org.camelcookbook.transformation.json package. The Spring XML files are located under src/main/resources/META-INF/spring and prefixed with json. To use Camel's JSON Component, you need to add a dependency element for the camel-xstream library, which provides an implementation for the JSON Data Format using the XStream library. Add the following to the dependencies section of your Maven POM: <dependency> <groupId>org.apache.camel</groupId> <artifactId>camel-xstream</artifactId> <version>${camel-version}</version> </dependency>   How to do it... Reference the Data Format within your route, choosing the marshal (Java to JSON), or unmarshal (JSON to Java) statement, as appropriate: In the XML DSL, this is written as follows: <route> <from uri="direct:marshal"/> <marshal> <json/> </marshal> <to uri="mock:marshalResult"/> </route> In the Java DSL, this same route is expressed as: from("direct:marshal") .marshal().json() .to("mock:marshalResult");   How it works... Using Camel JSON simplifies translating your data between JSON and Java. This is convenient when you are dealing with REST endpoints and need Java processors in Camel to do custom message processing later on in the route. Camel JSON provides a wrapper around the JSON libraries for instantiating the Java binding for the JSON data, eliminating more boilerplate code from your integration flows. There's more... Camel JSON works with the XStream library by default, and can be configured to use other JSON libraries, such as Jackson or GSon. These other libraries provide additional features, more customization, and more flexibility that can be leveraged by Camel. To use them, include their respective Camel components, for example, camel-jackson, and specify the library within the json element: <dataFormats> <json id="myJson" library="Jackson"/> </dataFormats>   See also Camel JSON: http://camel.apache.org/json.html Available Data Formats: http://camel.apache.org/data-format.html   Transforming from XML to JSON Camel provides an XML JSON Component that converts your data back and forth between XML and JSON in a single step, without an intermediate Java object representation. It provides a Camel Data Format that allows you to marshal (XML to JSON), or unmarshal (JSON to XML) your data. This recipe will show you how to use the XML JSON Component to convert from XML to JSON, and back. Getting ready Java code for this recipe is located in the org.camelcookbook.transformation.xmljson package. Spring XML files are located under src/main/resources/META-INF/spring and prefixed with xmljson. To use Camel's XML JSON Component, you need to add a dependency element for the camel-xmljson library, which provides an implementation for the XML JSON Data Format. Add the following to the dependencies section of your Maven POM: <dependency> <groupId>org.apache.camel</groupId> <artifactId>camel-xmljson</artifactId> <version>${camel-version}</version> </dependency>   How to do it... Reference the xmljson Data Format within your route, choosing the marshal(XML to JSON), or unmarshal(JSON to XML) statement, as appropriate: In the XML DSL, this is written as follows: <route> <from uri="direct:marshal"/> <marshal> <xmljson/> </marshal> <to uri="mock:marshalResult"/> </route> In the Java DSL, this same route is expressed as: from("direct:marshal") .marshal().xmljson() .to("mock:marshalResult");   How it works... Using the Camel XML JSON Component simplifies translating your data between XML and JSON, making it convenient to use when you are dealing with REST endpoints. The XML JSON Data Format wraps around the Json-lib library, which provides the core translation capabilities, eliminating more boilerplate code from your integration flows. There's more... You may need to configure XML JSON if you want to fine-tune the output of your transformation. For example, consider the following JSON: [{"@category":"PROGRAMMING","title":{"@lang":"en","#text": "Apache Camel Developer's Cookbook"},"author":[ "Scott Cranton","Jakub Korab"],"year":"2013","price":"49.99"}] This will be converted as follows, by default, which may not be exactly what you want (notice the <a> and <e> elements): <?xml version="1.0" encoding="UTF-8"?> <a> <e category="PROGRAMMING"> <author> <e>Scott Cranton</e> <e>Jakub Korab</e> </author> <price>49.99</price> <title lang="en">Apache Camel Developer's Cookbook</title> <year>2013</year> </e> </a> To configure XML JSON to use <bookstore> as the root element instead of <a>, use <book> for the individual elements instead of <e>, and expand the multiple author values to use a sequence of <author> elements, you would need to tune the configuration of the Data Format before referencing it in your route. In the XML DSL, the definition of the Data Format and the route that uses it is written as follows: <dataFormats> <xmljson id="myXmlJson" rootName="bookstore" elementName="book" expandableProperties="author author"/> </dataFormats> <route> <from uri="direct:unmarshalBookstore"/> <unmarshal ref="myXmlJson"/> <to uri="mock:unmarshalResult"/> </route> In the Java DSL, the same thing is expressed as: XmlJsonDataFormat xmlJsonFormat = new XmlJsonDataFormat(); xmlJsonFormat.setRootName("bookstore"); xmlJsonFormat.setElementName("book"); xmlJsonFormat.setExpandableProperties( Arrays.asList("author", "author")); from("direct:unmarshalBookstore") .unmarshal(xmlJsonFormat) .to("mock:unmarshalBookstoreResult"); This will result in the previous JSON being unmarshalled as follows: <?xml version="1.0" encoding="UTF-8"?> <bookstore> <book category="PROGRAMMING"> <author>Scott Cranton</author> <author>Jakub Korab</author><price>49.99</price> <title lang="en">Apache Camel Developer's Cookbook</title> <year>2013</year> </book> </bookstore>   See also Camel XML JSON: http://camel.apache.org/xmljson.html Available Data Formats: http://camel.apache.org/data-format.html Json-lib: http://json-lib.sourceforge.net   Summary We saw how Apache Camel is flexible in allowing us to transform and convert messages in various formats. This makes Apache Camel an ideal choice for integrating different systems together. Resources for Article: Further resources on this subject: Drools Integration Modules: Spring Framework and Apache Camel [Article] Installing Apache Karaf [Article] Working with AMQP [Article]
Read more
  • 0
  • 0
  • 9453

article-image-sap-hana-architecture
Packt
20 Dec 2013
12 min read
Save for later

SAP HANA Architecture

Packt
20 Dec 2013
12 min read
(For more resources related to this topic, see here.) Understanding the SAP HANA architecture Architecture is the key for SAP HANA to be a game changing innovative technology. SAP HANA has been designed so well architecture-wise such that it makes a lot of difference when compared to other traditional databases available today. This section explains us the various components of SAP HANA and its functionalities. Getting ready Enterprise application requirements have become more demanding—complex reports with high computation on huge volumes of transaction data and also business data of other formats (both structured and semi-structured). Data is being written or updated, and also read from the database in parallel. Thus, integration of both transactional and analytical data into single database is required, where SAP HANA has evolved. Columnar storage exploits modern hardware and technology (multiple CPU cores, large main memory, and caches) in achieving the requirements of enterprise applications. Apart from this, it should also support procedural logic where certain tasks cannot be completed with simple SQL. How it works… The SAP HANA database consists of several services (servers). Index server is the most important component of all the servers. Other servers are name server, preprocessor server, statistics server, and XS Engine: Index server: This server holds the actual data and the engines for processing the data. When SQL or MDX is fired against the SAP HANA system in the case of authenticated sessions and transactions, an index server takes care of these commands and processes them. Name server: This server holds complete information about the system landscape. Name server is responsible for the topology of the SAP HANA system. In a distributed system, SAP HANA instances will be running on multiple hosts. In this kind of setup, the name server knows where the components are running and how data is spread on different servers. Preprocessor server: This server comes into the picture during text data analysis. Index server utilizes the capabilities of preprocessor server in text data analysis and searching. This helps to extract the information on which text search capabilities are based. Statistics server: This server helps in collecting the data for the system monitor and helps you know the health of the SAP HANA system. The statistics server is responsible for collecting the data related to status, resource allocation/consumption and performance of the SAP HANA system. Monitoring the clients and getting the status of various alert monitors use the data collected by Statistics server. This server also provides a history of measurement data for further analysis. XS Engine: The XS Engine allows external applications and application developers to access the SAP HANA system via the XS Engine clients, for example, a web browser accesses SAP HANA apps built by application developers via HTTP. Application developers build applications by using the XS Engine, and the users access the app via HTTP by using a web browser. The persistent model in the SAP HANA database is converted into a consumption model for clients to access it via HTTP. This allows an organization to host system services that are a part of the SAP HANA database (for example, Search service, a built-in web server that provides access to static content in the repository). The following diagram shows the architecture of SAP HANA: There's is more... Let us continue learning about the different components: SAP Host Agent: According to the new approach from SAP, the SAP Host Agent should be installed on all machines that are related to the SAP landscape. It is used by Adaptive Computing Controller (ACC) to manage the system and Software Update Manager (SUM) for automatic updates. LM-structure: LM-structure for SAP HANA contains the information about current installation details. This information will be used by SUM during automatic updates. SAP Solution Manager diagnostic agent: This agent provides all the data to SAP Solution Manager (SAP SOLMAN) to monitor the SAP HANA system. After the SAP SOLMAN is integrated with the SAP HANA system, this agent provides information about the database at a glance, which includes the database state and general information about the system, such as alerts, CPU, or memory and disk usage. SAP HANA Studio repository: This helps the end users to update the SAP HANA studio to higher versions. The SAP HANA Studio repository is the code that does this process. Software Update Manager for SAP HANA: This helps in automatic updates of SAP HANA from the SAP Marketplace and patching the SAP host agent. It also allows distribution of the Studio repository to the end users. See also http://help.sap.com/hana/SAP_HANA_Installation_Guide_en.pdf SAP Notes:1793303 and 1514967 Explaining IMCE and its components We have seen the architecture of SAP HANA and its components. In this section, we will learn about IMCE (in-memory computing engine) and how its components and its functionalities. Getting Ready The SAP in-memory computing engine (formerly Business Analytic Engine (BAE)) is the core engine for SAP's next generation high-performance, in-memory solutions as it leverages technologies such as in-memory computing, columnar databases, massively parallel processing (MPP), and data compression, to allow organizations to instantly explore and analyze large volumes of transactional and analytical data from across the enterprise in real time. How it works... In-memory computing allows the processing of massive quantities of real-time data in the main memory of the server, providing immediate results from analyses and transactions. The SAP in-memory computing database delivers the following capabilities: In-memory computing functionality with native support for row and columnar datastores providing full ACID (atomicity, consistency, isolation, and durability) transactional capabilities Integrated lifecycle management capabilities and data integration capabilities to access SAP and non-SAP data sources SAP IMCE Studio, which includes tools for data modeling, data and life cycle management, and data security The SAP IMCE that resides at the heart of SAP HANA is an integrated database and calculation layer that allows the processing of massive quantities of real-time data in the main memory to provide immediate results from analysis and transactions. Like any standard database, the SAP IMCE not only supports industry standards such as SQL and MDX, but also incorporates a high-performance calculation engine that embeds procedural language support directly into the database kernel. This approach is designed to remove the need to read data from the database, process it, and then write data back to the database, that is, process the data near the database and return the results. The IMCE is an in-memory, column-oriented database technology. It is a powerful calculation engine at the heart of SAP HANA. As data resides in the Random Access Memory (RAM), highly accelerated performance can be achieved compared to systems that read data from disks. The heart lies within the IMCE, which allows us to create and perform calculations on data. SAP IMCE Studio includes tools for data modeling activities, data and life cycle management, and also tools that are related to data security. The following diagram shows the components of IMCE alone: There's more… SAP HANA database has the following two engines: Column-based store: This engine stores the huge amounts of relational data in column-optimized tables, which are aggregated and used in analytical operations. Row-based store: This engine stores the relational data in rows, similar to the storage mechanism of traditional database systems. The row store is more optimized for write operations and has a lower compression rate. Also, the query performance is lower when compared to the column-based store. The engine that is used to store data can be selected on a per-table basis at the time of creating a table. Tables in the row-based store are loaded at start up time. In the case of column-based stores, tables can be either loaded at start up or on demand, that is, during normal operation of the SAP HANA database. Both engines share a common persistence layer, which provides data persistency that is consistent across both engines. Like a traditional database, we have page management and logging in SAP HANA. The changes made to the in-memory database pages are persisted through savepoints. These savepoints are written to those data volumes on the persistent storage for which the storage medium is hard drives. All transactions committed in the SAP HANA database are stored/saved/referenced by the logger of the persistency layer in a log entry written to the log volumes on the persistent storage. To get high I/O performance and low latency, log volumes use the flash technology storage. The relational engines can be accessed through a variety of interfaces. The SAP HANA database supports SQL (JDBC/ODBC), MDX (ODBO), and BICS (SQLDBC). The calculation engine performs all the calculations in the database. No data moves into the application layer until calculations are completed. It also contains the business functions library that is called by applications to perform calculations based on the business rules and logic. The SAP HANA-specific SQL script language is an extension of SQL that can be used to push down data-intensive application logic into the SAP HANA database for specific requirements. Session management This component creates and manages sessions and connections for the database clients. When a session is created, a set of parameters are maintained. These parameters are like auto-commit settings or the current transaction isolation level. After establishing a session, database clients communicate with the SAP HANA database using SQL statements. SAP HANA database treats all the statements as transactions while processing them. Each new session created will be assigned to a new transaction. Transaction manager The transaction manager is the component that coordinates database transactions, takes care of controlling transaction isolation, and keeps track of running and closed transactions. The transaction manager informs the involved storage engines about the running or closed transactions, so that they can execute necessary actions, when a transaction is committed or rolled back. The transaction manager cooperates with the persistence layer to achieve atomic and durable transactions. The client requests are analyzed and executed by a set of components summarized as request processing and execution control. The client requests are analyzed by a request parser, and then it is dispatched to the responsible component. The transaction control statements are forwarded to the transaction manager. The data definition statements are sent to the metadata manager. The object invocations are dispatched to the object store. The data manipulation statements are sent to the optimizer, which creates an optimized execution plan that is given to the execution layer. The SAP HANA database also has built-in support for domain-specific models (such as for financial planning domain) and it offers scripting capabilities that allow application-specific calculations to run inside the database. It has its own scripting language named SQLScript that is designed to enable optimizations and parallelization. This SQLScript is based on free functions that operate on tables by using SQL queries for set processing. The SAP HANA database also contains a component called the planning engine that allows financial planning applications to execute basic planning operations in the database layer. For example, while applying filters/transformations, a new version of a dataset will be created as a copy of an existing one. An example of planning operation is disaggregation operation in which based on a distribution function; target values from higher to lower aggregation levels are distributed. Metadata manager Metadata manager helps to access metadata. SAP HANA database's metadata consists of a variety of objects, such as definitions of tables, views and indexes, SQLScript function definitions, and object store metadata. All these types of metadata are stored in one common catalog for all the SAP HANA database stores. Metadata is stored in tables in the row store. The SAP HANA features such as transaction support and multi-version concurrency control (MVCC) are also used for metadata management. Central metadata is shared across the servers in the case of a distributed database systems. The background mechanism of metadata storage and sharing is hidden from the components that use the metadata manager. As row-based tables and columnar tables can be combined in one SQL statement, both the row and column engines must be able to consume the intermediate results. The main difference between the two engines is the way they process data: the row store operators process data in a row-at-a-time fashion, whereas column store operations (such as scan and aggregate) require the entire column to be available in contiguous memory locations. To exchange intermediate results created by each other, the row store provides results to the column store. The result materializes as complete rows in the memory, while the column store can expose results using the iterators interface needed by the row store. Persistence layer The persistence layer is responsible for durability and atomicity of transactions. The persistent layer ensures that the database is restored to the most recent committed state after a restart, and makes sure that transactions are either completely executed or completely rolled back. To achieve this in an efficient way, the persistence layer uses a combination of write-ahead logs, shadow paging, and savepoints. Moreover, the persistence layer also offers interfaces for writing and reading data. It also contains SAP HANA's logger that manages the transaction log. Authorization manager The authorization manager is invoked by other SAP HANA database components to check the required privileges for users to execute the requested operations. Privileges to other users or roles can be granted. A privilege grants the right to perform a specified operation (such as create, update, select, and execute data manipulation languages) on a specified object such as a table, view, and SQLScript function. Analytic privileges represent filters or hierarchy, and they drill down limitations for analytic queries. Analytic privileges such as granting access to values with a certain combination of dimension attributes are supported in SAP HANA. Users are authenticated either by the SAP HANA database itself (log in with username and password), or authentication can be delegated to external authentication providers third-party such as an LDAP directory. See also SAP HANA in-memory analytics and in-memory computing available at http://scn.sap.com/people/vitaliy.rudnytskiy/blog/2011/03/22/time-to-update-your-sap-hana-vocabulary Summary This article explains the SAP architecture and the IMCE feature in brief. Resources for Article: Further resources on this subject: SAP HANA integration with Microsoft Excel [Article] Data Migration Scenarios in SAP Business ONE Application- part 2 [Article] Data Migration Scenarios in SAP Business ONE Application- part 1 [Article]
Read more
  • 0
  • 0
  • 9284

article-image-knowing-sql-injection-attacks-and-securing-our-android-applications-them
Packt
20 Dec 2013
10 min read
Save for later

Knowing the SQL-injection attacks and securing our Android applications from them

Packt
20 Dec 2013
10 min read
(For more resources related to this topic, see here.) Enumerating SQL-injection vulnerable content providers Just like web applications, Android applications may use untrusted input to construct SQL queries and do so in a way that's exploitable. The most common case is when applications do not sanitize input for any SQL and do not limit access to content providers. Why would you want to stop a SQL-injection attack? Well, let's say you're in the classic situation of trying to authorize users by comparing a username supplied by querying a database for it. The code would look similar to the following: public boolean isValidUser(){ u_username = EditText( some user value ); u_password = EditText( some user value ); //some un-important code here... String query = "select * from users_table where username = '" + u_username + "' and password = '" + u_password +"'"; SQLiteDatabase db //some un-important code here... Cursor c = db.rawQuery( p_query, null ); return c.getCount() != 0; } What's the problem in the previous code? Well, what happens when the user supplies a password '' or '1'='1'? The query being passed to the database then looks like the following: select * from users_table where username = '" + u_username + "' and password = '' or '1'='1' " The preceding bold characters indicate the part that was supplied by the user; this query forms what's known in Boolean algebra as a logical tautology; meaning no matter what table or data the query is targeted at, it will always be set to true, which means that all the rows in the database will meet the selection criteria. This then means that all the rows in users_table will be returned and as result, even if a nonvalid password ' or '1'=' is supplied, the c.getCount() call will always return a nonzero count, leading to an authentication bypass! Given that not many Android developers would use the rawQuery call unless they need to pull off some really messy SQL queries, I've included another code snippet of a SQL-injection vulnerability that occurs more often in real-world applications. So when auditing Android code for injection vulnerabilities, a good idea would be to look for something that resembles the following: public Cursor query(Uri uri, String[] projection , String selection ,String[] selectionArgs , String sortOrder ) { SQLiteDBHelper sdbh = new StatementDBHelper(this.getContext()); Cursor cursor; try { //some code has been omitted cursor = sdbh .query(projection,selection,selectionArgs,sortOrder); } finally { sdbh.close(); } return cursor; } In the previous code, none of the projection, selection, selectionArgs, or sortOrder variables are sourced directly from external applications. If the content provider is exported and grants URI permissions or, as we've seem before, does not require any permissions, it means that attackers will be able to inject arbitrary SQL to augment the way the malicious query is evaluated. Let's look at how you actually go about attacking SQL-injection vulnerable content providers using drozer. How to do it... In this recipe, I'll talk about two kinds of SQL-injection vulnerabilities: one is when the select clause of a SQL statement is injectable and the other is when the projection is injectable. Using drozer, it is pretty easy to find select-clause-injectable content providers: dz> run app.provider.query [URI] –-selection "1=1" The previous will try to inject what's called a logical tautology into the SQL statement being parsed by the content provider and eventually the database query parser. Due to the nature of the module being used here, you can tell whether or not it actually worked, because it should return all the data from the database; that is, the select-clause criteria is applied to every row and because it will always return true, every row will be returned! You could also try any values that would always be true: dz> run app.provider.query [URI] –-selection "1-1=0" dz> run app.provider.query [URI] –-selection "0=0" dz> run app.provider.query [URI] –-selection "(1+random())*10 > 1" The following is an example of using a purposely vulnerable content provider: dz> run app.provider.query content://com.example. vulnerabledatabase.contentprovider/statements –-selection "1=1" It returns the entire table being queried, which is shown in the following screenshot: You can, of course, inject into the projection of the SELECT statement, that is, the part before FROM in the statement, that is, SELECT [projection] FROM [table] WHERE [select clause]. Securing application components Application components can be secured both by making proper use of the AndroidManifest.xml file and by forcing permission checks at code level. These two factors of application security make the permissions framework quite flexible and allow you to limit the number of applications accessing your components in quite a granular way. There are many measures that you can take to lock down access to your components, but what you should do before anything else is make sure you understand the purpose of your component, why you need to protect it, and what kind of risks your users face should a malicious application start firing off intents to your app and accessing its data. This is called a risk-based approach to security, and it is suggested that you first answer these questions honestly before configuring your AndroidManifest.xml file and adding permission checks to your apps. In this recipe, I have detailed some of the measures that you can take to protect generic components, whether they are activities, broadcast receivers, content providers, or services. How to do it... To start off, we need to review your Android application AndroidManifest.xml file. The android:exported attribute defines whether a component can be invoked by other applications. If any of your application components do not need to be invoked by other applications or need to be explicitly shielded from interaction with the components on the rest of the Android system—other than components internal to your application—you should add the following attribute to the application component's XML element: <[component name] android_exported="false"> </[component name]> Here the [component name] would either be an activity, provider, service, or receiver. How it works… Enforcing permissions via the AndroidManifest.xml file means different things to each of the application component types. This is because of the various inter-process communications ( IPC ) mechanisms that can be used to interact with them. For every application component, the android:permission attribute does the following: Activity : Limits the application components which are external to your application that can successfully call startActivity or startActivityForResult to those with the required permission Service : Limits the external application components that can bind (by calling bindService()) or start (by calling startService()) the service to those with the specified permission Receiver : Limits the number of external application components that can send broadcasted intents to the receiver with the specified permission Provider : Limits access to data that is made accessible via the content provider The android:permission attribute of each of the component XML elements overrides the <application> element's android:permission attribute. This means that if you haven't specified any required permissions for your components and have specified one in the <application> element, it will apply to all of the components contained in it. Though specifying permissions via the <application> element is not something developers do too often because of how it affects the friendliness of the components toward the Android system itself (that is, if you override an activity's required permissions using the <application> element), the home launcher will not be able to start your activity. That being said, if you are paranoid enough and don't need any unauthorized interaction to happen with your application or its components, you should make use of the android:permission attribute of the <application> tag. When you define an <intent-filter> element on a component, it will automatically be exported unless you explicitly set exported="false". However, this seemed to be a lesser-known fact, as many developers were inadvertently opening their content providers to other applications. So, Google responded by changing the default behavior for <provider> in Android 4.2. If you set either android:minSdkVersion or android:targetSdkVersion to 17, the exported attribute on <provider> will default to false. Defending against the SQL-injection attack The previous chapter covered some of the common attacks against content providers, one of them being the infamous SQL-injection attack. This attack leverages the fact that adversaries are capable of supplying SQL statements or SQL-related syntax as part of their selection arguments, projections, or any component of a valid SQL statement. This allows them to extract more information from a content provider than they are not authorized. The best way to make sure adversaries will not be able to inject unsolicited SQL syntax into your queries is to avoid using SQLiteDatabase.rawQuery() instead opting for a parameterized statement. Using a compiled statement, such as SQLiteStatement, offers both binding and escaping of arguments to defend against SQL-injection attacks. Also, there is a performance benefit due to the fact the database does not need to parse the statement for each execution. An alternative to SQLiteStatement is to use the query, insert, update, and delete methods on SQLiteDatabase as they offer parameterized statements via their use of string arrays. When we describe parameterized statement, we are describing an SQL statement with a question mark where values will be inserted or binded. Here's an example of parameterized SQL insert statement: INSERT VALUES INTO [table name] (?,?,?,?,...) Here [table name] would be the name of the relevant table in which values have to be inserted. How to do it... For this example, we are using a simple Data Access Object ( DAO ) pattern, where all of the database operations for RSS items are contained within the RssItemDAO class: When we instantiate RssItemDAO, we compile the insertStatement object with a parameterized SQL insert statement string. This needs to be done only once and can be re-used for multiple inserts: public class RssItemDAO { private SQLiteDatabase db; private SQLiteStatement insertStatement; private static String COL_TITLE = "title"; private static String TABLE_NAME = "RSS_ITEMS"; private static String INSERT_SQL = "insert into " + TABLE_NAME + " (content, link, title) values (?,?,?)"; public RssItemDAO(SQLiteDatabase db) { this.db = db; insertStatement = db.compileStatement(INSERT_SQL); } The order of the columns noted in the INSERT_SQL variable is important, as it directly maps to the index when binding values. In the preceding example, content maps to index 0, link maps to index 1, and title to index 2. Now, when we come to insert a new RssItem object to the database, we bind each of the properties in the order they appear in the statement: public long save(RssItem item) { insertStatement.bindString(1, item.getContent()); insertStatement.bindString(2, item.getLink()); insertStatement.bindString(3, item.getTitle()); return insertStatement.executeInsert(); } Notice that we call executeInsert, a helper method that returns the ID of the newly created row. It's as simple as that to use a SQLiteStatement statement. This shows how to use SQLiteDatabase.query to fetch RssItems that match a given search term: public List<RssItem> fetchRssItemsByTitle(String searchTerm) { Cursor cursor = db.query(TABLE_NAME, null, COL_TITLE + "LIKE ?", new String[] { "%" + searchTerm + "%" }, null, null, null); // process cursor into list List<RssItem> rssItems = new ArrayList<RssItemDAO.RssItem>(); cursor.moveToFirst(); while (!cursor.isAfterLast()) { // maps cursor columns of RssItem properties RssItem item = cursorToRssItem(cursor); rssItems.add(item); cursor.moveToNext(); } return rssItems; } We use LIKE and the SQL wildcard syntax to match any part of the text with a title column. Summary There were a lot of technical details in this article. Firstly, we learned about the components that are vulnerable to SQL-injection attacks. We then figured out how to secure our Android applications from the exploitation attacks. Finally, we learned how to defend our applications from the SQL-injection attacks. Resources for Article: Further resources on this subject: Android Native Application API [Article] So, what is Spring for Android? [Article] Creating Dynamic UI with Android Fragments [Article]
Read more
  • 0
  • 0
  • 25689
article-image-common-asynctask-issues
Packt
20 Dec 2013
7 min read
Save for later

Common AsyncTask issues

Packt
20 Dec 2013
7 min read
(For more resources related to this topic, see here.) Fragmentation issues AsyncTask has evolved with new releases of the Android platform, resulting in behavior that varies with the platform of the device running the task, which is a part of the wider issue of fragmentation. The simple fact is that if we target a broad range of API levels, the execution characteristics of our AsyncTasks—and therefore, the behavior of our apps—can vary considerably on different devices. So what can we do to reduce the likelihood of encountering AsyncTask issues due to fragmentation? The most obvious approach is to deliberately target devices running at least Honeycomb, by setting a minSdkVersion of 11 in the Android Manifest file. This neatly puts us in the category of devices, which, by default, execute AsyncTasks serially, and therefore, much more predictably. However, this significantly reduces the market reach of our apps. At the time of writing in September 2013, more than 34 percent of Android devices in the wild run a version of Android in the danger zone between API levels 4 and 10. A second option is to design our code carefully and test exhaustively on a range of devices—always commendable practices of course, but as we've seen, concurrent programming is hard enough without the added complexity of fragmentation, and invariably, subtle bugs will remain. A third solution that has been suggested by the Android development community is to reimplement AsyncTaskin a package within your own project, then extend your own AsyncTask class instead of the SDK version. In this way, you are no longer at the mercy of the user's device platform, and can regain control of your AsyncTasks. Since the source code for AsyncTask is readily available, this is not difficult to do. Activity lifecycle issues Having deliberately moved any long-running tasks off the main thread, we've made our applications nice and responsive—the main thread is free to respond very quickly to any user interaction. Unfortunately, we have also created a potential problem for ourselves, because the main thread is able to finish the Activity before our background tasks complete. Activity might finish for many reasons, including configuration changes caused the by the user rotating the device (the default behavior of Activity on a change in orientation is to restart with an entirely new instance of the activity). If we continue processing a background task after the Activity has finished, we are probably doing unnecessary work, and therefore wasting CPU and other resources (including battery life), which could be put to better use. Also, any object references held by the AsyncTask will not be eligible for garbage collection until the task explicitly nulls those references or completes and is itself eligible for GC ( garbage collection ). Since our AsyncTask probably references the Activity or parts of the View hierarchy, we can easily leak a significant amount of memory in this way. A common usage of AsyncTask is to declare it as an anonymous inner class of the host Activity, which creates an implicit reference to the Activity and an even bigger memory leak. There are two approaches for preventing these resource wastage problems. Handling lifecycle issues with early cancellation First, and most obviously, we can synchronize our AsyncTask lifecycle with that of the Activity by canceling running tasks when our Activity is finishing. When an Activity finishes, its lifecycle callback methods are invoked on the main thread. We can check to see why the lifecycle method is being called, and if the Activity is finishing, cancel the background tasks. The most appropriate Activity lifecycle method for this is onPause, which is guaranteed to be called before the Activity finishes. protected void onPause() { super.onPause(); if ((task != null) && (isFinishing())) task.cancel(false); } If the Activity is not finishing—say because it has started another Activity and is still on the back stack—we might simply allow our background task to continue to completion. Handling lifecycle issues with retained headless fragments If the Activity is finishing because of a configuration change, it may still be useful to complete the background task and display the results in the restarted Activity. One pattern for achieving this is through the use of retained Fragments. Fragments were introduced to Android at API level 11, but are available through a support library to applications targeting earlier API levels. All of the downloadable examples use the support library, and target API levels 7 through 19. To use Fragments, our Activity must extend the FragmentActivity class. The Fragment lifecycle is closely bound to that of the host Activity, and a fragment will normally be disposed when the activity restarts. However, we can explicitly prevent this by invoking setRetainInstance (true) on our Fragment so that it survives across Activity restarts. Typically, a Fragment will be responsible for creating and managing at least a portion of the user interface of an Activity, but this is not mandatory. A Fragment that does not manage a view of its own is known as a headless Fragment. Isolating our AsyncTask in a retained headless Fragment makes it less likely that we will accidentally leak references to objects such as the View hierarchy, because the AsyncTask will no longer directly interact with the user interface. To demonstrate this, we'll start by defining an interface that our Activity will implement: public interface AsyncListener<Progress, Result> { void onPreExecute(); void onProgressUpdate(Progress... progress); void onPostExecute(Result result); void onCancelled(Result result); } Next, we'll create a retained headless Fragment, which wraps our AsyncTask. For brevity, doInBackground is omitted, as it is unchanged from the previous examples—see the downloadable samples for the complete code. public class PrimesFragment extends Fragment { private AsyncListener<Integer,BigInteger> listener; private PrimesTask task; public void onCreate(Bundle savedInstanceState) { super.onCreate(savedInstanceState); setRetainInstance(true); task = new PrimesTask(); task.execute(2000); } public void onAttach(Activity activity) { super.onAttach(activity); listener = (AsyncListener<Integer,BigInteger>)activity; } public void onDetach() { super.onDetach(); listener = null; } class PrimesTask extends AsyncTask<Integer, Integer, BigInteger>{ protected void onPreExecute() { if (listener != null) listener.onPreExecute(); } protected void onProgressUpdate(Integer... values) { if (listener != null) listener.onProgressUpdate(values); } protected void onPostExecute(BigInteger result) { if (listener != null) listener.onPostExecute(result); } protected void onCancelled(BigInteger result) { if (listener != null) listener.onCancelled(result); } // … doInBackground elided for brevity … } } We're using the Fragment lifecycle methods (onAttach and onDetach) to add or remove the current Activity as a listener, and PrimesTask delegates directly to it from all of its main-thread callbacks. Now, all we need is the host Activity that implements AsyncListener and uses PrimesFragment to implement its long-running task. The full source code is available to download from the Packt Publishing website, so we'll just take a look at the highlights. First, the code in the button's OnClickListener now checks to see if Fragment already exists, and only creates one if it is missing: FragmentManager fm = getSupportFragmentManager(); PrimesFragment primes = (PrimesFragment)fm.findFragmentByTag("primes"); if (primes == null) { primes = new PrimesFragment(); FragmentTransaction transaction = fm.beginTransaction(); transaction.add(primes, "primes").commit(); } If our Activity has been restarted, it will need to re-display the progress dialog when a progress update callback is received, so we check and show it, if necessary, before updating the progress bar: public void onProgressUpdate(Integer... progress) { if (dialog == null) prepareProgressDialog(); progress.setProgress(progress[0]); } Finally, Activity will need to implement the onPostExecute and onCancelled callbacks defined by AsyncListener. Both methods will update the resultView as in the previous examples, then do a little cleanup—dismissing the dialog and removing Fragment as its work is now done: public void onPostExecute(BigInteger result) { resultView.setText(result.toString()); cleanUp(); } public void onCancelled(BigInteger result) { resultView.setText("cancelled at " + result); cleanUp(); } private void cleanUp() { dialog.dismiss(); dialog = null; FragmentManager fm = getSupportFragmentManager(); Fragment primes = fm.findFragmentByTag("primes"); fm.beginTransaction().remove(primes).commit(); } Summary In this article, we've taken a look at AsyncTask and how to use it to write responsive applications that perform operations without blocking the main thread. Resources for Article: Further resources on this subject: Android Native Application API [Article] So, what is Spring for Android? [Article] Creating Dynamic UI with Android Fragments [Article]
Read more
  • 0
  • 0
  • 12112

article-image-sql-server-analysis-services-administering-and-monitoring-analysis-services
Packt
20 Dec 2013
5 min read
Save for later

SQL Server Analysis Services – Administering and Monitoring Analysis Services

Packt
20 Dec 2013
5 min read
Our Data Engineering Byte Newsletter gives data engineers and practitioners what they often lack today: clear, real-world insights—where every byte tells a story.Subscribe here to stay ahead in data engineeringIf your environment has only one or a handful of SSAS instances, they can be managed by the same database administrators managing SQL Server and other database platforms. In large enterprises, there could be hundreds of SSAS instances managed by dedicated SSAS administrators. Regardless of the environment, you should become familiar with the configuration options as well as troubleshooting methodologies. In large enterprises, you might also be required to automate these tasks using the Analysis Management Objects (AMO) code. Analysis Services is a great tool for building business intelligence solutions. However, much like any other software, it does have its fair share of challenges and limitations. Most frequently encountered enterprise business intelligence system goals include quick provision of relevant data to the business users and assuring excellent query performance. If your cubes serve a large, global community of users, you will quickly learn that SSAS is optimized to run a single query as fast as possible. Once users send a multitude of heavy queries in parallel, you can expect to see memory, CPU, and disk-related performance counters to quickly rise, with a corresponding increase in query execution duration which, in turn, worsens user experience. Although you could build aggregations to improve query performance, doing so will lengthen cube processing time, and thereby, delay the delivery of essential data to decision makers. It might also be tempting to consider using ROLAP storage mode in lieu of MOLAP so that processing times are shorter, but MOLAP queries usually outperform ROLAP due to heavy compression rates. Hence, figuring out the right storage mode and appropriate level of aggregations is a great balancing act. If you cannot afford using ROLAP, and query performance is paramount to successful cube implementation, you should consider scaling your solution. You have two options for scaling, given as follows:Scaling up: This option means purchasing servers with more memory, more CPU cores, and faster disk drivesScaling out: This option means purchasing several servers of approximately the same capacity and distributing the querying workload across multiple servers using a load balancing toolSSAS lends itself best to the second option—scaling out. Later in this article you will learn how to separate processing and querying activities and how to ensure that all servers in the querying pool have the same data.SSAS instance configuration optionsAll Analysis Services configuration options are available in the msmdsrv.ini file found in the config folder under the SSAS installation directory. Instance administrators can also modify some, but not all configuration properties, using SQL Server Management Studio (SSMS). SSAS has a multitude of properties that are undocumented—this normally means that such properties haven't undergone thorough testing, even by the software's developers. Hence, if you don't know exactly what the configuration setting does, it's best to leave the setting at default value. Even if you want to test various properties on a sandbox server, make a copy of the configuration file prior to applying any changes.How to do it...To modify the SSAS instance settings using the configuration file, perform the following steps:Navigate to the config folder within your Analysis Services installation directory. By default, this will be C:\Program Files\Microsoft SQL Server\MSAS11.instance_name\OLAP\Config.Open the msmdsrv.ini file using Notepad or another text editor of your choice. The file is in the XML format, so every property is enclosed in opening and closing tags.Search for the property of interest, modify its value as desired, and save the changes.For example, in order to change the upper limit of the processing worker threads, you would look for the <ThreadPool><Process><MaxThreads> tag sequence and set the values as shown in the following excerpt from the configuration file:<Process>       <MinThreads>0</MinThreads>       <MaxThreads>250</MaxThreads>      <PriorityRatio>2</PriorityRatio>       <Concurrency>2</Concurrency>       <StackSizeKB>0</StackSizeKB>       <GroupAffinity/>     </Process>  To change the configuration using SSMS, perform the following steps:Connect to the SSAS instance using the instance administrator account and choose Properties. If your account does not have sufficient permissions, you will get an error that only administrators can edit server properties.Change the desired properties by altering the Value column on the General page of the resulting dialog, as shown in the following screenshot: Advanced properties are hidden by default. You must check the Show Advanced (All) Properties box to see advanced properties. You will not see all the properties in SSMS even after checking this box. The only way to edit some properties is by editing msmdsrv.ini as previously discussed.Make a note of the Reset Default button in the bottom-right corner. This button comes in handy if you've forgotten what the configuration values were before you changed them and want to revert to the default settings. The default values are shown in the dialog box, which can provide guidance as to which properties have been altered.Some configuration settings require restarting the SSAS instance prior to being executed. If this is the case, the Restart column will have a value of Yes.Once you're happy with your changes, click on OK and restart the instance if necessary. You can restart SSAS using the Services.msc applet from the command line using the NET STOP / NET START commands, or directly in SSMS by choosing the Restart option after right-clicking on the instance. How it works...Discussing every SSAS property would make this article extremely lengthy; doing so is well beyond the scope of the book. Instead, in this section, I will summarize the most frequently used properties. Often, synchronization has to copy large partition datafiles and aggregation files. If the timeout value is exceeded, synchronization fails. Increase the value of the <Network><Listener><ServerSendTimeout> and <Network><Listener><ServerReceiveTimeout> properties to allow a longer time span for copying each file. By default, SSAS can use a lazy thread to rebuild missing indexes and aggregations after you process partition data. If the <OLAP><LazyProcessing><Enabled> property is set to 0, the lazy thread is not used for building missing indexes—you must use an explicit processing command instead. The <OLAP><LazyProcessing><MaxCPUUsage> property throttles the maximum CPU that could be used by the lazy thread. If efficient data delivery is your topmost priority, you can exploit the ProcessData option instead of ProcessFull. To build aggregations after the data is loaded, you must set the partition's ProcessingMode property to LazyAggregations. The SSAS formula engine is single threaded, so queries that perform heavy calculations will only use one CPU core, even on a multiCPU computer. The storage engine is multithreaded; hence, queries that read many partitions will require many CPU cycles. If you expect storage engine heavy queries, you should lower the CPU usage threshold for LazyAggregations. By default, Analysis Services records subcubes requested for every 10th query in the query log table. If you'd like to design aggregations based on query logs, you should change the <Log><QueryLog><QueryLogSampling> property value to 1 so that the SSAS logs subcube requests for every query. SSAS can use its own memory manager or the Windows memory manager. If your SSAS instance consistently becomes unresponsive, you could try using the Windows memory manager. Set <Memory><MemoryHeapType> to 2 and <Memory><HeapTypeForObjects> to 0. The Analysis Services memory manager values are 1 for both the properties. You must restart the SSAS service for the changes to these properties to take effect. The <Memory><PreAllocate> property specifies the percentage of total memory to be reserved at SSAS startup. SSAS normally allocates memory dynamically as it is required by queries and processing jobs. In some cases, you can achieve performance improvement by allocating a portion of the memory when the SSAS service starts.Setting this value will increase the time required to start the service. The memory will not be released back to the operating system until you stop the SSAS service. You must restart the SSAS service for changes to this property to take effect.The <Log><FlightRecorder><FileSizeMB>and <Log><FlightRecorder><LogDurationSec> properties control the size and age of the FlightRecorder trace file before it is recycled. You can supply your own trace definition file to include the trace events and columns you wish to monitor using the <Log><FlightRecorder><TraceDefinitionFile> property. If FlightRecorder collects useful trace events, it can be an invaluable troubleshooting tool. By default, the file is only allowed to grow to 10 MB or 60 minutes. Long processing jobs can take up much more space, and their duration could be much longer than 60 minutes. Hence, you should adjust the settings as necessary for your monitoring needs. You should also adjust the trace events and columns to be captured by FlightRecorder. You should consider adjusting the duration to cover three days (in case the issue you are researching happens over a weekend). The <Memory><LowMemoryLimit> property controls the point—amount of memory used by SSAS—at which the cleaner thread becomes actively engaged in reclaiming memory from existing jobs. Each SSAS command (query, processing, backup, synchronization, and so on) is associated with jobs that run on threads and use system resources. We can lower the value of this setting to run more jobs in parallel (though the performance of each job could suffer). Two properties control the maximum amount of memory that a SSAS instance could use. Once memory usage reaches the value specified by <Memory><TotalMemoryLimit>, the cleaner thread becomes particularly aggressive at reclaiming memory. The <Memory><HardMemoryLimit> property specifies the absolute memory limit—SSAS will not use memory above this limit. These properties are useful if you have SSAS and other applications installed on the same server computer. You should reserve some memory for other applications and the operating system as well. When HardMemoryLimit is reached, SSAS will disconnect the active sessions, advising that the operation was cancelled due to memory pressure.All memory settings are expressed in percentages if the values are less than or equal to 100. Values above 100 are interpreted as kilobytes. All memory configuration changes require restart of the SSAS service to take effect.In the prior releases of Analysis Services, you could only specify the minimum and maximum number of threads used for queries and processing jobs. With SSAS 2012, you can also specify the limits for the input/output job threads using the <ThreadPool><IOProcess> property. The <Process><IndexBuildThreshold> property governs the minimum number of rows within a partition for which SSAS will build indexes. The default value is 4096. SSAS decides which partitions it needs to scan for each query based on the partition index files. If the partition does not have indexes, it will be scanned for all the queries. Normally, SSAS can read small partitions without greatly affecting query performance. But if you have many small partitions, you should lower the threshold to ensure each partition has indexes. The <Process><BufferRecordLimit> and <Process><BufferMemoryLimit> properties specify the number of records for each memory buffer and the maximum percentage of memory that can be used by a memory buffer. Lower the value of these properties to process more partitions in parallel. You should monitor processing using the SQL Profiler to see if some partitions included in the processing batch are being processed while the others are in waiting. The <ExternalConnectionTimeout> and <ExternalCommandTimeout> properties control how long an SSAS command should wait for connecting to a relational database or how long SSAS should wait to execute the relational query before reporting timeout. Depending on the relational source, it might take longer than 60 seconds (that is, the default value) to connect. If you encounter processing errors without being able to connect to the relational source, you should increase the ExternalConnectionTimeout value. It could also take a long time to execute a query; by default, the processing query will timeout after one hour. Adjust the value as needed to prevent processing failures. The contents of the <AllowedBrowsingFolders> property define the drives and directories that are visible when creating databases, collecting backups, and so on. You can specify multiple items separated using the pipe (|) character. The <ForceCommitTimeout> property defines how long a processing job's commit operation should wait prior to cancelling any queries/jobs which may interfere with processing or synchronization. A long running query can block synchronization or processing from committing its transaction. You can adjust the value of this property from its default value of 30 seconds to ensure that processing and queries don't step on each other. The <Port> property specifies the port number for the SSAS instance. You can use the hostname followed by a colon (:) and a port number for connecting to the SSAS instance in lieu of the instance name. Be careful not to supply the port number used by another application; if you do so, the SSAS service won't start. The <ServerTimeout> property specifies the number of milliseconds after which a query will timeout. The default value is 1 hour, which could be too long for analytical queries. If the query runs for an hour, using up system resources, it could render the instance unusable by any other connection. You can also define a query timeout value in the client application's connection strings. Client setting overrides the server-level property.There's more...There are many other properties you can set to alter SSAS instance behavior. For additional information on configuration properties, please refer to product documentation at http://technet.microsoft.com/en-us/library/ms174556.aspx.Creating and dropping databasesOnly SSAS instance administrators are permitted to create, drop, restore, detach, attach, and synchronize databases. This recipe teaches administrators how to create and drop databases.Getting readyLaunch SSMS and connect to your Analysis Services instance as an administrator. If you're not certain that you have administrative properties to the instance, right-click on the SSAS instance and choose Properties. If you can view the instance's properties, you are an administrator; otherwise, you will get an error indicating that only instance administrators can view and alter properties.How to do it...To create a database, perform the following steps:Right-click on the Databases folder and choose New Database. Doing so launches the New Database dialog shown in the following screenshot.Specify a descriptive name for the database, for example, Analysis_Services_Administration. Note that the database name can contain spaces. Each object has a name as well as an identifier. The identifier value is set to the object's original name and cannot be changed without dropping and recreating the database; hence, it is important to come up with a descriptive name from the very beginning. You cannot create more than one database with the same name on any SSAS instance.Specify the storage location for the database. By default, the database will be stored under the \OLAP\DATA folder of your SSAS installation directory. The only compelling reason to change the default is if your data drive is running out of disk space and cannot support the new database's storage requirements. Specify the impersonation setting for the database. You could also specify the impersonation property for each data source. Alternatively, each data source can inherit the DataSourceImpersonationInfo property from the database-level setting. You have four choices as follows:Specific user name (must be a domain user) and password: This is the most secure option but requires updating the password if the user changes the passwordAnalysis Services service accountCredentials of the current user: This option is specifically for data miningDefault: This option is the same as using the service account optionSpecify an optional description for the database.As with majority of other SSMS dialogs, you can script the XMLA command you are about to execute by clicking on the Script button.To drop an existing database, perform the following steps:Expand the Databases folder on the SSAS instance, right-click on the database, and choose Delete.The Delete objects dialog allows you to ignore errors; however, it is not applicable to databases. You can script the XMLA command if you wish to review it first.An alternative way of scripting the DELETE command is to right-click on the database and navigate to Script database as | Delete To | New query window. Monitoring SSAS instance using Activity ViewerUnlike other database systems, Analysis Services has no system databases. However, administrators still need to check the activity on the server, ensure that cubes are available and can be queried, and there is no blocking. You can exploit a tool named Analysis Services Activity Viewer 2008 to monitor SSAS Versions 2008 and later, including SSAS 2012. This tool is owned and maintained by the SSAS community and can be downloaded from www.codeplex.com. Activity Viewer allows viewing active and dormant sessions, current XMLA and MDX queries, locks, as well as CPU and I/O usage by each connection. Additionally, you can define rules to raise alerts when a particular condition is met.How to do it...To monitor an SSAS instance using Activity Viewer, perform the following steps:Launch the application by double-clicking on ActivityViewer.exe.Click on the Add New Connection button on the Overview tab. Specify the hostname and instance name or the hostname and port number for the SSAS instance and then click on OK.For each SSAS instance you connect to, Activity Viewer adds a new tab. Click on the tab for your SSAS instance. Here, you will see several pages as shown in the following screenshot: Alerts: This page shows any sessions that met the condition found in the Rules page.Users: This page displays one row for each user as well as the number of sessions, total memory, CPU, and I/O usage.Active Sessions: This page displays each session that is actively running an MDX, Data Mining Extensions (DMX), or XMLA query. This page allows you to cancel a specific session by clicking on the Cancel Session button.Current Queries: This page displays the actual command's text, number of kilobytes read and written by the command, and the amount of  CPU time used by the command. This page allows you to cancel a specific query by clicking on the Cancel Query button.Dormant Sessions: This page displays sessions that have a connection to the SSAS instance but are not currently running any queries. You can also disconnect a dormant session by clicking on the Cancel Session button.CPU: This page allows you to review the CPU time used by the session as well as the last command executed on the session.I/O: This page displays the number of reads and writes as well as the kilobytes read and written by each session.Objects: This page shows the CPU time and number of reads affecting each dimension and partition. This page also shows the full path to the object's parent; this is useful if you have the same naming convention for partitions in multiple measure groups. Not only do you see the partition name, but also the full path to the partition's measure group. This page also shows the number of aggregation hits for each partition. If you find that a partition is frequently queried and requires many reads, you should consider building aggregations for it.Locks: This page displays the locks currently in place, whether already granted or waiting. Be sure to check the Lock Status column—the value of 0 indicates that the lock request is currently blocked.Rules: This page allows defining conditions that will result in an alert. For example, if the session is idle for over 30 minutes or if an MDX query takes over 30 minutes, you should get alerted. How it works...Activity Viewer monitors Analysis Services using Dynamic Management Views (DMV). In fact, capturing queries executed by Activity Viewer using SQL Server Profiler is a good way of familiarizing yourself with SSAS DMV's. For example, the Current Queries page checks the $system.DISCOVER_COMMANDS DMV for any actively executing commands by running the following query:SELECT SESSION_SPID,COMMAND_CPU_TIME_MS,COMMAND_ELAPSED_TIME_MS,   COMMAND_READ_KB,COMMAND_WRITE_KB, COMMAND_TEXT FROM $system.DISCOVER_COMMANDS WHERE COMMAND_ELAPSED_TIME_MS > 0 ORDER BY COMMAND_CPU_TIME_MS DESCThe Active Sessions page checks the $system.DISCOVER_SESSIONS DMV with the session status set to 1 using the following query:SELECT SESSION_SPID,SESSION_USER_NAME, SESSION_START_TIME,   SESSION_ELAPSED_TIME_MS,SESSION_CPU_TIME_MS, SESSION_ID FROM $SYSTEM.DISCOVER_SESSIONS WHERE SESSION_STATUS = 1 ORDER BY SESSION_USER_NAME DESCThe Dormant sessions page runs a very similar query to that of the Active Sessions page, except it checks for sessions with SESSION_STATUS=0—sessions that are currently not running any queries. The result set is also limited to top 10 sessions based on idle time measured in milliseconds. The Locks page examines all the columns of the $system.DISCOVER_LOCKS DMV to find all requested locks as well as lock creation time, lock type, and lock status. As you have already learned, the lock status of 0 indicates that the request is blocked, whereas the lock status of 1 means that the request has been granted. Analysis Services blocking can be caused by conflicting operations that attempt to query and modify objects. For example, a long running query can block a processing or synchronization job from completion because processing will change the data values. Similarly, a command altering the database structure will block queries. The database administrator or instance administrator can explicitly issue the LOCK XMLA command as well as the BEGIN TRANSACTION command. Other operations request locks implicitly. The following table documents most frequently encountered Analysis Services lock types: Lock type identifierDescriptionAcquired for2Read lockProcessing to read metadata.4Write lockProcessing to write data after it is read from relational sources.8Commit sharedDuring the processing, restore or synchronization commands.16Commit exclusiveCommitting the processing, restore, or synchronization transaction when existing files are replaced by new files. 
Read more
  • 0
  • 0
  • 20720

article-image-background-animation
Packt
19 Dec 2013
4 min read
Save for later

Background Animation

Packt
19 Dec 2013
4 min read
(For more resources related to this topic, see here.) Background-color animation Animating the background color of an element is a great way to draw our user's eyes to the object we want them to see. Another use for animating the background color of an element is to show that something has happened to the element. It's typically used in this way if the state of the object changes (added, moved, deleted, and so on), or if it requires attention to fix a problem. Due to the lack of support in jQuery 2.0 for animating background-color, we'll be using jQuery UI to give us the functionality we need to create this effect. Introducing the animate method The animate() method is one of the most useful methods jQuery has to offer in its bag of tricks in the animation realm. With it, we’re able to do things like, move an element across the page or alter and animating the properties of colors, backgrounds, text, fonts, the box model, position, display, lists, tables, generated content, and so on. Time for action – animating the body background-color Following the steps below, we're going to start by creating an example that changes the body background color. Start by creating a new file (using our template) called background-color.html and save it in our jquery-animation folder. Next, we'll need to include the jQuery UI library by adding this line directly under our jQuery library by adding this line: <script src = "js/jquery-ui.min.js"></script> A custom or stable build of jQuery UI can be downloaded from http://jqueryui.com, or you can link to the library using one of the three Content Delivery Networks (CDN) below. For fastest access to the library, go to http://jqueryui.com, scroll to the very bottom and look for the Quick Access section. Using the jQuery UI library JS file there will work just fine for our needs for the examples in this article. Media Template: http://code.jquery.com Google: http://developers.google.com/speed/libraries/devguide#jquery-ui Microsoft: http://asp.net/ajaxlibrary/cdn.ashx#jQuery_Releases_on_the_CDN_0 CDNJS: http://cdnjs.com/libraries/jquery Then, we'll add the following jQuery code to the anonymous function: var speed = 1500; $( "body").animate({ backgroundColor: "#D68A85" },speed); $( "body").animate({ backgroundColor: "#E7912D" },speed); $( "body").animate({ backgroundColor: "#CECC33" },speed); $( "body").animate({ backgroundColor: "#6FCD94" },speed); $( "body").animate({ backgroundColor: "#3AB6F1" },speed); $( "body").animate({ backgroundColor: "#8684D8" },speed); $( "body").animate({ backgroundColor: "#DD67AE" },speed); What just happened? First we added in the jQuery UI library to our page. This was needed because of the lack of support for animating the background color in the current version of jQuery. Next, we added in the code that will animate our background. We then set the speed variable to 1500 (milliseconds) so that we can control the duration of our animation. Lastly, using the animate() method, we set the background color of the body element and set duration to the variable we set above named speed. We duplicated the same line several times, changing only the hexadecimal value of the background color. The following screenshot is an illustration of colors the entire body background color animates through: Chaining together jQuery methods It's important to note that jQuery methods (animate() in this case) can be chained together. Our code mentioned previously would look like the following if we chained the animate() methods together: $("body")   .animate({ backgroundColor: "#D68A85"}, speed)  //red   .animate({ backgroundColor: "#E7912D"}, speed)  //orange   .animate({ backgroundColor: "#CECC33"}, speed)  //yellow   .animate({ backgroundColor: "#6FCD94"}, speed)  //green   .animate({ backgroundColor: "#3AB6F1"}, speed)  //blue   .animate({ backgroundColor: "#8684D8"}, speed)  //purple   .animate({ backgroundColor: "#DD67AE"}, speed); //pink Here's another example of chaining methods together: (selector).animate(properties).animate(properties).animate(properties) Have a go hero – extending our script with a loop In this example we used the animate() method and with some help from jQuery UI, we were able to animate the body background color of our page. Have a go at extending the script to use a loop, so that the colors continually animate without stopping once the script gets to the end of the function. Pop quiz – chaining with the animate() method Q1. Which code will properly animate our body background color from red to blue using chaining? $("body")   .animate({ background: "red"}, "fast")   .animate({ background: "blue"}, "fast"); $("body")   .animate({ background-color: "red"}, "slow")   .animate({ background-color: "blue"}, "slow"); $("body")   .animate({ backgroundColor:"red" })   .animate({ backgroundColor:"blue" }); $("body")   .animate({ backgroundColor,"red" }, "slow")   .animate({ backgroundColor,"blue" }, "slow");
Read more
  • 0
  • 0
  • 2520
article-image-going-isometric
Packt
19 Dec 2013
12 min read
Save for later

Going Isometric

Packt
19 Dec 2013
12 min read
(For more resources related to this topic, see here.) Cartesian to isometric equations A very important thing to understand here is that the level data still remains the same 2D array, and we will be altering only the rendering process. Later on, we will need to update the level data to accommodate large tiles, which will contain items that are bigger than the current tile size. Our two-dimensional top-down coordinates for a tile can be called Cartesian coordinates. The relationship between Cartesian and isometric coordinates is shown in the following code: //Cartesian to isometric: x_Iso = x_Cart - y_Cart; y_Iso = ( x_Cart + y_Cart ) / 2; //Isometric to Cartesian: x_Cart = ( 2 * y_Iso + x_Iso ) / 2; y_Cart = ( 2 * y_Iso – x_Iso ) / 2; Now that is very simple isn't it? We will use an IsoHelper class for this conversion where we can pass through a point and get back to the converted point. An isometric view via a matrix transformation Although the equations are simple and straightforward, the art needed for an isometric tile is a bit complicated. The artist needs to create the rhombus-shaped tile art with pixel precision and mostly tileable in all four directions. An alternative approach is to use the square tile itself and skew them dynamically using the corresponding code. Let us try to create the isometric view for the level data with the same tiles using this approach. The transformation matrix for isometric transformation is as follows, which is essentially a rotation of 45 degrees and scaling by half in Y axis: var m:Matrix = new Matrix(1,0.5,-1,0.5,0,0); The code for the IsometricLevel class, is shared as follows. You should initialize this class from the Starling document class using new Starling (IsometricLevel, stage). The following approach just applies the isometric transformation matrix to the RenderTexture image. Minor changes in the init function are shown in the following code: var m:Matrix = new Matrix(1,0.5,-1,0.5,0,0); for(var i_int=0;i<levelData.length;i++){ for(var j_int=0;j<levelData[0].length;j++){ img=new Image(texAtlas.getTexture(paddedName(levelData[i][j]))); img.x=j*tileWidth+borderX; img.y=i*tileWidth+borderY; rTex.draw(img); } } m.translate( 300, 0 ); rTexImage.transformationMatrix = m; We apply the transformation matrix to the RenderTexture image and translate it by 300 pixels so that the whole of it is visible. Skewing will make a part of the image to be out of the visible area of the screen. We will get the following result: An alternate approach is to apply the transformation matrix to each individual tile image, find the corresponding isometric coordinates, and move and place individual tiles accordingly as shown in the following code: var m:Matrix = new Matrix(1,0.5,-1,0.5,0,0); var pt_Point=new Point(); for(var i_int=0;i<levelData.length;i++){ for(var j_int=0;j<levelData[0].length;j++){ img = new Image(texAtlas.getTexture(paddedName(levelData[i][j]))); img.transformationMatrix = m; pt.x=j*tileWidth+borderX; pt.y=i*tileWidth+borderY; pt=IsoHelper.cartToIso(pt); img.x=pt.x+300; img.y=pt.y; rTex.draw(img); } } Here, we use the convenient cartToIso(pt) conversion function of our IsoHelper class to find the corresponding isometric coordinates to our Cartesian coordinates. We are offsetting the drawing by 300 pixels to handle the skewing offset for the image. This approach will work in some cases, but not all top-down tiles can be simply skewed and made into an isometric tile. For example, consider a tree in the top-down view, it will simply look like a skewed tree graphic after we apply the isometric transformation. So, the right approach is to create an isometric tile art specifically and use isometric equations to place them correctly. Let us use the isometric tiles provided in the assets pack to create a sample level. Implementing the isometric view via isometric art Please refer to the SampleIsometricDemo source folder, which implements a sample level of our game using isometric art and the previously mentioned equations. There are some differences in the approach that I will be explaining in the following sections. Most of it has to do with the change in level data, altering the registration point of larger tiles, and handling depth. We also need to offset the image drawing so that it fits in the screen area. We use a variable called screenOffset for this purpose. The render code is as follows: var pt_Point=new Point(); for(var i_int=0;i<groundArray.length;i++){ for(var j_int=0;j<groundArray[0].length;j++){ //draw the ground img=new Image(texAtlas.getTexture(String(groundArray[i][j]).split(".")[0])); pt.x=j*tileWidth; pt.y=i*tileWidth; pt=IsoHelper.cartToIso(pt); img.x=pt.x+screenOffset.x; img.y=pt.y+screenOffset.y; rTex.draw(img); //draw overlay if(overlayArray[i][j]!="*"){ img=new Image(texAtlas.getTexture(String(overlayArray[i][j]).split(".")[0]));])); img.x=pt.x+screenOffset.x; img.y=pt.y+screenOffset.y; if(regPoints[overlayArray[i][j]]!=null){ img.x+=regPoints[overlayArray[i][j]].x; img.y-=regPoints[overlayArray[i][j]].y; } rTex.draw(img); } } } The result is shown in the following screenshot: Level data structure The level data for our isometric level is not just a simple 2D array with index numbers any more, but a combination of multiple data structures. We have a 2D array for the ground tiles, another 2D array for overlay tiles, and a dictionary to store altered registration points of the overlay tiles. Ground tiles are those tiles which exactly fit the isometric tile dimensions, which in this case is 80 x 40, and makes up the bottom-most layer of the isometric level. These tiles won't take part in any depth sorting as they are always rendered below all other items that populate the level. Overlay tiles are items which may not fit into the isometric tile dimensions and have height, for instance, buildings, trees, bushes, rocks, and so on. Some of these can be fit into tile dimensions, but are kept as such that we have various advantages using the following approach: We are free to place an overlay tile over any ground tile, which adds to flexibility We would need a lot of tiles if we try to fit overlay tiles and ground tiles together for all permutations and combinations Effects such as tinting can be applied independently to the overlay tiles Depth handling becomes much easier Overlay tiles which are smaller than the tile size reduce the game size Altering registration points Starling considers all images as rectangular blocks with their registration point at the top-left corner. The registration point is the point which can be considered as the (0,0) of that image. Traditional Flash had given us the capability to alter the registration points by embedding images inside Sprite or MovieClip. We can still do the same, but it will require unnecessary creation of a lot of Sprites. Alternately, we can use the pivotX and pivotY properties of Starling objects for the same result too. In our isometric level, we will need to precisely place overlay tiles inside the isometric grid space. An overlay tile does not have any standard size as it can be any item— a tree, building, character, and so on. So, placing them correctly is a tricky thing and very specific to the tile concerned. This leads us to have independent registration points for each overlay tile. We use a dictionary structure to save these values and use those values as offsets while placing overlay tiles. For example, we need to place a bush image, nonwalk0009.png, exactly at the middle of an isometric grid, which means moving it 12 pixels to the left and 19 pixels to the top for proper alignment. We save (12,19) as a new point inside our dictionary for ID nonwalk0009.png, as follows: regPoints["nonwalk0009.png"]=new Point(12,19); Finding a tile's precise placement point needs to involve visual interaction; hence, we will build a level editor, which makes this easier. Depth sorting An isometric view needs us to handle the depth of items manually. For ground tiles, there is no depth issue as they always form the lowest layer over which all the overlay items and characters are drawn. But overlay tiles and characters need to be drawn at specific depths for it to look appropriate. By depth, I mean the order at which the images are drawn. An image drawn later will overlap the one drawn earlier, thereby making it seem in front of the latter. For a level which does not change or without any moving items, we need to find the depth only once for the initial render. But for a level with moving characters or vehicles, we need to find every frame in the game loop and render. The current sample level does not change over time, so we can simply render the level by looping through the array. Any overlay item placed at a higher I or J value will be rendered later, and hence will be shown in front, where I and J are array indices. Thus, items placed at higher indices appear closer to the camera, that is, for the same I, a higher J is closer to the camera and vice versa. When we have a moving item, we need to find the corresponding array position it occupies based on its current screen position. By using these new found array indices, we can compare with the overlay tile's indices and decide on the drawing sequence. The code to find array indices from the screen position is as follows: //capture screen position var screenPos_Point=new Point(hero.x,hero.y); //convert to cartesian coordinates var cartPos_Point=IsoHelper.isoToCart(screenPos); //find tile indices from cartesian values var tilePos_Point=IsoHelper.getTileIndices(screenPos,tileWidth); Understanding isometric movement Isometric movement is very straightforward to implement. All we need to do is move the item in top-down Cartesian coordinates and draw it on the screen after converting into isometric coordinates. For example, if our character is at a point, heroCart in the Cartesian system, then the following code moves him/her to the right: heroCart.x+=heroSpeed; //convert to isometric coordinates heroIso=IsoHelper.cartToIso(heroCart); heroImage.x=heroIso.x; heroImage.y=heroIso.y; rTex.draw(heroImage); Detecting isometric collision Collision detection for any tile-based game is done based on tiles. When designing, we will make sure that certain tiles are walkable while certain tiles are nonwalkable, which means that the characters can move over some tiles but not over others. So, when we calculate the movements of any character, we first make sure that the character won't end up on a nonwalkable tile. Thus, after each movement, we check if the resulting position falls in a nonwalkable tile by finding array indices as mentioned previously. If the result is true, we will ignore the movement, else we will proceed with the movement and update the on-screen character position. heroCart.x+=heroSpeed; //find new tile point var tilePos_Point=IsoHelper.getTileIndices(heroCart,tileWidth); //this checks if new tile position is occupied, else proceeds if(checkWalkable(tilePos)){ //convert to isometric coordinates heroIso=IsoHelper.cartToIso(heroCart); heroImage.x=heroIso.x; heroImage.y=heroIso.y; rTex.draw(heroImage); } You may be wondering that the hero character should need some special considerations to be drawn correctly as the right depth, but by the way we draw things, it gets handled automatically. We do not allow the hero to move onto a nonwalkable tile, that is, bushes, trees, and so on. So, any tile remains walkable or nonwalkable. The character gets drawn on top of a walkable tile, which does not contain any overlay items, and hence it will occupy the right depth. In this method, a full tile has to be made either walkable or nonwalkable, but this may not be the case for all games. We may need to have tiles, which block entry from a specific direction or block exit in a particular direction as a fence along one border of a tile. In such cases, the tile is still walkable, but valid movement is also checked by tracking the direction in which the character is moving. For our game, the first method will be used along with the four-way freedom of movement. In an isometric view, movement can be either in four directions or eight directions, which in turn is called a four-way movement or an eight-way movement respectively. A four-way movement is when we move along the X or Y axis alone on the Cartesian space. An eight-way movement happens when, in addition to four - way, we also move the item diagonally. Logic still remains the same. Summary In this article, we learned about the isometric projection and the equations that help us to implement it based on the simpler Cartesian system. We implemented a sample isometric level using isometric art as well as learned about matrix-based fake isometric rendering. We analyzed the IsoHelper class, which facilitates easy conversion between Cartesian and isometric coordinates and also helps in finding array indices. We learned why altering the registration points is essential for perfectly placing the overlay tiles and we found that our level data needs to track these registration points as well. We also learned how depth sorting, collision detection, and isometric movement are done based on our tile-based approach. Resources for Article: Further resources on this subject: Introduction to Game Development Using Unity 3D [Article] Flash Game Development: Making of Astro-PANIC! [Article] Collision Detection and Physics in Panda3D Game Development [Article]
Read more
  • 0
  • 0
  • 18817

article-image-platform-service-and-cloudbees
Packt
19 Dec 2013
10 min read
Save for later

Platform as a Service and CloudBees

Packt
19 Dec 2013
10 min read
(For more resources related to this topic, see here.) Platform as a Service (PaaS) is a crossover between IaaS and SaaS. This is a fuzzy definition, but it defines well the existing actors in this industry well and possible confusions. A general presentation of PaaS uses a pyramid. Depending on what the graphics try to demonstrate, the pyramid can be drawn upside down, as shown in the following diagram: Cloud pyramids The pyramid on the left-hand side shows XaaS platforms based on the target users' profiles. It demonstrates that IaaS is the basis for all Cloud services. It provides the required flexibility for PaaS to support applications that are exposed as SaaS to the end users. Some SaaS actually don't use a PaaS and directly rely on IaaS, but that doesn't really matter here. The pyramid on the right-hand side represents the providers and the three levels suggests the number of providers in each category. IaaS only makes sense for highly concentrated, large-scale providers. PaaS can have more actors, probably focused on some ecosystem, but the need is to have a neutral and standard platform that is actually attractive for developers. SaaS is about all the possible applications running in Cloud. The top-level shape should then be far larger than what the graphic shows. So, which platform? With the previous definition of platform, you just have a faint idea; your understanding about PaaS is more than IaaS and less than SaaS. The missing definition is to know what the platform is about. A platform is a standardization of the runtime for which a developer is waiting to do his/her job. This depends on the software ecosystem you're considering. For a Java EE developer, a platform means having at least a servlet container, managing DataSource to access the database, and having few comparable resources wrapped as standard Java EE APIs. A Play! framework developer will consider this as overweight and only ask for a JVM with web socket's support. A PHP developer will expect a Linux/Apache/MySQL/PHP (LAMP) stack, similar to the one he/she has been using for years, with a traditional server hosting service. So, depending on the development ecosystem you're considering, platforms don't have the exact same meaning, but they all share a common principle. A platform is the common denominator for a software language ecosystem, where the application is all that a specific developer will write or choose on their own. Java EE developers will ask for a container, and Ruby developers will ask for an RVM environment. What they run on top is their own choice. With this definition, you understand that a platform is about the standardization of runtime for a software ecosystem. Maybe some of you have patched OpenJDK to enable some magic features in the JVM (really?), but most of us just use the standard Oracle Java distribution. Such a standardization makes it possible to share resources and engineering skills on a large scale, to reduce cost, and provide a reliable runtime. Cloud and clustering Another consideration for a platform is clustering. Cloud is based on slicing resources into small virtual elements and letting the users select as many as they need. In most cases, this requires the application to support a clustering mode, as using more resources will require you to scale out on multiple hosts. Clustering has never been a trivial thing, and many developers aren't familiar with the related constraints. The platform can help them by providing specialized services to distribute the load around the cluster's nodes. Some PaaS such as CloudBees or Google App Engine provide such features, while some don't. This is the major difference between PaaS offers. Some are IaaS-like preinstalled middleware services, while some offer a highly integrated platform. A typical issue faced is that of state management. Java EE developers rely on HttpSession to store user's data and retrieve them on subsequent interaction. Modern frameworks tend to be stateless, but the state needs to be managed anyway. PaaS has to provide options to developers, so that they can choose the best strategy to match their own business requirements. This is a typical clustering issue that is well addressed by PaaS because the technical solutions (sticky session, session replication, distributed storage engines, and so on) have been implemented once with all the required skills to do it right, and can be used by all platform users. Thanks to a PaaS, you don't need to be a clustering guru. This doesn't mean that it will magically let your legacy application scale out, but it gives you adequate tools to design the application for scalability. Private versus public Clouds Many companies are interested in Cloud, thanks to the press for publishing all product announcements as the new revolution, and would like to benefit from them but as a private resource. If you go back to the comparison in the Preface with an electricity production, this may make sense if you're well established. Amazon or Google should have private power plants to supply giant data centers can make sense—anyway it doesn't seems that they do but as backends. For most of companies, this would be a surprising company choice. The main reason is that the principle of the Cloud relies on the last letter of XaaS (S) that stands for Service. You can install an OpenStack or VMware farm on your data center, but then you won't have an IaaS. You will have some virtualization and flexibility that probably is far better than traditional dedicated hardware, but you miss the major change. You still will have to hire operators to administer the servers and software stack. You will even have a more complex software stack (search for an OpenStack administrator and you'll understand). Using Cloud makes sense because there are thousands of users all around the world sharing the same lower-level resources, and a centralized, highly specialized team to manage them all. Building your own, private PaaS is yet another challenge. This is not a simple middleware stack. This is not about providing virtual machine images with a preinstalled Tomcat server. What about maintenance, application scalability, deployment APIs, clustering, backup, data replication, high availability,monitoring, and support? Support is a major added value of cloud services—I'm not just saying this because I'm a support engineer—but because when something fails, you need someone to help. You can't just wait with the promise for a patch provided by the community. The guy who's running your application needs to have significant knowledge of the platform. That's one reason that CloudBees is focusing on Java first, as this is the ecosystem and environment we know best (even we have some Erlang and Ruby engineers whose preferred game is to troll on this displeasing language). With a private Cloud, you probably can have level-one support with an internal support team, but you can't handle all the issues. As for resource concentration, to build an impressive knowledge base. All those topics are ignored in most cases as people only focus on the app:deploy automation, as opposed to the old-style deployments to dedicated hardware. If this is what you're looking for, you should know that Maven was able to do this for years on all the Java EE containers using cargo. You can check the same at http://cargo.codehaus.org. Cloud isn't just about abstracting the runtime behind an API; it's about changing the way in which developers manage and access runtime so that it becomes a service they can consume without any need to worry about what's happening behind the scene. Security The reason that companies claim to prefer a private cloud solution is security. Amazon datacenters are far more secure than any private datacenter, due to both strong security policy and anonymous user data. Security is not about exploiting encryption algorithms, like in Hollywood movies, but about social attacks that are far more fragile. Few companies take care of administrative, financial, familial, or personal safety. Thanks to the combination of VPN, HTTPS, fixed IPs, and firewall filters, you can safely deploy an application on Amazon Cloud as an extension to your own network, to access data from your legacy Oracle or SAP mainframe hosted in your datacenter. As a mobile application demonstrates, your data is already going out from your private network. There's no concrete reason why your backend application can't be hosted outside your walls. CloudBees – embrace the development stack CloudBees PaaS has something special in its DNA that you won't find in other PaaS; focusing on the Java ecosystem first, even with polyglot support, CloudBees understands well the Java ecosystem's complexity and its underlying practices. Heroku was one of the first successful PaaS, focusing on Ruby runtime. Deployment of a Ruby application is just about sending source code to the platform using the following command: git push heroku master Ruby is a pleasant ecosystem because there are no such long debates on building and provisioning tools that we know of, unlike in JavaWorld, GemFile, and Rake, period. In the Java ecosystem, there is a need to generate, compile the source code, and then sometime post the process classes, hence a large set of build tools are required. There's also a need to provision runtime with dozens of dependencies, so a set of dependency management tools, inter-project relations, and so on are required. With Agile development practices, automated testing has introduced a huge set of test frameworks that developers want to integrate into the deployment process. The Java platform is not just about hosting a JVM or a servlet container, it's about managing Ant, Maven, SBT, or Gradle builds, as well as Grails-, Play-, Clojure-, and Scala-specific tooling. It's about hosting dependency repositories. It's about handling complex build processes to include multiple levels of testing and code analysis. The CloudBees platform has two major components: RUN@cloud is a PaaS, as described earlier, to host applications and provide high-level runtime services DEV@cloud is a continuous integration and deployment SaaS based on Jenkins Jenkins is not the subject of this article, but it is the de facto standard for but not limited to continuous integration in the Java ecosystem. With a large set of plugins, it can be extended to support a large set of tools, processes, and views about your project. The CloudBees team includes major Jenkins committers (including myself #selfpromotion), and so it has a deep knowledge on Jenkins ecosystem and is best placed to offer it as a Cloud service. We also can help you to diagnose your project workflow by applying the best continuous integration and deployment practices. This also helps you to get more efficient and focused results on your actual business development. The following screenshot displays the continuous Cloud delivery concept in CloudBees: With some CloudBees-specific plugins to help, DEV@cloud Jenkins creates a smooth code-build-deploy pipeline, comparable to Heroku's Git push, but with full control over the intermediary process to convert your source code to a runnable application. This is such a significant component to build a full stack for Java developers that CloudBees is the official provider for the continuous integration service for Google App Engine (http://googleappengine.blogspot.fr/2012/10/jenkins-meet-google-app-engine.html), Cloud Foundry (http://blog.cloudfoundry.com/2013/02/28/continuous-integration-to-cloud-foundry-com-using-jenkins-in-the-cloud/), and Amazon. Summary This article introduced the Cloud principles and benefits, and compared CloudBees to its competitors. Resources for Article: Further resources on this subject: Framework Comparison: Backbase AJAX framework Vs Other Similar Framework (Part 2) [Article] Integrating Spring Framework with Hibernate ORM Framework: Part 2 [Article] Working with Zend Framework 2.0 [Article]
Read more
  • 0
  • 0
  • 2872
Modal Close icon
Modal Close icon