How-To Tutorials

(For more resources related to this topic, see here.) Creating a character is not an easy job, it can be done by creative insight or careful calculation. The main hero is the soul of the game and whose avatar is used by a player to explore the game world. Players identify themselves with that picture on the screen, empathizing with it, enjoying, grumbling, and taking all the situations to heart. Therefore, the objective is to generate an emotional connection between the character and the player. It is not necessary to provoke only positive feelings; ironically, sometimes the player can hate his avatar. A graphic look of the character can induce a wide spectrum of emotions, and these emotions can be calculated in advance. This is based on the fact that humans usually have nearly the same reactions; in this case, the rules of their behavior are pretty conspicuous. To illustrate the principles, I created the Scale of attractiveness, made of four main entries. At the very left, there is the Cute mark, in the middle is Brutal, followed by Human likeness, and to the right we see Scary. So, the chart begins with something very adorable and sweet but end with a scary character. Each position has its own collection of qualities. Making characters cute Cuteness is one of the most popular and demanded features that game designers want for their creations. Protagonists of majority casual games are cute to some degree; they look and act as sweet and comely creatures. It is simply impossible to ignore them and not fall in love with them. Recall the famous image of Nintendo's Mario, he is 100 percent cute. What is the secret of such popularity? First, the cuteness is not about beauty (it is hard to call an alligator's baby truly attractive, but it definitely is cute), which depends on personal taste and preferences, but about some basic patterns and proportions. Here is a citation from Natalie Angie's article The Cute Factor published in The New York Times: "Scientists who study the evolution of visual signaling have identified a wide and still expanding assortment of features and behaviors that make something look cute: bright forward-facing eyes set low on a big round face, a pair of big round ears, floppy limbs and a side-to-side, teeter-totter gait, among many others." All listed features are general descriptions of one class of creatures on Earth: little babies and cubs. They are small, their heads are noticeably bigger than their body, their limbs are short, eyes are large, and so on. When we see something like that, a special system inside us tends to react commonly. It says that probably in front of us is a defenseless young creature that needs protection, care, and tenderness; a list of positive senses is switched on. Figuratively, we are filled with light. By introducing a cute character in a game or other media, the authors simply exploits one of the natural human reactions. This is possible because it is pretty unconditional, the brain only needs some basic patterns, and the factual meaning of an object is totally irrelevant in this case. Thus, we consider something as cute despite the fact it is not a baby at all. Kittens are super cute, but adult cats can be cute too because they are small, have round and smooth bodies, and big eyes. Another popular example is owls, they have big round heads and large expressive eyes, making them one of the cutest birds on the planet. Moreover, some mechanical objects are cute as well: majority European compact cars from the 1950s are adorable, remember the BMW Isetta, Fiat 500, original Mini, and VW Beetle? All of them look so nice and sweet, that you want to hug them, cover them with a plaid, and give some milk in a plate, as though they are small mechanical babies of bigger adult cars. The industrial design in that period was inclined toward cuteness (may be it correlated with the baby boom). Even utility vehicle such as buses and trucks were cute, in addition to household devices such as radio sets and refrigerators. But the most amazing thing was cute weapons. Of course, I'm not talking about the real ones, but imaginary ray guns that appeared in sci-fi art and in the form of toys were definitely adorable. It is clear why illustrators prefer to use objects from previous decades in their pictures, the final illustrations look warmer. Therefore, to create a cute character, you need to follow some evident visual rules: Short body Rounded angles Curved contours and chubby figures Smooth surfaces without folds and wrinkles Big head with large forehead and small low jaw Small mouth and teeth Large eyes (or they stand wide) with big pupils Wide-open eyes with eyebrows lifted up Animals with big nose Short arms, legs, and fingers without visual joints Making characters scary Generally, cuteness is necessary for a protagonist to have a corner in the player's heart. An antagonist must give birth to opposite feelings: loath and fear. Good enemies are creepy characters. To choose their visual appearance, let's again exploit some ancient mechanics from the human brain. There are a lot of alarm systems that alert us when something looks or behaves suspiciously. Deep-seated fears of various forms are inside most people. A game, of course, should not provoke a real panic action, but can tickle some sensitive zones, playing with associations. Creepiness is the complete opposite of cuteness, it gives not a feeling of warmth, but that of cold. To find an effective scary factor, it is good to look at common fears. Traditionally, many people try to avoid insects or even have phobias. Attributes of such creatures are interpreted as unpleasant or frightening. The only exception is ladybugs (they are round with white dots on their head and large eyes) and butterflies, which are primarily associated with petals of bright flowers. It is important to note that in most cases, insects are not aggressive and harmful, but they remind us of creatures from our worst nightmares, giving us the creeps (few examples are mole crickets and earwigs, eek!) Besides them, there are other types of arthropods with high potential of creepiness (and some of them are really dangerous!): spiders, scorpions, and centipedes. They have adverse visual features such as jointed legs, spikes, exoskeletons, tails, multiple eyes, mandibles, and pincers. The key visual element is a gad, associated with cuts, injuries, and so on, that contradicts a cute creature's properties, which has no acute angles. Moreover, fishes, reptiles, birds, and mammals look dangerous when they show their teeth, canines, tusks, horns, clutches, or sharp beaks. A predator is frightening because it shows it threatens with its weaponry: the potential danger is pretty obvious and their current intention is questionable. Now, the eyes comes into play. If they are fixed at you and are not blinking, it is most likely that the predator is paying attention to you and that is super scary. Furthermore, the eyes can be red because the reflection of light creates such an image. Dangerous creatures are fast so they can move and attack quickly; this means their limbs are pretty long, but bodies are narrow and streamlined. The following figure shows a scary creature: Besides aggressive elements, other unpleasant properties can be used to increase the emotional impact; for example, the character can be additionally disgusting if it is covered with strange skin and even mucus. That turns on the dread of biological substances and the fear of germs and parasites. Squeamishness is one of the protection systems of a human, and sometimes it is very unconditional. Such an approach was used in Ridley Scott's science fiction classics Alien: apparently the xenomorph was inspired by various creepy creatures, including arthropods and reptiles. In addition, it had a very disgusting feature: toxic saliva was always dripping from his mouth, causing the viewers to feel terror and revulsion, a doubled negative emotion. While creating a scary character, remember some basic features it needs to express through its design: Long and skinny body Nonhumanoid structure Many angles Many legs or at least noticeable joints Acute elements, such as spikes, clutches, and horns Small head Naked eyes that stay very close and can be red Weaponry openly displayed (biological tools such as pincers, real guns, and grenades) Unpleasant skin with verruca, folds, wrinkles, and some mucus Warning color that can mean that the character is venomous Making characters brutal Brutality, at the middle of the scale of attractiveness, describes the properties of a character that should exercise some heroic duties, being a soldier or a mercenary. It is obvious that such a person cannot have cute characteristics, otherwise it would look comical. Adorable creatures are associated with something very young, but the heroic character should be an adult. He must demonstrate strength and confidence with a little aggression. So, his look should be a little scary, but only a little, as far as he is not a creepy creature from the end of the scale. Since the brutal hero performs various acts of bravery, he must be fast and agile. So, his anatomical proportions should be close to hyperbolic athletic ones like heavy action heroes from the 1980s, featuring a well-developed muscular system and military toys. The following figure shows such a heavy action hero: The apparent illustration of a brutal character is Duke Nukem, a protagonist of the game series of the same name originally developed by Apogee Software. He is brutal and fearless, and definitely not cute. A bunch of good examples is included in the game Gears of War from Epic Games. The members of Delta Squad are canonically brutal and tough guys. Such type of characters generally are used in action games such as shooters. Figuratively speaking, they are mix made of a human and an armed vehicle, since they hold heavy weapons and armor. The following are the basic visual properties of such a hero (it is important to note that the brutality is gender independent, and although many such characters are men, nobody has forbidden you from creating a strong woman protagonist): Figures with no acute angles or rounded corners, but ones that are roughed down Strong legs Heavy feet to lean on the ground reliably Big hands with tenacious fingers to hold weapons and other objects Wide chest with ram-like powerful shoulders that are bigger than the legs Normal head with mid-sized forehead and a big low jaw Naked big eyes It is pretty apparent that the extreme position on the scale is too categorical a benchmark to follow, whereas in most cases, nobody needs super cute or extremely scary characters. Something more universal is a mix of different properties, which is far more expressive. It is interesting that, as a rule, designs of a protagonist are situated between cuteness and brutality, so the character is anatomically adapted to make complex actions: move fast, climb ladders, fight, among others, and at the same time being pretty attractive. Summary The game lets you create a universe and settles it with characters, describing the rules of their behavior. This is a real gift for your imagination! And it is not about realism. The game universe can be based on your own principles and artistic taste. It can be cartoon like or gloomy and dark. It all depends on the mood of the story you are going to tell the players. Now, you know how to create a cute character and why you should be careful with real anatomical proportions. If the animation does not look scary, you can easily animate the protagonist and other characters. Resources for Article: Further resources on this subject: Development of iPhone Applications [Article] iPhone JavaScript: Installing Frameworks [Article] iPhone User Interface: Starting, Stopping, and Multitasking Applications [Article]

(For more resources related to this topic, see here.) To put some graphics on the screen; the first step for us would be creating a new game window class that will use Direct2D. This new game window class will derive from our original game window class, while adding the Direct2D functionality. Open Visual Studio. Add a new class to the project called GameWindow2D. We need to change its declaration to: public class GameWindow2D : GameWindow, IDispoable As you can see, it inherits from the GameWindow class meaning that it has all of the public and protected members of the GameWindow class, as though we had implemented them again in this class. It also implements the IDisposable interface, just as the GameWindow class does. Also, don't forget to add a reference to SlimDX to this project if you haven't already. We need to add some using statements to the top of this class file as well. They are all the same using statements that the GameWindow class has, plus one more. The new one is SlimDX.Direct2D. They are as follows: using System.Windows.Forms; using System.Diagnostics; using System.Drawing; using System; using SlimDX; using SlimDX.Direct2D; using SlimDX.Windows; Next, we need to create a handful of member variables: WindowRenderTarget m_RenderTarget; Factory m_Factory; PathGeometry m_Geometry; SolidColorBrush m_BrushRed; SolidColorBrush m_BrushGreen; SolidColorBrush m_BrushBlue; The first variable is a WindowRenderTarget object. The term render target is used to refer to the surface we are going to draw on. In this case, it is our game window. However, this is not always the case. Games can render to other places as well. For example, rendering into a texture object is used to create various effects. One example would be a simple security camera effect. Say, we have a security camera in one room and a monitor in another room. We want the monitor to display what our security camera sees. To do this, we can render the camera's view into a texture, which can then be used to texture the screen of the monitor. Of course, this has to be re-done in every frame so that the monitor screen shows what the camera is currently seeing. This idea is useful in 2D too. Back to our member variables, the second one is a Factory object that we will be using to set up our Direct2D stuff. It is used to create Direct2D resources such as RenderTargets. The third variable is a PathGeometry object that will hold the geometry for the first thing we will draw, which will be a rectangle. The last three variables are all SolidColorBrush objects. We use these to specify the color we want to draw something with. There is a little more to them than that, but that's all we need right now. The constructor Let's turn our attention now to the constructor of our Direct2D game window class. It will do two things. Firstly, it will call the base class constructor (remember the base class is the original GameWindow class), and it will then get our Direct2D stuff initialized. The following is the initial code for our constructor: public GameWindow2D(string title, int width, int height,   bool fullscreen)     : base(title, width, height, fullscreen) {     m_Factory = new Factory();     WindowRenderTargetProperties properties = new       WindowRenderTargetProperties();     properties.Handle = FormObject.Handle;     properties.PixelSize = new Size(width, height);     m_RenderTarget = new WindowRenderTarget(m_Factory,       properties); } In the preceding code, the line starting with a colon is calling the constructor of the base class for us. This ensures that everything inherited from the base class is initialized. In the body of the constructor, the first line creates a new Factory object and stores it in our m_Factory member variable. Next, we create a WindowRenderTargetProperties object and store the handle of our RenderForm object in it. Note that FormObject is one of the properties defined in our GameWindow base class. Remember that the RenderForm object is a SlimDX object that represents a window for us to draw on. The next line saves the size of our game window in the PixelSize property. The WindowRenderTargetProperties object is basically how we specify the initial configuration for a WindowRenderTarget object when we create it. The last line in our constructor creates our WindowRenderTarget object, storing it in our m_RenderTarget member variable. The two parameters we pass in are our Factory object and the WindowRenderTargetProperties object we just created. A WindowRenderTarget object is a render target that refers to the client area of a window. We use the WindowRenderTarget object to draw in a window. Creating our rectangle Now that our render target is set up, we are ready to draw stuff, but first we need to create something to draw! So, we will add a bit more code at the bottom of our constructor. First, we need to initialize our three SolidColorBrush objects. Add these three lines of code at the bottom of the constructor: m_BrushRed = new SolidColorBrush(m_RenderTarget, new Color4(1.0f,   1.0f, 0.0f, 0.0f)); m_BrushGreen = new SolidColorBrush(m_RenderTarget, new   Color4(1.0f, 0.0f, 1.0f, 0.0f)); m_BrushBlue = new SolidColorBrush(m_RenderTarget, new Color4(1.0f,   0.0f, 0.0f, 1.0f)); This code is fairly simple. For each brush, we pass in two parameters. The first parameter is the render target we will use this brush on. The second parameter is the color of the brush, which is an ARGB (Alpha Red Green Blue) value. The first parameter we give for the color is 1.0f. The f character on the end indicates that this number is of the float data type. We set alpha to 1.0 because we want the brush to be completely opaque. A value of 0.0 will make it completely transparent, and a value of 0.5 will be 50 percent transparent. Next, we have the red, green, and blue parameters. These are all float values in the range 0.0 to 1.0 as well. As you can see for the red brush, we set the red channel to 1.0f and the green and blue channels are both set to 0.0f. This means we have maximum red, but no green or blue in our color. With our SolidColorBrush objects set up, we now have three brushes we can draw with, but we still lack something to draw! So, let's fix that by adding some code to make our rectangle. Add this code to the end of the constructor: m_Geometry = new PathGeometry(m_RenderTarget.Factory); using (GeometrySink sink = m_Geometry.Open()) {     int top = (int) (0.25f * FormObject.Height);     int left = (int) (0.25f * FormObject.Width);     int right = (int) (0.75f * FormObject.Width);     int bottom = (int) (0.75f * FormObject.Height);     PointF p0 = new Point(left, top);     PointF p1 = new Point(right, top);     PointF p2 = new Point(right, bottom);     PointF p3 = new Point(left, bottom);     sink.BeginFigure(p0, FigureBegin.Filled);     sink.AddLine(p1);     sink.AddLine(p2);     sink.AddLine(p3);     sink.EndFigure(FigureEnd.Closed);     sink.Close(); } This code is a bit longer, but it's still fairly simple. The first line creates a new PathGeometry object and stores it in our m_Geometry member variable. The next line starts the using block and creates a new GeometrySink object that we will use to build the geometry of our rectangle. The using block will automatically dispose of the GeometrySink object for us when program execution reaches the end of the using block. The using blocks only work with objects that implement the IDisposable interface. The next four lines calculate where each edge of our rectangle will be. For example, the first line calculates the vertical position of the top edge of the rectangle. In this case, we are making the rectangle's top edge be 25 percent of the way down from the top of the screen. Then, we do the same thing for the other three sides of our rectangle. The second group of four lines of code creates four Point objects and initializes them using the values we just calculated. These four Point objects represent the corners of our rectangle. A point is also often referred to as a vertex. When we have more than one vertex, we call them vertices (pronounced as vert-is-ces). The final group of code has six lines. They use the GeometrySink and the Point objects we just created to set up the geometry of our rectangle inside the PathGeometry object. The first line uses the BeginFigure() method to begin the creation of a new geometric figure. The next three lines each add one more line segment to the figure by adding another point or vertex to it. With all four vertices added, we then call the EndFigure() method to specify that we are done adding vertices. The last line calls the Close() method to specify that we are finished adding geometric figures, since we can have more than one if we want. In this case, we are only adding one geometric figure, our rectangle. Drawing our rectangle Since our rectangle never changes, we don't need to add any code to our UpdateScene() method. We will override the base class's UpdateScene() method anyway, in case we need to add some code in here later, which is given as follows: public override void UpdateScene(double frameTime) {     base.UpdateScene(frameTime); } As you can see, we only have one line of code in this override modifier of the base class's UpdateScene() method. It simply calls the base class's version of this method. This is important because the base class's UpdateScene() method contains our code that gets the latest user input data each frame. Now, we are finally ready to write the code that will draw our rectangle on the screen! We will override the RenderScene() method so we can add our custom code. The following is the code: public override void RenderScene() {     if ((!this.IsInitialized) || this.IsDisposed)     {         return;     }     m_RenderTarget.BeginDraw();     m_RenderTarget.Clear(ClearColor);     m_RenderTarget.FillGeometry(m_Geometry, m_BrushBlue);     m_RenderTarget.DrawGeometry(m_Geometry, m_BrushRed, 1.0f);     m_RenderTarget.EndDraw(); } First, we have an if statement, which happens to be identical to the one we put in the base class's RenderScene() method. This is because we are not calling the base class's RenderScene() method, since the only code in it is this if statement. Not calling the base class version of this method will give us a slight performance boost, since we don't have the overhead of that function call. We could do the same thing with the UpdateScene() method as well. In this case we didn't though, because the base class version of that method has a lot more code in it. In your own projects you may want to copy and paste that code into your override of the UpdateScene() method. The next line of code calls the render target's BeginDraw() method to tell it that we are ready to begin drawing. Then, we clear the screen on the next line by filling it with the color stored in the ClearColor property that is defined by our GameWindow base class. The last three lines draw our geometry twice. First, we draw it using the FillGeometry() method of our render target. This will draw our rectangle filled in with the specified brush (in this case, solid blue). Then, we draw the rectangle a second time, but this time with the DrawGeometry() method. This draws only the lines of our shape but doesn't fill it in, so this draws a border on our rectangle. The extra parameter on the DrawGeometry() method is optional and specifies the width of the lines we are drawing. We set it to 1.0f, which means the lines will be one-pixel wide. And the last line calls the EndDraw() method to tell the render target that we are finished drawing. Cleanup As usual, we need to clean things up after ourselves when the program closes. So, we need to add override of the base class's Dispose(bool) method. We've already done this a few times, so it should be somewhat familiar and is not shown here. Our blue rectangle with a red border As you might guess, there is a lot more you can do with drawing geometry. You can draw curved line segments and draw shapes with gradient brushes too for example. You can also draw text on the screen using the render target's DrawText() method. But since we have limited space on these pages, we're going to look at how to draw bitmap images on the screen. These images are something that make up the graphics of most 2D games. Summary In this article, we first made a simple demo application that drew a rectangle on the screen. Then, we got a bit more ambitious and built a 2D tile-based game world. Resources for Article: Further resources on this subject: HTML5 Games Development: Using Local Storage to Store Game Data [Article] Flash Game Development: Creation of a Complete Tetris Game [Article] Interface Designing for Games in iOS [Article]

(For more resources related to this topic, see here.) In this article, we will cover the following recipes: Using a function loader to access the latest OpenGL functionality Using GLM for mathematics Determining the GLSL and OpenGL version Compiling a shader Linking a shader program The OpenGL Shading Language(GLSL) Version 4 brings unprecedented power and flexibility to programmers interested in creating modern, interactive, and graphical programs. It allows us to harness the power of modern Graphics Processing Units(GPUs) in a straightforward way by providing a simple yet powerful language and API. Of course, the first step towards using GLSL is to create a program that utilizes the latest version of the OpenGL API. GLSL programs don't stand on their own; they must be a part of a larger OpenGL program. In this article, we will provide some tips and techniques for getting a basic program up and running. First, let's start with some background. The OpenGL Shading Language The GLSL is now a fundamental and integral part of the OpenGL API. Going forward, every program written using the OpenGL API will internally utilize one or several GLSL programs. These "mini-programs" are often referred to as shader programs. A shader program usually consists of several components called shaders. Each shader executes within a different section of the OpenGL pipeline. Each shader runs on the GPU, and as the name implies, (typically) implement the algorithms related to the lighting and shading effects of an image. However, shaders are capable of doing much more than just implementing a shading algorithm. They are also capable of performing animation, tessellation, or even generalized computation. The field of study dubbed GPGPU(General Purpose Computing on Graphics Processing Units) is concerned with utilization of GPUs (often using specialized APIs such as CUDA or OpenCL) to perform general purpose computations such as fluid dynamics, molecular dynamics, cryptography, and so on. With compute shaders, introduced in OpenGL 4.3, we can now do GPGPU within OpenGL. Shader programs are designed for direct execution on the GPU and are executed in parallel. For example, a fragment shader might be executed once for every pixel, with each execution running simultaneously on a separate GPU thread. The number of processors on the graphics card determines how many can be executed at one time. This makes shader programs incredibly efficient, and provides the programmer with a simple API for implementing highly parallel computation. The computing power available in modern graphics cards is impressive. The following table shows the number of shader processors available for several models in the NVIDIA GeForce series cards (source: http://en.wikipedia.org/wiki/Comparison_of_Nvidia_graphics_processing_units). Model Unified Shader Processors GeForce GTS 450 192 GeForce GTX 480 480 GeForce GTX 780 2304 Shader programs are intended to replace parts of the OpenGL architecture referred to as the fixed-function pipeline. Prior to OpenGL Version 2.0, the shading algorithm was "hard-coded" into the pipeline and had only limited configurability. This default lighting/shading algorithm was a core part of the fixed-function pipeline. When we, as programmers, wanted to implement more advanced or realistic effects, we used various tricks to force the fixed-function pipeline into being more flexible than it really was. The advent of GLSL will help by providing us with the ability to replace this "hard-coded" functionality with our own programs written in GLSL, thus giving us a great deal of additional flexibility and power. In fact, recent (core) versions of OpenGL not only provide this capability, but they require shader programs as part of every OpenGL program. The old fixed-function pipeline has been deprecated in favor of a new programmable pipeline, a key part of which is the shader program written in GLSL. Profiles – Core vs. Compatibility OpenGL Version 3.0 introduced a deprecation model, which allowed for the gradual removal of functions from the OpenGL specification. Functions or features can be marked as deprecated, meaning that they are expected to be removed from a future version of OpenGL. For example, immediate mode rendering using glBegin/glEndwas marked deprecated in version 3.0 and removed in version 3.1. In order to maintain backwards compatibility, the concept of compatibility profiles was introduced with OpenGL 3.2. A programmer that is writing code intended to be used with a particular version of OpenGL (with older features removed) would use the so-called core profile. Someone who also wanted to maintain compatibility with older functionality could use the compatibility profile. It may be somewhat confusing that there is also the concept of a forward compatible context, which is distinguished slightly from the concept of a core/compatibility profile. A context that is considered forward compatible basically indicates that all deprecated functionality has been removed. In other words, if a context is forward compatible, it only includes functions that are in the core, but not those that were marked as deprecated. Some window APIs provide the ability to select forward compatible status along with the profile. The steps for selecting a core or compatibility profile are window system API dependent. For example, in GLFW, one can select a forward compatible, 4.3 core profile using the following code: glfwWindowHint( GLFW_CONTEXT_VERSION_MAJOR, 4 ); glfwWindowHint( GLFW_CONTEXT_VERSION_MINOR, 3 ); glfwWindowHint(GLFW_OPENGL_FORWARD_COMPAT, GL_TRUE); glfwWindowHint(GLFW_OPENGL_PROFILE, GLFW_OPENGL_CORE_PROFILE); GLFWwindow *window = glfwCreateWindow(640, 480, "Title", NULL, NULL); All programs in this article are designed to be compatible with a forward compatible OpenGL 4.3 core profile. Using a function loader to access the latest OpenGL functionality The OpenGL ABI (application binary interface) is frozen to OpenGL version 1.1 on Windows. Unfortunately for Windows developers, that means that it is not possible to link directly to functions that are provided in newer versions of OpenGL. Instead, one must get access to these functions by acquiring a function pointer at runtime. Getting access to the function pointers is not difficult, but requires somewhat tedious work, and has a tendency to clutter your code. Additionally, Windows typically comes with a standard OpenGL gl.h file that also conforms to OpenGL 1.1. The OpenGL wiki states that Microsoft has no plans to ever update the gl.hand opengl32.lib that come with their compilers. Thankfully, others have provided libraries that manage all of this for us by transparently providing the needed function pointers, while also exposing the needed functionality in header files. There are several libraries available that provide this kind of support. One of the oldest and most common is GLEW (OpenGL Extension Wrangler). However, there are a few serious issues with GLEW that might make it less desirable, and insufficient for my purposes when writing this article. First, at time of writing, it doesn't yet support core profiles properly, and for this article, I want to focus only on the latest non-deprecated functionality. Second, it provides one large header file that includes everything from all versions of OpenGL. It might be preferable to have a more streamlined header file that only includes functions that we might use. Finally, GLEW is distributed as a library that needs to be compiled separately and linked into our project. It is often preferable to have a loader that can be included into a project simply by adding the source files and compiling them directly into our executable, avoiding the need to support another link-time dependency. In this recipe, we'll use the OpenGL Loader Generator (GLLoadGen), available from https://bitbucket.org/alfonse/glloadgen/wiki/Home. This very flexible and efficient library solves all three of the issues described in the previous paragraph. It supports core profiles and it can generate a header that includes only the needed functionality, and also generates just a couple of files (a source file and a header) that we can add directly into our project. Getting ready To use GLLoadGen, you'll need Lua. Lua is a lightweight embeddable scripting language that is available for nearly all platforms. Binaries are available at http://luabinaries.sourceforge.net, and a fully packaged install for Windows (LuaForWindows) is available at: https://code.google.com/p/luaforwindows Download the GLLoadGen distribution from: https://bitbucket.org/alfonse/glloadgen/downloads. The distribution is compressed using 7zip, which is not widely installed, so you may need to install a 7zip utility, available at http://7-zip.org/. Extract the distribution to a convenient location on your hard drive. Since GLLoadGen is written in Lua, there's nothing to compile, once the distribution is uncompressed, you're ready to go. How to do it... The first step is to generate the header and source files for the OpenGL version and profile of choice. For this example, we'll generate files for an OpenGL 4.3 core profile. We can then copy the files into our project and compile them directly alongside our code: To generate the header and source files, navigate to the GLLoadGen distribution directory, and run GLLoadGen with the following arguments: lua LoadGen.lua -style=pointer_c -spec=gl -version=4.3 -profile=core core_4_3 The previous step should generate two files: gl_core_4_3.c and gl_core_4_3.h. Move these files into your project and include gl_core_4_3.c in your build. Within your program code, you can include the gl_core_4_3.h file whenever you need access to the OpenGL functions. However, in order to initialize the function pointers, you need to make sure to call a function to do so. The needed function is called ogl_LoadFunctions. Somewhere just after the GL context is created (typically in an initialization function), and before any OpenGL functions are called, use the following code: int loaded = ogl_LoadFunctions(); if(loaded == ogl_LOAD_FAILED) { //Destroy the context and abort return; } int num_failed = loaded - ogl_LOAD_SUCCEEDED; printf("Number of functions that failed to load: %i.n", num_failed); That's all there is to it! How it works... The lua command in step 1 generates a pair of files, that is; a header and a source file. The header provides prototypes for all of the selected OpenGL functions and redefines them as function pointers, and defines all of the OpenGL constants as well. The source file provides initialization code for the function pointers as well as some other utility functions. We can include the gl_core_4_3.h header file wherever we need prototypes for OpenGL functions, so all function entry points are available at compile time. At run time, the ogl_LoadFunctions()function will initialize all available function pointers. If some functions fail to load, the number of failures can be determined by the subtraction operation shown in step 2. If a function is not available in the selected OpenGL version, the code may not compile, because only function prototypes for the selected OpenGL version and profile are available in the header (depending on how it was generated). The command line arguments available to GLLoadGen are fully documented here: https://bitbucket.org/alfonse/glloadgen/wiki/Command_Line_Options. The previous example shows the most commonly used setup, but there's a good amount of flexibility built into this tool. Now that we have generated this source/header pair, we no longer have any dependency on GLLoadGen and our program can be compiled without it. This is a significant advantage over tools such as GLEW. There's more... GLLoadGen includes a few additional features that are quite useful. We can generate more C++ friendly code, manage extensions, and generate files that work without the need to call an initialization function. Generating a C++ loader GLLoadGen supports generation of C++ header/source files as well. This can be selected via the -style parameter. For example, to generate C++ files, use -style=pointer_cpp as in the following example: lua LoadGen.lua -style=pointer_cpp -spec=gl -version=4.3 -profile=core core_4_3 This will generate gl_core_4_3.cpp and gl_core_4_3.hpp. This places all OpenGL functions and constants within the gl::namespace, and removes their gl(or GL) prefix. For example, to call the function glBufferData, you might use the following syntax. gl::BufferData(gl::ARRAY_BUFFER, size, data, gl::STATIC_DRAW); Loading the function pointers is also slightly different. The return value is an object rather than just a simple integer and LoadFunctions is in the gl::sys namespace. gl::exts::LoadTest didLoad = gl::sys::LoadFunctions(); if(!didLoad) { // Clean up (destroy the context) and abort. return; } printf("Number of functions that failed to load: %i.n", didLoad.GetNumMissing()); -load styles GLLoadGen supports the automatic initialization of function pointers. This can be selected using the noload_c or noload_cpp options for the style parameter. With these styles, there is no need to call the initialization function ogl_LoadFunctions. The pointers are loaded automatically, the first time a function is called. This can be convenient, but there's very little overhead to loading them all at initialization. Using Extensions GLLoadGen does not automatically support extensions. Instead, you need to ask for them with command line parameters. For example, to request ARB_texture_view and ARB_vertex_attrib_binding extensions, you might use the following command. lua LoadGen.lua -style=pointer_c -spec=gl -version=3.3 -profile=core core_3_3 -exts ARB_texture_view ARB_vertex_attrib_binding The -exts parameter is a space-separated list of extensions. GLLoadGen also provides the ability to load a list of extensions from a file (via the -extfile parameter) and provides some common extension files on the website. You can also use GLLoadGen to check for the existence of an extension at run-time. For details, see the GLLoadGen wiki. See also GLEW, an older, and more common loader and extension manager, available from glew.sourceforge.net. Using GLM for mathematics Mathematics is core to all of computer graphics. In earlier versions, OpenGL provided support for managing coordinate transformations and projections using the standard matrix stacks (GL_MODELVIEW and GL_PROJECTION). In recent versions of core OpenGL however, all of the functionality supporting the matrix stacks has been removed. Therefore, it is up to us to provide our own support for the usual transformation and projection matrices, and then to pass them into our shaders. Of course, we could write our own matrix and vector classes to manage this, but some might prefer to use a ready-made, robust library. One such library is GLM(OpenGL Mathematics) written by Christophe Riccio. Its design is based on the GLSL specification, so the syntax is very similar to the mathematical support in GLSL. For experienced GLSL programmers, this makes GLM very easy to use and familiar. Additionally, it provides extensions that include functionality similar to some of the much missed OpenGL functions such as glOrtho, glRotate, or gluLookAt. Getting ready Since GLM is a header-only library, installation is simple. Download the latest GLM distribution from http://glm.g-truc.net. Then, unzip the archive file, and copy the glm directory contained inside to anywhere in your compiler's include path. How to do it... To use the GLM libraries, it is simply a matter of including the core header file, and headers for any extensions. For this example, we'll include the matrix transform extension as follows: #include <glm/glm.hpp> #include <glm/gtc/matrix_transform.hpp> Then the GLM classes are available in the glm namespace. The following is an example of how you might go about making use of some of them: glm::vec4 position = glm::vec4( 1.0f, 0.0f, 0.0f, 1.0f ); glm::mat4 view = glm::lookAt( glm::vec3(0.0,0.0,5.0), glm::vec3(0.0,0.0,0.0), glm::vec3(0.0,1.0,0.0) ); glm::mat4 model(1.0f); // The identity matrix model = glm::rotate( model, 90.0f, glm::vec3(0.0f,1.0f,0.0) ); glm::mat4 mv = view * model; glm::vec4 transformed = mv * position; How it works... The GLM library is a header-only library. All of the implementation is included within the header files. It doesn't require separate compilation and you don't need to link your program to it. Just placing the header files in your include path is all that's required! The previous example first creates a vec4(four coordinate vector) representing a position. Then it creates a 4 x 4 view matrix by using the glm::lookAt function. This works in a similar fashion to the old gluLookAt function. Here, we set the camera's location at (0, 0, 5), looking towards the origin, with the "up" direction in the direction of the y-axis. We then go on to create the model matrix by first storing the identity matrix in the variable model(via the single argument constructor), and multiplying by a rotation matrix using the glm::rotate function. The multiplication here is implicitly done by the glm::rotate function. It multiplies its first parameter by the rotation matrix (on the right) that is generated by the function. The second parameter is the angle of rotation (in degrees), and the third parameter is the axis of rotation. Since before this statement, model is the identity matrix, the net result is that model becomes a rotation matrix of 90 degrees around the y-axis. Finally, we create our modelview matrix (mv) by multiplying the view and model variables, and then using the combined matrix to transform the position. Note that the multiplication operator has been overloaded to behave in the expected way. There's more... It is not recommended to import all of the GLM namespace by using the following command: using namespace glm; This will most likely cause a number of namespace clashes. Instead, it is preferable to import symbols one at a time, as needed. For example: #include <glm/glm.hpp> using glm::vec3; using glm::mat4; Using the GLM types as input to OpenGL GLM supports directly passing a GLM type to OpenGL using one of the OpenGL vector functions (with the suffix v). For example, to pass a mat4 named proj to OpenGL we can use the following code: glm::mat4 proj = glm::perspective( viewAngle, aspect, nearDist, farDist ); glUniformMatrix4fv(location, 1, GL_FALSE, &proj[0][0]); See also The Qt SDK includes many classes for vector/matrix mathematics, and is another good option if you're already using Qt The GLM website http://glm.g-truc.net has additional documentation and examples

(For more resources related to this topic, see here.) The latest version of the example code for this article can be found at http://github.com/CamelCookbook/camel-cookbook-examples. You can also download the example code files for all Packt books you have purchased from your account at https://www.packtpub.com. If you purchased this book elsewhere, you can visit https://www.packtpub.com/books/content/support and register to have the files e-mailed directly to you. In this article we will explore a number of ways in which Camel performs message content transformation: Let us first look at some important concepts regarding transformation of messages in Camel: Using the transform statement. This allows you to reference Camel Expression Language code within the route to do message transformations. Calling a templating component, such as Camel's XSLT or Velocity template style components. This will typically reference an external template resource that is used in transforming your message. Calling a Java method (for example, beanref), defined by you, within a Camel route to perform the transformation. This is a special case processor that can invoke any referenced Java object method. Camel's Type Converter capability that can automatically cast data from one type to another transparently within your Camel route. This capability is extensible, so you can add your own Type Converters. Camel's Data Format capability that allows us to use built-in, or add our own, higher order message format converters. A Camel Data Format goes beyond simple data type converters, which handle simple data type translations such as String to int, or File to String. Data Formats are used to translate between a low-level representation (XML) and a high-level one (Java objects). Other examples include encrypting/decrypting data, and compressing/decompressing data. For more, see http://camel.apache.org/data-format.html. A number of Camel architectural concepts are used throughout this article. Full details can be found at the Apache Camel website at http://camel.apache.org. The code for this article is contained within the camel-cookbook-transformation module of the examples. Transforming using a Simple Expression When you want to transform a message in a relatively straightforward way, you use Camel's transform statement along with one of the Expression Languages provided by the framework. For example, Camel's Simple Expression Language provides you with a quick, inline mechanism for straightforward transformations. This recipe will show you how to use Camel's Simple Expression Language to transform the message body. Getting ready The Java code for this recipe is located in the org.camelcookbook.transformation.simple package. Spring XML files are located under src/main/resources/META-INF/spring and are prefixed with simple. How to do it... In a Camel route, use a transform DSL statement containing the Expression Language code to do your transformation. In the XML DSL, this is written as follows: <route> <from uri="direct:start"/> <transform> <simple>Hello ${body}</simple> </transform> </route> In the Java DSL, the same route is expressed as: from("direct:start") .transform(simple("Hello ${body}")); In this example, the message transformation prefixes the incoming message with the phrase Hello using the Simple Expression Language. The processing step after the transform statement will see the transformed message content in the body of the exchange. How it works... Camel's Simple Expression Language is quite good at manipulating the String content through its access to all aspects of the message being processed, through its rich String and logical operators. The result of your Simple Expression becomes the new message body after the transform step. This includes predicates such as using Simple's logical operators, to evaluate a true or false condition; the results of that Boolean operation become the new message body containing a String: "true" or "false". The advantage of using a distinct transform step within a route, as opposed to embedding it within a processor, is that the logic is clearly visible to other programmers. Ensure that the expression embedded within your route is kept simple so as to not distract the next developer from the overall purpose of the integration. It is best to move more complex (or just lengthy) transformation logic into its own subroute, and invoke it using direct: or seda:. There's more... The transform statement will work with any Expression Language available in Camel, so if you need more powerful message processing capabilities you can leverage scripting languages such as Groovy or JavaScript (among many others) as well. The Transforming inline with XQuery recipe will show you how to use the XQuery Expression Language to do transformations on XML messages. See also Message Translator: http://camel.apache.org/message-translator.html Camel Expression capabilities: http://camel.apache.org/expression.html Camel Simple Expression Language: http://camel.apache.org/simple.html Languages supported by Camel: http://camel.apache.org/languages.html The Transforming inline with XQuery recipe   Transforming inline with XQuery Camel supports the use of Camel's XQuery Expression Language along with the transform statement as a quick and easy way to transform an XML message within a route. This recipe will show you how to use an XQuery Expression to do in-route XML transformation. Getting ready The Java code for this recipe is located in the org.camelcookbook.transformation.xquery package. Spring XML files are located under src/main/resources/META-INF/spring and prefixed with xquery. To use the XQuery Expression Language, you need to add a dependency element for the camel-saxon library, which provides the implementation for the XQuery Expression Language. Add the following to the dependencies section of your Maven POM: <dependency> <groupId>org.apache.camel</groupId> <artifactId>camel-saxon</artifactId> <version>${camel-version}</version> </dependency>   How to do it... In the Camel route, specify a transform statement followed by the XQuery Expression Language code to do your transformation. In the XML DSL, this is written as: <route> <from uri="direct:start"/> <transform> <xquery> <books>{ for $x in /bookstore/book where $x/price>30 order by $x/title return $x/title }</books> </xquery> </transform> </route> When using the XML DSL, remember to XML encode the XQuery embedded XML elements. Therefore, < becomes &lt; and > becomes &gt;. In the Java DSL, the same route is expressed as: from("direct:start") .transform(xquery("<books>{ for $x in /bookstore/book " + "where $x/price>30 order by $x/title " + "return $x/title }</books>")); Feed the following input XML message through the transformation: <bookstore> <book category="COOKING"> <title lang="en">Everyday Italian</title> <author>Giada De Laurentiis</author> <year>2005</year> <price>30.00</price> </book> <book category="CHILDREN"> <title lang="en">Harry Potter</title> <author>J K. Rowling</author> <year>2005</year> <price>29.99</price> </book> <book category="PROGRAMMING"> <title lang="en">Apache Camel Developer's Cookbook</title> <author>Scott Cranton</author> <author>Jakub Korab</author> <year>2013</year> <price>49.99</price> </book> <book category="WEB"> <title lang="en">Learning XML</title> <author>Erik T. Ray</author> <year>2003</year> <price>39.95</price> </book> </bookstore> The resulting message will be: <books> <title lang="en">Apache Camel Developer's Cookbook</title> <title lang="en">Learning XML</title> </books> The processing step after the transform statement will see the transformed message content in the body of the exchange. How it works... Camel's XQuery Expression Language is a good way to inline XML transformation code within your route. The result of the XQuery Expression becomes the new message body after the transform step. All of the message's body, headers, and properties are made available to the XQuery Processor, so you can reference them directly within your XQuery statement. This provides you with a powerful mechanism for transforming XML messages. If you are more comfortable with XSLT, take a look at the Transforming with XSLT recipe. In-lining the transformation within your integration route can sometimes be an advantage as you can clearly see what is being changed. However, when the transformation expression becomes so complex that it starts to overwhelm the integration route, you may want to consider moving the transformation expression outside of the route. See the Transforming using a Simple Expression recipe for another inline transformation example, and see the Transforming with XSLT recipe for an example of externalizing your transformation. You can fetch the XQuery Expression from an external file using Camel's resource reference syntax. To reference an XQuery file on the classpath you can specify: <transform> <xquery>resource:classpath:/path/to/myxquery.xml</xquery> </transform> This is equivalent to using XQuery as an endpoint: <to uri="xquery:classpath:/path/to/myxquery.xml"/>   There's more... The XQuery Expression Language allows you to pass in headers associated with the message. These will show up as XQuery variables that can be referenced within your XQuery statements. Consider, from the previous example, to allow the value of the books that are filtered to be passed in with the message body, that is, parameterize the XQuery, you can modify the XQuery statement as follows: <transform> <xquery> declare variable $in.headers.myParamValue as xs:integer external; <books value='{$in.headers.myParamValue}'&gt;{ for $x in /bookstore/book where $x/price>$in.headers.myParamValue order by $x/title return $x/title }&lt;/books&gt; </xquery> </transform> Message headers will be associated with an XQuery variable called in.headers.<name of header>. To use this in your XQuery, you need to explicitly declare an external variable of the same name and XML Schema (xs:) type as the value of the message header. The transform statement will work with any Expression Language enabled within Camel, so if you need more powerful message processing capabilities you can leverage scripting languages such as Groovy or JavaScript (among many others) as well. The Transforming using a Simple Expression recipe will show you how to use the Simple Expression Language to do transformations on String messages. See also Message Translator: http://camel.apache.org/message-translator.html Camel Expression capabilities: http://camel.apache.org/expression.html Camel XQuery Expression Language: http://camel.apache.org/xquery.html XQuery language: http://www.w3.org/XML/Query/ Languages supported by Camel: http://camel.apache.org/languages.html The Transforming with XSLT recipe The Transforming using a Simple Expression recipe   Transforming with XSLT When you want to transform an XML message using XSLT, use Camel's XSLT Component. This is similar to the Transforming inline with XQuery recipe except that there is no XSLT Expression Language, so it can only be used as an endpoint. This recipe will show you how to transform a message using an external XSLT resource. Getting ready The Java code for this recipe is located in the org.camelcookbook.transformation.xslt package. Spring XML files are located under src/main/resources/META-INF/spring and prefixed with xslt. How to do it... In a Camel route, add the xslt processor step into the route at the point where you want the XSLT transformation to occur. The XSLT file must be referenced as an external resource, and depending on where the file is located, prefixed with either classpath:(default if not using a prefix), file:, or http:. In the XML DSL, this is written as: <route> <from uri="direct:start"/> <to uri="xslt:book.xslt"/> </route> In the Java DSL, the same route is expressed as: from("direct:start") .to("xslt:book.xslt"); The next processing step in the route will see the transformed message content in the body of the exchange. How it works... The following example shows how the preceding steps will process an XML file. Consider the following input XML message: <bookstore> <book category="COOKING"> <title lang="en">Everyday Italian</title> <author>Giada De Laurentiis</author> <year>2005</year> <price>30.00</price> </book> <book category="CHILDREN"> <title lang="en">Harry Potter</title> <author>J K. Rowling</author> <year>2005</year> <price>29.99</price> </book> <book category="PROGRAMMING"> <title lang="en">Apache Camel Developer's Cookbook</title> <author>Scott Cranton</author> <author>Jakub Korab</author> <year>2013</year> <price>49.99</price> </book> <book category="WEB"> <title lang="en">Learning XML</title> <author>Erik T. Ray</author> <year>2003</year> <price>39.95</price> </book> </bookstore> Process this with the following XSLT contained in books.xslt: <?xml version="1.0" encoding="UTF-8"?> <xsl:stylesheet version="1.0" > <xsl:output omit-xml-declaration="yes"/> <xsl:template match="/"> <books> <xsl:apply-templates select="/bookstore/book/title[../price>30]"> <xsl:sort select="."/> </xsl:apply-templates> </books> </xsl:template> <xsl:template match="node()|@*"> <xsl:copy> <xsl:apply-templates select="node()|@*"/> </xsl:copy> </xsl:template> </xsl:stylesheet> The result will appear as follows: <books> <title lang="en">Apache Camel Developer's Cookbook</title> <title lang="en">Learning XML</title> </books> The Camel XSLT Processor internally runs the message body through a registered Java XML transformer using the XSLT file referenced by the endpoint. This processor uses Camel's Type Converter capabilities to convert the input message body type to one of the supported XML source models in the following order of priority: StAXSource (off by default; this can be enabled by setting allowStAX=true on the endpoint URI) SAXSource StreamSource DOMSource Camel's Type Converter can convert from most input types (String, File, byte[], and so on) to one of the XML source types for most XML content loaded through other Camel endpoints with no extra work on your part. The output data type for the message is, by default, a String, and is configurable using the output parameter on the xslt endpoint URI. There's more... The XSLT Processor passes in headers, properties, and parameters associated with the message. These will show up as XSLT parameters that can be referenced within your XSLT statements. You can pass in the names of the books as parameters to the XSLT template; to do so, modify the previous XLST as follows: <xsl:param name="myParamValue"/> <xsl:template match="/"> <books> <xsl:attribute name="value"> <xsl:value-of select="$myParamValue"/> </xsl:attribute> <xsl:apply-templates select="/bookstore/book/title[../price>$myParamValue]"> <xsl:sort select="."/> </xsl:apply-templates> </books> </xsl:template> The Exchange instance will be associated with a parameter called exchange; the IN message with a parameter called in; and the message headers, properties, and parameters will be associated XSLT parameters with the same name. To use these in your XSLT, you need to explicitly declare a parameter of the same name in your XSLT file. In the previous example, it is possible to use either a message header or exchange property called myParamValue. See also Message Translator: http://camel.apache.org/message-translator.html Camel XSLT Component: http://camel.apache.org/xslt Camel Type Converter: http://camel.apache.org/type-converter.html XSL working group: http://www.w3.org/Style/XSL/ The Transforming inline with XQuery recipe   Transforming from Java to XML with JAXB Camel's JAXB Component is one of a number of components that can be used to convert your XML data back and forth from Java objects. It provides a Camel Data Format that allows you to use JAXB annotated Java classes, and then marshal (Java to XML) or unmarshal (XML to Java) your data. JAXB is a Java standard for translating between XML data and Java that is used by creating annotated Java classes that bind, or map, to your XML data schema. The framework takes care of the rest. This recipe will show you how to use the JAXB Camel Data Format to convert back and forth from Java to XML. Getting ready The Java code for this recipe is located in the org.camelcookbook.transformation.jaxb package. The Spring XML files are located under src/main/resources/META-INF/spring and prefixed with jaxb. To use Camel's JAXB Component, you need to add a dependency element for the camel-jaxb library, which provides the implementation for the JAXB Data Format. Add the following to the dependencies section of your Maven POM: <dependency> lt;groupId>org.apache.camel</groupId> <artifactId>camel-jaxb</artifactId> <version>${camel-version}</version> lt;/dependency>   How to do it... The main steps for converting between Java and XML are as follows: Given a JAXB annotated model, reference that model within a named Camel Data Format. Use that named Data Format within your Camel route using the marshal and unmarshal DSL statements. Create an annotated Java model using standard JAXB annotations. There are a number of external tools that can automate this creation from existing XML or XSD (XML Schema) files: @XmlAccessorType(XmlAccessType.FIELD) @XmlType(name = "", propOrder = { "title", "author", "year", "price" } ) @XmlRootElement(name = "book") public class Book { @XmlElement(required = true) protected Book.Title title; @XmlElement(required = true) protected List<String> author; protected int year; protected double price; // getters and setters } Instantiate a JAXB Data Format within your Camel route that refers to the Java package(s) containing your JAXB annotated classes. In the XML DSL, this is written as: <camelContext > <dataFormats> <jaxb id="myJaxb" contextPath="org.camelcookbook .transformation.myschema"/> </dataFormats> <!-- route definitions here --> </camelContext> In the Java DSL, the Data Format is defined as: public class JaxbRouteBuilder extends RouteBuilder { @Override public void configure() throws Exception { DataFormat myJaxb= new JaxbDataFormat( "org.camelcookbook.transformation.myschema"); // route definitions here } } Reference the Data Format within your route, choosing marshal(Java to XML) or unmarshal(XML to Java) as appropriate. In the XML DSL, this routing logic is written as: <route> <from uri="direct:unmarshal"/> <unmarshal ref="myJaxb"/> </route> In the Java DSL, this is expressed as: from("direct:unmarshal").unmarshal(myJaxb);   How it works... Using Camel JAXB to translate your XML data back and forth to Java makes it much easier for the Java processors defined later on in your route to do custom message processing. This is useful when the built-in XML translators (for example, XSLT or XQuery) are not enough, or you just want to call existing Java code. Camel JAXB eliminates the boilerplate code from your integration flows by providing a wrapper around the standard JAXB mechanisms for instantiating the Java binding for the XML data. There's more... Camel JAXB works just fine with existing JAXB tooling like the maven-jaxb2-plugin plugin, which can automatically create JAXB-annotated Java classes from an XML Schema (XSD). See also Camel JAXB: http://camel.apache.org/jaxb.html Available Data Formats: http://camel.apache.org/data-format.html JAXB Specification: http://jcp.org/en/jsr/detail?id=222   Transforming from Java to JSON Camel's JSON Component is used when you need to convert your JSON data back and forth from Java. It provides a Camel Data Format that, without any requirement for an annotated Java class, allows you to marshal (Java to JSON) or unmarshal (JSON to Java) your data. There is only one step to using Camel JSON to marshal and unmarshal XML data. Within your Camel route, insert the marshal(Java to JSON), or unmarshal(JSON to Java) statement, and configure it to use the JSON Data Format. This recipe will show you how to use the camel-xstream library to convert from Java to JSON, and back. Getting ready The Java code for this recipe is located in the org.camelcookbook.transformation.json package. The Spring XML files are located under src/main/resources/META-INF/spring and prefixed with json. To use Camel's JSON Component, you need to add a dependency element for the camel-xstream library, which provides an implementation for the JSON Data Format using the XStream library. Add the following to the dependencies section of your Maven POM: <dependency> <groupId>org.apache.camel</groupId> <artifactId>camel-xstream</artifactId> <version>${camel-version}</version> </dependency>   How to do it... Reference the Data Format within your route, choosing the marshal (Java to JSON), or unmarshal (JSON to Java) statement, as appropriate: In the XML DSL, this is written as follows: <route> <from uri="direct:marshal"/> <marshal> <json/> </marshal> <to uri="mock:marshalResult"/> </route> In the Java DSL, this same route is expressed as: from("direct:marshal") .marshal().json() .to("mock:marshalResult");   How it works... Using Camel JSON simplifies translating your data between JSON and Java. This is convenient when you are dealing with REST endpoints and need Java processors in Camel to do custom message processing later on in the route. Camel JSON provides a wrapper around the JSON libraries for instantiating the Java binding for the JSON data, eliminating more boilerplate code from your integration flows. There's more... Camel JSON works with the XStream library by default, and can be configured to use other JSON libraries, such as Jackson or GSon. These other libraries provide additional features, more customization, and more flexibility that can be leveraged by Camel. To use them, include their respective Camel components, for example, camel-jackson, and specify the library within the json element: <dataFormats> <json id="myJson" library="Jackson"/> </dataFormats>   See also Camel JSON: http://camel.apache.org/json.html Available Data Formats: http://camel.apache.org/data-format.html   Transforming from XML to JSON Camel provides an XML JSON Component that converts your data back and forth between XML and JSON in a single step, without an intermediate Java object representation. It provides a Camel Data Format that allows you to marshal (XML to JSON), or unmarshal (JSON to XML) your data. This recipe will show you how to use the XML JSON Component to convert from XML to JSON, and back. Getting ready Java code for this recipe is located in the org.camelcookbook.transformation.xmljson package. Spring XML files are located under src/main/resources/META-INF/spring and prefixed with xmljson. To use Camel's XML JSON Component, you need to add a dependency element for the camel-xmljson library, which provides an implementation for the XML JSON Data Format. Add the following to the dependencies section of your Maven POM: <dependency> <groupId>org.apache.camel</groupId> <artifactId>camel-xmljson</artifactId> <version>${camel-version}</version> </dependency>   How to do it... Reference the xmljson Data Format within your route, choosing the marshal(XML to JSON), or unmarshal(JSON to XML) statement, as appropriate: In the XML DSL, this is written as follows: <route> <from uri="direct:marshal"/> <marshal> <xmljson/> </marshal> <to uri="mock:marshalResult"/> </route> In the Java DSL, this same route is expressed as: from("direct:marshal") .marshal().xmljson() .to("mock:marshalResult");   How it works... Using the Camel XML JSON Component simplifies translating your data between XML and JSON, making it convenient to use when you are dealing with REST endpoints. The XML JSON Data Format wraps around the Json-lib library, which provides the core translation capabilities, eliminating more boilerplate code from your integration flows. There's more... You may need to configure XML JSON if you want to fine-tune the output of your transformation. For example, consider the following JSON: [{"@category":"PROGRAMMING","title":{"@lang":"en","#text": "Apache Camel Developer's Cookbook"},"author":[ "Scott Cranton","Jakub Korab"],"year":"2013","price":"49.99"}] This will be converted as follows, by default, which may not be exactly what you want (notice the <a> and <e> elements): <?xml version="1.0" encoding="UTF-8"?> <a> <e category="PROGRAMMING"> <author> <e>Scott Cranton</e> <e>Jakub Korab</e> </author> <price>49.99</price> <title lang="en">Apache Camel Developer's Cookbook</title> <year>2013</year> </e> </a> To configure XML JSON to use <bookstore> as the root element instead of <a>, use <book> for the individual elements instead of <e>, and expand the multiple author values to use a sequence of <author> elements, you would need to tune the configuration of the Data Format before referencing it in your route. In the XML DSL, the definition of the Data Format and the route that uses it is written as follows: <dataFormats> <xmljson id="myXmlJson" rootName="bookstore" elementName="book" expandableProperties="author author"/> </dataFormats> <route> <from uri="direct:unmarshalBookstore"/> <unmarshal ref="myXmlJson"/> <to uri="mock:unmarshalResult"/> </route> In the Java DSL, the same thing is expressed as: XmlJsonDataFormat xmlJsonFormat = new XmlJsonDataFormat(); xmlJsonFormat.setRootName("bookstore"); xmlJsonFormat.setElementName("book"); xmlJsonFormat.setExpandableProperties( Arrays.asList("author", "author")); from("direct:unmarshalBookstore") .unmarshal(xmlJsonFormat) .to("mock:unmarshalBookstoreResult"); This will result in the previous JSON being unmarshalled as follows: <?xml version="1.0" encoding="UTF-8"?> <bookstore> <book category="PROGRAMMING"> <author>Scott Cranton</author> <author>Jakub Korab</author><price>49.99</price> <title lang="en">Apache Camel Developer's Cookbook</title> <year>2013</year> </book> </bookstore>   See also Camel XML JSON: http://camel.apache.org/xmljson.html Available Data Formats: http://camel.apache.org/data-format.html Json-lib: http://json-lib.sourceforge.net   Summary We saw how Apache Camel is flexible in allowing us to transform and convert messages in various formats. This makes Apache Camel an ideal choice for integrating different systems together. Resources for Article: Further resources on this subject: Drools Integration Modules: Spring Framework and Apache Camel [Article] Installing Apache Karaf [Article] Working with AMQP [Article]

(For more resources related to this topic, see here.) Understanding the SAP HANA architecture Architecture is the key for SAP HANA to be a game changing innovative technology. SAP HANA has been designed so well architecture-wise such that it makes a lot of difference when compared to other traditional databases available today. This section explains us the various components of SAP HANA and its functionalities. Getting ready Enterprise application requirements have become more demanding—complex reports with high computation on huge volumes of transaction data and also business data of other formats (both structured and semi-structured). Data is being written or updated, and also read from the database in parallel. Thus, integration of both transactional and analytical data into single database is required, where SAP HANA has evolved. Columnar storage exploits modern hardware and technology (multiple CPU cores, large main memory, and caches) in achieving the requirements of enterprise applications. Apart from this, it should also support procedural logic where certain tasks cannot be completed with simple SQL. How it works… The SAP HANA database consists of several services (servers). Index server is the most important component of all the servers. Other servers are name server, preprocessor server, statistics server, and XS Engine: Index server: This server holds the actual data and the engines for processing the data. When SQL or MDX is fired against the SAP HANA system in the case of authenticated sessions and transactions, an index server takes care of these commands and processes them. Name server: This server holds complete information about the system landscape. Name server is responsible for the topology of the SAP HANA system. In a distributed system, SAP HANA instances will be running on multiple hosts. In this kind of setup, the name server knows where the components are running and how data is spread on different servers. Preprocessor server: This server comes into the picture during text data analysis. Index server utilizes the capabilities of preprocessor server in text data analysis and searching. This helps to extract the information on which text search capabilities are based. Statistics server: This server helps in collecting the data for the system monitor and helps you know the health of the SAP HANA system. The statistics server is responsible for collecting the data related to status, resource allocation/consumption and performance of the SAP HANA system. Monitoring the clients and getting the status of various alert monitors use the data collected by Statistics server. This server also provides a history of measurement data for further analysis. XS Engine: The XS Engine allows external applications and application developers to access the SAP HANA system via the XS Engine clients, for example, a web browser accesses SAP HANA apps built by application developers via HTTP. Application developers build applications by using the XS Engine, and the users access the app via HTTP by using a web browser. The persistent model in the SAP HANA database is converted into a consumption model for clients to access it via HTTP. This allows an organization to host system services that are a part of the SAP HANA database (for example, Search service, a built-in web server that provides access to static content in the repository). The following diagram shows the architecture of SAP HANA: There's is more... Let us continue learning about the different components: SAP Host Agent: According to the new approach from SAP, the SAP Host Agent should be installed on all machines that are related to the SAP landscape. It is used by Adaptive Computing Controller (ACC) to manage the system and Software Update Manager (SUM) for automatic updates. LM-structure: LM-structure for SAP HANA contains the information about current installation details. This information will be used by SUM during automatic updates. SAP Solution Manager diagnostic agent: This agent provides all the data to SAP Solution Manager (SAP SOLMAN) to monitor the SAP HANA system. After the SAP SOLMAN is integrated with the SAP HANA system, this agent provides information about the database at a glance, which includes the database state and general information about the system, such as alerts, CPU, or memory and disk usage. SAP HANA Studio repository: This helps the end users to update the SAP HANA studio to higher versions. The SAP HANA Studio repository is the code that does this process. Software Update Manager for SAP HANA: This helps in automatic updates of SAP HANA from the SAP Marketplace and patching the SAP host agent. It also allows distribution of the Studio repository to the end users. See also http://help.sap.com/hana/SAP_HANA_Installation_Guide_en.pdf SAP Notes:1793303 and 1514967 Explaining IMCE and its components We have seen the architecture of SAP HANA and its components. In this section, we will learn about IMCE (in-memory computing engine) and how its components and its functionalities. Getting Ready The SAP in-memory computing engine (formerly Business Analytic Engine (BAE)) is the core engine for SAP's next generation high-performance, in-memory solutions as it leverages technologies such as in-memory computing, columnar databases, massively parallel processing (MPP), and data compression, to allow organizations to instantly explore and analyze large volumes of transactional and analytical data from across the enterprise in real time. How it works... In-memory computing allows the processing of massive quantities of real-time data in the main memory of the server, providing immediate results from analyses and transactions. The SAP in-memory computing database delivers the following capabilities: In-memory computing functionality with native support for row and columnar datastores providing full ACID (atomicity, consistency, isolation, and durability) transactional capabilities Integrated lifecycle management capabilities and data integration capabilities to access SAP and non-SAP data sources SAP IMCE Studio, which includes tools for data modeling, data and life cycle management, and data security The SAP IMCE that resides at the heart of SAP HANA is an integrated database and calculation layer that allows the processing of massive quantities of real-time data in the main memory to provide immediate results from analysis and transactions. Like any standard database, the SAP IMCE not only supports industry standards such as SQL and MDX, but also incorporates a high-performance calculation engine that embeds procedural language support directly into the database kernel. This approach is designed to remove the need to read data from the database, process it, and then write data back to the database, that is, process the data near the database and return the results. The IMCE is an in-memory, column-oriented database technology. It is a powerful calculation engine at the heart of SAP HANA. As data resides in the Random Access Memory (RAM), highly accelerated performance can be achieved compared to systems that read data from disks. The heart lies within the IMCE, which allows us to create and perform calculations on data. SAP IMCE Studio includes tools for data modeling activities, data and life cycle management, and also tools that are related to data security. The following diagram shows the components of IMCE alone: There's more… SAP HANA database has the following two engines: Column-based store: This engine stores the huge amounts of relational data in column-optimized tables, which are aggregated and used in analytical operations. Row-based store: This engine stores the relational data in rows, similar to the storage mechanism of traditional database systems. The row store is more optimized for write operations and has a lower compression rate. Also, the query performance is lower when compared to the column-based store. The engine that is used to store data can be selected on a per-table basis at the time of creating a table. Tables in the row-based store are loaded at start up time. In the case of column-based stores, tables can be either loaded at start up or on demand, that is, during normal operation of the SAP HANA database. Both engines share a common persistence layer, which provides data persistency that is consistent across both engines. Like a traditional database, we have page management and logging in SAP HANA. The changes made to the in-memory database pages are persisted through savepoints. These savepoints are written to those data volumes on the persistent storage for which the storage medium is hard drives. All transactions committed in the SAP HANA database are stored/saved/referenced by the logger of the persistency layer in a log entry written to the log volumes on the persistent storage. To get high I/O performance and low latency, log volumes use the flash technology storage. The relational engines can be accessed through a variety of interfaces. The SAP HANA database supports SQL (JDBC/ODBC), MDX (ODBO), and BICS (SQLDBC). The calculation engine performs all the calculations in the database. No data moves into the application layer until calculations are completed. It also contains the business functions library that is called by applications to perform calculations based on the business rules and logic. The SAP HANA-specific SQL script language is an extension of SQL that can be used to push down data-intensive application logic into the SAP HANA database for specific requirements. Session management This component creates and manages sessions and connections for the database clients. When a session is created, a set of parameters are maintained. These parameters are like auto-commit settings or the current transaction isolation level. After establishing a session, database clients communicate with the SAP HANA database using SQL statements. SAP HANA database treats all the statements as transactions while processing them. Each new session created will be assigned to a new transaction. Transaction manager The transaction manager is the component that coordinates database transactions, takes care of controlling transaction isolation, and keeps track of running and closed transactions. The transaction manager informs the involved storage engines about the running or closed transactions, so that they can execute necessary actions, when a transaction is committed or rolled back. The transaction manager cooperates with the persistence layer to achieve atomic and durable transactions. The client requests are analyzed and executed by a set of components summarized as request processing and execution control. The client requests are analyzed by a request parser, and then it is dispatched to the responsible component. The transaction control statements are forwarded to the transaction manager. The data definition statements are sent to the metadata manager. The object invocations are dispatched to the object store. The data manipulation statements are sent to the optimizer, which creates an optimized execution plan that is given to the execution layer. The SAP HANA database also has built-in support for domain-specific models (such as for financial planning domain) and it offers scripting capabilities that allow application-specific calculations to run inside the database. It has its own scripting language named SQLScript that is designed to enable optimizations and parallelization. This SQLScript is based on free functions that operate on tables by using SQL queries for set processing. The SAP HANA database also contains a component called the planning engine that allows financial planning applications to execute basic planning operations in the database layer. For example, while applying filters/transformations, a new version of a dataset will be created as a copy of an existing one. An example of planning operation is disaggregation operation in which based on a distribution function; target values from higher to lower aggregation levels are distributed. Metadata manager Metadata manager helps to access metadata. SAP HANA database's metadata consists of a variety of objects, such as definitions of tables, views and indexes, SQLScript function definitions, and object store metadata. All these types of metadata are stored in one common catalog for all the SAP HANA database stores. Metadata is stored in tables in the row store. The SAP HANA features such as transaction support and multi-version concurrency control (MVCC) are also used for metadata management. Central metadata is shared across the servers in the case of a distributed database systems. The background mechanism of metadata storage and sharing is hidden from the components that use the metadata manager. As row-based tables and columnar tables can be combined in one SQL statement, both the row and column engines must be able to consume the intermediate results. The main difference between the two engines is the way they process data: the row store operators process data in a row-at-a-time fashion, whereas column store operations (such as scan and aggregate) require the entire column to be available in contiguous memory locations. To exchange intermediate results created by each other, the row store provides results to the column store. The result materializes as complete rows in the memory, while the column store can expose results using the iterators interface needed by the row store. Persistence layer The persistence layer is responsible for durability and atomicity of transactions. The persistent layer ensures that the database is restored to the most recent committed state after a restart, and makes sure that transactions are either completely executed or completely rolled back. To achieve this in an efficient way, the persistence layer uses a combination of write-ahead logs, shadow paging, and savepoints. Moreover, the persistence layer also offers interfaces for writing and reading data. It also contains SAP HANA's logger that manages the transaction log. Authorization manager The authorization manager is invoked by other SAP HANA database components to check the required privileges for users to execute the requested operations. Privileges to other users or roles can be granted. A privilege grants the right to perform a specified operation (such as create, update, select, and execute data manipulation languages) on a specified object such as a table, view, and SQLScript function. Analytic privileges represent filters or hierarchy, and they drill down limitations for analytic queries. Analytic privileges such as granting access to values with a certain combination of dimension attributes are supported in SAP HANA. Users are authenticated either by the SAP HANA database itself (log in with username and password), or authentication can be delegated to external authentication providers third-party such as an LDAP directory. See also SAP HANA in-memory analytics and in-memory computing available at http://scn.sap.com/people/vitaliy.rudnytskiy/blog/2011/03/22/time-to-update-your-sap-hana-vocabulary Summary This article explains the SAP architecture and the IMCE feature in brief. Resources for Article: Further resources on this subject: SAP HANA integration with Microsoft Excel [Article] Data Migration Scenarios in SAP Business ONE Application- part 2 [Article] Data Migration Scenarios in SAP Business ONE Application- part 1 [Article]

Our Data Engineering Byte Newsletter gives data engineers and practitioners what they often lack today: clear, real-world insights—where every byte tells a story.Subscribe here to stay ahead in data engineeringIf your environment has only one or a handful of SSAS instances, they can be managed by the same database administrators managing SQL Server and other database platforms. In large enterprises, there could be hundreds of SSAS instances managed by dedicated SSAS administrators. Regardless of the environment, you should become familiar with the configuration options as well as troubleshooting methodologies. In large enterprises, you might also be required to automate these tasks using the Analysis Management Objects (AMO) code. Analysis Services is a great tool for building business intelligence solutions. However, much like any other software, it does have its fair share of challenges and limitations. Most frequently encountered enterprise business intelligence system goals include quick provision of relevant data to the business users and assuring excellent query performance. If your cubes serve a large, global community of users, you will quickly learn that SSAS is optimized to run a single query as fast as possible. Once users send a multitude of heavy queries in parallel, you can expect to see memory, CPU, and disk-related performance counters to quickly rise, with a corresponding increase in query execution duration which, in turn, worsens user experience. Although you could build aggregations to improve query performance, doing so will lengthen cube processing time, and thereby, delay the delivery of essential data to decision makers. It might also be tempting to consider using ROLAP storage mode in lieu of MOLAP so that processing times are shorter, but MOLAP queries usually outperform ROLAP due to heavy compression rates. Hence, figuring out the right storage mode and appropriate level of aggregations is a great balancing act. If you cannot afford using ROLAP, and query performance is paramount to successful cube implementation, you should consider scaling your solution. You have two options for scaling, given as follows:Scaling up: This option means purchasing servers with more memory, more CPU cores, and faster disk drivesScaling out: This option means purchasing several servers of approximately the same capacity and distributing the querying workload across multiple servers using a load balancing toolSSAS lends itself best to the second option—scaling out. Later in this article you will learn how to separate processing and querying activities and how to ensure that all servers in the querying pool have the same data.SSAS instance configuration optionsAll Analysis Services configuration options are available in the msmdsrv.ini file found in the config folder under the SSAS installation directory. Instance administrators can also modify some, but not all configuration properties, using SQL Server Management Studio (SSMS). SSAS has a multitude of properties that are undocumented—this normally means that such properties haven't undergone thorough testing, even by the software's developers. Hence, if you don't know exactly what the configuration setting does, it's best to leave the setting at default value. Even if you want to test various properties on a sandbox server, make a copy of the configuration file prior to applying any changes.How to do it...To modify the SSAS instance settings using the configuration file, perform the following steps:Navigate to the config folder within your Analysis Services installation directory. By default, this will be C:\Program Files\Microsoft SQL Server\MSAS11.instance_name\OLAP\Config.Open the msmdsrv.ini file using Notepad or another text editor of your choice. The file is in the XML format, so every property is enclosed in opening and closing tags.Search for the property of interest, modify its value as desired, and save the changes.For example, in order to change the upper limit of the processing worker threads, you would look for the <ThreadPool><Process><MaxThreads> tag sequence and set the values as shown in the following excerpt from the configuration file:<Process>       <MinThreads>0</MinThreads>       <MaxThreads>250</MaxThreads>      <PriorityRatio>2</PriorityRatio>       <Concurrency>2</Concurrency>       <StackSizeKB>0</StackSizeKB>       <GroupAffinity/>     </Process>  To change the configuration using SSMS, perform the following steps:Connect to the SSAS instance using the instance administrator account and choose Properties. If your account does not have sufficient permissions, you will get an error that only administrators can edit server properties.Change the desired properties by altering the Value column on the General page of the resulting dialog, as shown in the following screenshot: Advanced properties are hidden by default. You must check the Show Advanced (All) Properties box to see advanced properties. You will not see all the properties in SSMS even after checking this box. The only way to edit some properties is by editing msmdsrv.ini as previously discussed.Make a note of the Reset Default button in the bottom-right corner. This button comes in handy if you've forgotten what the configuration values were before you changed them and want to revert to the default settings. The default values are shown in the dialog box, which can provide guidance as to which properties have been altered.Some configuration settings require restarting the SSAS instance prior to being executed. If this is the case, the Restart column will have a value of Yes.Once you're happy with your changes, click on OK and restart the instance if necessary. You can restart SSAS using the Services.msc applet from the command line using the NET STOP / NET START commands, or directly in SSMS by choosing the Restart option after right-clicking on the instance. How it works...Discussing every SSAS property would make this article extremely lengthy; doing so is well beyond the scope of the book. Instead, in this section, I will summarize the most frequently used properties. Often, synchronization has to copy large partition datafiles and aggregation files. If the timeout value is exceeded, synchronization fails. Increase the value of the <Network><Listener><ServerSendTimeout> and <Network><Listener><ServerReceiveTimeout> properties to allow a longer time span for copying each file. By default, SSAS can use a lazy thread to rebuild missing indexes and aggregations after you process partition data. If the <OLAP><LazyProcessing><Enabled> property is set to 0, the lazy thread is not used for building missing indexes—you must use an explicit processing command instead. The <OLAP><LazyProcessing><MaxCPUUsage> property throttles the maximum CPU that could be used by the lazy thread. If efficient data delivery is your topmost priority, you can exploit the ProcessData option instead of ProcessFull. To build aggregations after the data is loaded, you must set the partition's ProcessingMode property to LazyAggregations. The SSAS formula engine is single threaded, so queries that perform heavy calculations will only use one CPU core, even on a multiCPU computer. The storage engine is multithreaded; hence, queries that read many partitions will require many CPU cycles. If you expect storage engine heavy queries, you should lower the CPU usage threshold for LazyAggregations. By default, Analysis Services records subcubes requested for every 10th query in the query log table. If you'd like to design aggregations based on query logs, you should change the <Log><QueryLog><QueryLogSampling> property value to 1 so that the SSAS logs subcube requests for every query. SSAS can use its own memory manager or the Windows memory manager. If your SSAS instance consistently becomes unresponsive, you could try using the Windows memory manager. Set <Memory><MemoryHeapType> to 2 and <Memory><HeapTypeForObjects> to 0. The Analysis Services memory manager values are 1 for both the properties. You must restart the SSAS service for the changes to these properties to take effect. The <Memory><PreAllocate> property specifies the percentage of total memory to be reserved at SSAS startup. SSAS normally allocates memory dynamically as it is required by queries and processing jobs. In some cases, you can achieve performance improvement by allocating a portion of the memory when the SSAS service starts.Setting this value will increase the time required to start the service. The memory will not be released back to the operating system until you stop the SSAS service. You must restart the SSAS service for changes to this property to take effect.The <Log><FlightRecorder><FileSizeMB>and <Log><FlightRecorder><LogDurationSec> properties control the size and age of the FlightRecorder trace file before it is recycled. You can supply your own trace definition file to include the trace events and columns you wish to monitor using the <Log><FlightRecorder><TraceDefinitionFile> property. If FlightRecorder collects useful trace events, it can be an invaluable troubleshooting tool. By default, the file is only allowed to grow to 10 MB or 60 minutes. Long processing jobs can take up much more space, and their duration could be much longer than 60 minutes. Hence, you should adjust the settings as necessary for your monitoring needs. You should also adjust the trace events and columns to be captured by FlightRecorder. You should consider adjusting the duration to cover three days (in case the issue you are researching happens over a weekend). The <Memory><LowMemoryLimit> property controls the point—amount of memory used by SSAS—at which the cleaner thread becomes actively engaged in reclaiming memory from existing jobs. Each SSAS command (query, processing, backup, synchronization, and so on) is associated with jobs that run on threads and use system resources. We can lower the value of this setting to run more jobs in parallel (though the performance of each job could suffer). Two properties control the maximum amount of memory that a SSAS instance could use. Once memory usage reaches the value specified by <Memory><TotalMemoryLimit>, the cleaner thread becomes particularly aggressive at reclaiming memory. The <Memory><HardMemoryLimit> property specifies the absolute memory limit—SSAS will not use memory above this limit. These properties are useful if you have SSAS and other applications installed on the same server computer. You should reserve some memory for other applications and the operating system as well. When HardMemoryLimit is reached, SSAS will disconnect the active sessions, advising that the operation was cancelled due to memory pressure.All memory settings are expressed in percentages if the values are less than or equal to 100. Values above 100 are interpreted as kilobytes. All memory configuration changes require restart of the SSAS service to take effect.In the prior releases of Analysis Services, you could only specify the minimum and maximum number of threads used for queries and processing jobs. With SSAS 2012, you can also specify the limits for the input/output job threads using the <ThreadPool><IOProcess> property. The <Process><IndexBuildThreshold> property governs the minimum number of rows within a partition for which SSAS will build indexes. The default value is 4096. SSAS decides which partitions it needs to scan for each query based on the partition index files. If the partition does not have indexes, it will be scanned for all the queries. Normally, SSAS can read small partitions without greatly affecting query performance. But if you have many small partitions, you should lower the threshold to ensure each partition has indexes. The <Process><BufferRecordLimit> and <Process><BufferMemoryLimit> properties specify the number of records for each memory buffer and the maximum percentage of memory that can be used by a memory buffer. Lower the value of these properties to process more partitions in parallel. You should monitor processing using the SQL Profiler to see if some partitions included in the processing batch are being processed while the others are in waiting. The <ExternalConnectionTimeout> and <ExternalCommandTimeout> properties control how long an SSAS command should wait for connecting to a relational database or how long SSAS should wait to execute the relational query before reporting timeout. Depending on the relational source, it might take longer than 60 seconds (that is, the default value) to connect. If you encounter processing errors without being able to connect to the relational source, you should increase the ExternalConnectionTimeout value. It could also take a long time to execute a query; by default, the processing query will timeout after one hour. Adjust the value as needed to prevent processing failures. The contents of the <AllowedBrowsingFolders> property define the drives and directories that are visible when creating databases, collecting backups, and so on. You can specify multiple items separated using the pipe (|) character. The <ForceCommitTimeout> property defines how long a processing job's commit operation should wait prior to cancelling any queries/jobs which may interfere with processing or synchronization. A long running query can block synchronization or processing from committing its transaction. You can adjust the value of this property from its default value of 30 seconds to ensure that processing and queries don't step on each other. The <Port> property specifies the port number for the SSAS instance. You can use the hostname followed by a colon (:) and a port number for connecting to the SSAS instance in lieu of the instance name. Be careful not to supply the port number used by another application; if you do so, the SSAS service won't start. The <ServerTimeout> property specifies the number of milliseconds after which a query will timeout. The default value is 1 hour, which could be too long for analytical queries. If the query runs for an hour, using up system resources, it could render the instance unusable by any other connection. You can also define a query timeout value in the client application's connection strings. Client setting overrides the server-level property.There's more...There are many other properties you can set to alter SSAS instance behavior. For additional information on configuration properties, please refer to product documentation at http://technet.microsoft.com/en-us/library/ms174556.aspx.Creating and dropping databasesOnly SSAS instance administrators are permitted to create, drop, restore, detach, attach, and synchronize databases. This recipe teaches administrators how to create and drop databases.Getting readyLaunch SSMS and connect to your Analysis Services instance as an administrator. If you're not certain that you have administrative properties to the instance, right-click on the SSAS instance and choose Properties. If you can view the instance's properties, you are an administrator; otherwise, you will get an error indicating that only instance administrators can view and alter properties.How to do it...To create a database, perform the following steps:Right-click on the Databases folder and choose New Database. Doing so launches the New Database dialog shown in the following screenshot.Specify a descriptive name for the database, for example, Analysis_Services_Administration. Note that the database name can contain spaces. Each object has a name as well as an identifier. The identifier value is set to the object's original name and cannot be changed without dropping and recreating the database; hence, it is important to come up with a descriptive name from the very beginning. You cannot create more than one database with the same name on any SSAS instance.Specify the storage location for the database. By default, the database will be stored under the \OLAP\DATA folder of your SSAS installation directory. The only compelling reason to change the default is if your data drive is running out of disk space and cannot support the new database's storage requirements. Specify the impersonation setting for the database. You could also specify the impersonation property for each data source. Alternatively, each data source can inherit the DataSourceImpersonationInfo property from the database-level setting. You have four choices as follows:Specific user name (must be a domain user) and password: This is the most secure option but requires updating the password if the user changes the passwordAnalysis Services service accountCredentials of the current user: This option is specifically for data miningDefault: This option is the same as using the service account optionSpecify an optional description for the database.As with majority of other SSMS dialogs, you can script the XMLA command you are about to execute by clicking on the Script button.To drop an existing database, perform the following steps:Expand the Databases folder on the SSAS instance, right-click on the database, and choose Delete.The Delete objects dialog allows you to ignore errors; however, it is not applicable to databases. You can script the XMLA command if you wish to review it first.An alternative way of scripting the DELETE command is to right-click on the database and navigate to Script database as | Delete To | New query window. Monitoring SSAS instance using Activity ViewerUnlike other database systems, Analysis Services has no system databases. However, administrators still need to check the activity on the server, ensure that cubes are available and can be queried, and there is no blocking. You can exploit a tool named Analysis Services Activity Viewer 2008 to monitor SSAS Versions 2008 and later, including SSAS 2012. This tool is owned and maintained by the SSAS community and can be downloaded from www.codeplex.com. Activity Viewer allows viewing active and dormant sessions, current XMLA and MDX queries, locks, as well as CPU and I/O usage by each connection. Additionally, you can define rules to raise alerts when a particular condition is met.How to do it...To monitor an SSAS instance using Activity Viewer, perform the following steps:Launch the application by double-clicking on ActivityViewer.exe.Click on the Add New Connection button on the Overview tab. Specify the hostname and instance name or the hostname and port number for the SSAS instance and then click on OK.For each SSAS instance you connect to, Activity Viewer adds a new tab. Click on the tab for your SSAS instance. Here, you will see several pages as shown in the following screenshot: Alerts: This page shows any sessions that met the condition found in the Rules page.Users: This page displays one row for each user as well as the number of sessions, total memory, CPU, and I/O usage.Active Sessions: This page displays each session that is actively running an MDX, Data Mining Extensions (DMX), or XMLA query. This page allows you to cancel a specific session by clicking on the Cancel Session button.Current Queries: This page displays the actual command's text, number of kilobytes read and written by the command, and the amount of  CPU time used by the command. This page allows you to cancel a specific query by clicking on the Cancel Query button.Dormant Sessions: This page displays sessions that have a connection to the SSAS instance but are not currently running any queries. You can also disconnect a dormant session by clicking on the Cancel Session button.CPU: This page allows you to review the CPU time used by the session as well as the last command executed on the session.I/O: This page displays the number of reads and writes as well as the kilobytes read and written by each session.Objects: This page shows the CPU time and number of reads affecting each dimension and partition. This page also shows the full path to the object's parent; this is useful if you have the same naming convention for partitions in multiple measure groups. Not only do you see the partition name, but also the full path to the partition's measure group. This page also shows the number of aggregation hits for each partition. If you find that a partition is frequently queried and requires many reads, you should consider building aggregations for it.Locks: This page displays the locks currently in place, whether already granted or waiting. Be sure to check the Lock Status column—the value of 0 indicates that the lock request is currently blocked.Rules: This page allows defining conditions that will result in an alert. For example, if the session is idle for over 30 minutes or if an MDX query takes over 30 minutes, you should get alerted. How it works...Activity Viewer monitors Analysis Services using Dynamic Management Views (DMV). In fact, capturing queries executed by Activity Viewer using SQL Server Profiler is a good way of familiarizing yourself with SSAS DMV's. For example, the Current Queries page checks the $system.DISCOVER_COMMANDS DMV for any actively executing commands by running the following query:SELECT SESSION_SPID,COMMAND_CPU_TIME_MS,COMMAND_ELAPSED_TIME_MS,   COMMAND_READ_KB,COMMAND_WRITE_KB, COMMAND_TEXT FROM $system.DISCOVER_COMMANDS WHERE COMMAND_ELAPSED_TIME_MS > 0 ORDER BY COMMAND_CPU_TIME_MS DESCThe Active Sessions page checks the $system.DISCOVER_SESSIONS DMV with the session status set to 1 using the following query:SELECT SESSION_SPID,SESSION_USER_NAME, SESSION_START_TIME,   SESSION_ELAPSED_TIME_MS,SESSION_CPU_TIME_MS, SESSION_ID FROM $SYSTEM.DISCOVER_SESSIONS WHERE SESSION_STATUS = 1 ORDER BY SESSION_USER_NAME DESCThe Dormant sessions page runs a very similar query to that of the Active Sessions page, except it checks for sessions with SESSION_STATUS=0—sessions that are currently not running any queries. The result set is also limited to top 10 sessions based on idle time measured in milliseconds. The Locks page examines all the columns of the $system.DISCOVER_LOCKS DMV to find all requested locks as well as lock creation time, lock type, and lock status. As you have already learned, the lock status of 0 indicates that the request is blocked, whereas the lock status of 1 means that the request has been granted. Analysis Services blocking can be caused by conflicting operations that attempt to query and modify objects. For example, a long running query can block a processing or synchronization job from completion because processing will change the data values. Similarly, a command altering the database structure will block queries. The database administrator or instance administrator can explicitly issue the LOCK XMLA command as well as the BEGIN TRANSACTION command. Other operations request locks implicitly. The following table documents most frequently encountered Analysis Services lock types: Lock type identifierDescriptionAcquired for2Read lockProcessing to read metadata.4Write lockProcessing to write data after it is read from relational sources.8Commit sharedDuring the processing, restore or synchronization commands.16Commit exclusiveCommitting the processing, restore, or synchronization transaction when existing files are replaced by new files. 

(for more resources related to this topic, see here.) Mission briefing We've covered creating robots using a wheeled/track base. In this article, you will be introduced to some of the basics of servo motors and using the BeagleBone Black to control the speed and direction of your legged platform. Here is an image of a finished project: Why is it awesome? Even though you've learned to make your robot mobile by adding wheels or tracks, this mobile platform will only work well on smooth, flat surfaces. Often, you'll want your robot to work in environments where it is not smooth or flat; perhaps, you'll even want your robot to go upstairs or over curbs. In this article, you'll learn how to attach your board, both mechanically and electrically, to a platform with legs, so your projects can be mobile in many more environments. Robots that can walk: what could be more amazing than that? Your objectives In this article, you will learn: Connecting the BeagleBone Black to a mobile platform using a servo controller Creating a program in Linux to control the movement of the mobile platform Making your mobile platform truly mobile by issuing voice commands   Mission checklist In this article, you'll need to add a legged platform to make your project mobile. So, here is your parts' list: A legged robot: There are a lot of choices. As before, some are completely assembled, others have some assembly required, and you may even choose to buy the components and construct your own custom mobile platform. Also, as before, I'm going to assume that you don't want to do any soldering or mechanical machining yourself, so let's look at a several choices that are available completely assembled or can be assembled by simple tools (screwdriver and/or pliers). One of the easiest legged mobile platforms is one that has two legs and four servo motors. Here is an image of this type of platform: You'll use this platform in this article because it is the simplest to program and because it is the least expensive, requiring only four servos. To construct this platform, you must purchase the parts and then assemble it yourself. Find the instructions and parts list at http://www.lynxmotion.com/images/html/build112.htm. Another easy way to get all the mechanical parts (except servos) is to purchase a biped robot kit with six degrees of freedom (DOF). This will contain the parts needed to construct your four-servo biped. These six DOF bipeds can be purchased by searching eBay or by going to http://www.robotshop.com/2-wheeled-development-platforms-1.html. You'll also need to purchase the servo motors. For this type of robot, you can use standard size servos. I like the Hitec HS-311 or HS-322 for this robot. They are inexpensive but powerful enough. You can get those on Amazon or eBay. Here is an image of an HS-311: You'll need a mobile power supply for the BeagleBone Black. Again, I personally like the 5V cell phone rechargeable batteries that are available almost anywhere that supplies cell phones. Choose one that comes with two USB connectors, just in case you want to also use the powered USB hub. This one mounts well on the biped HW platform: You'll also need a USB cable to connect your battery to the BeagleBone Black, but you can just use the cable supplied with the BeagleBone Black. If you want to connect your powered USB hub, you'll need a USB to DC jack adapter for that as well. You'll also need a way to connect your batteries to the servo motor controller. Here is an image of a four AA battery holder, available at most electronics parts stores or from Amazon: Now that you have the mechanical parts for your legged mobile platform, you'll need some HW that will take the control signals from your BeagleBone Black and turn them into a voltage that can control the servo motors. Servo motors are controlled using a control signal called PWM. For a good overview of this type of control, see http://pcbheaven.com/wikipages/How_RC_Servos_Works/ or https://www.ghielectronics.com/docs/18/pwm. You can find tutorials that show you how to control servos directly using the BeagleBone Black's GPIO pins, for example, here at http://learn.adafruit.com/controlling-a-servowith-a-beaglebone-black/overview and http://www.youtube.com/watch?v=6gv3gWtoBWQ. For ease of use I chose to purchase a motor controller that can talk over USB and control the servo motor. These protect my board and make controlling many servos easy. My personal favorite for this application is a simple servo motor controller utilizing USB from Pololu that can control 18 servo motors. Here is an image of the unit: Again, make sure you order the assembled version. This piece of HW will turn USB commands into voltage that control your servo motors. Pololu makes a number of different versions of this controller, each able to control a certain number of servos. Once you've chosen your legged platform, simply count the number of servos you need to control, and chose the controller that can control that number of servos. One advantage of the 18 servo controller is the ease of connecting power to the unit via screw type connectors. Since you are going to connect this controller to your BeagleBone Black via USB, you'll also need a USB A to mini-B cable. Now that you have all the HW, let's walk through a quick tutorial on how a two-legged system with servos works and then some step-by-step instructions to make your project walk. Connecting the BeagleBone Black to the mobile platform using a servo controller Now that you have a legged platform and a servo motor controller, you are ready to make your project walk! Prepare for lift off Before you begin, you'll need some background on servo motors. Servo motors are somewhat similar to DC motors; however, there is an important difference. While DC motors are generally designed to move in a continuous way—rotating 360 degrees at a given speed—servos are generally designed to move within a limited set of angles. In other words, in the DC motor world, you generally want your motors to spin with continuous rotation speed that you control. In the servo world, you want your motor to move to a specific position that you control. Engage thrusters To make your project walk, you first need to connect the servo motor controller to the servos. There are two connections you need to make: the first to the servo motors, the second to the battery holder. In this section, you'll connect your servo controller to your PC to check to see if everything is working. First, connect the servos to the controller. Here is an image of your two-legged robot, and the four different servo connections: In order to be consistent, let's connect your four servos to the connections marked 0 through 3 on the controller using this configuration: 0 – left foot, 1 – left hip, 2 – right foot, and 3 – right hip. Here is an image of the back of the controller; it will tell you where to connect your servos: Connect these to the servo motor controller like this: the left foot to the top O connector, black cable to the outside (–), the left hip to the 1 connector, black cable out, right foot to the 2 connector, black cable out, and right hip to the 3 connector, black cable out. See the following image for a clearer description: Now you need to connect the servo motor controller to your battery. If you are using a standard 4 AA battery holder, connect it to the two green screw connectors, the black cable to the outside, and the red cable to the inside, as shown in the following image: Now you can connect the motor controller to your PC to see if you can talk with it.   Objective complete – mini debriefing Now that the HW is connected, you can use some SW provided by Polulu to control the servos. It is easiest to do this using your personal computer. First, download the Polulu SW from http://www.pololu.com/docs/0J40/3.a and install it based on the instructions on the website. Once it is installed, run the SW, and you should see the following screen: You first will need to change the configuration on Serial Settings, so select the Serial Settings tab, and you should see a screen as shown in the following screenshot: Make sure that the USB Chained option is selected; this will allow you to connect and control the motor controller over USB. Now go back to the main screen by selecting the Status tab, and now you can turn on the four servos. The screen should look like the following screenshot: Now you can use the sliders to control the servos. Make sure that the servo 0 moves the left foot, 1 the left hip, 2 the right foot, and 3 the right hip. You've checked the motor controllers and the servos, and you'll now connect the motor controller up to the BeagleBone Black control the servos from it. Remove the USB cable from the PC and connect it into the powered USB hub. The entire system will look like the following image: Let's now talk to the motor controller by downloading the Linux code from Pololu at http://www.pololu.com/docs/0J40/3.b. Perhaps, the best way is to log in to your Beagle Bone Black by using vncserver and a vncviewer window on your PC. To do this, log in to your BeagleBone Black using PuTTY, then type vncserver at the prompt to make sure vncserver is running. On your PC open the VNC Viewer application, enter your IP address, then press connect. Then enter your password that you created for the vncserver, and you should see the BeagleBone Black Viewer screen, which should look like this: Open a Firefox browser window and go to http://www.pololu.com/docs/0J40/3.b. Click on the Maestro Servo Controller Linux Software link. You will download the file maestro_linux_100507.tar.gz to the Download directory. Go to your download directory, move this file to your home directory by typing mv maestro_linux_100507.tar.gz .. and then you can go back to your home directory. Unpack the file by typing tar –xzfv maestro_linux_011507.tar.gz. This will create a directory called maestro_linux. Go to that directory by typing cd maestro_linux and then type ls. You should see something like this: The document README.txt will give you explicit instructions on how to install the SW. Unfortunately you can't run MaestroControlCenter on your BeagleBone Black. Our version of windowing doesn't support the graphics, but you can control your servos using the UscCmd command-line application. First type ./UscCmd --list and you should see the following: The unit sees your servo controller. If you just type ./UscCmd you can see all the commands you could send to your controller: Notice you can send a servo a specific target angle, although the target is not in angle values, so it makes it a bit difficult to know where you are sending your servo. Try typing ./UscCmd --servo 0, 10. The servo will most likely move to its full angle position. Type ./UscCmd --servo 0, 0 and it will stop the servo from trying to move. In the next section, you'll write some SW that will translate your angles to the commands that the servo controller will want to see. If you didn't run the Windows version of Maestro Controller and set the serial settings to USB Chained, your motor controller may not respond.

(For more resources related to this topic, see here.) Enumerating SQL-injection vulnerable content providers Just like web applications, Android applications may use untrusted input to construct SQL queries and do so in a way that's exploitable. The most common case is when applications do not sanitize input for any SQL and do not limit access to content providers. Why would you want to stop a SQL-injection attack? Well, let's say you're in the classic situation of trying to authorize users by comparing a username supplied by querying a database for it. The code would look similar to the following: public boolean isValidUser(){ u_username = EditText( some user value ); u_password = EditText( some user value ); //some un-important code here... String query = "select * from users_table where username = '" + u_username + "' and password = '" + u_password +"'"; SQLiteDatabase db //some un-important code here... Cursor c = db.rawQuery( p_query, null ); return c.getCount() != 0; } What's the problem in the previous code? Well, what happens when the user supplies a password '' or '1'='1'? The query being passed to the database then looks like the following: select * from users_table where username = '" + u_username + "' and password = '' or '1'='1' " The preceding bold characters indicate the part that was supplied by the user; this query forms what's known in Boolean algebra as a logical tautology; meaning no matter what table or data the query is targeted at, it will always be set to true, which means that all the rows in the database will meet the selection criteria. This then means that all the rows in users_table will be returned and as result, even if a nonvalid password ' or '1'=' is supplied, the c.getCount() call will always return a nonzero count, leading to an authentication bypass! Given that not many Android developers would use the rawQuery call unless they need to pull off some really messy SQL queries, I've included another code snippet of a SQL-injection vulnerability that occurs more often in real-world applications. So when auditing Android code for injection vulnerabilities, a good idea would be to look for something that resembles the following: public Cursor query(Uri uri, String[] projection , String selection ,String[] selectionArgs , String sortOrder ) { SQLiteDBHelper sdbh = new StatementDBHelper(this.getContext()); Cursor cursor; try { //some code has been omitted cursor = sdbh .query(projection,selection,selectionArgs,sortOrder); } finally { sdbh.close(); } return cursor; } In the previous code, none of the projection, selection, selectionArgs, or sortOrder variables are sourced directly from external applications. If the content provider is exported and grants URI permissions or, as we've seem before, does not require any permissions, it means that attackers will be able to inject arbitrary SQL to augment the way the malicious query is evaluated. Let's look at how you actually go about attacking SQL-injection vulnerable content providers using drozer. How to do it... In this recipe, I'll talk about two kinds of SQL-injection vulnerabilities: one is when the select clause of a SQL statement is injectable and the other is when the projection is injectable. Using drozer, it is pretty easy to find select-clause-injectable content providers: dz> run app.provider.query [URI] –-selection "1=1" The previous will try to inject what's called a logical tautology into the SQL statement being parsed by the content provider and eventually the database query parser. Due to the nature of the module being used here, you can tell whether or not it actually worked, because it should return all the data from the database; that is, the select-clause criteria is applied to every row and because it will always return true, every row will be returned! You could also try any values that would always be true: dz> run app.provider.query [URI] –-selection "1-1=0" dz> run app.provider.query [URI] –-selection "0=0" dz> run app.provider.query [URI] –-selection "(1+random())*10 > 1" The following is an example of using a purposely vulnerable content provider: dz> run app.provider.query content://com.example. vulnerabledatabase.contentprovider/statements –-selection "1=1" It returns the entire table being queried, which is shown in the following screenshot: You can, of course, inject into the projection of the SELECT statement, that is, the part before FROM in the statement, that is, SELECT [projection] FROM [table] WHERE [select clause]. Securing application components Application components can be secured both by making proper use of the AndroidManifest.xml file and by forcing permission checks at code level. These two factors of application security make the permissions framework quite flexible and allow you to limit the number of applications accessing your components in quite a granular way. There are many measures that you can take to lock down access to your components, but what you should do before anything else is make sure you understand the purpose of your component, why you need to protect it, and what kind of risks your users face should a malicious application start firing off intents to your app and accessing its data. This is called a risk-based approach to security, and it is suggested that you first answer these questions honestly before configuring your AndroidManifest.xml file and adding permission checks to your apps. In this recipe, I have detailed some of the measures that you can take to protect generic components, whether they are activities, broadcast receivers, content providers, or services. How to do it... To start off, we need to review your Android application AndroidManifest.xml file. The android:exported attribute defines whether a component can be invoked by other applications. If any of your application components do not need to be invoked by other applications or need to be explicitly shielded from interaction with the components on the rest of the Android system—other than components internal to your application—you should add the following attribute to the application component's XML element: <[component name] android_exported="false"> </[component name]> Here the [component name] would either be an activity, provider, service, or receiver. How it works… Enforcing permissions via the AndroidManifest.xml file means different things to each of the application component types. This is because of the various inter-process communications ( IPC ) mechanisms that can be used to interact with them. For every application component, the android:permission attribute does the following: Activity : Limits the application components which are external to your application that can successfully call startActivity or startActivityForResult to those with the required permission Service : Limits the external application components that can bind (by calling bindService()) or start (by calling startService()) the service to those with the specified permission Receiver : Limits the number of external application components that can send broadcasted intents to the receiver with the specified permission Provider : Limits access to data that is made accessible via the content provider The android:permission attribute of each of the component XML elements overrides the <application> element's android:permission attribute. This means that if you haven't specified any required permissions for your components and have specified one in the <application> element, it will apply to all of the components contained in it. Though specifying permissions via the <application> element is not something developers do too often because of how it affects the friendliness of the components toward the Android system itself (that is, if you override an activity's required permissions using the <application> element), the home launcher will not be able to start your activity. That being said, if you are paranoid enough and don't need any unauthorized interaction to happen with your application or its components, you should make use of the android:permission attribute of the <application> tag. When you define an <intent-filter> element on a component, it will automatically be exported unless you explicitly set exported="false". However, this seemed to be a lesser-known fact, as many developers were inadvertently opening their content providers to other applications. So, Google responded by changing the default behavior for <provider> in Android 4.2. If you set either android:minSdkVersion or android:targetSdkVersion to 17, the exported attribute on <provider> will default to false. Defending against the SQL-injection attack The previous chapter covered some of the common attacks against content providers, one of them being the infamous SQL-injection attack. This attack leverages the fact that adversaries are capable of supplying SQL statements or SQL-related syntax as part of their selection arguments, projections, or any component of a valid SQL statement. This allows them to extract more information from a content provider than they are not authorized. The best way to make sure adversaries will not be able to inject unsolicited SQL syntax into your queries is to avoid using SQLiteDatabase.rawQuery() instead opting for a parameterized statement. Using a compiled statement, such as SQLiteStatement, offers both binding and escaping of arguments to defend against SQL-injection attacks. Also, there is a performance benefit due to the fact the database does not need to parse the statement for each execution. An alternative to SQLiteStatement is to use the query, insert, update, and delete methods on SQLiteDatabase as they offer parameterized statements via their use of string arrays. When we describe parameterized statement, we are describing an SQL statement with a question mark where values will be inserted or binded. Here's an example of parameterized SQL insert statement: INSERT VALUES INTO [table name] (?,?,?,?,...) Here [table name] would be the name of the relevant table in which values have to be inserted. How to do it... For this example, we are using a simple Data Access Object ( DAO ) pattern, where all of the database operations for RSS items are contained within the RssItemDAO class: When we instantiate RssItemDAO, we compile the insertStatement object with a parameterized SQL insert statement string. This needs to be done only once and can be re-used for multiple inserts: public class RssItemDAO { private SQLiteDatabase db; private SQLiteStatement insertStatement; private static String COL_TITLE = "title"; private static String TABLE_NAME = "RSS_ITEMS"; private static String INSERT_SQL = "insert into " + TABLE_NAME + " (content, link, title) values (?,?,?)"; public RssItemDAO(SQLiteDatabase db) { this.db = db; insertStatement = db.compileStatement(INSERT_SQL); } The order of the columns noted in the INSERT_SQL variable is important, as it directly maps to the index when binding values. In the preceding example, content maps to index 0, link maps to index 1, and title to index 2. Now, when we come to insert a new RssItem object to the database, we bind each of the properties in the order they appear in the statement: public long save(RssItem item) { insertStatement.bindString(1, item.getContent()); insertStatement.bindString(2, item.getLink()); insertStatement.bindString(3, item.getTitle()); return insertStatement.executeInsert(); } Notice that we call executeInsert, a helper method that returns the ID of the newly created row. It's as simple as that to use a SQLiteStatement statement. This shows how to use SQLiteDatabase.query to fetch RssItems that match a given search term: public List<RssItem> fetchRssItemsByTitle(String searchTerm) { Cursor cursor = db.query(TABLE_NAME, null, COL_TITLE + "LIKE ?", new String[] { "%" + searchTerm + "%" }, null, null, null); // process cursor into list List<RssItem> rssItems = new ArrayList<RssItemDAO.RssItem>(); cursor.moveToFirst(); while (!cursor.isAfterLast()) { // maps cursor columns of RssItem properties RssItem item = cursorToRssItem(cursor); rssItems.add(item); cursor.moveToNext(); } return rssItems; } We use LIKE and the SQL wildcard syntax to match any part of the text with a title column. Summary There were a lot of technical details in this article. Firstly, we learned about the components that are vulnerable to SQL-injection attacks. We then figured out how to secure our Android applications from the exploitation attacks. Finally, we learned how to defend our applications from the SQL-injection attacks. Resources for Article: Further resources on this subject: Android Native Application API [Article] So, what is Spring for Android? [Article] Creating Dynamic UI with Android Fragments [Article]

(For more resources related to this topic, see here.) Fragmentation issues AsyncTask has evolved with new releases of the Android platform, resulting in behavior that varies with the platform of the device running the task, which is a part of the wider issue of fragmentation. The simple fact is that if we target a broad range of API levels, the execution characteristics of our AsyncTasks—and therefore, the behavior of our apps—can vary considerably on different devices. So what can we do to reduce the likelihood of encountering AsyncTask issues due to fragmentation? The most obvious approach is to deliberately target devices running at least Honeycomb, by setting a minSdkVersion of 11 in the Android Manifest file. This neatly puts us in the category of devices, which, by default, execute AsyncTasks serially, and therefore, much more predictably. However, this significantly reduces the market reach of our apps. At the time of writing in September 2013, more than 34 percent of Android devices in the wild run a version of Android in the danger zone between API levels 4 and 10. A second option is to design our code carefully and test exhaustively on a range of devices—always commendable practices of course, but as we've seen, concurrent programming is hard enough without the added complexity of fragmentation, and invariably, subtle bugs will remain. A third solution that has been suggested by the Android development community is to reimplement AsyncTaskin a package within your own project, then extend your own AsyncTask class instead of the SDK version. In this way, you are no longer at the mercy of the user's device platform, and can regain control of your AsyncTasks. Since the source code for AsyncTask is readily available, this is not difficult to do. Activity lifecycle issues Having deliberately moved any long-running tasks off the main thread, we've made our applications nice and responsive—the main thread is free to respond very quickly to any user interaction. Unfortunately, we have also created a potential problem for ourselves, because the main thread is able to finish the Activity before our background tasks complete. Activity might finish for many reasons, including configuration changes caused the by the user rotating the device (the default behavior of Activity on a change in orientation is to restart with an entirely new instance of the activity). If we continue processing a background task after the Activity has finished, we are probably doing unnecessary work, and therefore wasting CPU and other resources (including battery life), which could be put to better use. Also, any object references held by the AsyncTask will not be eligible for garbage collection until the task explicitly nulls those references or completes and is itself eligible for GC ( garbage collection ). Since our AsyncTask probably references the Activity or parts of the View hierarchy, we can easily leak a significant amount of memory in this way. A common usage of AsyncTask is to declare it as an anonymous inner class of the host Activity, which creates an implicit reference to the Activity and an even bigger memory leak. There are two approaches for preventing these resource wastage problems. Handling lifecycle issues with early cancellation First, and most obviously, we can synchronize our AsyncTask lifecycle with that of the Activity by canceling running tasks when our Activity is finishing. When an Activity finishes, its lifecycle callback methods are invoked on the main thread. We can check to see why the lifecycle method is being called, and if the Activity is finishing, cancel the background tasks. The most appropriate Activity lifecycle method for this is onPause, which is guaranteed to be called before the Activity finishes. protected void onPause() { super.onPause(); if ((task != null) && (isFinishing())) task.cancel(false); } If the Activity is not finishing—say because it has started another Activity and is still on the back stack—we might simply allow our background task to continue to completion. Handling lifecycle issues with retained headless fragments If the Activity is finishing because of a configuration change, it may still be useful to complete the background task and display the results in the restarted Activity. One pattern for achieving this is through the use of retained Fragments. Fragments were introduced to Android at API level 11, but are available through a support library to applications targeting earlier API levels. All of the downloadable examples use the support library, and target API levels 7 through 19. To use Fragments, our Activity must extend the FragmentActivity class. The Fragment lifecycle is closely bound to that of the host Activity, and a fragment will normally be disposed when the activity restarts. However, we can explicitly prevent this by invoking setRetainInstance (true) on our Fragment so that it survives across Activity restarts. Typically, a Fragment will be responsible for creating and managing at least a portion of the user interface of an Activity, but this is not mandatory. A Fragment that does not manage a view of its own is known as a headless Fragment. Isolating our AsyncTask in a retained headless Fragment makes it less likely that we will accidentally leak references to objects such as the View hierarchy, because the AsyncTask will no longer directly interact with the user interface. To demonstrate this, we'll start by defining an interface that our Activity will implement: public interface AsyncListener<Progress, Result> { void onPreExecute(); void onProgressUpdate(Progress... progress); void onPostExecute(Result result); void onCancelled(Result result); } Next, we'll create a retained headless Fragment, which wraps our AsyncTask. For brevity, doInBackground is omitted, as it is unchanged from the previous examples—see the downloadable samples for the complete code. public class PrimesFragment extends Fragment { private AsyncListener<Integer,BigInteger> listener; private PrimesTask task; public void onCreate(Bundle savedInstanceState) { super.onCreate(savedInstanceState); setRetainInstance(true); task = new PrimesTask(); task.execute(2000); } public void onAttach(Activity activity) { super.onAttach(activity); listener = (AsyncListener<Integer,BigInteger>)activity; } public void onDetach() { super.onDetach(); listener = null; } class PrimesTask extends AsyncTask<Integer, Integer, BigInteger>{ protected void onPreExecute() { if (listener != null) listener.onPreExecute(); } protected void onProgressUpdate(Integer... values) { if (listener != null) listener.onProgressUpdate(values); } protected void onPostExecute(BigInteger result) { if (listener != null) listener.onPostExecute(result); } protected void onCancelled(BigInteger result) { if (listener != null) listener.onCancelled(result); } // … doInBackground elided for brevity … } } We're using the Fragment lifecycle methods (onAttach and onDetach) to add or remove the current Activity as a listener, and PrimesTask delegates directly to it from all of its main-thread callbacks. Now, all we need is the host Activity that implements AsyncListener and uses PrimesFragment to implement its long-running task. The full source code is available to download from the Packt Publishing website, so we'll just take a look at the highlights. First, the code in the button's OnClickListener now checks to see if Fragment already exists, and only creates one if it is missing: FragmentManager fm = getSupportFragmentManager(); PrimesFragment primes = (PrimesFragment)fm.findFragmentByTag("primes"); if (primes == null) { primes = new PrimesFragment(); FragmentTransaction transaction = fm.beginTransaction(); transaction.add(primes, "primes").commit(); } If our Activity has been restarted, it will need to re-display the progress dialog when a progress update callback is received, so we check and show it, if necessary, before updating the progress bar: public void onProgressUpdate(Integer... progress) { if (dialog == null) prepareProgressDialog(); progress.setProgress(progress[0]); } Finally, Activity will need to implement the onPostExecute and onCancelled callbacks defined by AsyncListener. Both methods will update the resultView as in the previous examples, then do a little cleanup—dismissing the dialog and removing Fragment as its work is now done: public void onPostExecute(BigInteger result) { resultView.setText(result.toString()); cleanUp(); } public void onCancelled(BigInteger result) { resultView.setText("cancelled at " + result); cleanUp(); } private void cleanUp() { dialog.dismiss(); dialog = null; FragmentManager fm = getSupportFragmentManager(); Fragment primes = fm.findFragmentByTag("primes"); fm.beginTransaction().remove(primes).commit(); } Summary In this article, we've taken a look at AsyncTask and how to use it to write responsive applications that perform operations without blocking the main thread. Resources for Article: Further resources on this subject: Android Native Application API [Article] So, what is Spring for Android? [Article] Creating Dynamic UI with Android Fragments [Article]

(for more resources related to this topic, see here.) Introduction to options Options come in two variants, puts and calls. The call option gives the owner of the option the right, but not the obligation, to buy the underlying asset at the strike price. The put gives the holder of the contract, the right but not the obligation to sell the underlying asset. The Black-Scholes formula describes the European option, which can only be exercised on the maturity date, in contrast to for example American options. The buyer of the option pays a premium for this, to cover the risk taken from the counterpart side. Options have become very popular and they are traded on the major exchanges throughout the world, covering most asset-classes. The theory behind options can become complex pretty quick. In this article we'll look at the basics of options and how to explore them using code written in F#. Looking into contract specifications Options comes in a wide number of variations, some of them will be covered briefly below. The contract specifications for options will also depend on its type. Generally there are some properties that are more or less general to all of them. The general specifications are as follows: Side Quantity Strike price Expiration date Settlement terms The contract specifications, or know variables, are used then we valuate options. European options European options are the basic form of options that the other variants derive, American options and exotic options are some examples. We'll stick to European options in this article. American options American options are options that may be exercised on any trading day on or before expiry. Exotic options Exotic options are any of the broad category of options that may include complex financial structures and may be combinations of other instruments as well. Learning about Wiener processes Wiener processes are closely related to stochastic differential equations and volatility. Wiener processes or geometric Brownian motion, is defined as this: The formula describes the change in the stock price, or underlying, with a drift, μ, and a volatility, σ, and the Wiener process, Wt. This process is used to model the prices in Black-Scholes. We'll simulate market data using a Brownian motion, or Wiener process implemented in F# as a sequence. Sequences can be infinite and only the values used are evaluated, which suites or needs. We'll implement a generator function, to generate the Wiener process as a sequence as follows: // A normally distributed random generator let normd = new Normal(0.0, 1.0) let T = 1.0 let N = 500.0 let dt:float = T / N /// Sequences represent infinite number of elements // p -> probability mean // s -> scaling factor let W s = let rec loop x = seq { yield x; yield! loop (x + sqrt(dt)*normd.Sample()*s)} loop s;; Here we use the random function in normd.Sample(). Let's explain the parameters and the theory behind Brownian motion before looking at the implementation. The parameter T is the time used to create a discrete time increment dt. Notice that dt will assume there is 500 N:s, 500 items in the sequence, this is of course not always the case but will do fine in here. Next, we use recursion to create the sequence, where we add an increment to the previous value (x+...), where x c] xt-1. We can easily generate an arbitrary length of the sequence: > Seq.take 50 (W 55.00);; val it : seq<float> = seq [55.0; 56.72907873; 56.96071054;58.72850048; ...] Here we create a sequence of length 50. Let's plot the sequence to get a better understanding about the process. A Wiener process generated from the sequence generator above. Next we'll look at the code to generate the graph in the figure above. open System open System.Net open System.Windows.Forms open System.Windows.Forms.DataVisualization.Charting open Microsoft.FSharp.Control.WebExtensions open MathNet.Numerics.Distributions; // A normally distributed random generator let normd = new Normal(0.0, 1.0) // Create chart and form let chart = new Chart(Dock = DockStyle.Fill) let area = new ChartArea("Main") chart.ChartAreas.Add(area) let mainForm = new Form(Visible = true, TopMost = true, Width = 700, Height = 500) do mainForm.Text <- "Wiener process in F#" mainForm.Controls.Add(chart) // Create series for stock price let wienerProcess = new Series("process") do wienerProcess.ChartType <- SeriesChartType.Line do wienerProcess.BorderWidth <- 2 do wienerProcess.Color <- Drawing.Color.Red chart.Series.Add(wienerProcess) let random = new System.Random() let rnd() = random.NextDouble() let T = 1.0 let N = 500.0 let dt:float = T / N /// Sequences represent infinite number of elements let W s = let rec loop x = seq { yield x; yield! loop (x +/ sqrt(dt)*normd.Sample()*s)} loop s;; do (Seq.take 100 (W 55.00)) |> Seq.iter (wienerProcess.Points.Add>> ignore) Most of the code will be familiar to you at this stage, but the interesting part is the last line, where we can simply feed a chosen number of elements from the sequence into the Seq.iter which will plot the values, elegant and efficient. Learning the Black-Scholes formula The Black-Scholes formula was developed by Fischer Black and Myron Scholes in the 1970s. The Black-Scholes formula is a stochastic partial differential equation, which estimates the price an the option. The main idea behind the formula is the delta neutral portfolio. They created the theoretical delta neutral portfolio, to reduce the uncertainty involved. This was a necessary step to be able to come to the analytical formula which we’ll cover in this section. Below is the assumptions made under Black-Scholes: No arbitrage Possible to borrow money at a constant risk-free interest rate (throughout the holding of the option) Possible to buy, sell and short fractional amounts of underlying asset No transaction costs Price of underlying follows a Brownian Motion, constant drift and volatility No dividends paid from underlying security The simplest of the two variants is the one for call options. First the stock price is scaled using the cumulative distribution function with d1 as a parameter. Then the stock price is reduced by the discounted strike price scaled by the cumulative distribution function of d2. In other words, it’s the difference between the stock price and the strike using probability scaling of each and discounting the strike price. The formula for the put is a little more involved, but follows the same principles. The Black-Scholes formula are often separated into parts, where d1, d2 are the probability factors, describing the probability of the stock price being related to the strike price. The parameters used in the formula above can be summarized as follows: N – The cumulative distribution function T - Time to maturity, expressed in years S – The stock price, or other underlying K – The strike price r – The risk free interest rate σ – The volatility of the underlying Implementing Black-Scholes in F# Now that we've looked at the basics behind the Black-Scholes formula, and the parameters involved, we can implement it ourselves. The cumulative distribution function is implemented here to avoid dependencies and to illustrate that it's quite simple to implement it yourself too. Below is the Black-Scholes implemented in F#. It takes six arguments; the first is a call-put-flag that determines if it's a call or put option. The constants a1 to a5 are the Taylor series coefficients used in the approximation for the numerical implementation. let pow x n = exp(n * log(x)) type PutCallFlag = Put | Call /// Cumulative distribution function let cnd x = let a1 = 0.31938153 let a2 = -0.356563782 let a3 = 1.781477937 let a4 = -1.821255978 let a5 = 1.330274429 let pi = 3.141592654 let l = abs(x) let k = 1.0 / (1.0 + 0.2316419 * l) let w = (1.0-1.0/sqrt(2.0*pi)*exp(-l*l/2.0)*(a1*k+a2*k*k+a3*(pow k 3.0)+a4*(pow k 4.0)+a5*(pow k 5.0))) if x < 0.0 then 1.0 - w else w /// Black-Scholes // call_put_flag: Put | Call // s: stock price // x: strike price of option // t: time to expiration in years // r: risk free interest rate // v: volatility let black_scholes call_put_flag s x t r v = let d1=(log(s / x) + (r+v*v*0.5)*t)/(v*sqrt(t)) let d2=d1-v*sqrt(t) //let res = ref 0.0 match call_put_flag with | Put -> x*exp(-r*t)*cnd(-d2)-s*cnd(-d1) | Call -> s*cnd(d1)-x*exp(-r*t)*cnd(d2) Let's use the black_scholes function using some various numbers for call and put options. Suppose we want to know the price of an option, where the underlying is a stock traded at $58.60 with an annual volatility of 30%. The risk free interest rate is, let's say, 1%. Then we can use our formula, we defined previously to get the theoretical price according the Black-Scholes formula of a call option with 6 month to maturity (0.5 years): > black_scholes Call 58.60 60.0 0.5 0.01 0.3;; val it : float = 4.465202269 And the value for the put option, just by changing the flag to the function: > black_scholes Put 58.60 60.0 0.5 0.01 0.3;; val it : float = 5.565951021 Sometimes it's more convenient to express the time to maturity in number of days, instead of years. Let's introduce a helper function for that purpose. /// Convert the nr of days to years let days_to_years d = (float d) / 365.25 Note the number 365.25 which includes the factor for leap years. This is not necessary in our examples, but used for correctness. We can now use this function instead, when we know the time in days. > days_to_years 30;; val it : float = 0.08213552361 Let's use the same example above, but now with 20 days to maturity. > black_scholes Call 58.60 60.0 (days_to_years 20) 0.01 0.3;; val it : float = 1.065115482 > black_scholes Put 58.60 60.0 (days_to_years 20) 0.01 0.3;; val it : float = 2.432270266 Using Black-Scholes together with Charts Sometimes it's useful to be able to plot the price of an option until expiration. We can use our previously defined functions and vary the time left and plot the values coming out. In this example we'll make a program that outputs the graph seen below. Chart showing prices for call and put option as function of time /// Plot price of option as function of time left to maturity #r "System.Windows.Forms.DataVisualization.dll" open System open System.Net open System.Windows.Forms open System.Windows.Forms.DataVisualization.Charting open Microsoft.FSharp.Control.WebExtensions /// Create chart and form let chart = new Chart(Dock = DockStyle.Fill) let area = new ChartArea("Main") chart.ChartAreas.Add(area) chart.Legends.Add(new Legend()) let mainForm = new Form(Visible = true, TopMost = true, Width = 700, Height = 500) do mainForm.Text <- "Option price as a function of time" mainForm.Controls.Add(chart) /// Create series for call option price let optionPriceCall = new Series("Call option price") do optionPriceCall.ChartType <- SeriesChartType.Line do optionPriceCall.BorderWidth <- 2 do optionPriceCall.Color <- Drawing.Color.Red chart.Series.Add(optionPriceCall) /// Create series for put option price let optionPricePut = new Series("Put option price") do optionPricePut.ChartType <- SeriesChartType.Line do optionPricePut.BorderWidth <- 2 do optionPricePut.Color <- Drawing.Color.Blue chart.Series.Add(optionPricePut) /// Calculate and plot call option prices let opc = [for x in [(days_to_years 20)..(-(days_to_years 1))..0.0]do yield black_scholes Call 58.60 60.0 x 0.01 0.3] do opc |> Seq.iter (optionPriceCall.Points.Add >> ignore) /// Calculate and plot put option prices let opp = [for x in [(days_to_years 20)..(-(days_to_years 1))..0.0]do yield black_scholes Put 58.60 60.0 x 0.01 0.3] do opp |> Seq.iter (optionPricePut.Points.Add >> ignore) The code is just a modified version of the code seen in the previous article, with the options parts added. We have two series in this chart, one for call options and one for put options. We also add a legend for each of the series. The last part is the calculation of the prices and the actual plotting. List comprehensions are used for compact code, and the Black-Scholes formula is called for everyday until expiration, where the days are counted down by one day at each step. It's up to you as a reader to modify the code to plot various aspects of the option, such as the option price as a function of an increase in the underlying stock price etc. Introducing the greeks The greeks are partial derivatives of the Black-Scholes formula, with respect to a particular parameter such as time, rate, volatility or stock price. The greeks can be divided into two or more categories, with respect to the order of the derivatives. Below we'll look at the first and second order greeks. First order greeks In this section we'll present the first order greeks using the table below. Name Symbol Description Delta Δ Rate of change of option value with respect to change in the price of the underlying asset. Vega ν Rate of change of option value with respect to change in the volatility of the underlying asset. Referred to as the volatility sensitivity. Theta Θ Rate of change of option value with respect to time. The sensitivity with respect to time will decay as time elapses, phenomenon referred to as the "time decay." Rho ρ Rate of change of option value with respect to the interest rate. Second order greeks In this section we'll present the second order greeks using the table below. Name Symbol Description Gamma Γ Rate of change of delta with respect to change in the price of the underlying asset. Veta - Rate of change in Vega with respect to time. Vera - Rate of change in Rho with respect to volatility. Some of the second order greeks are omitted for clarity, we'll not cover these in this book. Implementing the greeks in F# Let's implement the greeks; Delta, Gamma, Vega, Theta and Rho. First we look at the formulas for each greek. In some of the cases they vary for calls and puts respectively. We need the derivative of the cumulative distribution function, which in fact is the normal distribution with zero mean and standard deviation of one: /// Normal distribution open MathNet.Numerics.Distributions; let normd = new Normal(0.0, 1.0) Delta Delta is the rate of change of option price with respect to change in the price of the underlying asset. /// Black-Scholes Delta // call_put_flag: Put | Call // s: stock price // x: strike price of option // t: time to expiration in years // r: risk free interest rate // v: volatility let black_scholes_delta call_put_flag s x t r v = let d1=(log(s / x) + (r+v*v*0.5)*t)/(v*sqrt(t)) match call_put_flag with | Put -> cnd(d1) - 1.0 | Call -> cnd(d1) Gamma Gamma is the rate of change of delta with respect to change in the price of the underlying asset. This is the 2nd derivative, with respect to price of the underlying asset. It measures the acceleration of the price of the option with respect to the underlying price. /// Black-Scholes Gamma // s: stock price // x: strike price of option // t: time to expiration in years // r: risk free interest rate // v: volatility let black_scholes_gamma s x t r v = let d1=(log(s / x) + (r+v*v*0.5)*t)/(v*sqrt(t)) normd.Density(d1) / (s*v*sqrt(t) Vega Vega is the rate of change of option value with respect to change in the volatility of the underlying asset. It is referred to as the volatility sensitivity. /// Black-Scholes Vega // s: stock price // x: strike price of option // t: time to expiration in years // r: risk free interest rate // v: volatility let black_scholes_vega s x t r v = let d1=(log(s / x) + (r+v*v*0.5)*t)/(v*sqrt(t)) s*normd.Density(d1)*sqrt(t) Theta Theta is the rate of change of option value with respect to time. The sensitivity with respect to time will decay as time elapses, phenomenon referred to as the “time decay.” /// Black-Scholes Theta // call_put_flag: Put | Call // s: stock price // x: strike price of option // t: time to expiration in years // r: risk free interest rate // v: volatility let black_scholes_theta call_put_flag s x t r v = let d1=(log(s / x) + (r+v*v*0.5)*t)/(v*sqrt(t)) let d2=d1-v*sqrt(t) let res = ref 0.0 match call_put_flag with | Put -> -(s*normd.Density(d1)*v)/(2.0*sqrt(t))+r*x*exp(-r*t)*cnd(-d2) | Call -> -(s*normd.Density(d1)*v)/(2.0*sqrt(t))-r*x*exp(-r*t)*cnd(d2) Rho Rho is rate of change of option value with respect to the interest rate. /// Black-Scholes Rho // call_put_flag: Put | Call // s: stock price // x: strike price of option // t: time to expiration in years // r: risk free interest rate // v: volatility let black_scholes_rho call_put_flag s x t r v = let d1=(log(s / x) + (r+v*v*0.5)*t)/(v*sqrt(t)) let d2=d1-v*sqrt(t) let res = ref 0.0 match call_put_flag with | Put -> -x*t*exp(-r*t)*cnd(-d2) | Call -> x*t*exp(-r*t)*cnd(d2) Investigating the sensitivity of the of the greeks Now that we have all the greeks implemented we'll investigate the sensitivity of some of them and see how they vary when the underlying stock price changes. The figure below is a surface plot with four of the greeks where time and underlying price is changing. The figure below is generated in MATLAB, and will not be generated in F#. We’ll use a 2D version of the graph to study the greeks below. Surface plot of Delta, Gamma, Theta and Rho of a call option. In this section we'll start by plotting the value of Delta for a call option where we vary the price of the underlying. This will result in the following 2D plot: A plot of call option delta versus price of underlying The result in the plot seen in figure above will be generated by the code presented next. We'll reuse most of the code from the example where we looked at the option prices for calls and puts. A slightly modified version is presented here, where the price of the underlying varies from $10.0 to $70.0. /// Plot delta of call option as function of underlying price #r "System.Windows.Forms.DataVisualization.dll" open System open System.Net open System.Windows.Forms open System.Windows.Forms.DataVisualization.Charting open Microsoft.FSharp.Control.WebExtensions /// Create chart and form let chart = new Chart(Dock = DockStyle.Fill) let area = new ChartArea("Main") chart.ChartAreas.Add(area) chart.Legends.Add(new Legend()) let mainForm = new Form(Visible = true, TopMost = true, Width = 700, Height = 500) do mainForm.Text <- "Option delta as a function of underlying price" mainForm.Controls.Add(chart) /// Create series for call option delta let optionDeltaCall = new Series("Call option delta") do optionDeltaCall.ChartType <- SeriesChartType.Line do optionDeltaCall.BorderWidth <- 2 do optionDeltaCall.Color <- Drawing.Color.Red chart.Series.Add(optionDeltaCall) /// Calculate and plot call delta let opc = [for x in [10.0..1.0..70.0] do yield black_scholes_delta Call x 60.0 0.5 0.01 0.3] do opc |> Seq.iter (optionDeltaCall.Points.Add >> ignore) We can extend the code to plot all four greeks, as in the figure with the surface plots, but here in 2D. The result will be a graph like seen in the figure below. Graph showing the for Greeks for a call option with respect to price change (x-axis). Code listing for visualizing the four greeks Below is the code listing for the entire program used to create the graph above. #r "System.Windows.Forms.DataVisualization.dll" open System open System.Net open System.Windows.Forms open System.Windows.Forms.DataVisualization.Charting open Microsoft.FSharp.Control.WebExtensions /// Create chart and form let chart = new Chart(Dock = DockStyle.Fill) let area = new ChartArea("Main") chart.ChartAreas.Add(area) chart.Legends.Add(new Legend()) let mainForm = new Form(Visible = true, TopMost = true, Width = 700, Height = 500) do mainForm.Text <- "Option delta as a function of underlying price" mainForm.Controls.Add(chart) We’ll create one series for each greek: /// Create series for call option delta let optionDeltaCall = new Series("Call option delta") do optionDeltaCall.ChartType <- SeriesChartType.Line do optionDeltaCall.BorderWidth <- 2 do optionDeltaCall.Color <- Drawing.Color.Red chart.Series.Add(optionDeltaCall) /// Create series for call option gamma let optionGammaCall = new Series("Call option gamma") do optionGammaCall.ChartType <- SeriesChartType.Line do optionGammaCall.BorderWidth <- 2 do optionGammaCall.Color <- Drawing.Color.Blue chart.Series.Add(optionGammaCall) /// Create series for call option theta let optionThetaCall = new Series("Call option theta") do optionThetaCall.ChartType <- SeriesChartType.Line do optionThetaCall.BorderWidth <- 2 do optionThetaCall.Color <- Drawing.Color.Green chart.Series.Add(optionThetaCall) /// Create series for call option vega let optionVegaCall = new Series("Call option vega") do optionVegaCall.ChartType <- SeriesChartType.Line do optionVegaCall.BorderWidth <- 2 do optionVegaCall.Color <- Drawing.Color.Purple chart.Series.Add(optionVegaCall) Next, we’ll calculate the values to plot for each greek: /// Calculate and plot call delta let opd = [for x in [10.0..1.0..70.0] do yield black_scholes_delta Call x 60.0 0.5 0.01 0.3] do opd |> Seq.iter (optionDeltaCall.Points.Add >> ignore) /// Calculate and plot call gamma let opg = [for x in [10.0..1.0..70.0] do yield black_scholes_gamma x 60.0 0.5 0.01 0.3] do opg |> Seq.iter (optionGammaCall.Points.Add >> ignore) /// Calculate and plot call theta let opt = [for x in [10.0..1.0..70.0] do yield black_scholes_theta Call x 60.0 0.5 0.01 0.3] do opt |> Seq.iter (optionThetaCall.Points.Add >> ignore) /// Calculate and plot call vega let opv = [for x in [10.0..1.0..70.0] do yield black_scholes_vega x 60.0 0.1 0.01 0.3] do opv |> Seq.iter (optionVegaCall.Points.Add >> ignore) Summary In this article, we looked into using F# for investigating different aspects of volatility. Volatility is an interesting dimension of finance where you quickly dive into complex theories and models. Here it's very much helpful to have a powerful tool such as F# and F# Interactive. We've just scratched the surface of options and volatility in this article. There is a lot more to cover, but that's outside the scope of this book. Most of the content here will be used in the trading system. resources for article: further resources on this subject: Working with Windows Phone Controls [article] Simplifying Parallelism Complexity in C# [article] Watching Multiple Threads in C# [article]

(For more resources related to this topic, see here.) System requirements The minimum system requirements needed to run Paint.NET are as follows: Windows 7 (recommended) or Windows XP .NET Framework 3.5 SP1 A 800 MHz processor 512 MB of RAM More than 200 MB of hard drive space A 64-bit CPU and a 64-bit edition of Windows for 64-bit support (this is optional) A 1024 x 768 screen resolution At present, there is no Mac version of this product. Downloading and installing Paint.NET To download Paint.NET, go to www.getpaint.net/download.html. In order for Paint.NET to pay a few bills and keep the servers going, the people who maintain Paint.NET have placed a few ads on the site that sometimes lead to other programs. You will most likely not want to download these programs. If you want to download only the Paint.NET program, make sure you click on the link that looks like the following: It can be rather confusing, so choose wisely. Once you hit the correct link, it will take you to one of the Paint.NET mirror download sites. Hit the download link that looks like the following: A ZIP file should begin downloading onto your computer. Once this is done, open the ZIP file and you will find a file named Paint.NET.3.5.11.Install.exe. Run this file and the installation will begin. The installer will take you through a series of steps. For simplicity, choose Quick Installation and then hit Next. Read the terms and conditions, and if you agree to them, choose I Agree and hit Next. As the program installs, it will give you an opportunity to donate to the program if you wish. Because Paint.NET is a freeware, it only survives on donations and the time people put into it. So, if you like the program, donating here is one way to help out. When the software has finished installing, click on Finish and Paint.NET will open automatically. Once the program launches, you will see a screen similar to the following screenshot: This is the entire Paint.NET work area. If you have used photo editing software like Photoshop, some of what you see in the preceding screenshot may look a bit familiar to you. If you are not used to an image editing program, all of these windows may look a little overwhelming. Work area windows Let's take a look at the Paint.NET work area: Paint.NET is comprised of a series of windows, each with their own function to help you with your workflow. The following is a list of the various windows available on Paint.NET; the numbers correspond to the labels given in the previous screenshot: The title bar The menu bar The toolbar The image canvas The Colors window The status bar The Layers window The History window The image list Summary In this article, we learned about Paint.NET and went over the minimum requirements of the software and how to install it. We also learned about the Paint.NET work area and enumerated the various windows associated with it. Resources for Article: Further resources on this subject: Painting – Multi-finger Paint [Article] ExpressionEngine: Creating a Photo Gallery [Article] Photo Stream with iCloud [Article]

(For more resources related to this topic, see here.) Impala Core Components: Here we will discuss following three important components: Impala Daemon Impala Statestore Impala Metadata and Metastore Putting together above components with Hadoop and application or command line interface, we can conceptualize them as below: Impala Execution Architecture: Essentially Impala daemons receives queries from variety of sources and distribute query load to other Impala daemons running on other nodes and while doing so interact with Statestore for node specific update and access Metastore, either stored in centralized database or in local cache. Now to complete the Impala execution we will discuss how Impala interacts with other components i.e. Hive, HDFS and HBase.  Impala working with Apache Hive: We have already discussed earlier about Impala Metastore using the centralized database as Metastore and Hive also uses the same MySQL or PostgreSQL database for same kind of data. Impala provides same SQL like queries interface use in Apache Hive. Because both Impala and Hive share same database as Metastore, Impala can access Hive specific tables definitions if Hive table definition use the same file format, compression codecs and Impala-supported data types in their column values. Apache Hive provides various kinds of file type processing support to Impala. When using other then text file format i.e. RCFile, Avro, SequenceFile the data must be loaded through Hive first and then Impala can query the data from these file formats. Impala can perform read operation on more types of data using SELECT statement than it can perform write operation using INSERT statement. The ANALYZE TABLE statement in Hive generates useful table and column statistics and Impala use these valuable statistics to optimize the queries. Impala working with HDFS: Impala table data is actually regular data files stored in HDFS (Hadoop Distributed File System) and Impala uses HDFS as its primary data storage medium.  As soon as a data file or a collection of files is available in specific folder of new table, Impala reads all of the files regardless of their name and new data is included in files with the name controlled by Impala. HDFS provides data redundancy through replication factor and Impala relies on such redundancy to access data on other datanodes in case it is not available on a specific datanode. We have already learnt earlier that Impala also maintains the information about physical location of the blocks about data files in HDFS,which helps data access in case of node failure. Impala working with HBase: HBase is a distributed, scalable, big data storage system, provides random, real-time read and write access to data stored on HDFS. HBase is a database storage system, sits on top of HDFS however like other traditional database storage system, HBase does not provide built-in SQL support however 3party applications can provide such functionality. To use HBase, first user defines tables in Impala and then maps them to the equivalent HBase tables. Once table relationship is established, users can submit queries into HBase table through Impala. Not only that join operations can be formed including HBase and Impala tables. Impala Security: Impala is designed & developed on run on top of Hadoop. So you must understand the Hadoop security model as well as the security provided in OS where Hadoop is running. If Hadoop is running on Linux then as Linux administrator and Hadoop administrator user can harden and tighten the security, which definitely can be taken in account with the security provided by Impala. Impala 1.1 or later uses Sentry Open Source Project to provide detailed authorization framework for Hadoop. Impala 1.1.1 supports auditing capabilities in cluster by creating auditing data, which can be collected from all nodes and then processing for further analysis and insight. Data Visualization using Impala: Visualizing data is as important as processing the data. Human brain perceives pictures fast then reading data in tables and because of it data visualization provides super fast understanding to large amount of data in split seconds. Reports, charts, interactive dashboards and any form of info-graphics are all part of data visualization and provide deeper understanding of results. To connect with 3rd party applications, Cloudera provides ODBC and JDBC connectors. These connectors are installed on machines where 3rd party applications are running and by configuring correct Impala server and port details on those connectors, 3rd party applications connect with Impala and submit those queries and then take results back to application. The result then displayed on 3rd party application, where it is rendered on graphics device for visualization or displayed in table format or processed further depending on application requirement. In this section we will cover few notable 3rd party applications, which can take advantage of Impala super fast query processing and than display amazing graphical results. Tableau and Impala: Tableau Software supporting Impala by providing access to tables on Impala using Impala ODBC connector provided by Tableau. Tableau is one of the most prominent data visualization software technologies in recent days and used by thousands of enterprises daily to get intelligence out of their data. Tableau software is available on Windows OS and an ODBC connector is provided by Cloudera to make this connection a reality. You can visit the link below to download Impala connector for Tableau: http://go.cloudera.com/tableau_connector_download Once Impala connector is installed on a machine where Tableau software is running, and configured correctly, Tableau software is ready to work with Impala. In this image below Tableau is connected to Impala server at port 21000, and then selected a table located at Impala: Once table is selected, particular fields are select and data is displayed in graphical format in various mind-blowing visualizations. The screenshot below displays one example of showing such visualization:   Microsoft Excel and Impala Microsoft Excel is one of the widely adopted data processing application used by business professional worldwide. You can connect Microsoft Excel with Impala using another ODBC connector provided by Simba Technology. Microstrategy and Impala Microstrategy is another big player in data analysis and visualization software and uses ODBC drive to connect with Impala to render amazing looking visualizations. The connectivity model between Microstrategy software and Cloudera Impala is shown as below:    Zoomdata and Impala: Zoomdata is considered to new generation of data user interface by addressing streams of data instead of sets of data. Zoomdata processing engine performs continuous mathematical operations across data streams in real-time to create visualization on multitude of devices. The visualization updates itself as the new data arrives and re-computed by Zoomdata. As shown in in the image below, you can see Zoomdata application uses Impala as a source of data, which is configured underneath to use of one the available connectors to connect with Impala: Once connection are made user can see amazing data visualization as shown below: Real-time Query with Impala on Hadoop: Impala is marketed as a product, which can do “Real-time queries on Hadoop” by its developer Cloudera. Impala is open source implementation based on above-mentioned Google Dremel technology, available free for anyone of use. Impala is available as package product, free to use or can be compiled from its source, which can run queries in memory to make them real-time and in some cases depending on type of data, if Parquet file format is used as input data source, it can expedite the query processing to multifold speed.  Real-time query subscription with Impala: Cloudera provides Real-time Query (RTQ) Subscription as an add-on to Cloudera Enterprise subscription. You can still use Impala as free open source product however taking RTQ subscription makes you take advantage of Cloudera paid service to extend its usability and resilience. By accepting RTQ subscription you cannot only have access to Cloudera Technical support, but also you can work with Impala development team to provide ample feedback to shape up the product design and implementation. Summary Thus concludes the discussion on the key components of Impala and their inner working. Resources for Article: Further resources on this subject: Securing the Hadoop Ecosystem [Article] Cloudera Hadoop and HP Vertica [Article] Hadoop and HDInsight in a Heartbeat [Article]

(For more resources related to this topic, see here.) Why use Lumion? The short answer is that Lumion is easy to use and the final product is of a good quality. The long answer is that every construction project needs technical drawings and documents. Although this technical information is fine for a construction crew, usually the client has no idea what a CAD plan means. They can have an idea where the kitchen or the living room will be, but translating that 2D information to 3D is not always easy in the client's mind. This can be an issue if we need to give a presentation or if we are trying to sell something that is not built yet. And truth be told, an image sells more than words. That's where Lumion comes in. Lumion is the fastest way to render high quality still pictures and videos, and it makes it so easy to import our 3D models from any 3D modeling software, such as SketchUp, AutoCAD, Revit, ArchiCAD, and 3ds Max, and create a scene in minutes. So, Lumion 3D is a distinct architectural visualization software not only because it is faster to render, but also because it is very user friendly and intuitive. Another reason why we can use Lumion to create architectural visualizations is because we can have a great idea of how our project will look in natural surroundings at any time of the day or season, and this in just a few minutes. Now if you are an architect, it is doubtless that you want to enhance your project characteristics in the best way possible. Lumion can help you achieve this in hours instead of the inevitable days and weeks of rendering time. The following screenshot is an example of what you can get with Lumion in just a few minutes: However, this tool is not exclusively meant for architects. For example, if you are an interior designer, you may want to present how the textures, furniture, and colors would look at different angles, places, moods, and light conditions. Lumion provides nice interior visualization with furniture, good lighting, and realistic textures. In conclusion, Lumion is a great tool that improves the process of creating a building, or an art, or an architectural project. The time we need to get those results is less in comparison to other solutions such as using 3ds Max and V-Ray. What can we get from Lumion? Asking what Lumion can us is a double-edged question. Looking at the previous screenshots, we can get an idea of the final result. The final quality depends only on your creativity and time. I have seen amazing videos created with Lumion, but then you may need a touch of other software to create eye-catching compositions. Now, the package that we get with Lumion is another thing. You already know that we can easily create beautiful still images and videos, but we need to bear in mind that Lumion is not a tool designed to create photo-realistic renders. Nevertheless, we can get so much from this application that you will forget photo-realistic renders. Lumion is a powerful and advanced 3D engine that allows us to work with heavier models, and we can make our scene come alive with the click-and-drag technique. To do this, Lumion comes with a massive library where we can find: 2409 models of trees, transports, exterior and interior models, and people 103 ambient, people, and machine sounds 518 types of materials and textures 28 types of landscape In addition to this extensive collection, there are more features that we can add; we can include realistic water in our scene (oceans, rivers, pools, waterfalls, and fountains), we can sculpt the landscape to adapt to our needs, and we can add rain, snow, fog, wind, and animate objects, and we can add camera effects. You just need a blank 3D model; import and start working because, as you can see, Lumion is well equipped with almost everything we need to create architectural visualisations. Lumion's 3D interface Now that we know what we can do with Lumion and the content available, we will take some time to explore Lumion and get our hands dirty. In my opinion and experience, it is much easier to learn something if at the same time we apply what we are learning. So, in the next section we are going to explore the Lumion interface with the menus and different settings. But to do that we will use a small tutorial as a quick start. By doing this, we will explore Lumion and at the same time see how easy it is to produce great results. We will see that Lumion is easier to learn and more accessible than other software. So go ahead and fire up Lumion and let's have a quick tour before we start working with it. I am going to explain to you what each tab does, to help you see how you can do simple tasks, such as saving and loading a scene, changing the settings, and creating a new scene. Let's start with the first tab that Lumion shows us. A look into the New tab On startup, Lumion goes straight to the New tab. The New tab, as the name indicates, is a good place to start when you want to create a new scene. We can create a new scene based on a few presets or just create an empty scene. We can choose from Night, Sunset, Sunny Day, Flatlands, Island, Lake, Desert, Cold Climate, and an Empty scene. I found these presets as a quick help to cut some time, because in the end everything we get from these presets, we can create in a few minutes. So, there is nothing special about them. When you start Lumion, this will be the first thing you will see: The nine presets you can find on the New tab Now, we will finally see the Objects menu and the following is what this menu looks like: The Objects menu Here is where the fun starts. We have at our disposal eight categories of objects and more, such as Nature, Transport, Sound, Effects, Indoor, People and Animals, Outdoor, and Lights and special objects. Each of these menus has subcategories. If you are working with the Lumion PRO version, you can choose from more than 2000 models. Even if you don't have this version, cheer up! You can still import your own models and textures. It is really simple to add a model. First, we need to select the category we want to use. So in this case, click on the Nature button. Now that we have this category selected, click on the thumbnail above the Place button and a new window with the Nature library will appear. We don't have just trees, we have grass, plants, flowers, rocks, and clusters. Now let me show you one trick. Click on the Grass tab and select Rough_Grass1_RT. Now that we are back to the Build mode, press the Ctrl key and click on the ground. We are randomly adding 10 copies of the object, which in this case is really handy. So, after playing a little with Lumion, we can get something like the following screenshot: Our scene after adding some trees, grass, and animals Just think, it took me about 30 minutes to create something like this. Now imagine what you can do. Let's save our scene and turn our attention to the right-hand side of the Lumion 3D interface, where we can find the menus as shown in the following screenshot: The Build mode button Starting from the top of the preceding screenshot we can see the blue rectangle with a question mark. If we put our mouse cursor over this rectangle, we can see a quick help guideline for our interface. The next button informs us that we are in the Build mode and if, for example, you are working in the Photo or Video mode, this button lets you go back to the scene. Lumion materials Lumion helps us with this important step by offering many types of materials: 518 materials that are ready to use. You may need to do some adjustments, but the major hard work was already done for you. The materials that we can assign to our model are as follows: Wood: 45 materials Wood floor : 67 materials Brick: 32 materials Tiles: 99 materials Ground: 39 materials Concrete: 43 materials Carpet: 20 materials Misc: 109 materials Asphalt: 12 materials Metal: 47 materials This is one of the reasons why it is so easy to create still images and videos with Lumion. Everything is set up for you, including parameters, details, and textures. However, we may need to do some minor adjustments, and for that, it is important to understand or at least have a basic notion of what each setting does. So, let's have a quick look at how we can configure materials in Lumion. The Landscape material The best way to explain this material is by showing you an example. So, let's say that along with the model, you also created a terrain like the one you can see in the following screenshot: The house along with a terrain model Import the terrain if needed, and add a new material to this terrain. Go to the Custom menu and click on the Landscape material. The Landscape material allows you to seamlessly blend or merge parts of the model with the landscape. Make sure that the terrain intersects with the ground so that it can be perfectly blended. The following screenshot shows this Landscape material applied to my 3D terrain: The terrain model merged with the landscape Adding this material not only allows you to use this cool feature, but as you can see in the picture, we can also start painting soil types of the landscape in the imported terrain. The other two materials that I want to introduce you to are the Standard and Water materials. The Standard material is a simple material without any textures or settings, and we can use this material to start something from scratch. The Water material can have several applications, but perhaps, the most common one is, for example, pools. Summary This article helped you in starting with Lumion, and gave you a taste of how easily and quickly you can get great images and videos. In particular, you have learned the basic steps to save and load scenes, import models, add materials, change the terrain and weather, and create photos and videos. You also learned how to use and configure the prebuilt materials in Lumion and found out how to use the Landscape material to create a terrain. Resources for Article: Further resources on this subject: The Spotfire Architecture Overview [Article] augmentedTi: The application architecture [Article] The architecture of JavaScriptMVC [Article]

(For more resources related to this topic, see here.) The final objective of the information in the BI reports and dashboards is to detect the cause-effect business behavior and trends, and trigger actions to solve them. These actions supported by visual information, via scorecards and dashboards. This process requires an interaction with several people. MicroStrategy includes the functionality to share our reports, scorecards, and dashboards, regardless of the location of the people. Reaching your audience MicroStrategy offers the option to share our reports via different channels that leverage the latest social technologies that are already present in the marketplace, that is, MicroStrategy integrates with Twitter and Facebook. The sharing is like avoiding any related costs and maintaining the design premise of the do-it-yourself approach without any help from specialized IT personnel. Main menu The main menu of MicroStrategy shows a column named Status. When we click on that column, as shown in the following screenshot, the Share option appears: The Share button The other option is the Share button within our reports, that is, the view that we want to share. Select the Share button located at the bottom of the screen, as shown in the following screenshot: The share options are the same, regardless of the location where you activate the option; the various alternate menus are shown in the following screenshot: E-mail sharing While selecting the e-mail option from the Scorecards-Dashboards model, the system will ask you for the e-mail programs that you want to use in order to send an e-mail; in our case, we select Outlook. MicroStrategy automatically prepares an e-mail with a link to share it. You can modify the text, and select the recipients of the e-mail, as shown in the following screenshot: The recipients of the e-mail will click on the URL that is included in the e-mail, send it by this schema, and the user will be able to analyze the report in a read-only mode with only the Filters panel enabled. The following screenshot shows how the user will review the report. Also, the user is not allowed to make any modifications. This option does not require a MicroStrategy platform user account. When a user clicks on the link, he is able to edit the filters and perform their analyses, as well as switch to any available layout, in our case, scorecards and dashboards. As a result, any visualization object can be maximized and minimized for better analysis, as shown in the following screenshot: In this option, the report can be visualized in a fullscreen mode by clicking on the fullscreen button [] located at the top-right corner of the screen. In this sharing mode, the user is able to download the information in Excel and PDF formats for each visualization object. For instance, if you need all the data included in the grid for the stores in region 1 opened in the year 2000. Perform the following steps: In the browser, open the URL that is generated when you select the e-mail share option. Select the ScoreCard tab. In the Open Year filter, type 2012 and in the Region filter, type 1. Now, maximize the grid. Two icons will appear in the top-left corner of the screen: one for exporting the data to Excel and the other for exporting it to PDF for each visualization object, as shown in the following screenshot: Please keep in mind that these two export options only apply to a specific visualization object; it is not possible to export the complete report from this functionality that is offered to the consumer. Summary In this article, we learned how to share our scorecards and dashboards via several channels, such as e-mails, social networks (Twitter and Facebook), and blogs or corporate intranet sites. Resources for Article: Further resources on this subject: Participating in a business process (Intermediate) [Article] Self-service Business Intelligence, Creating Value from Data [Article] Exploring Financial Reporting and Analysis [Article]

(For more resources related to this topic, see here.) PWM allows us to drive a certain pin to on and off at desired frequency and duty cycle. This way we can pulse our LEDs much faster than our eyes can react, and while we only see a dimmer or a brighter LED, if one would look at it with a high speed camera, one would see that the LED still only turns on or off. Our eyes just perceive this differently. There are three basic terms you need to understand about PWM: Frequency: This defines how many full on/off "cycles" are generated in a second Period: This defines how long one complete pulse cycle takes Duty cycle: This specifies the time the signal is high during one full period Think about the following figure: In the preceding figure, we have PWM generating first a 50 percent duty cycle and then dropping to 25 percent. The effective time the LED spends as on and off can be controlled with very high precision, and this way we can achieve smooth brightness fluctuations. Let's try doing just that. First, we will design a schematic with two LEDs that are connected to two different PWMs. Nothing fancy here either really, we have a current limiting resistor after the LEDs and that's it. This time we will be using input from both headers, as PWM1 and PWM2 are located on P9 and P8, respectively. Our third and last program in this article will be called racing_PWMs.py. As usual, we need to include the Adafruit library here as well, this time the PWM part of it: import Adafruit_BBIO.PWM as PWM When you initialize a PWM, you can initialize it with two to four parameters listed as follows: Channel (header pin) Duty (as in percent, 0-100) Frequency (periods in a second, default is 2000) And polarity (0 or 1. With this you can invert the duty cycle, default is 0) So, we will initialize both our channels: PWM.start("P9_14", 50, 100) #50% duty and 100hz cycle PWM.start("P8_13", 50, 100) At this point, the LEDs will light up, and now we can start changing our duty cycle in a loop to adjust the average voltage. Full listing of racing_PWMs.py is as follows: #!/usr/bin/python import time import Adafruit_BBIO.PWM as PWM sleep_time=0.005 #The lower the value the faster the activity PWM.start("P9_14", 50, 100) #50% duty and 100hz cycle PWM.start("P8_13", 50, 100) while True: for i in range(100,1, -1): PWM.set_duty_cycle("P9_14", i) # Dimming PWM.set_duty_cycle("P8_13", abs(i-100)+1) # Getting brighter time.sleep(sleep_time) for i in range(1, 100): PWM.set_duty_cycle("P9_14", i) PWM.set_duty_cycle("P8_13", abs(i-100)+1 ) time.sleep(sleep_time) When you run the program, you should see both LEDs racing (blinking) in opposite phases. As you see, the Adafruit BBIO library is extremely useful and easy to use. And so far we have only used two functionalities it provides. Actually, the library also supports easy access to SPI and I2C communication and Analog to Digital Converter(ADC) as well. Summary In this article we went through foundations of input and output on a very basic level. We talked about the general purpose I/O pins, and how they can be used to control external components, such as LEDs or buttons. As you must have gathered already, with proper application of voltage and care, we can operate many of the basic electronic components. You should now feel comfortable with the basic building blocks that can help you build some external inputs and outputs to your Beagle programs. Resources for Article: Further resources on this subject: Home Security by BeagleBone [Article] Introducing BeagleBoard [Article] Viewing on Mobile Devices [Article]