How-To Tutorials

Adding the tank model For tank battles, we will be using a 3D model available for download from the App Hub website (http://create.msdn.com) in the Simple Animation CODE SAMPLE available at http://xbox.create.msdn.com/en-US/education/catalog/sample/simple_animation. Our first step will be to add the model to our content project in order to bring it into the game. Time for action – adding the tank model We can add the tank model to our project by following these steps: Download the 7089_06_GRAPHICSPACK.ZIP file from the book's website and extract the contents to a temporary folder. Select the .fbx file and the two .tga files from the archive and copy them to the Windows clipboard. Switch to Visual Studio and expand the Tank BattlesContent (Content) project. Right-click on the Models folder and select Paste to copy the files on the clipboard into the folder. Right-click on engine_diff_tex.tga inside the Models folder and select Exclude From Project. Right click on turret_alt_diff_tex.tga inside the Models folder and select Exclude From Project. What just happened? Adding a model to our game is like adding any other type of content, though there are a couple of pitfalls to watch out for. Our model includes two image files (the .tga files&emdash;an image format commonly associated with 3D graphics files because the format is not encumbered by patents) that will provide texture maps for the tank's surfaces. Unlike the other textures we have used, we do not want to include them as part of our content project. Why not? The content processor for models will parse the .fbx file (an Autodesk file format used by several 3D modeling packages) at compile time and look for the textures it references in the directory the model is in. It will automatically process these into .xnb files that are placed in the output folder &endash; Models, for our game. If we were to also include these textures in our content project, the standard texture processor would convert the image just like it does with the textures we normally use. When the model processor comes along and tries to convert the texture, an .xnb file with the same name will already exist in the Models folder, causing compile time errors. Incidentally, even though the images associated with our model are not included in our content project directly, they still get built by the content pipeline and stored in the output directory as .xnb files. They can be loaded just like any other Texture2D object with the Content.Load() method. Free 3D modeling software There are a number of freely available 3D modeling packages downloadable on the Web that you can use to create your own 3D content. Some of these include:   Blender: A free, open source 3D modeling and animation package. Feature rich, and very powerful. Blender can be found at http://www.blender.org. Wings 3D: Free, open source 3D modeling package. Does not support animation, but includes many useful modeling features. Wings 3D can be found at http://wings3d.com. Softimage Mod Tool: A modeling and animation package from Autodesk. The Softimage Mod Tool is available freely for non-commercial use. A version with a commercial-friendly license is also available to XNA Creator's Club members at http://usa.autodesk.com/adsk/servlet/pc/item?id=13571257&siteID=123112.         Building tanks Now that the model is part of our project, we need to create a class that will manage everything about a tank. While we could simply load the model in our TankBattlesGame class, we need more than one tank, and duplicating all of the items necessary to handle both tanks does not make sense. Time for action – building the Tank class We can build the Tank class using the following steps: Add a new class file called Tank.cs to the Tank Battles project. Add the following using directives to the top of the Tank.cs class file: using Microsoft.Xna.Framework; using Microsoft.Xna.Framework.Graphics; Add the following fields to the Tank class: #region Fields private Model model; private GraphicsDevice device; private Vector3 position; private float tankRotation; private float turretRotation; private float gunElevation; private Matrix baseTurretTransform; private Matrix baseGunTransform; private Matrix[] boneTransforms; #endregion Add the following properties to the Tank class: #region Properties public Vector3 Position { get { return position; } set { position = value; } } public float TankRotation { get { return tankRotation; } set { tankRotation = MathHelper.WrapAngle(value); } } public float TurretRotation { get { return turretRotation; } set { turretRotation = MathHelper.WrapAngle(value); } } public float GunElevation { get { return gunElevation; } set { gunElevation = MathHelper.Clamp( value, MathHelper.ToRadians(-90), MathHelper.ToRadians(0)); } } #endregion Add the Draw() method to the Tank class, as follows: #region Draw public void Draw(ArcBallCamera camera) { model.Root.Transform = Matrix.Identity * Matrix.CreateScale(0.005f) * Matrix.CreateRotationY(TankRotation) * Matrix.CreateTranslation(Position); model.CopyAbsoluteBoneTransformsTo(boneTransforms); foreach (ModelMesh mesh in model.Meshes) { foreach (BasicEffect basicEffect in mesh.Effects) { basicEffect.World = boneTransforms[mesh.ParentBone. Index]; basicEffect.View = camera.View; basicEffect.Projection = camera.Projection; basicEffect.EnableDefaultLighting(); } mesh.Draw(); } } #endregion In the declarations area of the TankBattlesGame class, add a new List object to hold a list of Tank objects, as follows: List tanks = new List(); Create a temporary tank so we can see it in action by adding the following to the end of the LoadContent() method of the TankBattlesGame class: tanks.Add( new Tank( GraphicsDevice, Content.Load(@"Modelstank"), new Vector3(61, 40, 61))); In the Draw() method of the TankBattlesGame class, add a loop to draw all of the Tank objects in the tank's list after the terrain has been drawn, as follows: foreach (Tank tank in tanks) { tank.Draw(camera); } Execute the game. Use your mouse to rotate and zoom in on the tank floating above the top of the central mountain in the scene, as shown in the following screenshot: What just happened? The Tank class stores the model that will be used to draw the tank in the model field. Just as with our terrain, we need a reference to the game's GraphicsDevice in order to draw our model when necessary. In addition to this information, we have fields (and corresponding properties) to represent the position of the tank, and the rotation angle of three components of the model. The first, TankRotation, determines the angle at which the entire tank is rotated. As the turret of the tank can rotate independently of the direction in which the tank itself is facing, we store the rotation angle of the turret in TurretRotation. Both TankRotation and TurretRotation contain code in their property setters to wrap their angles around if we go past a full circle in either direction. The last angle we want to track is the elevation angle of the gun attached to the turret. This angle can range from 0 degrees (pointing straight out from the side of the turret) to -90 degrees (pointing straight up). This angle is stored in the GunElevation property. The last field added in step 3 is called boneTransforms, and is an array of matrices. We further define this array while defining the Tank class' constructor by creating an empty array with a number of elements equal to the number of bones in the model. But what exactly are bones? When a 3D artist creates a model, they can define joints that determine how the various pieces of the model are connected. This process is referred to as "rigging" the model, and a model that has been set up this way is sometimes referred to as "rigged for animation". The bones in the model are defined with relationships to each other, so that when a bone higher up in the hierarchy moves, all of the lower bones are moved in relation to it. Think for a moment of one of your fingers. It is composed of three distinct bones separated by joints. If you move the bone nearest to your palm, the other two bones move as well – they have to if your finger bones are going to stay connected! The same is true of the components in our tank. When the tank rotates, all of its pieces rotate as well. Rotating the turret moves the cannon, but has no effect on the body or the wheels. Moving the cannon has no effect on any other parts of the model, but it is hinged at its base, so that rotating the cannon joint makes the cannon appear to elevate up and down around one end instead of spinning around its center. We will come back to these bones in just a moment, but let's first look at the current Draw() method before we expand it to account for bone-based animation. Model.Root refers to the highest level bone in the model's hierarchy. Transforming this bone will transform the entire model, so our basic scaling, rotation, and positioning happen here. Notice that we are drastically scaling down the model of the tank, to a scale of 0.005f. The tank model is quite large in raw units, so we need to scale it to a size that is in line with the scale we used for our terrain. Next, we use the boneTransforms array we created earlier by calling the model's CopyAbsoluteBoneTransformsTo() method. This method calculates the resultant transforms for each of the bones in the model, taking into account all of the parent bones above it, and copies these values into the specified array. We then loop through each mesh in the model. A mesh is an independent piece of the model, representing a movable part. Each of these meshes can have multiple effects tied to it, so we loop through those as well, using an instance of BasicEffect created on the spot to render the meshes. In order to render each mesh, we establish the mesh's world location by looking up the mesh's parent bone transformation and storing it in the World matrix. We apply our View and Projection matrices just like before, and enable default lighting on the effect. Finally, we draw the mesh, which sends the triangles making up this portion of the model out to the graphics card. The tank model The tank model we are using is from the Simple Animation sample for XNA 4.0, available on Microsoft's MSDN website at http://xbox.create.msdn.com/en-US/education/catalog/sample/simple_animation. Bringing things down to earth You might have noticed that our tank is not actually sitting on the ground. In fact, we have set our terrain scaling so that the highest point in the terrain is at 30 units, while the tank is positioned at 40 units above the X-Z plane. Given a (X,Z) coordinate pair, we need to come up with a way to determine what height we should place our tank at, based on the terrain. Time for action – terrain heights To place our tank appropriately on the terrain, we first need to calculate, then place our tank there. This is done in the following steps: Add a helper method to the Terrain class to calculate the height based on a given coordinate as follows: #region Helper Methods public float GetHeight(float x, float z) { int xmin = (int)Math.Floor(x); int xmax = xmin + 1; int zmin = (int)Math.Floor(z); int zmax = zmin + 1; if ( (xmin < 0) || (zmin < 0) || (xmax > heights.GetUpperBound(0)) || (zmax > heights.GetUpperBound(1))) { return 0; } Vector3 p1 = new Vector3(xmin, heights[xmin, zmax], zmax); Vector3 p2 = new Vector3(xmax, heights[xmax, zmin], zmin); Vector3 p3; if ((x - xmin) + (z - zmin) <= 1) { p3 = new Vector3(xmin, heights[xmin, zmin], zmin); } else { p3 = new Vector3(xmax, heights[xmax, zmax], zmax); } Plane plane = new Plane(p1, p2, p3); Ray ray = new Ray(new Vector3(x, 0, z), Vector3.Up); float? height = ray.Intersects(plane); return height.HasValue ? height.Value : 0f; } #endregion In the LoadContent() method of the TankBattlesGame class, modify the statement that adds a tank to the battlefield to utilize the GetHeight() method as follows: tanks.Add( new Tank( GraphicsDevice, Content.Load(@"Modelstank"), new Vector3(61, terrain.GetHeight(61,61), 61))); Execute the game and view the tank, now placed on the terrain as shown in the following screenshot: What just happened? You might be tempted to simply grab the nearest (X, Z) coordinate from the heights[] array in the Terrain class and use that as the height for the tank. In fact, in many cases that might work. You could also average the four surrounding points and use that height, which would account for very steep slopes. The drawbacks with those approaches will not be entirely evident in Tank Battles, as our tanks are stationary. If the tanks were mobile, you would see the elevation of the tank jump between heights jarringly as the tank moved across the terrain because each virtual square of terrain that the tank entered would have only one height. In the GetHeight() method that we just saw, we take a different approach. Recall that the way our terrain is laid out, it grows along the positive X and Z axes. If we imagine looking down from a positive Y height onto our terrain with an orientation where the X axis grows to the right and the Z axis grows downward, we would have something like the following: As we discussed when we created our index buffer, our terrain is divided up into squares whose corners are exactly 1 unit apart. Unfortunately, these squares do not help us in determining the exact height of any given point, because each of the four points of the square can theoretically have any height from 0 to 30 in the case of our terrain scale. Remember though, that each square is divided into two triangles. The triangle is the basic unit of drawing for our 3D graphics. Each triangle is composed of three points, and we know that three points can be used to define a plane. We can use XNA's Plane class to represent the plane defined by an individual triangle on our terrain mesh. To do so, we just need to know which triangle we want to use to create the plane. In order to determine this, we first get the (X, Z) coordinates (relative to the view in the preceding figure) of the upper-left corner of the square our point is located in. We determine this point by dropping any fractional part of the x and z coordinates and storing the values in xmin and zmin for later use. We check to make sure that the values we will be looking up in the heights[] array are valid (greater than zero and less than or equal to the highest element in each direction in the array). This could happen if we ask for the height of a position that is outside the bounds of our map's height. Instead of crashing the game, we will simply return a zero. It should not happen in our code, but it is better to account for the possibility than be surprised later. We define three points, represented as Vector3 values p1, p2, and p3. We can see right away that no matter which of the two triangles we pick, the (xmax, zmin) and (xmin, zmax) points will be included in our plane, so their values are set right away. To decide which of the final two points to use, we need to determine which side of the central dividing line the point we are looking for lies in. This actually turns out to be fairly simple to do for the squares we are using. In the case of our triangle, if we eliminate the integer portion of our X and Z coordinates (leaving only the fractional part that tells us how far into the square we are), the sum of both of these values will be less than or equal to the size of one grid square (1 in our case) if we are in the upper left triangle. Otherwise our point is in the right triangle. The code if ((x - xmin) + (z - zmin) <= 1) performs this check, and sets the value of p3 to either (xmin, zmin) or (xmax, zmax) depending on the result. Once we have our three points, we ask XNA to construct a Plane using them, and then we construct another new type of object we have not yet used – an object of the Ray class. A Ray has a base point, represented by a Vector3, and a direction – also represented by a Vector3. Think of a Ray as an infinitely long arrow that starts somewhere in our world and heads off in a given direction forever. In the case of the Ray we are using, the starting point is at the zero point on the Y axis, and the coordinates we passed into the method for X and Z. We specify Vector3.Up as the direction the Ray is pointing in. Remember from the FPS camera that Vector3.Up has an actual value of (0, 1, 0), or pointing up along the positive Y axis. The Ray class has an Intersects() method that returns the distance from the origin point along the Ray where the Ray intersects a given Plane. We must assign the return value of this method to a float? instead of a normal float. You may not be familiar with this notation, but the question mark at the end of the type specifies that the value is nullable—that is, it might contain a value, but it could also just contain a null value. In the case of the Ray.Intersects() method, the method will return null if the object of Ray class does not intersect the object of the Plane class at any point. This should never happen with our terrain height code, but we need to account for the possibility. When using a nullable float, we need to check to make sure that the variable actually has a value before trying to use it. In this case, we use the HasValue property of the variable. If it does have one, we return it. Otherwise we return a default value of zero.

Python Geospatial Development Build a complete and sophisticated mapping application from scratch using Python tools for GIS development Build applications for GIS development using Python Analyze and visualize Geo-Spatial data Comprehensive coverage of key GIS concepts Recommended best practices for storing spatial data in a database Draw maps, place data points onto a map, and interact with maps A practical tutorial with plenty of step-by-step instructions to help you develop a mapping application from scratch         Read more about this book       (For more resources on Python, see here.) Working with Shapely geometries Shapely is a very capable library for performing various calculations on geo-spatial data. Let's put it through its paces with a complex, real-world problem. Task: Identify parks in or near urban areas The U.S. Census Bureau makes available a Shapefile containing something called Core Based Statistical Areas (CBSAs), which are polygons defining urban areas with a population of 10,000 or more. At the same time, the GNIS website provides lists of placenames and other details. Using these two datasources, we will identify any parks within or close to an urban area. Because of the volume of data we are potentially dealing with, we will limit our search to California. Feel free to download the larger data sets if you want, though you will have to optimize the code or your program will take a very long time to check all the CBSA polygon/placename combinations. Let's start by downloading the necessary data. Go to the TIGER website at http://census.gov/geo/www/tiger Click on the 2009 TIGER/Line Shapefiles Main Page link, then follow the Download the 2009 TIGER/Line Shapefiles now link. Choose California from the pop-up menu on the right, and click on Submit. A list of the California Shapefiles will be displayed; the Shapefile you want is labelled Metropolitan/Micropolitan Statistical Area. Click on this link, and you will download a file named tl_2009_06_cbsa.zip. Once the file has downloaded, uncompress it and place the resulting Shapefile into a convenient location so that you can work with it. You now need to download the GNIS placename data for California. Go to the GNIS website: http://geonames.usgs.gov/domestic Click on the Download Domestic Names hyperlink, and then choose California from the pop-up menu. You will be prompted to save the CA_Features_XXX.zip file. Do so, then decompress it and place the resulting CA_Features_XXX.txt file into a convenient place. The XXX in the above file name is a date stamp, and will vary depending on when you download the data. Just remember the name of the file as you'll need to refer to it in your source code. We're now ready to write the code. Let's start by reading through the CBSA urban area Shapefile and extracting the polygons that define the boundary of each urban area: shapefile = osgeo.ogr.Open("tl_2009_06_cbsa.shp")layer = shapefile.GetLayer(0)for i in range(layer.GetFeatureCount()): feature = layer.GetFeature(i) geometry = feature.GetGeometryRef() ... Make sure you add directory paths to your osgeo.ogr.Open() statement (and to the file() statement below) to match where you've placed these files. Using what we learned in the previous section, we can convert this geometry into a Shapely object so that we can work with it: wkt = geometry.ExportToWkt()shape = shapely.wkt.loads(wkt) Next, we need to scan through the CA_Features_XXX.txt file to identify the features marked as a park. For each of these features, we want to extract the name of the feature and its associated latitude and longitude. Here's how we might do this: f = file("CA_Features_XXX.txt", "r")for line in f.readlines(): chunks = line.rstrip().split("|") if chunks[2] == "Park": name = chunks[1] latitude = float(chunks[9]) longitude = float(chunks[10]) ... Remember that the GNIS placename database is a pipedelimited text file. That's why we have to split the line up using line.rstrip().split("|"). Now comes the fun part—we need to figure out which parks are within or close to each urban area. There are two ways we could do this, either of which will work: We could use the shape.distance() method to calculate the distance between the shape and a Point object representing the park's location: We could dilate the polygon using the shape.buffer() method, and then see if the resulting polygon contained the desired point: The second option is faster when dealing with a large number of points as we can pre-calculate the dilated polygons and then use them to compare against each point in turn. Let's take this option: # findNearbyParks.pyimport osgeo.ogrimport shapely.geometryimport shapely.wktMAX_DISTANCE = 0.1 # Angular distance; approx 10 km.print "Loading urban areas..."urbanAreas = {} # Maps area name to Shapely polygon.shapefile = osgeo.ogr.Open("tl_2009_06_cbsa.shp")layer = shapefile.GetLayer(0)for i in range(layer.GetFeatureCount()): feature = layer.GetFeature(i) name = feature.GetField("NAME") geometry = feature.GetGeometryRef() shape = shapely.wkt.loads(geometry.ExportToWkt()) dilatedShape = shape.buffer(MAX_DISTANCE) urbanAreas[name] = dilatedShapeprint "Checking parks..."f = file("CA_Features_XXX.txt", "r")for line in f.readlines(): chunks = line.rstrip().split("|") if chunks[2] == "Park": parkName = chunks[1] latitude = float(chunks[9]) longitude = float(chunks[10]) pt = shapely.geometry.Point(longitude, latitude) for urbanName,urbanArea in urbanAreas.items(): if urbanArea.contains(pt): print parkName + " is in or near " + urbanNamef.close() Don't forget to change the name of the CA_Features_XXX.txt file to match the actual name of the file you downloaded. You may also need to change the path names to the tl_2009_06_CBSA.shp file and the CA_Features file if you placed them in a different directory. If you run this program, you will get a master list of all the parks that are in or close to an urban area: % python findNearbyParks.pyLoading urban areas...Checking parks...Imperial National Wildlife Refuge is in or near El Centro, CATwinLakesStateBeach is in or near Santa Cruz-Watsonville, CAAdmiralWilliamStandleyState Recreation Area is in or near Ukiah, CAAgate Beach County Park is in or near San Francisco-Oakland-Fremont, CA... Note that our program uses angular distances to decide if a park is in or near a given urban area. An angular distance is the angle (in decimal degrees) between two rays going out from the center of the Earth to the Earth's surface. Because a degree of angular measurement (at least for the latitudes we are dealing with here) roughly equals 100 km on the Earth's surface, an angular measurement of 0.1 roughly equals a real distance of 10 km. Using angular measurements makes the distance calculation easy and quick to calculate, though it doesn't give an exact distance on the Earth's surface. If your application requires exact distances, you could start by using an angular distance to filter out the features obviously too far away, and then obtain an exact result for the remaining features by calculating the point on the polygon's boundary that is closest to the desired point, and then calculating the linear distance between the two points. You would then discard the points that exceed your desired exact linear distance. Converting and standardizing units of geometry and distance Imagine that you have two points on the Earth's surface with a straight line drawn between them: Each point can be described as a coordinate using some arbitrary coordinate system (for example, using latitude and longitude values), while the length of the straight line could be described as the distance between the two points. Given any two coordinates, it is possible to calculate the distance between them. Conversely, you can start with one coordinate, a desired distance and a direction, and then calculate the coordinates for the other point. Of course, because the Earth's surface is not flat, we aren't really dealing with straight lines at all. Rather, we are calculating geodetic or Great Circle distances across the surface of the Earth. The pyproj Python library allows you to perform these types of calculations for any given datum. You can also use pyproj to convert from projected coordinates back to geographic coordinates, and vice versa, allowing you to perform these sorts of calculations for any desired datum, coordinate system, and projection. Ultimately, a geometry such as a line or a polygon consists of nothing more than a list of connected points. This means that, using the above process, you can calculate the geodetic distance between each of the points in any polygon and total the results to get the actual length for any geometry. Let's use this knowledge to solve a realworld problem.  

In this article, by Douglas Reynolds author of the book Learning Grunt, you will learn about Grunt plugins as they are core of the Grunt functionality and are an important aspect of Grunt as plugins are what we use in order to design an automated build process. (For more resources related to this topic, see here.) Common Grunt plugins and their purposes At this point, you should be asking yourself, what plugins can benefit me the most and why? Once you ask these questions, you may find that a natural response will be to ask further questions such as "what plugins are available?" This is exactly the intended purpose of this section, to introduce useful Grunt plugins and describe their intended purpose. contrib-watch This is, in the author's opinion, probably the most useful plugin available. The contrib-watch plugin responds to changes in files defined by you and runs additional tasks upon being triggered by the changed file events. For example, let's say that you make changes to a JavaScript file. When you save, and these changes are persisted, contrib-watch will detect that the file being watched has changed. An example workflow might be to make and save changes in a JavaScript file, then run Lint on the file. You might paste the code into a Lint tool, such as http://www.jslint.com/, or you might run an editor plugin tool on the file to ensure that your code is valid and has no defined errors. Using Grunt and contrib-watch, you can configure contrib-watch to automatically run a Grunt linting plugin so that every time you make changes to your JavaScript files, they are automatically lined. Installation of contrib-watch is straight forward and accomplished using the following npm-install command: npm install grunt-contrib-watch –save-dev The contrib-watch plugin will now be installed into your node-module directory. This will be located in the root of your project, you may see the Angular-Seed project for an example. Additionally, contrib-watch will be registered in package.json, you will see something similar to the following in package.json when you actually run this command: "devDependencies": { "grunt": "~0.4.5", "grunt-contrib-watch": "~0.4.0" } Notice the tilde character (~) in the grunt-contrib-watch: 0.5.3 line, the tilde actually specifies that the most recent minor version should be used when updating. Therefore, for instance, if you updated your npm package for the project, it would use 0.5.4 if available; however, it would not use 0.6.x as that is a higher minor version. There is also a caret character (^) that you may see. It will allow updates matching the most recent major version. In the case of the 0.5.3 example, 0.6.3 will be allowed, while 1.x versions are not allowed. At this point, contrib-watch is ready to be configured into your project in a gruntfile, we will look more into the gruntfile later. It should be noted that this, and many other tasks, can be run manually. In the case of contrib-watch, once installed and configured, you can run the grunt watch command to start watching the files. It will continue to watch until you end your session. The contrib-watch has some useful options. While we won't cover all of the available options, the following are some notable options that you should be aware of. Make sure to review the documentation for the full listing of options: Options.event: This will allow you to configure contrib-watch to only trigger when certain event types occur. The available types are all, changed, added, and deleted. You may configure more than one type if you wish. The all will trigger any file change, added will respond to new files added, and deleted will be triggered on removal of a file. Options.reload: This will trigger the reload of the watch task when any of the watched files change. A good example of this is when the file in which the watch is configured changes, it is called gruntfile.js. This will reload the gruntfile and restart contrib-watch, watching the new version of gruntfile.js. Options.livereload: This is different than reload, therefore, don't confuse the two. Livereload starts a server that enables live reloading. What this means is that when files are changed, your server will automatically update with the changed files. Take, for instance, a web application running in a browser. Rather than saving your files and refreshing your browser to get the changed files, livereload automatically reloads your app in the browser for you. contrib-jshint The contrib-jshint plugin is to run automated JavaScript error detection and will help with identifying potential problems in your code, which may surface during the runtime. When the plugin is run, it will scan your JavaScript code and issue warnings on the preconfigured options. There are a large number of error messages that jshint might provide and it can be difficult at times to understand what exactly a particular message might be referring to. Some examples are shown in the following: The array literal notation [] is preferable The {a} is already defined Avoid arguments.{a} Bad assignment Confusing minuses The list goes on and there are resources such as http://jslinterrors.com/, whose purpose is to help you understand what a particular warning/error message means. Installation of contrib-jshint follows the same pattern as other plugins, using npm to install the plugin in your project, as shown in the following: npm install grunt-contrib-jshint --save-dev This will install the contrib-jshint plugin in your project's node-modules directory and register the plugin in your devDependencies section of the package.json file at the root of your project. It will be similar to the following: "devDependencies": { "grunt": "~0.4.5", "grunt-contrib-jshint": "~0.4.5" } Similar to the other plugins, you may manually run contrib-jshint using the grunt jshint command. The contrib-jshint is jshint, therefore, any of the options available in jshint may be passed to contrib-jshint. Take a look at http://jshint.com/docs/options/ for a complete listing of the jshint options. Options are configured in the gruntfile.js file that we will cover in detail later in this book. Some examples of options are as follows: curly: This enforces that curly braces are used in code blocks undef: This ensures that all the variables have been declared maxparams: This checks to make sure that the number of arguments in a method does not exceed a certain limit The contrib-jshint allows you to configure the files that will be linted, the order in which the linting will occur, and even control linting before and after the concatenation. Additionally, contrib-jshint allows you to suppress warnings in the configuration options using the ignore_warning option. contrib-uglify Compression and minification are important for reducing the file sizes and contributing for better loading times to improve the performance. The contrib-uglify plugin provides the compression and minification utility by optimizing the JavaScript code and removing unneeded line breaks and whitespaces. It does this by parsing JavaScript and outputting regenerated, optimized, and code with shortened variable names for example. The contrib-uglify plugin is installed in your project using the npm install command, just as you will see with all the other Grunt plugins, as follows: npm install grunt-contrib-uglify --save-dev After you run this command, contrib-uglify will be installed in your node-modules directory at the root of your application. The plugin will also be registered in your devDependencies section of package.json. You should see something similar to the following in devDependencies: "devDependencies": { "grunt": "~0.4.5", "grunt-contrib-uglify": "~0.4.0" } In addition to running as an automated task, contrib-uglify plugin may be run manually by issuing the grunt uglify command. The contrib-uglify plugin is configured to process specific files as defined in the gruntfile.js configuration file. Additionally, contrib-uglify will have defined output destination files that will be created for the processed minified JavaScript. There is also a beautify option that can be used to revert the minified code, should you wish to easily debug your JavaScript. A useful option that is available in conntrib-uglify is banners. Banners allow you to configure banner comments to be added to the minified output files. For example, a banner could be created with the current date and time, author, version number, and any other important information that should be included. You may reference your package.json file in order to get information, such as the package name and version, directly from the package.json configuration file. Another notable option is the ability to configure directory-level compiling of files. You achieve this through the configuration of the files option to use wildcard path references with file extension, such as **/*.js. This is useful when you want to minify all the contents in a directory. contrib-less The contrib-less is a plugin that compiles LESS files into CSS files. The LESS provides extensibility to standard CSS by allowing variables, mixins (declaring a group of style declarations at once that can be reused anywhere in the stylesheet), and even conditional logic to manage styles throughout the document. Just as with other plugins, contrib-less is installed to your project using the npm install command with the following command: npm install grunt-contrib-less –save-dev The npm install will add contrib-less to your node-modules directory, located at the root of your application. As we are using --save-dev, the task will be registered in devDependencies of package.json. The registration will look something similar to the following: "devDependencies": { "grunt": "~0.4.5", "grunt-contrib-less": "~0.4.5" } Typical of Grunt tasks, you may also run contrib-less manually using the grunt less command. The contrib-less will be configured using the path and file options to define the location of source and destination output files. The contrib-less plugin can also be configured with multiple environment-type options, for example dev, test, and production, in order to apply different options that may be needed for different environments. Some typical options used in contrib-less include the following: paths: These are the directories that should be scanned compress: This shows whether to compress output to remove the whitespace plugins: This is the mechanism for including additional plugins in the flow of processing banner: This shows the banner to use in the compiled destination files There are several more options that are not listed here, make sure to refer to the documentation for the full listing of contrib-less options and example usage. Summary In this article we have covered some basic grunt plugins. Resources for Article: Further resources on this subject: Grunt in Action [article] Optimizing JavaScript for iOS Hybrid Apps [article] Welcome to JavaScript in the full stack [article]

One of the most exciting new directions that Android has gone in recently, is the extension of the platform from phones and tablets to televisions, car dashboards and wearables such as watches. These new devices allow us to provide added functionality to our existing apps, as well as creating wholly original apps designed specifically for these new environments. This article is really concerned with explaining the idiosyncrasies of each platform and the guidelines Google is keen for us to follow. This is particularly vital when it comes to developing apps that people will use when driving, as safety has to be of prime importance. There are also certain technical issues that need to be addressed when developing wearable apps, such as the pairing of the device with a handset and the entirely different UI and methods of use. In this article, you will: Create wearable AVDs Connect a wearable emulator to a handset with adb commands Connect a wearable emulator to a handset emulator Create a project with both mobile and wearable modules Use the Wearable UI Library Create shape-aware layouts Create and customize cards for wearables Understand wearable design principles (For more resources related to this topic, see here.) Android Wear Creating or adapting apps for wearables is probably the most complicated factors and requires a little more setting up than the other projects. However, wearables often give us access to one of the more fun new sensors, the heart rate monitor. In seeing how this works, we also get to see, how to manage sensors in general. Do not worry if you do not have access to an Android wearable device, as we will be constructing AVDs. You will ideally have an actual Android 5 handset, if you wish to pair it with the AVD. If you do not, it is still possible to work with two emulators but it is a little more complex to set up. Bearing this in mind, we can now prepare our first wearable app. Constructing and connecting to a wearable AVD It is perfectly possible to develop and test wearable apps on the emulator alone, but if we want to test all wearable features, we will need to pair it with a phone or a tablet. The next exercise assumes that you have an actual device. If you do not, still complete tasks 1 through 4 and we will cover how the rest can be achieved with an emulator a little later on. Open Android Studio. You do not need to start a project at this point. Start the SDK Manager and ensure you have the relevant packages installed. Open the AVD Manager. Create two new Android Wear AVDs, one round and one square, like so: Ensure USB Debugging is selected on your handset. Install the Android Wear app from the Play Store at this URL: https://play.google.com/store/apps/details?id=com.google.android.wearable.app. Connect it to your computer and start one of the AVDs we just created. Locate and open the folder containing the adb.exe file. It will probably be something like userAppDataLocalAndroidsdkplatform-tools. Using Shift + right-click, select Open command window here. In the command window, issue the following command: adb -d forward tcp:5601 tcp:5601 Launch the companion app and follow the instructions to pair the two devices. Being able to connect a real-world device to an AVD is a great way to develop form factors without having to own the devices. The wearable companion app simplifies the process of connecting the two. If you have had the emulator running for any length of time, you will have noticed that many actions, such as notifications, are sent to the wearable automatically. This means that very often our apps will link seamlessly with a wearable device, without us having to include code to pre-empt this. The adb.exe (Android Debug Bridge) is a vital part of our development toolkit. Most of the time, the Android Studio manages it for us. However, it is useful to know that it is there and a little about how to interact with it. We used it here to manually open a port between our wearable AVD and our handset. There are many adb commands that can be issued from the command prompt and perhaps the most useful is adb devices, which lists all currently debuggable devices and emulators, and is very handy when things are not working, to see if an emulator needs restarting. Switching the ADB off and on can be achieved using adb kill-server and adb start-server respectively. Using adb help will list all available commands. The port forwarding command we used in Step 10, needs to be issued every time the phone is disconnected from the computer. Without writing any code as such, we have already seen some of the features that are built into an Android Wear device and the way that the Wear UI differs from most other Android devices. Even if you usually develop with the latest Android hardware, it is often still a good idea to use an emulator, especially for testing the latest SDK updates and pre-releases. If you do not have a real device, then the next, small section will show you how to connect your wearable AVD to a handset AVD. Connecting a wearable AVD with another emulator Pairing two emulators is very similar to pairing with a real device. The main difference is the way we install the companion app without access to the Play Store. Follow these steps to see how it is done: Start up, an AVD. This will need to be targeting Google APIs as seen here: Download the com.google.android.wearable.app-2.apk. There are many places online where it can be found with a simple search, I used www.file-upload.net/download. Place the file in your sdk/platform-tools directory. Shift + right-click in this folder and select Open command window here. Enter the following command: adb install com.google.android.wearable.app-2.apk. Start your wearable AVD. Enter adb devices into the command prompt, making sure that both emulators are visible with an output similar to this: List of devices attached emulator-5554 device emulator-5555 device Enter adb telnet localhost 5554 at the command prompt, where 5554 is the phone emulator. Next, enter adb redir add tcp:5601:5601. You can now use the Wear app on the handheld AVD to connect to the watch. As we've just seen, setting up a Wear project takes a little longer than some of the other exercises we have performed. Once set up though, the process is very similar to that of developing for other form factors, and something we can now get on with. Creating a wearable project All of the apps that we have developed so far, have required just a single module, and this makes sense as we have only been building for single devices. In this next step, we will be developing across two devices and so will need two modules. This is very simple to do, as you will see in these next steps. Start a new project in the Android Studio and call it something like Wearable App. On the Target Android Devices screen, select both Phone and Tablet and Wear, like so: You will be asked to add two Activities. Select Blank Activity for the Mobile Activity and Blank Wear Activity for Wear. Everything else can be left as it is. Run the app on both round and square virtual devices. The first thing you will have noticed is the two modules, mobile and wear. The first is the same as we have seen many times, but there are a few subtle differences with the wear module and it is worth taking a little look at. The most important difference is the WatchViewStub class. The way it is used can be seen in the activity_main.xml and MainActivity.java files of the wear module. This frame layout extension is designed specifically for wearables and detects the shape of the device, so that the appropriate layout is inflated. Utilizing the WatchViewStub is not quite as straightforward, as one might imagine, as the appropriate layout is only inflated after the WatchViewStub has done its thing. This means that, to access any views within the layout, we need to employ a special listener that is called once the layout has been inflated. How this OnLayoutInflatedListener() works can be seen by opening the MainActivity.java file in the wear module and examining the onCreate() method, which will look like this: @Override protected void onCreate(Bundle savedInstanceState) { super.onCreate(savedInstanceState); setContentView(R.layout.activity_main); final WatchViewStub stub = (WatchViewStub) findViewById(R.id.watch_view_stub); stub.setOnLayoutInflatedListener(new WatchViewStub.OnLayoutInflatedListener() { @Override public void onLayoutInflated(WatchViewStub stub) { mTextView = (TextView) stub.findViewById(R.id.text); } }); } Other than the way that wearable apps and devices are set up for developing, the other significant difference is the UI. The widgets and layouts that we use for phones and tablets are not suitable, in most cases, for the diminished size of a watch screen. Android provides a whole new set of UI components, that we can use and this is what we will look at next. Designing a UI for wearables As well as having to consider the small size of wearable when designing layouts, we also have the issue of shape. Designing for a round screen brings its own challenges, but fortunately the Wearable UI Library makes this very simple. As well as the WatchViewStub, that we encountered in the previous section that inflates the correct layout, there is also a way to design a single layout that inflates in such a way, that it is suitable for both square and round screens. Designing the layout The project setup wizard included this library for us automatically in the build.gradle (Module: wear) file as a dependency: compile 'com.google.android.support:wearable:1.1.0' The following steps demonstrate how to create a shape-aware layout with a BoxInsetLayout: Open the project we created in the last section. You will need three images that must be placed in the drawable folder of the wear module: one called background_image of around 320x320 px and two of around 50x50 px, called right_icon and left_icon. Open the activity_main.xml file in the wear module. Replace its content with the following code: <android.support.wearable.view.BoxInsetLayout android_background="@drawable/background_image" android_layout_height="match_parent" android_layout_width="match_parent" android_padding="15dp"> </android.support.wearable.view.BoxInsetLayout> Inside the BoxInsetLayout, add the following FrameLayout: <FrameLayout android_id="@+id/wearable_layout" android_layout_width="match_parent" android_layout_height="match_parent" android_padding="5dp" app_layout_box="all"> </FrameLayout> Inside this, add these three views: <TextView android_gravity="center" android_layout_height="wrap_content" android_layout_width="match_parent" android_text="Weather warning" android_textColor="@color/black" /> <ImageView android_layout_gravity="bottom|left" android_layout_height="60dp" android_layout_width="60dp" android_src="@drawable/left_icon" /> <ImageView android_layout_gravity="bottom|right" android_layout_height="60dp" android_layout_width="60dp" android_src="@drawable/right_icon" /> Open the MainActivity.java file in the wear module. In the onCreate() method, delete all lines after the line setContentView(R.layout.activity_main);. Now, run the app on both square and round emulators. As we can see, the BoxInsetLayout does a fine job of inflating our layout regardless of screen shape. How it works is very simple. The BoxInsetLayout creates a square region, that is as large as can fit inside the circle of a round screen. This is set with the app:layout_box="all" instruction, which can also be used for positioning components, as we will see in a minute. We have also set the padding of the BoxInsetLayout to 15 dp and that of the FrameLayout to 5 dp. This has the effect of a margin of 5 dp on round screens and 15 dp on square ones. Whether you use the WatchViewStub and create separate layouts for each screen shape or BoxInsetLayout and just one layout file depends entirely on your preference and the purpose and design of your app. Whichever method you choose, you will no doubt want to add Material Design elements to your wearable app, the most common and versatile of these being the card. In the following section, we will explore the two ways that we can do this, the CardScrollView and the CardFragment. Adding cards The CardFragment class provides a default card view, providing two text views and an image. It is beautifully simple to set up, has all the Material Design features such as rounded corners and a shadow, and is suitable for nearly all purposes. It can be customized, as we will see, although the CardScrollView is often a better option. First, let us see, how to implement a default card for wearables: Open the activity_main.xml file in the wear module of the current project. Delete or comment out the the text view and two image views. Open the MainActivity.java file in the wear module. In the onCreate() method, add the following code: FragmentManager fragmentManager = getFragmentManager(); FragmentTransaction fragmentTransaction = fragmentManager.beginTransaction(); CardFragment cardFragment = CardFragment.create("Short title", "with a longer description"); fragmentTransaction.add(R.id.wearable_layout, cardFragment); fragmentTransaction.commit(); Run the app on one or other of the wearable emulators to see how the default card looks. The way we created the CardFragment itself, is also very straightforward. We used two string parameters here, but there is a third, drawable parameter and if the line is changed to CardFragment cardFragment = CardFragment.create("TITLE", "with description and drawable", R.drawable.left_icon); then we will get the following output: This default implementation for cards on wearable is fine for most purposes and it can be customized by overriding its onCreateContentView() method. However, the CardScrollView is a very handy alternative, and this is what we will look at next. Customizing cards The CardScrollView is defined from within our layout and furthermore it detects screen shape and adjusts the margins to suit each shape. To see how this is done, follow these steps: Open the activity_main.xml file in the wear module. Delete or comment out every element, except the root BoxInsetLayout. Place the following CardScrollView inside the BoxInsetLayout: <android.support.wearable.view.CardScrollView android_id="@+id/card_scroll_view" android_layout_height="match_parent" android_layout_width="match_parent" app_layout_box="bottom"> </android.support.wearable.view.CardScrollView> Inside this, add this CardFrame: <android.support.wearable.view.CardFrame android_layout_width="match_parent" android_layout_height="wrap_content"> </android.support.wearable.view.CardFrame> Inside the CardFrame, add a LinearLayout. Add some views to this, so that the preview resembles the layout here: Open the MainActivity.java file. Replace the code we added to the onCreate() method with this: CardScrollView cardScrollView = (CardScrollView) findViewById(R.id.card_scroll_view); cardScrollView.setCardGravity(Gravity.BOTTOM); You can now test the app on an emulator, which will produce the following result: As can be seen in the previous image, the Android Studio has preview screens for both wearable shapes. Like some other previews, these are not always what you will see on a device, but they allow us to put layouts together very quickly, by dragging and dropping widgets. As we can see, the CardScrollView and CardFrame are even easier to implement than the CardFragment and also far more flexible, as we can design almost any layout we can imagine. We assigned app:layout_box here again, only this time using bottom, causing the card to be placed as low on the screen as possible. It is very important, when designing for such small screens, to keep our layouts as clean and simple as possible. Google's design principles state that wearable apps should be glanceable. This means that, as with a traditional wrist watch, the user should be able to glance at our app and immediately take in the information and return to what they were doing. Another of Google's design principle—Zero to low interaction—is only a single tap or swipe a user needs to do to interact with our app. With these principles in mind, let us create a small app, with some actual functionality. In the next section, we will take advantage of the new heart rate sensor found in many wearable devices and display current beats-per-minute on the display. Accessing sensor data The location of an Android Wear device on the user's wrist, makes it the perfect piece of hardware for fitness apps, and not surprisingly, these apps are immensely popular. As with most features of the SDK, accessing sensors is pleasantly simple, using classes such as managers and listeners and requiring only a few lines of code, as you will see by following these steps: Open the project we have been working on in this article. Replace the background image with one that might be suitable for a fitness app. I have used a simple image of a heart. Open the activity_main.xml file. Delete everything, except the root BoxInsetLayout. Place this TextView inside it: <TextView android_id="@+id/text_view" android_layout_width="match_parent" android_layout_height="wrap_content" android_layout_gravity="center_vertical" android_gravity="center" android_text="BPM" android_textColor="@color/black" android_textSize="42sp" /> Open the Manifest file in the wear module. Add the following permission inside the root manifest node: <uses-permission android_name="android.permission.BODY_SENSORS" /> Open the MainActivity.java file in the wear module. Add the following fields: private TextView textView; private SensorManager sensorManager; private Sensor sensor; Implement a SensorEventListener on the Activity: public class MainActivity extends Activity implements SensorEventListener { Implement the two methods required by the listener. Edit the onCreate() method, like this: @Override protected void onCreate(Bundle savedInstanceState) { super.onCreate(savedInstanceState); setContentView(R.layout.activity_main); textView = (TextView) findViewById(R.id.text_view); sensorManager = ((SensorManager) getSystemService(SENSOR_SERVICE)); sensor = sensorManager.getDefaultSensor(Sensor.TYPE_HEART_RATE); } Add this onResume() method: protected void onResume() { super.onResume(); sensorManager.registerListener(this, this.sensor, 3); } And this onPause() method: @Override protected void onPause() { super.onPause(); sensorManager.unregisterListener(this); } Edit the onSensorChanged() callback, like so: @Override public void onSensorChanged(SensorEvent event) { textView.setText("" + (int) event.values[0]); } If you do not have access to a real device, you can download a sensor simulator from here:     https://code.google.com/p/openintents/wiki/SensorSimulator The app is now ready to test. We began by adding a permission in the AndroidManifest.xml file in the appropriate module; this is something we have done before and need to do any time we are using features, that require the user's permission before installing. The inclusion of a background image may seem necessary, but an appropriate background is a real aid to glancability as the user can tell instantly which app they are looking at. It should be clear, from the way the SensorManager and the Sensor are set up in the onCreate() method, that all sensors are accessed in the same way and different sensors can be accessed with different constants. We used TYPE_HEART_RATE here, but any other sensor can be started with the appropriate constant, and all sensors can be managed with the same basic structures as we found here, the only real difference being the way each sensor returns SensorEvent.values[]. A comprehensive list of all sensors, and descriptions of the values they produce can be found at http://developer.android.com/reference/android/hardware/Sensor.html. As with any time our apps utilize functions that run in the background, it is vital that we unregister our listeners, whenever they are no longer needed, in our Activity's onPause() method. We didn't use the onAccuracyChanged() callback here, but its purpose should be clear and there are many possible apps where its use is essential. This concludes our exploration of wearable apps and how they are put together. Such devices continue to become more prevalent and the possibility of ever more imaginative uses is endless. Providing we consider why and how people use smart watches, and the like, and develop to take advantage of the location of these devices by programming glanceable interfaces that require the minimum of interactivity, Android Wear seems certain to grow in popularity and use, and the developers will continue to produce ever more innovative apps. Summary In this article, we have explored Android wearable apps and how they are put together. Despite their diminutive size and functionality, wearables offer us an enormous range of possibilities. We know now how to create and connect wearable AVDs and how to develop easily for both square and round devices. We then designed the user interface for both round and square screens. To learn more about Android UI interface designing and developing android applications, the following books published by Packt Publishing (https://www.packtpub.com/) are recommended: Android User Interface Development: Beginner's Guide, found at https://www.packtpub.com/application-development/android-user-interface-development-beginners-guide Android 4: New Features for Application Development, found at https://www.packtpub.com/application-development/android-4-new-features-application-development Resources for Article: Further resources on this subject: Understanding Material Design [Article] Speaking Java – Your First Game [Article] Mobile Game Design Best Practices [Article]

(For more resources related to this topic, see here.) The ASP.NET Web API is a framework that you can make use of to build web services that use HTTP as the protocol. You can use the ASP.NET Web API to return data based on the data requested by the client, that is, you can return JSON or XML as the format of the data. Layers of an application The ASP.NET Framework runs on top of the managed environment of the .NET Framework. The Model, View, and Controller (MVC) architectural pattern is used to separate the concerns of an application to facilitate testing, ease the process of maintenance of the application's code, and to provide better support for change. The model represents the application's data and the business objects; the view is the presentation layer component, and the controller binds the model and the view together. The following figure illustrates the components of Model View Architecture: The MVC Architecture The ASP.NET Web API architecture The ASP.NET Web API is a lightweight web-based architecture that uses HTTP as the application protocol. Routing in the ASP.NET Web API works a bit differently compared to the way it works in ASP.NET MVC. The basic difference between routing in MVC and routing in a Web API is that, Web API uses the HTTP method, and not the URI path, to select the action. The Web API Framework uses a routing table to determine which action is to be invoked for a particular request. You need to specify the routing parameters in the WebApiConfig.cs file that resides in the App_Start directory. Here's an example that shows how routing is configured: routes.MapHttpRoute(    name: "Packt API Default",     routeTemplate: "api/{controller}/{id}",     defaults: new { id = RouteParameter.Optional } ); The following code snippet illustrates how routing is configured by action names: routes.MapHttpRoute(     name: "PacktActionApi",     routeTemplate: "api/{controller}/{action}/{id}",     defaults: new { id = RouteParameter.Optional } ); The ASP.NET Web API generates structured data such as JSON and XML as responses. It can route the incoming requests to the actions based on HTTP verbs and not only action names. Also, the ASP.NET Web API can be hosted outside of the ASP.NET runtime environment and the IIS Web Server context. Routing in ASP.NET Web API Routing in the ASP.NET Web API is very much the same as in the ASP.NET MVC. The ASP.NET Web API routes URLs to a controller. Then, the control is handed over to the action that corresponds to the HTTP verb of the request message. Note that the default route template for an ASP.NET Web API project is {controller}/{id}—here the {id} parameter is optional. Also, the ASP.NET Web API route templates may optionally include an {action} parameter. It should be noted that unlike the ASP.NET MVC, URLs in the ASP.NET Web API cannot contain complex types. It should also be noted that complex types must be present in the HTTP message body, and that there can be one, and only one, complex type in the HTTP message body. Note that the ASP.NET MVC and the ASP.NET Web API are two distinctly separate frameworks which adhere to some common architectural patterns. In the ASP.NET Web API framework, the controller handles all HTTP requests. The controller comprises a collection of action methods—an incoming request to the Web API framework, the request is routed to the appropriate action. Now, the framework uses a routing table to determine the action method to be invoked when a request is received. Here is an example: routes.MapHttpRoute(     name: "Packt Web API",     routeTemplate: "api/{controller}/{id}",     defaults: new { id = RouteParameter.Optional } ); Refer to the following UserController class. public class UserController <UserAuthentication>: BaseApiController<UserAuthentication> {     public void GetAllUsers() { }     public IEnumerable<User> GetUserById(int id) { }     public HttpResponseMessage DeleteUser(int id){ } } The following table illustrates the HTTP method and the corresponding URI, Actions, and so on: HTTP Method URI Action Parameter GET api/users GetAllUsers None GET api/users/1 GetUserByID 1 POST api/users     DELETE api/users/3 DeleteUser 3 The Web API Framework matches the segments in the URI path to the template. The following steps are performed: The URI is matched to a route template. The respective controller is selected. The respective action is selected. The IHttpControllerSelector.SelectController method selects the controller, takes an HttpRequestMessage instance and returns an HttpControllerDescriptor. After the controller has been selected, the Web API framework selects the action by invoking the IHttpActionSelector.SelectAction method. This method in turn accepts HttpControllerContext and returns HttpActionDescriptor. You can also explicitly specify the HTTP method for an action by decorating the action method using the HttpGet, HttpPut, HttpPost, or HttpDelete attributes. Here is an example: public class UsersController : ApiController {     [HttpGet]     public User FindUser(id) {} } You can also use the AcceptVerbs attribute to enable HTTP methods other than GET, PUT, POST, and DELETE. Here is an example: public class UsersController : ApiController {     [AcceptVerbs("GET", "HEAD")]     public User FindUser(id) { } } You can also define route by an action name. Here is an example: routes.MapHttpRoute(     name: "PacktActionApi",     routeTemplate: "api/{controller}/{action}/{id}",     defaults: new { id = RouteParameter.Optional } ); You can also override the action name by using the ActionName attribute. The following code snippet illustrates two actions: one that supports GET and the other that supports POST: public class UsersController : ApiController {     [HttpGet]     [ActionName("Token")]     public HttpResponseMessage GetToken(int userId);     [HttpPost]     [ActionName("Token")]     public void AddNewToken(int userId); }  

In this article by Ryan O'Neill, author of the book Learning Linux Binary Analysis, ELF will be discussed. In order to reverse-engineer Linux binaries, we must understand the binary format itself. ELF has become the standard binary format for UNIX and UNIX-Flavor OS's. Binary formats such as ELF are not generally a quick study, and to learn ELF requires some degree of application of the different components that you learn as you go. Programming things that perform tasks such as binary parsing will require learning some ELF, and in the act of programming such things, you will in-turn learn ELF better and more proficiently as you go along. ELF is often thought to be a dry and complicated topic to learn, and if one were to simply read through the ELF specs without applying them through the spirit of creativity, then indeed it would be. ELF is really quite an incredible composition of computer science at work, with program layout, program loading, dynamic linking, symbol table lookups, and many other tightly orchestrated components. (For more resources related to this topic, see here.) ELF section headers Now that we've looked at what program headers are, it is time to look at section headers. I really want to point out here the distinction between the two; I often hear people calling sections "segments" and vice versa. A section is not a segment. Segments are necessary for program execution, and within segments are contained different types of code and data which are separated within sections and these sections always exist, and usually they are addressable through something called section-headers. Section-headers are what make sections accessible, but if the section-headers are stripped (Missing from the binary), it doesn't mean that the sections are not there. Sections are just data or code. This data or code is organized across the binary in different sections. The sections themselves exist within the boundaries of the text and data segment. Each section contains either code or data of some type. The data could range from program data, such as global variables, or dynamic linking information that is necessary for the linker. Now, as mentioned earlier, every ELF object has sections, but not all ELF objects have section headers. Usually this is because the executable has been tampered with (Such as the section headers having been stripped so that debugging is harder). All of GNU's binutils, such as objcopy, objdump, and other tools such as gdb, rely on the section-headers to locate symbol information that is stored in the sections specific to containing symbol data. Without section-headers, tools such as gdb and objdump are nearly useless. Section-headers are convenient to have for granular inspection over what parts or sections of an ELF object we are viewing. In fact, section-headers make reverse engineering a lot easier, since they provide us with the ability to use certain tools that require them. If, for instance, the section-header table is stripped, then we can't access a section such as .dynsym, which contains imported/exported symbols describing function names and offsets/addresses. Even if a section-header table has been stripped from an executable, a moderate reverse engineer can actually reconstruct a section-header table (and even part of a symbol table) by getting information from certain program headers, since these will always exist in a program or shared library. We discussed the dynamic segment earlier and the different DT_TAGs that contain information about the symbol table and relocation entries. This is what a 32 bit ELF section-header looks like: typedef struct {     uint32_t   sh_name; // offset into shdr string table for shdr name     uint32_t   sh_type; // shdr type I.E SHT_PROGBITS     uint32_t   sh_flags; // shdr flags I.E SHT_WRITE|SHT_ALLOC     Elf32_Addr sh_addr;  // address of where section begins     Elf32_Off  sh_offset; // offset of shdr from beginning of file     uint32_t   sh_size;   // size that section takes up on disk     uint32_t   sh_link;   // points to another section     uint32_t   sh_info;   // interpretation depends on section type     uint32_t   sh_addralign; // alignment for address of section     uint32_t   sh_entsize;  // size of each certain entries that may be in section } Elf32_Shdr; Let's take a look at some of the most important section types, once again allowing room to study the ELF(5) man pages and the official ELF specification for more detailed information about the sections. .text The .text section is a code section that contains program code instructions. In an executable program where there are also phdr, this section would be within the range of the text segment. Because it contains program code, it is of the section type SHT_PROGBITS. .rodata The rodata section contains read-only data, such as strings, from a line of C code: printf("Hello World!n"); These are stored in this section. This section is read-only, and therefore must exist in a read-only segment of an executable. So, you would find .rodata within the range of the text segment (Not the data segment). Because this section is read-only, it is of the type SHT_PROGBITS. .plt The Procedure linkage table (PLT) contains code that is necessary for the dynamic linker to call functions that are imported from shared libraries. It resides in the text segment and contains code, so it is marked as type SHT_PROGBITS. .data The data section, not to be confused with the data segment, will exist within the data segment and contain data such as initialized global variables. It contains program variable data, so it is marked as SHT_PROGBITS. .bss The bss section contains uninitialized global data as part of the data segment, and therefore takes up no space on the disk other than 4 bytes, which represents the section itself. The data is initialized to zero at program-load time, and the data can be assigned values during program execution. The bss section is marked as SHT_NOBITS, since it contains no actual data. .got The Global offset table (GOT) section contains the global offset table. This works together with the PLT to provide access to imported shared library functions, and is modified by the dynamic linker at runtime. This section has to do with program execution and is therefore marked as SHT_PROGBITS. .dynsym The dynsym section contains dynamic symbol information imported from shared libraries. It is contained within the text segment and is marked as type SHT_DYNSYM. .dynstr The dynstr section contains the string table for dynamic symbols; this has the name of each symbol in a series of null terminated strings. .rel.* Relocation sections contain information about how the parts of an ELF object or process image need to be fixed up or modified at linking or runtime. .hash The hash section, sometimes called .gnu.hash, contains a hash table for symbol lookup. The following hash algorithm is used for symbol name lookups in Linux ELF: uint32_t dl_new_hash (const char *s) {         uint32_t h = 5381;           for (unsigned char c = *s; c != ' '; c = *++s)                 h = h * 33 + c;           return h; } .symtab The symtab section contains symbol information of the type ElfN_Sym. The symtab section is marked as type SHT_SYMTAB as it contains symbol information. .strtab This section contains the symbol string table that is referenced by the st_name entries within the ElfN_Sym structs of .symtab, and is marked as type SHT_STRTAB since it contains a string table. .shstrtab The shstrtab section contains the section header string table, which is a set of null terminated strings containing the names of each section, such as .text, .data, and so on. This section is pointed to by the ELF file header entry called e_shstrndx, which holds the offset of .shstrtab. This section is marked as SHT_STRTAB since it contains a string table. .ctors and .dtors The .ctors (constructors) and .dtors (destructors) sections contain code for initialization and finalization, which is to be executed before and after the actual main() body of program code, and then after the main program code. The __constructor__ function attribute is often used by hackers and virus writers to implement a function that performs an anti-debugging trick, such as calling PTRACE_TRACEME, so that the process traces itself and no debuggers can attach themselves to it. This way, the anti-debugging mechanism gets executed before the program enters main(). There are many other section names and types, but we have covered most of the primary ones found in a dynamically linked executable. One can now visualize how an executable is laid out with both phdrs and shdrs: ELF Relocations From the ELF(5) man pages: Relocation is the process of connecting symbolic references with symbolic definitions.  Relocatable files must have information that describes how to modify their section contents, thus allowing executable and shared object files to hold the right information for a process's program image. Relocation entries are these data. The process of relocation relies on symbols, which is why we covered symbols first. An example of relocation might be a couple of relocatable objects (ET_REL) being linked together to create an executable. obj1.o wants to call a function, foo(), located in obj2.o. Both obj1.o and obj2.o are being linked to create a fully working executable; they are currently Position independent code (PIC), but once relocated to form an executable, they will no longer be position independent since symbolic references will be resolved into symbolic definitions. The term "relocated" means exactly that: a piece of code or data is being relocated from a simple offset in an object file to some memory address location in an executable, and anything that references that relocated code or data must also be adjusted. Let's take a quick look at a 32 bit relocation entry: typedef struct {     Elf32_Addr r_offset;     uint32_t   r_info; } Elf32_Rel; And some relocation entries require an addend: typedef struct {     Elf32_Addr r_offset;     uint32_t   r_info;     int32_t    r_addend; } Elf32_Rela; Following is the description of the preceding snippet: r_offset: This points to the location (offset or address) that requires the relocation action (which is going to be some type of modification) r_info: This gives both the symbol table index with respect to which the relocation must be made, and the type of relocation to apply r_addend: This specifies a constant addend used to compute the value stored in the relocatable field Let's take a look at the source code for obj1.o: _start() {   foo(); } We see that it calls the function foo(), however foo() is not located within the source code or the compiled object file, so there will be a relocation entry necessary for symbolic reference: ryan@alchemy:~$ objdump -d obj1.o obj1.o:     file format elf32-i386 Disassembly of section .text: 00000000 <func>:    0:  55                     push   %ebp    1:  89 e5                  mov    %esp,%ebp    3:  83 ec 08               sub    $0x8,%esp    6:  e8 fc ff ff ff         call   7 <func+0x7>    b:  c9                     leave     c:  c3                     ret   As we can see, the call to foo() is highlighted and simply calls to nowhere; 7 is the offset of itself. So, when obj1.o, which calls foo() (located in obj2.o), is linked with obj2.o to make an executable, a relocation entry is there to point at offset 7, which is the data that needs to be modified, changing it to the offset of the actual function, foo(), once the linker knows its location in the executable during link time: ryan@alchemy:~$ readelf -r obj1.o Relocation section '.rel.text' at offset 0x394 contains 1 entries:  Offset     Info    Type            Sym.Value  Sym. Name 00000007  00000902 R_386_PC32        00000000   foo As we can see, a relocation field at offset 7 is specified by the relocation entry's r_offset field. R_386_PC32 is the relocation type; to understand all of these types, read the ELF specs as we will only be covering some. Each relocation type requires a different computation on the relocation target being modified. R_386_PC32 says to modify the target with S + A – P. The following list explains all these terms: S is the value of the symbol whose index resides in the relocation entry A is the addend found in the relocation entry P is the place (section offset or address) where the storage unit is being relocated (computed using r_offset) If we look at the final output of our executable after compiling obj1.o and obj2.o, as shown in the following code snippet: ryan@alchemy:~$ gcc -nostdlib obj1.o obj2.o -o relocated ryan@alchemy:~$ objdump -d relocated test:     file format elf32-i386 Disassembly of section .text: 080480d8 <func>:  80480d8:  55                     push   %ebp  80480d9:  89 e5                  mov    %esp,%ebp  80480db:  83 ec 08               sub    $0x8,%esp  80480de:  e8 05 00 00 00         call   80480e8 <foo>  80480e3:  c9                     leave   80480e4:  c3                     ret     80480e5:  90                     nop  80480e6:  90                     nop  80480e7:  90                     nop 080480e8 <foo>:  80480e8:  55                     push   %ebp  80480e9:  89 e5                  mov    %esp,%ebp  80480eb:  5d                     pop    %ebp  80480ec:  c3                     ret We can see that the call instruction (the relocation target) at 0x80480de has been modified with the 32 bit offset value of 5, which points to foo(). The value 5 is the result of the R386_PC_32 relocation action: S + A – P: 0x80480e8 + 0xfffffffc – 0x80480df = 5 0xfffffffc is the same as -4 if a signed integer, so the calculation can also be seen as: 0x80480e8 + (0x80480df + sizeof(uint32_t)) To calculate an offset into a virtual address, use the following computation: address_of_call + offset + 5 (Where 5 is the length of the call instruction) Which in this case is 0x80480de + 5 + 5 = 0x80480e8. An address may also be computed into an offset with the following computation: address – address_of_call – 4 (Where 4 is the length of a call instruction – 1) Relocatable code injection based binary patching Relocatable code injection is a technique that hackers, virus writers, or anyone who wants to modify the code in a binary may utilize as a way to sort of re-link a binary after it has already been compiled. That is, you can inject an object file into an executable, update the executables symbol table, and perform the necessary relocations on the injected object code so that it becomes a part of the executable. A complicated virus might use this rather than just appending code at the end of an executable or finding existing padding. This technique requires extending the text segment to create enough padding room to load the object file. The real trick though is handling the relocations and applying them properly. I designed a custom reverse engineering tool for ELF that is named Quenya. Quenya has many features and capabilities, and one of them is to inject object code into an executable. Why do this? Well, one reason would be to inject a malicious function into an executable, and then hijack a legitimate function and replace it with the malicious one. From a security point of view, one could do hot-patching and apply a legitimate patch to a binary rather than doing something malicious. Let's pretend we are an attacker and we want to infect a program that calls puts() to print "Hello World", and our goal is to hijack puts() so that it calls evil_puts(). First, we would need to write a quick PIC object that can write a string to standard output: #include <sys/syscall.h> int _write (int fd, void *buf, int count) {   long ret;     __asm__ __volatile__ ("pushl %%ebxnt"                         "movl %%esi,%%ebxnt"                         "int $0x80nt" "popl %%ebx":"=a" (ret)                         :"0" (SYS_write), "S" ((long) fd),                         "c" ((long) buf), "d" ((long) count));   if (ret >= 0) {     return (int) ret;   }   return -1; } int evil_puts(void) {         _write(1, "HAHA puts() has been hijacked!n", 31); } Now, we compile evil_puts.c into evil_puts.o, and inject it into our program, hello_world: ryan@alchemy:~/quenya$ ./hello_world Hello World This program calls the following: puts(“Hello Worldn”); We now use Quenya to inject and relocate our evil_puts.o file into hello_world: [Quenya v0.1@alchemy] reloc evil_puts.o hello_world 0x08048624  addr: 0x8048612 0x080485c4 _write addr: 0x804861e 0x080485c4  addr: 0x804868f 0x080485c4  addr: 0x80486b7 Injection/Relocation succeeded As we can see, the function write() from our evil_puts.o has been relocated and assigned an address at 0x804861e in the executable, hello_world. The next command, hijack, overwrites the global offset table entry for puts() with the address of evil_puts(): [Quenya v0.1@alchemy] hijack binary hello_world evil_puts puts Attempting to hijack function: puts Modifying GOT entry for puts Succesfully hijacked function: puts Commiting changes into executable file [Quenya v0.1@alchemy] quit And Whammi! ryan@alchemy:~/quenya$ ./hello_world HAHA puts() has been hijacked! We have successfully relocated an object file into an executable and modified the executable's control flow so that it executes the code that we injected. If we use readelf -s on hello_world, we can actually now see a symbol called evil_puts(). For the readers interest, I have included a small snippet of code that contains the ELF relocation mechanics in Quenya; it may be a little bit obscure without knowledge of the rest of the code base, but it is also somewhat straightforward if you've paid attention to what we learned about relocations. It is just a snippet and does not show any of the other important aspects such as modifying the executables symbol table: case SHT_RELA: for (j = 0; j < obj.shdr[i].sh_size / sizeof(Elf32_Rela); j++, rela++) {   rela = (Elf32_Rela *)(obj.mem + obj.shdr[i].sh_offset);       /* symbol table */                            symtab = (Elf32_Sym *)obj.section[obj.shdr[i].sh_link];               /* symbol we are applying relocation to */       symbol = &symtab[ELF32_R_SYM(rela->r_info)];        /* section to modify */       TargetSection = &obj.shdr[obj.shdr[i].sh_info];       TargetIndex = obj.shdr[i].sh_info;        /* target location */       TargetAddr = TargetSection->sh_addr + rela->r_offset;              /* pointer to relocation target */       RelocPtr = (Elf32_Addr *)(obj.section[TargetIndex] + rela->r_offset);              /* relocation value */       RelVal = symbol->st_value;       RelVal += obj.shdr[symbol->st_shndx].sh_addr;       switch (ELF32_R_TYPE(rela->r_info))       {         /* R_386_PC32      2    word32  S + A - P */         case R_386_PC32:               *RelocPtr += RelVal;                   *RelocPtr += rela->r_addend;                   *RelocPtr -= TargetAddr;                   break;              /* R_386_32        1    word32  S + A */            case R_386_32:                *RelocPtr += RelVal;                   *RelocPtr += rela->r_addend;                   break;       }  } As shown in the preceding code, the relocation target that RelocPtr points to is modified according to the relocation action requested by the relocation type (such as R_386_32). Although relocatable code binary injection is a good example of the idea behind relocations, it is not a perfect example of how a linker actually performs it with multiple object files. Nevertheless, it still retains the general idea and application of a relocation action. Later on, we will talk about shared library (ET_DYN) injection, which brings us now to the topic of dynamic linking. Summary In this article we discussed different types of ELF section headers and ELF relocations. Resources for Article:   Further resources on this subject: Advanced User Management [article] Creating Code Snippets [article] Advanced Wireless Sniffing [article]

All module settings are company specific. If you have multiple companies in Dynamics GP, you will need to set up each module in each company separately. This is often advantageous, as different companies may require different settings for various features. If you would like to copy setup from one company to another, Microsoft Dynamics GP KnowledgeBase article 872709?9? 9? 9? 9? 9? describes how to do this: https://mbs.microsoft.com/knowledgebase/KBDisplay.aspx?scid=kb;en-us;872709 (CustomerSource or PartnerSource login required). The Sales Order Processing module, also commonly referred to as SOP, bridges the gap between the Inventory and Receivables modules in Dynamics GP. In SOP, you can enter quotes, orders, back orders, invoices, and returns, with detailed inventory and non-inventory items. SOP also integrates to the Purchase Order Processing module with the ability to automatically create purchase orders for sales orders that you do not have stock to fill. Setup for Sales Order Processing consists of the following steps: Sales Order Processing Setup Sales Document Setup User-Defined Fields SOP Document Numbers Sales Order Processing Setup Options E-mail Settings Customer Items Sales Order Processing Setup To begin setting up SOP, navigate to Microsoft Dynamics GP | Tools | Setup | Sales | Sales Order Processing. The following is a list of the fields on the Sales Order Processing Setup window: Display Item Unit Cost: Unmarking this will show the unit cost of items entered on sales order transactions as zero. Some companies prefer to hide item costs from users; however, this option hides them only on the Sales Transaction Entry window, not anywhere else in the system. So, unless a user has extremely limited access in Dynamics GP, this is not as useful as it sounds. Track Voided Transactions in History: It is recommended to leave this option marked, as there may often be a reason to look at voided sales transactions. Calculate Kit Price based on Component Cost: This option applies to pricing methods based on cost, which are not too common. Selecting this option will cause the cost of the kit (and thus the price) to be recalculated based on the cost of each component in the kit. Display Quantity Distribution Warning: Selecting this enables a warning during transaction entry that helps users avoid mistakes when entering quantities for transactions. It is recommended to leave this checked. Search for New Rates During Transfer Process: This setting only applies to Multicurrency transactions. If it is checked, then during the transfer of sales documents (for example from quote to order), Dynamics GP will look for an updated exchange rate. If this setting is not selected, the system will still verify that the exchange rate is valid, but will not update exchange rates unless they are expired. Track Master Numbers and Next Master Number: Dynamics GP offers the ability to track related transactions with a Master Number. For example, a quote, a resulting order, and two partial invoices may all have their own individual numbers but share one master number, thus allowing for easy lookups of all related transactions. It is recommended to leave this option selected, even if you do not foresee a need to use master numbers. The Next Master Number can be anything you would like, although there is typically no reason to change it from the default value of 1. Prices Not Required in Price List: This option works together with the Inventory module and, if selected, allows users to enter items without a price level set up. This may sometimes be needed, but it is better to leave this unchecked and create price lists for all items to keep setup consistent. If this option is selected and no Password is supplied, any user can enter an item without a price level on a transaction. With a Password selected, a user will be prompted to enter the password before they can continue. Convert Functional Price: This option is only available if Prices Not Required in Price List is selected. With this checked, if a price cannot be found for the item in the currency that is being used on the transaction, the functional currency price will be converted as needed, using the current exchange rate. Data Entry Defaults: These are defaults to help speed up data entry: Quantity Shortage: Choose what users will see as the default option when they enter a quantity that is greater than what is in stock. Document Date: When entering a new SOP transaction, the Document Date will default to either the User Date (this is at the top of every window next to the User ID and Company Name) or the date of the previously saved transaction. For companies that typically enter all invoices with the same date (for example, the end of the previous month), it is best to default this to the Previous Doc. Date. For companies that want to have the current date defaulted for new transactions, choose User Date. Price Warning: You can decide whether to warn users if the price being used on a transaction they are entering is a default price for an item because a price has not been set up for the customer's price level. This may be useful if there should be a price set up on the item price list for each customer price level and can help avoid mistakes. If you choose to give a warning, using the Message option is recommended, as it will give users an indication of what the problem is; the Beep option may not be enough to catch a user's attention when you consider the typical noise level in an office and the number of beeps various applications generate. Requested Ship Date: When SOP orders are entered into Dynamics GP, a Requested Ship Date automatically populates to help determine when orders should be shipped and when purchase orders may need to be placed. A user can change the requested ship date on each line item for an order; however, if there is a typical default you can use this setting to help speed up data entry. Choices are either Document Date (order date) or number of Days After Doc. Date. If the latter is chosen, a box will appear next to this option where you can fill in the number of days (up to 999). Document Defaults: Similar to a customer with multiple Address IDs, each transaction type in SOP can have multiple document types set up to follow different rules. During transaction entry, users select which transaction type and document type to use for the proper set of rules to be followed. This section allows you to optionally set up a default for each transaction type, as well as the Site ID and Checkbook ID, to help speed up data entry. If you decide to set these up, you will need to come back to this section after you have created the various document types for each transaction type. If there is a transaction type you will not be using, or that you want to make sure users have to proactively select during transaction entry, you can leave it blank. Posting Accounts From: If Item is chosen, the system will first look at the item's accounts, then at the Inventory series of the company posting accounts to determine what GL accounts to use for SOP transactions. If Customer is chosen, the system will first look at the customer's accounts, then at the Sales series of the company posting accounts to determine what GL accounts to use for SOP transactions. Maintain History: It is recommended to keep all history and leave all of these selections checked. Decimal Places for Non-Inventoried Items: If non-inventory items are entered on SOP transactions, these settings will determine how many decimal places to use for Quantities and Currency. If Multicurrency is enabled, use the expansion button (blue arrow) next to the Currency field to enter these for each currency. The following screenshot shows a typical Sales Order Processing Setup window: Sales Document Setup You can set up as many document types as you need for each Sales Order Processing transaction type. One reason to set up different document types is to set up different transaction rules. Another reason is to use different numbering schemes or be able to segregate transactions for reporting purposes. Dynamics GP has two types of transactions that are basically the same—a fulfillment order and an invoice. In most places you will see these listed together as Fulfillment Order/Invoice. When using Workflow with the SOP module, a fulfillment order becomes a separate transaction type that, once completed, turns into an invoice automatically. To start setting up document types, click the Sales Document Setup button at the bottom of the Sales Order Processing Setup window. You will see a list of the available transaction types: The sections below will go over the setup for each transaction type. Quote Quotes are typically the start of the sales process and can be entered for customers or prospects. Prospects can be created "on the fly", as they are needed in Dynamics GP, and then transferred to customers if they accept a quote and place an order. The following list explains the fields on the Sales Quote Setup window: Quote ID: An ID for this quote document type—up to 15 characters. This ID is what users will need to type or select on a sales transaction when using this quote document type. Quote ID Next Number: If you are setting up multiple Quote IDs and would like for each to have its own numbering scheme, you can enter the next quote number here. Otherwise, if all quotes share one numbering scheme that can be entered during SOP Document Numbers setup, and you can leave this blank. Days to Expire: The default number of days a quote is valid for. After expiration, the user will not be able to transfer quotes to orders. Users can change this as needed on individual quotes. Comment ID: This is the default Comment ID for a quote. For example, if all quotes are set to expire in 30 days, you could create a comment that says Prices are guaranteed for 30 days from quote date and enter the corresponding Comment ID here. The comment would automatically populate on every quote created with this Quote ID. Format: Dynamics GP has four report formats available for each SOP transaction type: Blank Paper, Short Form, Long Form, and Other Form. If you need to set up multiple Quote IDs that show different information when printed, each Quote ID can be defaulted to use a different report format. Transfer Quote to Order: If this quote can be transferred to an order, check the box and enter an Order ID. Transfer Quote to Fulfillment Order/Invoice: If this quote can be transferred to a fulfillment order or invoice, check the box and enter a Fulfillment Order/Invoice ID. Default Quantities: This determines whether the item quantities for the quote default to Quantity to Invoice or Quantity to Order. This setting will depend on whether you plan to typically transfer a quote to an order or an invoice. Use Prospects: Leave this checked if you want to allow prospects to be used on quotes, otherwise only customers will be allowed. Allow Repeating Documents: A quote can be set up to repeat. This is more typical for orders, but may be needed at times for quotes. This setting determines whether this Quote ID will be allowed to repeat. Options and Password scrolling list: each of the Options listed has an optional Password, saved in clear text. Delete Documents: With this unchecked, once saved, a quote cannot be deleted. Edit Printed Documents: With this unchecked, once a quote is printed, it cannot be changed. Override Document Numbers: If users can change the quote numbers when they are creating them, check this option (once created a quote number cannot be changed). Void Documents: Select this if users are allowed to void quotes. If so, make sure that you have selected Track Voided Transactions in History on the Sales Order Processing Setup window. Order In Dynamics GP, a sales transaction can start out as an order, or quotes can be transferred to orders. Depending on setup, orders can allocate inventory items, thus making them unavailable for other orders or invoices. The following is a list of the fields on the Sales Order Setup window: Order ID: An ID for the order document type—the maximum length allowed is 15 characters. This is what users will need to type or select when using this order document type. Order ID Next Number: If you are setting up multiple Order IDs and would like for each to have its own numbering scheme, you can enter the Next Number here. Otherwise, if all orders will share one numbering scheme, that numbering scheme can be entered during SOP Document Numbers setup. Comment ID: The default Comment ID for an order. Format: Dynamics GP has four report formats available for each SOP transaction type: Blank Paper, Short Form, Long Form, and Other Form. If you need to set up multiple Order IDs that show different information when printed, each Order ID can be defaulted to use a different report format. Allocate by: This setting determines how (or if) inventory is allocated for the Order ID. Allocated inventory is still considered On Hand stock, but it is not available for other orders or invoices. Line Item: Select this if you want to allocate inventory as each line is entered. Users will need to choose an action on each line with a quantity shortage, which could significantly slow down order entry if most of the orders being entered do not have inventory in stock. Document/Batch: Inventory is not allocated as orders are entered and a separate allocation process is run either for each order or batch of orders. This gives less visibility of inventory availability during order entry; however, this can greatly speed it up because each line item is not checked and dealt with individually. None: Orders are not allocated at all and only after an order is transferred to an invoice or fulfillment order is inventory allocated. Transfer Order to Back Order: If this order type can be transferred to a back order, check this selection and enter a Back Order ID. Many companies do not use back orders, and simply use the back ordered quantity on orders to track back ordered items. Transfer Order to Fulfillment Order/Invoice: There is no checkbox here, as the basic functionality of an order is to transfer to an invoice. The only choice is what Fulfillment Order/Invoice ID to use. Options section: Allow Repeating Documents: If this Order ID can be set up to be repeated, mark this checkbox. Allocate by has to be set to Document/Batch or None to enable this option. Use Separate Fulfillment Process: If this option is checked, a separate step to fulfill orders will be needed prior to being able to transfer them to invoices. Allow all Back Ordered Items to Print on Invoice: If this is selected, all back ordered items will be transferred to an invoice with a fulfilled quantity of zero, allowing them to print on the invoice. Most companies like to show only items that are being billed on invoices, so they would leave this option unchecked. Credit Limit Hold ID: Holds can offer an additional level of control in Sales Order Processing. Override Quantity to Invoice with Quantity Fulfilled: Marking this option will set the Quantity to Invoice to be the same as Quantity Fulfilled, if Quantity Fulfilled is not zero. If this option is checked, Enable Quantity Cancelled in Sales Order Fulfillment becomes available. If Transfer Order to Back Order is selected, Enable Quantity to Back Order in Sales Order Fulfillment will also be enabled. Options and Password scrolling list: Each of these Options has an optional Password, saved in clear text. Allow Invoicing of Unfulfilled or Partially Fulfilled Orders: This option is activated only when the Use Separate Fulfillment Process option is selected. Otherwise, if Use Separate Fulfillment Process is not selected, invoicing an unfulfilled or partially fulfilled order will not be allowed, even though you can select this option. Delete Documents: With this unchecked, a saved order cannot be deleted. Edit Printed Documents: With this unchecked, once an order is printed, it cannot be changed. Override Document Numbers: If users can change the order numbers when they are creating them, check this option (once created, an order number cannot be changed). Void Documents: Select this option if users are allowed to void orders. If so, make sure that you have selected Track Voided Transactions in History in the Sales Order Processing Setup window. The following screenshot shows a typical Sales Order Setup window:

  Scribus 1.3.5: Beginner's Guide You might be fully interested in free software, may be running Linux or any other system except Apple Mac OS or Microsoft Windows, and in this case, you don't have much choice except for Scribus, Scribus, or Scribus. This is mostly because proprietary equivalent software such as Adobe InDesign or Quark Xpress is not available for Linux-based platforms. Desktop publishing software versus text processors If you have already used layout software before, these arguments are not new to you. However, if you come from any other computer-assisted profession, you may be surprised at the way such software is organized. Especially, most of you would have certainly used text processors such as Microsoft Word, OpenOffice.org Writer, and maybe Microsoft Publisher. Once you go deeper into the details, you'll see how Scribus is different. I've heard many people explain that they were trying Scribus, because they thought or heard it was a better piece of software. Text processors are very qualitative when it's time to handle text (and this is an important point) but not when there is a need to customize a document. Just take a look around: you can identify any magazine or any book collection because of their visual identity, which is made possible by the Desktop Publishing set of software. Could you identify as easily the origin of a Microsoft Word or OpenOffice document? I'm not sure, because all of these documents will be very similar. Generally, you won't use a layout program if you need to save time and work very quickly, because it is not intended to save time, but to let you be as free as possible to create a unique document: the one that will make you change the world, or the one that will help you improve the communication of your company and make it more efficient. Scribus will give you everything to be as productive as possible. However, every time you need to choose a color, every time you need to add a shape, or every time you need to change the text settings, every single little task that you will find yourself doing to get the best graphically designed final document will add to the time taken. This is a very important point if you want your layout project to succeed. I have experienced many projects where people really underestimated the time taken to perform these tasks. To help you create your document, remember that a layout program is not based on text handling, but on the page. In Scribus, the page is an object that you'll be able to manipulate. On the page, you'll add shapes or frames that you'll place precisely, one by one, and each of these will have their own properties. Especially in a layout program, images are drastically apart from the text, whereas in a text processor both will be in the same flow. This again results in a different way of considering the elements you will have and may change the way you work. This is for the best, and once you get used to this, once you have the major but quite simple software possibilities integrated, and once you have the print process specificities in your work, you'll be more free than you've ever been to create a unique document. This document will be the result of your own creativity and not only the default settings defined by a product or another. To InDesign and Xpress users If you've already used a layout program, you will certainly have questions such as: Is this software as good as mine? Can I import what I've done with my actual software so that I won't have to do everything again? Will I have many things to learn to be as productive as I actually am? For the first question, Scribus is in some ways very good and has very original features but in some other ways it is less than perfect. The real question is: what do you already use in the software you have and does Scribus have it? I used to be an Xpress teacher and I've often met graphic designers who don't even use styles or master page and Scribus has it. Scribus can use spot color, set bleeds, and many other features required. As an answer to the second question I could simply say "No"—mainly "No". As far as I know, it's always the tricky part in whatever software you use. Scribus will soon be able to import Xpress tags and IDXML, but it is still in development and is actually not usable; if you use Microsoft Publisher, there is really no way. As for the last question, I don't think there are so many things to learn. Scribus has an original user interface but can be inspired by some de facto standards. And mainly, the principles are the same in Scribus and in InDesign or Xpress. Of course, you will use some of your habits, but in two or three days of Scribus testing, everything will be perfect again and you'll feel comfortable with it. Shortcuts will certainly be the most difficult to learn. Xpress users, especially, use them a lot and even InDesign users use them for text handling. Scribus shortcut defaults are much simpler. You can use the Keyboard Shortcuts category of Preferences to change them. Simply select the function for the shortcut you want to change in the Action list, click on the User Defined Key option, click on the Set Key button, and perform the shortcut you'd like to assign. If it is already being used, you won't be able to assign it unless you find where it is assigned and erase it. Applying master pages: In Scribus, unlike in InDesign, the left-hand side master page can be applied to a right-hand side page. Scribus never automates the way master pages are applied, except when creating the document. So, if you're confused by that, don't worry; you'll be able to do what you want even if you have chosen the bad side. Frame conversion and text to outlines: In Scribus, frames are central. Adobe InDesign, in some ways tries to avoid them by using a single tool for text edit and text frame, and at the same time it can import pictures without requiring a frame. But in any case, a frame is made even if automatically. Another good feature with Scribus frames is that they can easily be converted to any other kind of frame. So, if you created a Text Frame and want to put an image into it, you can still do so without deleting and drawing a new frame. This is very important because the default frame shape is set to rectangle and cannot be changed. Importing several pictures In Scribus, it is actually impossible to import several pictures (as it can be done in InDesign) at once. This can be done with Scribus Python scripting. There are already some scripts for this on the Scribus wiki at http://wiki.scribus.net. Check for the script that suits your needs. Summary In this article we saw how Scribus is different from other kinds of software. Further resources on this subject: Scribus: Managing Colors [Article] Scribus: Importing Images [Article] Scribus: Creating a Layout [Article] Working with Colors in Scribus [Article] Scribus: Manipulate and Place Objects in a Layout [Article]

This article by Javier Collado expands the set of web scraping techniques shown in his previous article by looking closely into a more complex problem that cannot be solved with the tools that were explained there. For those who missed out on that article, here's the link. Web Scraping with Python This article will show how to extract the desired information using the same three steps when the web page is not written directly using HTML, but is auto-generated using JavaScript to update the DOM tree. As you may remember from that article, web scraping is the ability to extract information automatically from a set of web pages that were designed only to display information nicely to humans; but that might not be suitable when a machine needs to retrieve that information. The three basic steps that were recommended to be followed when performing a scraping task were the following: Explore the website to find out where the desired information is located in the HTML DOM tree Download as many web pages as needed Parse downloaded web pages and extract the information from the places found in the exploration step What should be taken into account when the content is not directly coded in the HTML DOM tree? The main difference, as you probably have already noted, is that using the downloading methods that were suggested in the previous article (urllib2 or mechanize) just don't work. This is because they generate an HTTP request to get the web page and deliver the received HTML directly to the scraping script. However, the pieces of information that are auto-generated by the JavaScript code are not yet in the HTML file because the code is not executed in any virtual machine as it happens when the page is displayed in a web browser. Hence, instead of relying on a library that generates HTTP requests, we need a library that behaves as a real web browser, or even better, a library that interacts with a real web browser. So that we are sure that we obtain the same data as we see when manually opening a page in a web browser. Please remember that the aim of web scraping is actually parsing the data that a human user sees, so interacting with a real web browser would be a really nice feature. Is there any tool out there to perform that? Fortunately, the answer is yes. In particular, there are a couple of tools used for web testing automation that can be used to solve the JavaScript execution problem: Selenium and Windmill . For the code samples in the sections below, Windmill is used. Any choice would be fine as both of them are well documented and stable tools ready to be used for production. Let's now follow the same three steps that were suggested in the previous article to solve the scraping of the contents of a web page that is partly generated using JavaScript code. Explore Imagine that you are a fan of NASA Image of the day gallery. You want to get a list of the names of all the images in the gallery together with the link to the whole resolution picture just in case you decide to download it later to use as a desktop wallpaper. The first thing to do is to locate the data that has to be extracted on the desired web page. In the case of the Image of the day gallery (see screenshot below), there are three elements that are important to note: Title of the image that is being currently displayed Link to the image full resolution file Next link to make it possible navigate through all the images To find out the location of each piece of interesting information, as it was already suggested in the previous article, it's better to use a tool such as Firebug whose inspect functionality can be really useful. The following picture, for example, shows the location of the image title inside an h3 tag: The other two fields can be located as easily as the title, so no further explanation will be given here. Please refer to the previous article for further information. Download As explained in the introduction, to download the content of the web page, we will use Windmill as it allows the JavaScript code to execute in the web browser before getting the page content. Because Windmill is mostly a testing library, instead of writing a script that calls the Windmill API, I will write a test case for Windmill to navigate through all the image web pages. The code for the test should be as follows: 1 def test_scrape_iotd_gallery(): 2 """ 3 Scrape NASA Image of the Day Gallery 4 """ 5 # Extra data massage for BeautifulSoup 6 my_massage = get_massage() 7 8 # Open main gallery page 9 client = WindmillTestClient(__name__) 10 client.open(url='http://www.nasa.gov/multimedia/imagegallery/iotd.html') 11 12 # Page isn't completely loaded until image gallery data 13 # has been updated by javascript code 14 client.waits.forElement(xpath=u"//div[@id='gallery_image_area']/img", 15 timeout=30000) 16 17 # Scrape all images information 18 images_info = {} 19 while True: 20 image_info = get_image_info(client, my_massage) 21 22 # Break if image has been already scrapped 23 # (that means that all images have been parsed 24 # since they are ordered in a circular ring) 25 if image_info['link'] in images_info: 26 break 27 28 images_info[image_info['link']] = image_info 29 30 # Click to get the information for the next image 31 client.click(xpath=u"//div[@class='btn_image_next']") 32 33 # Print results to stdout ordered by image name 34 for image_info in sorted(images_info.values(), 35 key=lambda image_info: image_info['name']): 36 print ("Name: %(name)sn" 37 "Link: %(link)sn" % image_info) As it can be seen, the usage of Windmill is similar to other libraries such as mechanize. For example, first of all a client object has to be created to interact with the browser, (line 9) and later, the main web page, that is going to be used to navigate through all the information, has to be opened (line 10). Nevertheless, it also includes some facilities that take into account JavaScript code as shown at line 14. In this line, the waits.forElement method has been used to look for DOM element that is filled by the JavaScript code so when that element, in this case the big image in the image gallery, is displayed, the rest of the script can proceed. It is important to note here that the web page processing doesn't start when the page is downloaded (this happens after line 10), but when there's some evidence that JavaScript code has finished the DOM tree manipulation. For navigating through all the pages that contain the information needed, this is just a matter of pressing over the next arrow (line 30). As the images are ordered in a circular buffer, the point when it is decided to stop is when the same image link has been parsed twice (line 25). To execute the script, instead of launching it as we would normally do for a python script, we should call it through the Windmill script to properly initialize the environment: $ windmill firefox test=nasa_iotd.py As it can be seen in the following screenshot, Windmill takes care of opening a browser (Firefox in this case) window and a controller window in which it's possible to see the commands that the script is executing (several clicks on next in the example): The controller window is really interesting because not only does it display the progress of the test cases, but also allows to enter/record actions interactively, which is a nice feature when trying things out. In particular, the recording may be used under some situations to replace Firebug in the exploration step. This is because the captured actions may be stored in a script without spending much time in xpath expressions. For more information about how to use Windmill and the complete API, please refer to the Windmill documentation.

In our previous Scripty2 article, we saw the basics of the Scripty2 library, we saw the UI elements available, the fx transitions and some effects. We even build an small example of a image slide effect. This article is going to be a little more complex, just a little, so we are able to build a fully working image slideshow. If you haven't read our previous article, it would be interesting and useful if you do so before continuing with this one. You can find the article here: Scripty2 in Action Well, now we can continue. But first, why don't we take a look at what we are going to build in this article: As we can see in the image we have three main elements. The biggest one will be where the main images are placed, there is also one element where we will make the slide animation to take place. There's also a caption and some descriptive text for the image and, to the right, we have the miniatures slider, clicking on them will make the main image to also slide out and the new one to appear. Seems difficult to achieve? Don't worry, we will make it step by step, so it's very easy to follow. Our main steps are going to be these: First we we will create the html markup and css necessary, so our page looks like the design. Next step will be to create the slide effect for the available images to the right. And our final step will be to make the thumbnail slider to work. Good, enough talking, let's start with the action. Creating the necessary html markup and css We will start by creating an index.html file, with some basic html in it, just like this: <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd"><html xml_lang="es-ES" lang="es-ES" ><head><meta http-equiv="content-type" content="text/html; charset=utf-8" /><title>Scripty2</title><link rel="stylesheet" href="css/reset.css" type="text/css" /><link rel="stylesheet" href="css/styles.css" type="text/css" /> </head><body id="article"> <div id="gallery"> </div> <script type="text/javascript" src="js/prototype.s2.min.js"></script> </body></html> Let's take a look at what we have here, first we are including a reset.css file, this time, for a change, we are going to use the Yahoo one, which we can find in this link: http://developer.yahoo.com/yui/3/cssreset/ We will create a folder called css, and a reset.css file inside it, where we will place the yahoo code. Note that we could also link the reset directly from the yahoo site: <link rel="stylesheet" type="text/css" href="http://yui.yahooapis.com/3.1.1/build/cssreset/reset-min.css"> Just below the link to the reset.css file, we find a styles.css file. This file will be the place where we are going to place our own styles. Next we find a div: <div id="gallery"> </div> This will be our placeholder for the entire slideshow gallery. Then one last thing, the link to the Scripty2 library: <script type="text/javascript" src="js/prototype.s2.min.js"></script> Next step will be to add some styles in our styles.css file: html, body{ background-color: #D7D7D7; }#gallery{ font-family: arial; font-size: 12px; width: 470px; height: 265px; background-color: #2D2D2D; border: 1px solid #4F4F4F; margin: 50px auto; position: relative;} Mostly, we are defining a background colour for the page, and then some styles for our gallery div like its width, height, margin and also a background colour. With all this our page will look like the following screenshot: We will need to keep working on this, first in our index.html file, we need a bit more of code: <div id="gallery"> <div id="photos_container"> <img src="./img/image_1.jpg" title="Lorem ipsum" /> <img src="./img/image_1.jpg" title="Lorem ipsum" /> <img src="./img/image_1.jpg" title="Lorem ipsum" /> <img src="./img/image_1.jpg" title="Lorem ipsum" /> </div> <div id="text_container"> <p><strong>Lorem Ipsum</strong><br/>More lorem ipsum but smaller text</p> </div> <div id="thumbnail_container"> <img src="./img/thumbnail_1.jpg" title="Lorem ipsum"/> <img src="./img/thumbnail_1.jpg" title="Lorem ipsum"/> <img src="./img/thumbnail_1.jpg" title="Lorem ipsum"/> <img src="./img/thumbnail_1.jpg" title="Lorem ipsum"/> </div></div> We have placed three more divs inside our main gallery one. One of them would be the photos-photos_container, where we place our big photos. The other two will be one for the text- text_container, and the other for the thumbnail images, that will be thumbnail_container. After we have finished with our html, we need to work with our css. Remember we are going to do it in our styles.css: #photos_container{ width: 452px; height: 247px; background-color: #fff; border: 1px solid #fff; margin-top: 8px; margin-left: 8px; overflow: hidden;}#text_container{ width: 377px; height: 55px; background-color: #000000; color: #ffffff; position: absolute; z-index: 2; bottom: 9px; left: 9px;}#text_container p{ padding: 5px; }#thumbnail_container{ width: 75px; height: 247px; background-color: #000000; position: absolute; z-index: 3; top: 9px; right: 9px;}#thumbnail_container img{ margin: 13px 13px 0px 13px; }strong{ font-weight: bold; } Here we have added styles for each one of our divs, just quite basic styling for the necessary elements, so our page now looks a bit more like the design: Though this looks a bit better than before, it still does nothing and that's going to be our next step. Creating the slide effect First we need some modifications to our html code, this time, though we could use id tags in order to identify elements, we are going to use rel tags. So we can see a different way of doing things. Let's then modify the html: <div id="photos_container"> <img src="./img/image_1.jpg" title="Lorem ipsum" rel="1"/> <img src="./img/image_1.jpg" title="Lorem ipsum" rel="2"/> <img src="./img/image_1.jpg" title="Lorem ipsum" rel="3"/> <img src="./img/image_1.jpg" title="Lorem ipsum" rel="4"/> </div> <div id="text_container"> <p><strong>Lorem Ipsum</strong><br/>More lorem ipsum but smaller text</p> </div> <div id="thumbnail_container"> <img src="./img/thumbnail_1.jpg" title="Lorem ipsum" rel="1"/> <img src="./img/thumbnail_1.jpg" title="Lorem ipsum" rel="2"/> <img src="./img/thumbnail_1.jpg" title="Lorem ipsum" rel="3"/> <img src="./img/thumbnail_1.jpg" title="Lorem ipsum" rel="4"/> </div> Note the difference, we have added rel tags to each one of the images in every one of the divs. Now we are going to add a script after the next line: <script type="text/javascript" src="js/prototype.s2.min.js"></script> The script is going to look like this: <script type="text/javascript"> $$('#thumbnail_container img').each(function(image){ image.observe('click', function(){ $$('#photos_container img[rel="'+this. readAttribute('rel')+'"]').each(function(big_image){ alert(big_image.readAttribute('title')); }) }); });</script> First we select all the images inside the #thumbnail_container div: $$('#thumbnail_container img') Then we use the each function to loop through the results of this selection, and add the click event to them: image.observe('click', function(){ This event will fire a function, that will in turn select the bigger images, the one in which the rel attribute is equal to the rel attribute of the thumbnail: $$('#photos_container img[rel="'+this.readAttribute('rel')+'"]').each(function(big_image){ We know this will return only one value, but as the selected could, theoretically, return more than one value, we need to use the each function again. Note that we use the this.readAttribute('rel') function to read the value of the rel attribute of the thumbnail image. Then we use the alert function to show the title value of the big image: alert(big_image.readAttribute('title')); If we now click on any of the thumbnails, we will get an alert, just like this: We have done this just to check if our code is working and we are able to select the image we want. Now we are going to change this alert for something more interesting. But first, we need to do some modifications to our styles.css file, we will modify our photos container styles: #photos_container{ width: 452px; height: 247px; background-color: #fff; border: 1px solid #fff; margin-top: 8px; margin-left: 8px; overflow: hidden; position: relative;} Only to add the position relate in it, this way we will be able to absolute position the images inside it, adding these styles: #photos_container img{ position: absolute;} With these changes we are done with the styles for now, return to the index.html file, we are going to introduce some modifications here too: <script type="text/javascript" src="js/prototype.s2.min.js"></script> <script type="text/javascript"> var current_image = 1; $$('#photos_container img').each(function(image){ var pos_y = (image.readAttribute('rel') * 247) - 247; image.setStyle({top: pos_y+'px'}); })... I've placed the modifications in bold, so we can concentrate on them. What have we here? We are creating a new variable, current_image, so we can save which is the current active image. At page load this will be the first one, so we are placing 1 to its value. Next we loop through all of the big images, reading its rel attribute, to know which one of the images are we working with. We are multiplying this rel attribute with 247, which is the height of our div. We are also subtracting 247 so the first image is at 0 position. note Just after this, we are using prototype's setStyle function to give each image its proper top value. Now, I'm going to comment one of the css styles: #photos_container{ width: 452px; height: 247px; background-color: #fff; border: 1px solid #fff; margin-top: 8px; margin-left: 8px; //overflow: hidden; position: relative;} You don't need to do this, I' will do it for you, so we can see the result of our previous coding, and then I will turn back and leave it as it was before, prior to continuing. So our page would look like the next image: As we see in the image, all images are one on top of the others, this way we will be able to move them in the y axis. We are going to achieve that by modifying this code: $$('#thumbnail_container img').each(function(image){ image.observe('click', function(){ $$('#photos_container img[rel="'+this. readAttribute('rel')+'"]').each(function(big_image){ alert(big_image.readAttribute('title')); }) }); });

When the iPhone first came out there was only one screen size to worry about 320, 480. Then the Retina screen was introduced doubling the screen's resolution. Apple quickly introduced the iPhone 5 and added an extra 88 points to the bottom. With the most recent line of iPhones two more sizes were added to the mix. Before even mentioning the iPad line that is already five different combinations of heights and widths to account for. To help remedy this growing number of sizes and resolutions Apple introduced Auto Layout with iOS 6. Auto Layout is a dynamic way of laying out views with constraints and rules to let the content fit on multiple screen sizes; think "responsive" for mobile. Lots of layouts are possible with Auto Layout but some require an extra bit of work. One of the more common, albeit tricky, arrangements is to have evenly spaced elements. Having the view scale up to different resolutions and look great on all devices isn't hard and can be done in both Interface Builder or manually in code. Let's walk through how to evenly space views with Auto Layout using Xcode's Interface Builder. Using Interface Builder The easiest way to play around and test layout in IB is to create a new Single View Application iOS project.   Open Main.storyboard and select ViewController on the left. Don't worry that it is showing a square view since we will be laying everything out dynamically. The first addition to the view will be the three `UIView`s we will be evenly spacing. Add them along the view from left to right and assign different colors to each. This will make it easier to distinguish them later. Don't worry about where they are we will fix the layout soon enough.   Spacer View Layout Ideally we would be able to add constraints that evenly space these out directly. Unfortunately, you can not set *equals* restrictions on constraints, only on views. What that means is we have to create spacer views in between our content and then set equal constraints on those. Add four more views, one between the edges and the content views.   Before we add our first constraint let's name each view so we can have a little more context when adding their attributes. One of the most frustrating things when working with Auto Layout is seeing the little red arrow telling you something is wrong. Let's try and incrementally add constraints and get back to a working state as quickly as possible. The first item we want to add will constrain the Left Content view using the spacer. Select the Left Spacer and add left 20, top 20, and bottom 20 constraints.   To fix this first error we need to assign a width to the spacer. While we will be removing it later it makes sense to always have a clean slate when moving on to another view. Add in a width (50) constraint and let IB automatically and update its frame.   Now do the same thing to the Right Spacer.   Content View Layout We will remove the width constraints when everything else is working correctly. Consider them temporary placeholders for now. Next lets lay out the Left Content view. Add a left 0, top 20, bottom 20, width 20 constraint to it.   Follow the same method on the Right Content view.   Twice more follow the same procedure for the Middle Spacer Views giving them left/right 0, top 20, bottom 20, width 50 constraints.   Finally, let's constrain the Middle Content view. Add left 0, top 20, right 0, bottom 20 constraints to it and lay it out.   Final Constraints Remember when I said it was tricky? Maybe a better way to describe this process is long and tedious. All of the setup we have done so far was to set us up in a good position to give the constraints we actually want. If you look at the view it doesn't look very special and it won't even resize the right way yet. To start fix this we bring in the magic constraint of this entire example, Equal Widths on the spacer views. Go ahead and delete the four explicit Width constraints on the spacer views and add an Equal Width constraint to each. Select them all at the same time then add the constraint so they work off of each other.   Finally, set explicit widths on the three content views. This is where you can start customizing the layout to have it look the way you want. For my view I want the three views to be 75 points wide so I removed all of the Width constraints and added them back in for each. Now set the background color of the four spacer views to clear and hide them. Running the app on different size simulators will produce the same result: the three content views remain the same width and stay evenly spaced out along the screen. Even when you rotate the device the views remain spaced out correctly.   Try playing around with different explicit widths of the content views. The same technique can be used to create very dynamic layouts for a variety of applications. For example, this procedurecan be used to create a table cell with an image on the left, text in the middle, and a button on the right. Or it can make one row in a calculator that sizes to fit the screen of the device. What are you using it for? About the author Joe Masilotti is a test-driven iOS developer living in Brooklyn, NY. He contributes to open-source testing tools on GitHub and talks about development, cooking, and craft beer on Twitter.

Overview of the tutorial You will begin by creating an ODBC datasource for accessing data on the Postgres server. Using the User DSN created you will be connecting to the Postgres server data. You will derive a dataset from the table which you will be using to display in a datagrid view on a form in a windows application. We start with the Categories table that was migrated from MS SQL Server 2008. This table with all of its columns is shown in the Postgres studio in the next figure. Creating the ODBC DSN Navigate to Start | Control Panel | Administrative Tools | Data Sources (ODBC) to bring up the ODBC Database Manager window. Click on Add.... In the Create New Data Source scroll down to EnterpriseDB 8.2 under the list heading Name as shown. Click Finish. The EnterpriseDB ODBC Driver page gets displayed as shown. Accept the default name for the Data Source(DSN) or, if you prefer, change the name. Here the default is accepted. The Database, Server, User Name, Port and the Password should all be available to you [Read article 1]. If you click on the option button Datasource you display a window with two pages as shown. Make no changes to the pages and accept defaults but make sure you review the pages. Click OK and you will be back in the EnterpriseDB Driver window. If you click on the button Global the Global Settings window gets displayed (not shown). These are logging options as the page describes. Click Cancel to the Global Settings window. Click on the Test button and verify that the connection was successful. Click on the Save button and save the DSN under the list heading User DSN. The DSN EnterpriseDB enters the list of DSN's created as shown here. Create a Windows Forms application and Establish a connection to Postgres Open Visual Studio 2008 from its shortcut. Click File | New | Project... and open the New Project window. Choose a windows forms project for Framework 2.0. Besides Framework 2.0 you can also create projects in other versions in Visual Studio 2008. In Server Explorer window double click the Connection icon as shown. This brings up the Add Connection window as shown. Click on Change... button to display the Change Data Source window. Scroll up and select Microsoft ODBC Data Source as shown. Click OK. Click on the drop-down handle for the option Use user or system data source name and choose EnterpriseDB you created earlier as shown. Insert User Name and Password and click on the Test Connection button. You should get a connection succeeded message as shown. Click OK on the message screen as well as to the add connection window. The connection appears in the Visual Studio 2008 in the Server Explorer as shown.     Displaying data from the table Drag and drop a DataGridView under Data in the Toolbox onto the form as shown (shown with SmartTasks handle clicked) Click on Choose Data Source handle to display a drop-down menu as shown below. Click on Add Project Data Source at the bottom. This displays the Choose a Data Source Type page of the Data Source Configuration Wizard. Accept the default datasource type and click Next. In the Choose Your Data Connection page of the wizard choose the ODBC.localhost.PGNorthwind as shown in the drop-down list. Click Next in the page that gets displayed and accept the default to save the connection string to the application configuration file as shown. Click Next. In the Choose Your Database Objects page, expand Tables and choose the categories table as shown. The default Dataset name can be changed. Herein the default is accepted. Click Finish. The DatagridView on Form1 gets displayed with two columns and a row but can be extended to the right by using drag handles to reveal all the four columns as shown. Three other objects PGNorthwindDataSet, CategoriesBindingSource, and CategoriesTableAdapter are also added to the control tray as shown. The PGNorthwindDataset.xsd file gets added to the project. Now build the project and run. The Form 1 gets displayed with the data from the PGNorthwind database as shown. In the design view of the form few more tasks have been added as shown. Here you can Add Query... to filter the data displayed; Edit the details of the columns and you can choose to add a column if you had chosen fewer columns from the original table. For example, Edit Column brings up its editor as shown where you can make changes to the styles if you desire to do so. The next figure shows slightly modified form by editing the columns and resizing the cell heights as shown. Summary A step-by-step procedure was described to display the data stored in a table in the Postgres database in a Windows Forms application. Procedure to create an ODBC DSN was also described. Using this ODBC DSN a connection was established to the Postgres server in Visual Studio 2008.

In this article by Edwood Ng and Vineeth Mohan, authors of the book Lucene 4 Cookbook, we will cover the following recipes: Performing advanced filtering Creating a custom filter Searching with QueryParser TermQuery and TermRangeQuery BooleanQuery PrefixQuery and WildcardQuery PhraseQuery and MultiPhraseQuery FuzzyQuery (For more resources related to this topic, see here.) When it comes to search application, usability is always a key element that either makes or breaks user impression. Lucene does an excellent job of giving you the essential tools to build and search an index. In this article, we will look into some more advanced techniques to query and filter data. We will arm you with more knowledge to put into your toolbox so that you can leverage your Lucene knowledge to build a user-friendly search application. Performing advanced filtering Before we start, let us try to revisit these questions: what is a filter and what is it for? In simple terms, a filter is used to narrow the search space or, in another words, search within a search. Filter and Query may seem to provide the same functionality, but there is a significant difference between the two. Scores are calculated in querying to rank results, based on their relevancy to the search terms, while a filter has no effect on scores. It's not uncommon that users may prefer to navigate through a hierarchy of filters in order to land on the relevant results. You may often find yourselves in a situation where it is necessary to refine a result set so that users can continue to search or navigate within a subset. With the ability to apply filters, we can easily provide such search refinements. Another situation is data security where some parts of the data in the index are protected. You may need to include an additional filter behind the scene that's based on user access level so that users are restricted to only seeing items that they are permitted to access. In both of these contexts, Lucene's filtering features will provide the capability to achieve the objectives. Lucene has a few built-in filters that are designed to fit most of the real-world applications. If you do find yourself in a position where none of the built-in filters are suitable for the job, you can rest assured that Lucene's expansibility will allow you to build your own custom filters. Let us take a look at Lucene's built-in filters: TermRangeFilter: This is a filter that restricts results to a range of terms that are defined by lower bound and upper bound of a submitted range. This filter is best used on a single-valued field because on a tokenized field, any tokens within a range will return by this filter. This is for textual data only. NumericRangeFilter: Similar to TermRangeFilter, this filter restricts results to a range of numeric values. FieldCacheRangeFilter: This filter runs on top of the number of range filters, including TermRangeFilter and NumericRangeFilter. It caches filtered results using FieldCache for improved performance. FieldCache is stored in the memory, so performance boost can be upward of 100x faster than the normal range filter. Because it uses FieldCache, it's best to use this on a single-valued field only. This filter will not be applicable for multivalued field and when the available memory is limited, since it maintains FieldCache (in memory) on filtered results. QueryWrapperFilter: This filter acts as a wrapper around a Query object. This filter is useful when you have complex business rules that are already defined in a Query and would like to reuse for other business purposes. It constructs a Query to act like a filter so that it can be applied to other Queries. Because this is a filter, scoring results from the Query within is irrelevant. PrefixFilter: This filter restricts results that match what's defined in the prefix. This is similar to a substring match, but limited to matching results with a leading substring only. FieldCacheTermsFilter: This is a term filter that uses FieldCache to store the calculated results in memory. This filter works on a single-valued field only. One use of it is when you have a category field where results are usually shown by categories in different pages. The filter can be used as a demarcation by categories. FieldValueFilter: This filter returns a document containing one or more values on the specified field. This is useful as a preliminary filter to ensure that certain fields exist before querying. CachingWrapperFilter: This is a wrapper that adds a caching layer to a filter to boost performance. Note that this filter provides a general caching layer; it should be applied on a filter that produces a reasonably small result set, such as an exact match. Otherwise, larger results may unnecessarily drain the system's resources and can actually introduce performance issues. If none of the above filters fulfill your business requirements, you can build your own, extending the Filter class and implementing its abstract method getDocIdSet (AtomicReaderContext, Bits). How to do it... Let's set up our test case with the following code: Analyzer analyzer = new StandardAnalyzer(); Directory directory = new RAMDirectory(); IndexWriterConfig config = new   IndexWriterConfig(Version.LATEST, analyzer); IndexWriter indexWriter = new IndexWriter(directory, config); Document doc = new Document(); StringField stringField = new StringField("name", "",   Field.Store.YES); TextField textField = new TextField("content", "",   Field.Store.YES); IntField intField = new IntField("num", 0, Field.Store.YES); doc.removeField("name"); doc.removeField("content"); doc.removeField("num"); stringField.setStringValue("First"); textField.setStringValue("Humpty Dumpty sat on a wall,"); intField.setIntValue(100); doc.add(stringField); doc.add(textField); doc.add(intField); indexWriter.addDocument(doc); doc.removeField("name"); doc.removeField("content"); doc.removeField("num"); stringField.setStringValue("Second"); textField.setStringValue("Humpty Dumpty had a great fall."); intField.setIntValue(200); doc.add(stringField); doc.add(textField); doc.add(intField); indexWriter.addDocument(doc); doc.removeField("name"); doc.removeField("content"); doc.removeField("num"); stringField.setStringValue("Third"); textField.setStringValue("All the king's horses and all the king's men"); intField.setIntValue(300); doc.add(stringField); doc.add(textField); doc.add(intField); indexWriter.addDocument(doc); doc.removeField("name"); doc.removeField("content"); doc.removeField("num"); stringField.setStringValue("Fourth"); textField.setStringValue("Couldn't put Humpty together   again."); intField.setIntValue(400); doc.add(stringField); doc.add(textField); doc.add(intField); indexWriter.addDocument(doc); indexWriter.commit(); indexWriter.close(); IndexReader indexReader = DirectoryReader.open(directory); IndexSearcher indexSearcher = new IndexSearcher(indexReader); How it works… The preceding code adds four documents into an index. The four documents are: Document 1 Name: First Content: Humpty Dumpty sat on a wall, Num: 100 Document 2 Name: Second Content: Humpty Dumpty had a great fall. Num: 200 Document 3 Name: Third Content: All the king's horses and all the king's men Num: 300 Document 4 Name: Fourth Content: Couldn't put Humpty together again. Num: 400 Here is our standard test case: IndexReader indexReader = DirectoryReader.open(directory); IndexSearcher indexSearcher = new IndexSearcher(indexReader); Query query = new TermQuery(new Term("content", "humpty")); TopDocs topDocs = indexSearcher.search(query, FILTER, 100); System.out.println("Searching 'humpty'"); for (ScoreDoc scoreDoc : topDocs.scoreDocs) {    doc = indexReader.document(scoreDoc.doc);    System.out.println("name: " + doc.getField("name").stringValue() +        " - content: " + doc.getField("content").stringValue() + " - num: " + doc.getField("num").stringValue()); } indexReader.close(); Running the code as it is will produce the following output, assuming the FILTER variable is declared: Searching 'humpty' name: First - content: Humpty Dumpty sat on a wall, - num: 100 name: Second - content: Humpty Dumpty had a great fall. - num: 200 name: Fourth - content: Couldn't put Humpty together again. - num: 400 This is a simple search on the word humpty. The search would return the first, second, and fourth sentences. Now, let's take a look at a TermRangeFilter example: TermRangeFilter termRangeFilter = TermRangeFilter.newStringRange("name", "A", "G", true, true); Applying this filter to preceding search (by setting FILTER as termRangeFilter) will produce the following output: Searching 'humpty' name: First - content: Humpty Dumpty sat on a wall, - num: 100 name: Fourth - content: Couldn't put Humpty together again. - num: 400 Note that the second sentence is missing from the results due to this filter. This filter removes documents with name outside of A through G. Both first and fourth sentences start with F that's within the range so their results are included. The second sentence's name value Second is outside the range, so the document is not considered by the query. Let's move on to NumericRangeFilter: NumericRangeFilter numericRangeFilter = NumericRangeFilter.newIntRange("num", 200, 400, true, true); This filter will produce the following results: Searching 'humpty' name: Second - content: Humpty Dumpty had a great fall. - num: 200 name: Fourth - content: Couldn't put Humpty together again. - num: 400 Note that the first sentence is missing from results. It's because its num 100 is outside the specified numeric range 200 to 400 in NumericRangeFilter. Next one is FieldCacheRangeFilter: FieldCacheRangeFilter fieldCacheTermRangeFilter = FieldCacheRangeFilter.newStringRange("name", "A", "G", true, true); The output of this filter is similar to the TermRangeFilter example: Searching 'humpty' name: First - content: Humpty Dumpty sat on a wall, - num: 100 name: Fourth - content: Couldn't put Humpty together again. - num: 400 This filter provides a caching layer on top of TermRangeFilter. Results are similar, but performance is a lot better because the calculated results are cached in memory for the next retrieval. Next is QueryWrapperFiler: QueryWrapperFilter queryWrapperFilter = new QueryWrapperFilter(new TermQuery(new Term("content", "together"))); This example will produce this result: Searching 'humpty' name: Fourth - content: Couldn't put Humpty together again. - num: 400 This filter wraps around TermQuery on term together on the content field. Since the fourth sentence is the only one that contains the word "together" search results is limited to this sentence only. Next one is PrefixFilter: PrefixFilter prefixFilter = new PrefixFilter(new Term("name", "F")); This filter produces the following: Searching 'humpty' name: First - content: Humpty Dumpty sat on a wall, - num: 100 name: Fourth - content: Couldn't put Humpty together again. - num: 400 This filter limits results where the name field begins with letter F. In this case, the first and fourth sentences both have the name field that begins with F (First and Fourth); hence, the results. Next is FieldCacheTermsFilter: FieldCacheTermsFilter fieldCacheTermsFilter = new FieldCacheTermsFilter("name", "First"); This filter produces the following: Searching 'humpty' name: First - content: Humpty Dumpty sat on a wall, - num: 100 This filter limits results with the name containing the word first. Since the first sentence is the only one that contains first, only one sentence is returned in search results. Next is FieldValueFilter: FieldValueFilter fieldValueFilter = new FieldValueFilter("name1"); This would produce the following: Searching 'humpty' Note that there are no results because this filter limits results in which there is at least one value on the filed name1. Since the name1 field doesn't exist in our current example, no documents are returned by this filter; hence, zero results. Next is CachingWrapperFilter: TermRangeFilter termRangeFilter = TermRangeFilter.newStringRange("name", "A", "G", true, true); CachingWrapperFilter cachingWrapperFilter = new CachingWrapperFilter(termRangeFilter); This wrapper wraps around the same TermRangeFilter from above, so the result produced is similar: Searching 'humpty' name: First - content: Humpty Dumpty sat on a wall, - num: 100 name: Fourth - content: Couldn't put Humpty together again. - num: 400 Filters work in conjunction with Queries to refine the search results. As you may have already noticed, the benefit of Filter is its ability to cache results, while Query calculates in real time. When choosing between Filter and Query, you will want to ask yourself whether the search (or filtering) will be repeated. Provided you have enough memory allocation, a cached Filter will always provide a positive impact to search experiences. Creating a custom filter Now that we've seen numerous examples on Lucene's built-in Filters, we are ready for a more advanced topic, custom filters. There are a few important components we need to go over before we start: FieldCache, SortedDocValues, and DocIdSet. We will be using these items in our example to help you gain practical knowledge on the subject. In the FieldCache, as you already learned, is a cache that stores field values in memory in an array structure. It's a very simple data structure as the slots in the array basically correspond to DocIds. This is also the reason why FieldCache only works for a single-valued field. A slot in an array can only hold a single value. Since this is just an array, the lookup time is constant and very fast. The SortedDocValues has two internal data mappings for values' lookup: a dictionary mapping an ordinal value to a field value and a DocId to an ordinal value (for the field value) mapping. In the dictionary data structure, the values are deduplicated, dereferenced, and sorted. There are two methods of interest in this class: getOrd(int) and lookupTerm(BytesRef). The getOrd(int) returns an ordinal for a DocId (int) and lookupTerm(BytesRef) returns an ordinal for a field value. This data structure is the opposite of the inverted index structure, as this provides a DocId to value lookup (similar to FieldCache), instead of value to a DocId lookup. DocIdSet, as the name implies, is a set of DocId. A FieldCacheDocIdSet subclass we will be using is a combination of this set and FieldCache. It iterates through the set and calls matchDoc(int) to find all the matching documents to be returned. In our example, we will be building a simple user security Filter to determine which documents are eligible to be viewed by a user based on the user ID and group ID. The group ID is assumed to be hereditary, where as a smaller ID inherits rights from a larger ID. For example, the following will be our group ID model in our implementation: 10 – admin 20 – manager 30 – user 40 – guest A user with group ID 10 will be able to access documents where its group ID is 10 or above. How to do it... Here is our custom Filter, UserSecurityFilter: public class UserSecurityFilter extends Filter {   private String userIdField; private String groupIdField; private String userId; private String groupId;   public UserSecurityFilter(String userIdField, String groupIdField, String userId, String groupId) {    this.userIdField = userIdField;    this.groupIdField = groupIdField;    this.userId = userId;    this.groupId = groupId; }   public DocIdSet getDocIdSet(AtomicReaderContext context, Bits acceptDocs) throws IOException {    final SortedDocValues userIdDocValues = FieldCache.DEFAULT.getTermsIndex(context.reader(), userIdField);    final SortedDocValues groupIdDocValues = FieldCache.DEFAULT.getTermsIndex(context.reader(), groupIdField);      final int userIdOrd = userIdDocValues.lookupTerm(new BytesRef(userId));    final int groupIdOrd = groupIdDocValues.lookupTerm(new BytesRef(groupId));      return new FieldCacheDocIdSet(context.reader().maxDoc(), acceptDocs) {      @Override      protected final boolean matchDoc(int doc) {        final int userIdDocOrd = userIdDocValues.getOrd(doc);        final int groupIdDocOrd = groupIdDocValues.getOrd(doc);        return userIdDocOrd == userIdOrd || groupIdDocOrd >= groupIdOrd;      }    }; } } This Filter accepts four arguments in its constructor: userIdField: This is the field name for user ID groupIdField: This is the field name for group ID userId: This is the current session's user ID groupId: This is the current session's group ID of the user Then, we implement getDocIdSet(AtomicReaderContext, Bits) to perform our filtering by userId and groupId. We first acquire two SortedDocValues, one for the user ID and one for the group ID, based on the Field names we obtained from the constructor. Then, we look up the ordinal values for the current session's user ID and group ID. The return value is a new FieldCacheDocIdSet object implementing its matchDoc(int) method. This is where we compare both the user ID and group ID to determine whether a document is viewable by the user. A match is true when the user ID matches and the document's group ID is greater than or equal to the user's group ID. To test this Filter, we will set up our index as follows:    Analyzer analyzer = new StandardAnalyzer();    Directory directory = new RAMDirectory();    IndexWriterConfig config = new IndexWriterConfig(Version.LATEST, analyzer);    IndexWriter indexWriter = new IndexWriter(directory, config);    Document doc = new Document();    StringField stringFieldFile = new StringField("file", "", Field.Store.YES);    StringField stringFieldUserId = new StringField("userId", "", Field.Store.YES);    StringField stringFieldGroupId = new StringField("groupId", "", Field.Store.YES);      doc.removeField("file"); doc.removeField("userId"); doc.removeField("groupId");    stringFieldFile.setStringValue("Z:\shared\finance\2014- sales.xls");    stringFieldUserId.setStringValue("1001");    stringFieldGroupId.setStringValue("20");    doc.add(stringFieldFile); doc.add(stringFieldUserId); doc.add(stringFieldGroupId);    indexWriter.addDocument(doc);      doc.removeField("file"); doc.removeField("userId"); doc.removeField("groupId");    stringFieldFile.setStringValue("Z:\shared\company\2014- policy.doc");    stringFieldUserId.setStringValue("1101");    stringFieldGroupId.setStringValue("30");    doc.add(stringFieldFile); doc.add(stringFieldUserId);    doc.add(stringFieldGroupId);    indexWriter.addDocument(doc);    doc.removeField("file"); doc.removeField("userId");    doc.removeField("groupId");    stringFieldFile.setStringValue("Z:\shared\company\2014- terms-and-conditions.doc");    stringFieldUserId.setStringValue("1205");    stringFieldGroupId.setStringValue("40");    doc.add(stringFieldFile); doc.add(stringFieldUserId);    doc.add(stringFieldGroupId);    indexWriter.addDocument(doc);    indexWriter.commit();    indexWriter.close(); The setup adds three documents to our index with different user IDs and group ID settings in each document, as follows: UserSecurityFilter userSecurityFilter = new UserSecurityFilter("userId", "groupId", "1001", "40"); IndexReader indexReader = DirectoryReader.open(directory); IndexSearcher indexSearcher = new IndexSearcher(indexReader); Query query = new MatchAllDocsQuery(); TopDocs topDocs = indexSearcher.search(query, userSecurityFilter,   100); for (ScoreDoc scoreDoc : topDocs.scoreDocs) { doc = indexReader.document(scoreDoc.doc); System.out.println("file: " + doc.getField("file").stringValue() +" - userId: " + doc.getField("userId").stringValue() + " - groupId: " +       doc.getField("groupId").stringValue());} indexReader.close(); We initialize UserSecurityFilter with the matching names for user ID and group ID fields, and set it up with user ID 1001 and group ID 40. For our test and search, we use MatchAllDocsQuery to basically search without any queries (as it will return all the documents). Here is the output from the code: file: Z:sharedfinance2014-sales.xls - userId: 1001 - groupId: 20 file: Z:sharedcompany2014-terms-and-conditions.doc - userId: 1205 - groupId: 40 The search specifically filters by user ID 1001, so the first document is returned because its user ID is also 1001. The third document is returned because its group ID, 40, is greater than or equal to the user's group ID, which is also 40. Searching with QueryParser QueryParser is an interpreter tool that transforms a search string into a series of Query clauses. It's not absolutely necessary to use QueryParser to perform a search, but it's a great feature that empowers users by allowing the use of search modifiers. A user can specify a phrase match by putting quotes (") around a phrase. A user can also control whether a certain term or phrase is required by putting a plus ("+") sign in front of the term or phrase, or use a minus ("-") sign to indicate that the term or phrase must not exist in results. For Boolean searches, the user can use AND and OR to control whether all terms or phrases are required. To do a field-specific search, you can use a colon (":") to specify a field for a search (for example, content:humpty would search for the term "humpty" in the field "content"). For wildcard searches, you can use the standard wildcard character asterisk ("*") to match 0 or more characters, or a question mark ("?") for matching a single character. As you can see, the general syntax for a search query is not complicated, though the more advanced modifiers can seem daunting to new users. In this article, we will cover more advanced QueryParser features to show you what you can do to customize a search. How to do it.. Let's look at the options that we can set in QueryParser. The following is a piece of code snippet for our setup: Analyzer analyzer = new StandardAnalyzer(); Directory directory = new RAMDirectory(); IndexWriterConfig config = new IndexWriterConfig(Version.LATEST, analyzer); IndexWriter indexWriter = new IndexWriter(directory, config); Document doc = new Document(); StringField stringField = new StringField("name", "", Field.Store.YES); TextField textField = new TextField("content", "", Field.Store.YES); IntField intField = new IntField("num", 0, Field.Store.YES);   doc.removeField("name"); doc.removeField("content"); doc.removeField("num"); stringField.setStringValue("First"); textField.setStringValue("Humpty Dumpty sat on a wall,"); intField.setIntValue(100); doc.add(stringField); doc.add(textField); doc.add(intField); indexWriter.addDocument(doc);   doc.removeField("name"); doc.removeField("content"); doc.removeField("num"); stringField.setStringValue("Second"); textField.setStringValue("Humpty Dumpty had a great fall."); intField.setIntValue(200); doc.add(stringField); doc.add(textField); doc.add(intField); indexWriter.addDocument(doc);   doc.removeField("name"); doc.removeField("content"); doc.removeField("num"); stringField.setStringValue("Third"); textField.setStringValue("All the king's horses and all the king's men"); intField.setIntValue(300); doc.add(stringField); doc.add(textField); doc.add(intField); indexWriter.addDocument(doc);   doc.removeField("name"); doc.removeField("content"); doc.removeField("num"); stringField.setStringValue("Fourth"); textField.setStringValue("Couldn't put Humpty together again."); intField.setIntValue(400); doc.add(stringField); doc.add(textField); doc.add(intField); indexWriter.addDocument(doc);   indexWriter.commit(); indexWriter.close();   IndexReader indexReader = DirectoryReader.open(directory); IndexSearcher indexSearcher = new IndexSearcher(indexReader); QueryParser queryParser = new QueryParser("content", analyzer); // configure queryParser here Query query = queryParser.parse("humpty"); TopDocs topDocs = indexSearcher.search(query, 100); We add four documents and instantiate a QueryParser object with a default field and an analyzer. We will be using the same analyzer that was used in indexing to ensure that we apply the same text treatment to maximize matching capability. Wildcard search The query syntax for a wildcard search is the asterisk ("*") or question mark ("?") character. Here is a sample query: Query query = queryParser.parse("humpty*"); This query will return the first, second, and fourth sentences. By default, QueryParser does not allow a leading wildcard character because it has a significant performance impact. A leading wildcard would trigger a full scan on the index since any term can be a potential match. In essence, even an inverted index would become rather useless for a leading wildcard character search. However, it's possible to override this default setting to allow a leading wildcard character by calling setAllowLeadingWildcard(true). You can go ahead and run this example with different search strings to see how this feature works. Depending on where the wildcard character(s) is placed, QueryParser will produce either a PrefixQuery or WildcardQuery. In this specific example in which there is only one wildcard character and it's not the leading character, a PrefixQuery will be produced. Term range search We can produce a TermRangeQuery by using TO in a search string. The range has the following syntax: [start TO end] – inclusive {start TO end} – exclusive As indicated, the angle brackets ( [ and ] ) is inclusive of start and end terms, and curly brackets ( { and } ) is exclusive of start and end terms. It's also possible to mix these brackets to inclusive on one side and exclusive on the other side. Here is a code snippet: Query query = queryParser.parse("[aa TO c]"); This search will return the third and fourth sentences, as their beginning words are All and Couldn't, which are within the range. You can optionally analyze the range terms with the same analyzer by setting setAnalyzeRangeTerms(true). Autogenerated phrase query QueryParser can automatically generate a PhraseQuery when there is more than one term in a search string. Here is a code snippet: queryParser.setAutoGeneratePhraseQueries(true); Query query = queryParser.parse("humpty+dumpty+sat"); This search will generate a PhraseQuery on the phrase humpty dumpty sat and will return the first sentence. Date resolution If you have a date field (by using DateTools to convert date to a string format) and would like to do a range search on date, it may be necessary to match the date resolution on a specific field. Here is a code snippet on setting the Date resolution: queryParser.setDateResolution("date", DateTools.Resolution.DAY); queryParser.setLocale(Locale.US); queryParser.setTimeZone(TimeZone.getTimeZone("Am erica/New_York")); This example sets the resolution to day granularity, locale to US, and time zone to New York. The locale and time zone settings are specific to the date format only. Default operator The default operator on a multiterm search string is OR. You can change the default to AND so all the terms are required. Here is a code snippet that will require all the terms in a search string: queryParser.setDefaultOperator(QueryParser.Operator.AND); Query query = queryParser.parse("humpty dumpty"); This example will return first and second sentences as these are the only two sentences with both humpty and dumpty. Enable position increments This setting is enabled by default. Its purpose is to maintain a position increment of the token that follows an omitted token, such as a token filtered by a StopFilter. This is useful in phrase queries when position increments may be important for scoring. Here is an example on how to enable this setting: queryParser.setEnablePositionIncrements(true); Query query = queryParser.parse(""humpty dumpty""); In our scenario, it won't change our search results. This attribute only enables position increments information to be available in the resulting PhraseQuery. Fuzzy query Lucene's fuzzy search implementation is based on Levenshtein distance. It compares two strings and finds out the number of single character changes that are needed to transform one string to another. The resulting number indicates the closeness of the two strings. In a fuzzy search, a threshold number of edits is used to determine if the two strings are matched. To trigger a fuzzy match in QueryParser, you can use the tilde ~ character. There are a couple configurations in QueryParser to tune this type of query. Here is a code snippet: queryParser.setFuzzyMinSim(2f); queryParser.setFuzzyPrefixLength(3); Query query = queryParser.parse("hump~"); This example will return first, second, and fourth sentences as the fuzzy match matches hump to humpty because these two words are missed by two characters. We tuned the fuzzy query to a minimum similarity to two in this example. Lowercase expanded term This configuration determines whether to automatically lowercase multiterm queries. An analyzer can do this already, so this is more like an overriding configuration that forces multiterm queries to be lowercased. Here is a code snippet: queryParser.setLowercaseExpandedTerms(true); Query query = queryParser.parse(""Humpty Dumpty""); This code will lowercase our search string before search execution. Phrase slop Phrase search can be tuned to allow some flexibility in phrase matching. By default, phrase match is exact. Setting a slop value will give it some tolerance on terms that may not always be matched consecutively. Here is a code snippet that will demonstrate this feature: queryParser.setPhraseSlop(3); Query query = queryParser.parse(""Humpty Dumpty wall""); Without setting a phrase slop, this phrase Humpty Dumpty wall will not have any matches. By setting phrase slop to three, it allows some tolerance so that this search will now return the first sentence. Go ahead and play around with this setting in order to get more familiarized with its behavior. TermQuery and TermRangeQuery A TermQuery is a very simple query that matches documents containing a specific term. The TermRangeQuery is, as its name implies, a term range with a lower and upper boundary for matching. How to do it.. Here are a couple of examples on TermQuery and TermRangeQuery: query = new TermQuery(new Term("content", "humpty")); query = new TermRangeQuery("content", new BytesRef("a"), new BytesRef("c"), true, true); The first line is a simple query that matches the term humpty in the content field. The second line is a range query matching documents with the content that's sorted within a and c. BooleanQuery A BooleanQuery is a combination of other queries in which you can specify whether each subquery must, must not, or should match. These options provide the foundation to build up to logical operators of AND, OR, and NOT, which you can use in QueryParser. Here is a quick review on QueryParser syntax for BooleanQuery: "+" means required; for example, a search string +humpty dumpty equates to must match humpty and should match "dumpty" "-" means must not match; for example, a search string -humpty dumpty equates to must not match humpty and should match dumpty AND, OR, and NOT are pseudo Boolean operators. Under the hood, Lucene uses BooleanClause.Occur to model these operators. The options for occur are MUST, MUST_NOT, and SHOULD. In an AND query, both terms must match. In an OR query, both terms should match. Lastly, in a NOT query, the term MUST_NOT exists. For example, humpty AND dumpty means must match both humpty and dumpty, humpty OR dumpty means should match either or both humpty or dumpty, and NOT humpty means the term humpty must not exist in matching. As mentioned, rudimentary clauses of BooleanQuery have three option: must match, must not match, and should match. These options allow us to programmatically create Boolean operations through an API. How to do it.. Here is a code snippet that demonstrates BooleanQuery: BooleanQuery query = new BooleanQuery(); query.add(new BooleanClause( new TermQuery(new Term("content", "humpty")), BooleanClause.Occur.MUST)); query.add(new BooleanClause(new TermQuery( new Term("content", "dumpty")), BooleanClause.Occur.MUST)); query.add(new BooleanClause(new TermQuery( new Term("content", "wall")), BooleanClause.Occur.SHOULD)); query.add(new BooleanClause(new TermQuery( new Term("content", "sat")), BooleanClause.Occur.MUST_NOT)); How it works… In this demonstration, we will use TermQuery to illustrate the building of BooleanClauses. It's equivalent to this logic: (humpty AND dumpty) OR wall NOT sat. This code will return the second sentence from our setup. Because of the last MUST_NOT BooleanClause on the word "sat", the first sentence is filtered from the results. Note that BooleanClause accepts two arguments: a Query and a BooleanClause.Occur. BooleanClause.Occur is where you specify the matching options: MUST, MUST_NOT, and SHOULD. PrefixQuery and WildcardQuery PrefixQuery, as the name implies, matches documents with terms starting with a specified prefix. WildcardQuery allows you to use wildcard characters for wildcard matching. A PrefixQuery is somewhat similar to a WildcardQuery in which there is only one wildcard character at the end of a search string. When doing a wildcard search in QueryParser, it would return either a PrefixQuery or WildcardQuery, depending on the wildcard character's location. PrefixQuery is simpler and more efficient than WildcardQuery, so it's preferable to use PrefixQuery whenever possible. That's exactly what QueryParser does. How to do it... Here is a code snippet to demonstrate both Query types: PrefixQuery query = new PrefixQuery(new Term("content", "hum")); WildcardQuery query2 = new WildcardQuery(new Term("content", "*um*")); How it works… Both queries would return the same results from our setup. The PrefixQuery will match anything that starts with hum and the WildcardQuery would match anything that contains um. PhraseQuery and MultiPhraseQuery A PhraseQuery matches a particular sequence of terms, while a MultiPhraseQuery gives you an option to match multiple terms in the same position. For example, MultiPhrasQuery supports a phrase such as humpty (dumpty OR together) in which it matches humpty in position 0 and dumpty or together in position 1. How to do it... Here is a code snippet to demonstrate both Query types: PhraseQuery query = new PhraseQuery(); query.add(new Term("content", "humpty")); query.add(new Term("content", "together")); MultiPhraseQuery query2 = new MultiPhraseQuery(); Term[] terms1 = new Term[1];terms1[0] = new Term("content", "humpty"); Term[] terms2 = new Term[2];terms2[0] = new Term("content", "dumpty"); terms2[1] = new Term("content", "together"); query2.add(terms1); query2.add(terms2); How it works… The first Query, PhraseQuery, searches for the phrase humpty together. The second Query, MultiPhraseQuery, searches for the phrase humpty (dumpty OR together). The first Query would return sentence four from our setup, while the second Query would return sentence one, two, and four. Note that in MultiPhraseQuery, multiple terms in the same position are added as an array. FuzzyQuery A FuzzyQuery matches terms based on similarity, using the Damerau-Levenshtein algorithm. We are not going into the details of the algorithm as it is outside of our topic. What we need to know is a fuzzy match is measured in the number of edits between terms. FuzzyQuery allows a maximum of 2 edits. For example, between "humptX" and humpty is first edit and between humpXX and humpty are two edits. There is also a requirement that the number of edits must be less than the minimum term length (of either the input term or candidate term). As another example, ab and abcd would not match because the number of edits between the two terms is 2 and it's not greater than the length of ab, which is 2. How to do it... Here is a code snippet to demonstrate FuzzyQuery: FuzzyQuery query = new FuzzyQuery(new Term("content", "humpXX")); How it works… This Query will return sentences one, two, and four from our setup, as humpXX matches humpty within the two edits. In QueryParser, FuzzyQuery can be triggered by the tilde ( ~ ) sign. An equivalent search string would be humpXX~. Summary This gives you a glimpse of the various querying and filtering features that have been proven to build successful search engines. Resources for Article: Further resources on this subject: Extending ElasticSearch with Scripting [article] Downloading and Setting Up ElasticSearch [article] Lucene.NET: Optimizing and merging index segments [article]

[box type="note" align="" class="" width=""]This article is an excerpt from a book written by Shweta Sankhe-Savale titled Tableau Cookbook – Recipes for Data Visualization. This cookbook has simple recipes for creating visualizations in Tableau. It covers the fundamentals of data visualization such as getting familiarized with Tableau Desktop and also goes to more complex problems like creating dynamic analytics with parameters, and advanced calculations.[/box] In today’s tutorial, we will learn how to create a Treemap and a packed Bubble chart in Tableau. Treemaps Treemaps are useful for representing hierarchical (tree-structured) data as a part-to-whole relationship. It shows data as a set of nested rectangles, and each branch of the tree is given a rectangle, which represents the amount of data it comprises. These can then be further divided into smaller rectangles that represent sub branches, based on its proportion to the whole. We can show information via the color and size of the rectangles and find out patterns that would be difficult to spot in other ways. They make efficient use of the space and hence can display a lot of items in a single visualization simultaneously. Getting Ready We will create a Treemap to show the sales and profit across various product subcategories. Let's see how to create a Treemap. How to Do it We will first create a new sheet and rename it as Treemap. Next, we will drag Sales from the Measures pane and drop it into the Size shelf. We will then drag Profit from Measures pane and drop it into the Color shelf. Our Mark type will automatically change to show squares. Refer to the following image: 5. Next, we will drop Sub-Category into the Label shelf in the Marks card, and we will get the output as shown in the following image: How it Works In the preceding image, since we have placed Sales in the Size shelf, we are inferring this: the greater the size, the higher the sales value; the smaller the size, the smaller the sales value. Since the Treemap is sorted in descending order of Size, we will see the biggest block in the top left-hand side corner and the smaller block in the bottom right-hand side corner. Further, we placed Profit in the Color shelf. There are some subcategories where the profit is negative and hence Tableau selects the orange/blue diverging color. Thus, when the color blue is the darkest, it indicates the Most profit. However, the orange color indicates that a particular subcategory is in a loss scenario. So, in the preceding chart, Phones has the maximum number of sales. Further, Copiers has the highest profit. Tables, on the other hand, is non-profitable. Packed Bubble Charts A Packed bubble chart is a cluster of circles where we use dimensions to define individual bubbles, and the size and/or color of the individual circles represent measures. Bubble charts have many benefits and one of them is to let us spot categories easily and compare them to the rest of the data by looking at the size of the bubble. This simple data visualization technique can provide insight in a visually attractive format. The Packed Bubble chart in Tableau uses the Circle mark type. Getting Ready To create a packed bubble chart, we will continue with the same example that we saw in the Treemap recipe. In the following section, we will see how we can convert the Treemap we created earlier into a Packed Bubble chart. How to Do it Let us duplicate the Tree Map sheet name and rename it to Packed Bubble chart. Next, change the marks from Square to Circle from the Marks dropdown in the Marks card. The output will be as shown in the following image: How it works In the Packed Bubble chart, there is no specific sort of order for Bubbles. The size and/or color are what defines the chart; the bigger or darker the circle, the greater the value. So, in the preceding example, we have Sales in the Size shelf, Profit in the Color shelf, and Sub-Category in the Label shelf. Thus, when we look at it, we understand that Phones has the most sales. Further, Copiers has the highest profit. Tables, on the other hand, is non-profitable even though the size indicates that the sales are fairly good. We saw two ways to visualize data by using Treemap and Packed Bubble chart types in Tableau. If you found this post is useful, do check out the book Tableau Cookbook – Recipes for Data Visualization to create more such charts, interactive dashboards and other beautiful data visualizations with Tableau.      

In the first post of this series we saw how RxJS Observables are similar in use to Promises and we covered how powerful they are. In this post we'll experience the power Observables bring to the table. Now, as if our little app wasn't inspirational enough, we want to overload it with inspiration. Instead of 1, we want to show 10 Chuck Norris inspirations one by one, with a delay of 2 seconds each. Let's implement that with Promises first: Promise JsBin: http://jsbin.com/wupoya/1/edit?js,output import superagent from 'superagent'; let apiUrl, inspireHTML, addPromiscuousInspiration, add10Inspirations; apiUrl = `http://api.icndb.com/jokes/random?limitTo=[nerdy,explicit]`; inspireHTML = (parentId, inspiration) => { let parentNode, inspirationalNode; parentNode = document.getElementById(parentId); inspirationalNode = document.createElement('p'); inspirationalNode.innerHTML = inspiration; parentNode.insertBefore(inspirationalNode, parentNode.firstChild); }; addPromiscuousInspiration = () => { let promiscuousInspiration; promiscuousInspiration = new Promise((resolve, reject) => { superagent .get(apiUrl) .end((err, res) => { if (err) { return reject(err); } let data, inspiration; data = JSON.parse(res.text); inspiration = data.value.joke; console.log('Inspiration has arrived!'); return resolve(inspiration); }); }); promiscuousInspiration.then((inspiration) => { let parentId; parentId = `inspiration`; inspireHTML(parentId, inspiration); }, (err) => { console.error('Error while getting inspired: ', err); }); }; add10Inspirations = () => { let maxTries, tries, interval; maxTries = 10; tries = 1; interval = setInterval(() => { addPromiscuousInspiration(); if (tries < maxTries) { tries++; } else { clearInterval(interval); } }, 2000); }; add10Inspirations(); Note: From now on we are injecting inspirations into HTML (as you'd have guessed from the code). So keep the ES6/Babel and Output panels open in JsBin Well there, we solved the problem. The code is convoluted, but it is just plain JavaScript we already know, so I am not going to explain it step-by-step. Let's try to implement the same using Observable. Observable JsBin: http://jsbin.com/wupoya/3/edit?js,output import superagent from 'superagent'; import {Observable} from 'rx'; let apiUrl, inspireHTML, reactiveInspirations; apiUrl = `http://api.icndb.com/jokes/random?limitTo=[nerdy,explicit]`; inspireHTML = (parentId, inspiration) => { let parentNode, inspirationalNode; parentNode = document.getElementById(parentId); inspirationalNode = document.createElement('p'); inspirationalNode.innerHTML = inspiration; parentNode.insertBefore(inspirationalNode, parentNode.firstChild); }; reactiveInspirations = Observable.create((observer) => { let interval, maxTries, tries; maxTries = 10; tries = 0; interval = setInterval(() => { superagent .get(apiUrl) .end((err, res) => { if (err) { return observer.onError(err); } let data, inspiration; data = JSON.parse(res.text); inspiration = data.value.joke; observer.onNext(inspiration); }); if (tries < maxTries) { tries++; } else { observer.onCompleted(); } }, 2000); return () => { console.log('Releasing the Kraken!'); clearInterval(interval); }; }); reactiveInspirations.subscribe({ onNext: (inspiration) => { let parentId; parentId = `inspiration`; inspireHTML(parentId, inspiration); }, onError: (error) => { console.error('Error while getting inspired', error); }, onCompleted: () => { console.log('Done getting inspired!'); }, }); Easter Egg: There is a tiny little easter egg in this code. Try find it if you want some candy! Hint: Observable can stop right when asked to. There are better ways of doing this (and we'll use them later), but to try comparing apples to apples I took the same approach as Promises. We'll use an interval that makes a call to the server and gets us an inspiration. You may have noticed some major differences. The consumer of our Observable (subscriber) is not changed at all, it is still assuming the same thing it was earlier. And that is a big thing. The only change we made is in the creation of our Observable. Earlier we would create a single value and called it completed, but now we set an interval and emit values making an Ajax request every 2 seconds. Take another look at how we have written the Observable creator. Notice how we are clearing the interval we set up. We put the code responsible for clearing the interval in the dispose method, because its responsibility is to release resources. And then when we have made 10 requests, we simply execute onCompleted and all the resources that need to be cleared (the interval in this case) are released. If you still can't see the power this declarative way of disposing resources brings, let's assume another case. Assume you are the consumer (subscriber) of this observable, and now you want only five inspirations. But you can't change the implementation of the Observable. How would you go about it? We can go around counting the inspirations and ignore that after we've received five, but that means we waste five requests. We want to make only five requests to the server, and then we want to stop the interval and make no more requests. We can actually do that without making a single change to how the Observable is created. When we do a reactiveInspiration.subscribe, it returns us a Disposable object. We can call reactiveInspiration.subscribe(...).dispose() any time to stop the Observable right there and release all its resources. Turns out there are many such use cases which come up more than often when you're dealing with streams/collections of asynchronous operations. RxJS provides very nice API to deal with a lot of them. Instead of manually counting the number of items emitted by our Observable and then disposing it, we can use Observable.prototype.take operator. Observable.prototype.take takes a number as input, and call dispose on the Observable after it has emitted that many values. Here we go: JsBin: http://jsbin.com/wupoya/4/edit?js,output reactiveInspirations .take(5) .subscribe({ onNext: (inspiration) => { let parentId; parentId = `inspiration`; inspireHTML(parentId, inspiration); }, onError: (error) => { console.error('Error while getting inspired', error); }, onCompleted: () => { console.log('Done getting inspired!'); }, }); If you notice in the console, you would see Releasing the Kraken! logged after five inspirations, and no more requests will be made to the server. take is one of the many operators available on the Observable that we can use to declaratively manipulate asynchronous collections. Doing the same thing in the present implementation with promises would involve making changes all over the place. Some would argue that we could create a list of promises and use a library like Q to work over it sequentially, or sequentially create a list of promises with Q and then sequentially operate on it (or something better), but that's not the point. The point is that our use case is to handle a list of asynchronous operations, and Promises are not designed for it. At the end of the day both Promise and Observable are just abstractions for same operation, the one which makes the job easier wins. Easy here is not just the "easy to implement", but easy includes: easy to think easy to implement easy to read easy to maintain easy to extend easy to reuse I don't know about you but the expected behavior of a Promise for me is that it will execute when I want it to, not when I create it. That is what I expected of it when I first tried one. I mean they advertised it as a "unit of asynchronous operation" not as a "unit of asynchronous operation which is already done, here's the dump it took on you". I won't go into explaining each point about which code stands on which of the above points, since that's your job. In my opinion Observables encourage good architecture with maintainable/modular code. From this point on we'll abandon the Promise-based implementation and build little more on Observable implementations. For starters, let's clean it up and do it the right way. import superagent from 'superagent'; import {Observable} from 'rx'; let apiUrl, inspireHTML, getInspiration; apiUrl = `http://api.icndb.com/jokes/random?limitTo=[nerdy,explicit]`; inspireHTML = (parentId, inspiration) => { let parentNode, inspirationalNode; parentNode = document.getElementById(parentId); inspirationalNode = document.createElement('p'); inspirationalNode.innerHTML = inspiration; parentNode.insertBefore(inspirationalNode, parentNode.firstChild); }; getInspiration = () => new Promise((resolve, reject) => { superagent .get(apiUrl) .end((err, res) => { if (err) { return reject(err); } let data, inspiration; data = JSON.parse(res.text); inspiration = data.value.joke; resolve(inspiration); }); }); Observable .interval(2000) .take(10) .flatMap(getInspiration) .subscribe({ onNext: (inspiration) => { let parentId; parentId = `inspiration`; inspireHTML(parentId, inspiration); }, onError: (error) => { console.error('Error while getting inspired', error); }, onCompleted: () => { console.log('Done getting inspired!'); }, }); Promises are not bad. They are designed to represent single asynchronous operations and they do their job fine. (Another) Best thing about RxJS is interoperability. They play well with almost all (a)synchronous forms, including regular arrays, promises, generators, events, and even ES7 async functions. Promises in particular are smooth to work with in RxJS, because when we return a promise in an Observable chain, it gets automatically converted to an Observable. Let's dissect that code from bottom up again. At the bottom of that code is an Observable chain. Yup, a chain! It's a chain of small independent operations that do a single task in our operation. Let's start from the top of this chain. On the top of the chain sits an Observable.interval(2000). Observable.create that we used earlier is one of many ways to create an Observable. Many times (I'd say mostly) we use special purpose operators to create Observable from promises, callbacks, events, and intervals. Observable.interval(N) creates an Observable which emits a value every N milliseconds. It just constantly keeps emitting the values indefinitely. To ask it to stop after emitting 10 values, we use Observable.prototype.take(N). take will accept N values, and then ask its parent to stop. Take, like most other operators, creates an Observable, so we can chain it further. Next in the chain we have a flatMap. If you have never worked with Array-extras like map/filter/reduce, flatMap might be hard to understand at first. Array.prototype.map takes a function, and you can apply it to each item in the array, and return a new array from returned values of the functions. There is another method not included in the default native array extras (but is provided by utility libraries like underscore/lodash/ramda) called flatten. flatten takes an array, and if that array further contains arrays, it flattens them all and returns a single dimensional array. It converts arrays like [1, 2, 3, [4, 5]] to [1, 2, 3, 4, 5, 6]. flatMap is a combination of map and flatten. It takes a function, invokes it on every item in the collection, and then flattens the resulting collection. RxJs tries to mimic the Array-extra API, which makes working with asynchronous collections so much more natural and intuitive. So you have map, filter, reduce and friends available on Observables. flatMap is also great. It flattens the Observable within the Observable to a value and passes it down the chain (i.e creates another Observable that emits flattened values, but let's not get technical). To understand it well, let's take another look at our example: .flatMap(getInspiration). getInspiration returns a Promise. As I said earlier, Promises inside the Observable chain get automatically converted to Observable. So we can safely say that getInspiration returns an Observable. If we used a simple map instead of flatMap, it would just return an Observable for each value it gets from Observable.interval, so it would give us an Observable every 2 seconds. But flatMap goes a step ahead and flattens this Observable so it will resolve this Observable, and give us its value. What value does the Promise/Observable returned by getInspiration resolve to? It resolves to the inspiration. So that's what we get in the next part of the chain. Wow! Such complex task done with such a simple API. Next in the chain is our subscriber, i.e the end of the chain. Here we expect whatever end result we wanted. Here we just do some work on it (append it in our page). I hope it wasn't too much for you to grasp. If it was, perhaps you should go ahead and take a tour of this awesome rxjs tutorial. I am sure you are as impressed by now as I am. The ability to compose asynchronous operations like this brings great power and agility to our code. Plus it is so much fun to code this way. Let's add some more features for our inspirational app. Why not remove the 10 inspiration limit and instead add a button to stop getting inspired when we feel like we're done? JsBin: http://jsbin.com/jenule/1/edit?js,output import superagent from 'superagent'; import {Observable} from 'rx'; let apiUrl, inspireHTML, getInspiration, stopButtonId, stopGettingInspired; apiUrl = `http://api.icndb.com/jokes/random?limitTo=[nerdy,explicit]`; inspireHTML = (parentId, inspiration) => { let parentNode, inspirationalNode; parentNode = document.getElementById(parentId); inspirationalNode = document.createElement('p'); inspirationalNode.innerHTML = inspiration; parentNode.insertBefore(inspirationalNode, parentNode.firstChild); }; getInspiration = () => new Promise((resolve, reject) => { superagent .get(apiUrl) .end((err, res) => { if (err) { return reject(err); } let data, inspiration; data = JSON.parse(res.text); inspiration = data.value.joke; resolve(inspiration); }); }); stopButtonId = 'stop'; stopGettingInspired = Observable .fromEvent(document.getElementById(stopButtonId), 'click') .do((event) => event.preventDefault()); Observable .interval(2000) .takeUntil(stopGettingInspired) .flatMap(getInspiration) .subscribe({ onNext: (inspiration) => { let parentId; parentId = `inspiration`; inspireHTML(parentId, inspiration); }, onError: (error) => { console.error('Error while getting inspired', error); }, onCompleted: () => { console.log('Done getting inspired!'); }, }); Let's quickly go over what new we've added: Stop Button : We've added a stop button in the HTML with HTML id stop. In our JavaScript, we created a new Observable for click events on this button. Observable.fromEvent does that for you. I won't go in much detail here (I am already way out of my allotted space), but this is really powerful. Plus I promised you I'd show you events in action). This allow us to think of click events on our button as any other asynchronous collection, which we can combine with other Observable. That's what we do. We want to stop our Observable.interval Observable on clicking the stop button. take is one of many operators for restricting an Observable. Another one is takeUntil. takeUntil works exactly like take, but instead of taking a number, it takes another Observable. When this Observable emits a value, it stops its parent Observable. So in our case, when our stopGettingInspired Observable emits a value (it emits the click event when we click the button), our interval is stopped. That's some great extensibility! That's all the fun we going to have for today. If you felt like it was too much, I'd recommend you read this tutorial again: . Or if you felt it was too simple, I wrote a small (but use-able) RSS reader as a tutorial here. About the author Charanjit Singh is a freelance developer based out of Punjab, India. He can be found on GitHub @channikhabra.