How-To Tutorials

(For more resources related to this topic, see here.) In this article, we'll focus on recipes for designing high performance SOA Suite 11g applications. These recipes look at how you can design your applications for high performance and scalability, where high performance is defined as providing low response times even under load, and scalability is defined as the ability to expand to cope with large numbers of requests. While many of the recipes in other articles can be applied after the application has been designed and written, those in this article need to be applied while the application is being written, and may require that your application is implemented in a certain way. Designing an application with performance as a requirement from the start is much easier than trying to add performance to an application that is already live. So, the recipes in this article provide some of the best value for money in terms of getting the most performance out of your SOA Suite infrastructure. However, while this book focuses on decisions that should be made during the design stages of a development process, this article is not a list of general SOA Suite design patterns. As for many of the recipes in other articles, a lot of the focus in this article is on reducing the amount of time your application spends waiting on external services and the SOA Suite database tables. There are many aspects to the performance of a SOA Suite application, and the design guidelines depend very much on the particular business problems that your application is designed to solve. Factors such as payload size, number of external systems being orchestrated, data transformation complexity, and persistence requirements, all have an impact on the performance of your application. Performance is a relative term, with each application and use-case having its own requirements, but there are a number of basic principles that can help ensure that your application will have a good chance of meeting its goals. Design for peak loads, not average loads. Average loads can be very misleading; there are many situations in which the average load of a system is not a good indicator of the expected load. A good example of this would be a tax return system, where the usage for most of the year is very low, building into a peak in 30 or so days before people's tax returns are due. Smaller payloads are faster. When designing your application, try and limit the amount of payload data that goes through your composites and processes. It is often better to store the data in a database and send the key and metadata through the processes, only going to retrieve data when required. Understand your transaction boundaries. Many applications suffer performance problems because their transactions boundaries are in the wrong places, causing work to be redone unnecessarily when failures happen, or leaving data in an inconsistent state. Understand what causes your application to access the database, and why. Much of the performance overhead of Oracle SOA Suite applications is in repeated trips to the database. These trips add value by persisting state between steps or within processes, but the overuse of steps that cause database persistence is a common cause of performance problems. Follow standard web service design patterns, such as using asynchronous callbacks and stateless invocations, where you are using web services. Using BPEL process parallelization By having your BPEL process execute steps in parallel when there are no dependencies, you can increase the performance by spending less time waiting for external systems to complete. Getting ready You will need JDeveloper installed, and have an open BPEL project. How to do it... Follow these steps to use BPEL process parallelization: Expand the BPEL Constructs section in the component palette. Drag Flow from the palette onto the process. Click on the + icon next to the flow to expand it. Populate the flow with the process steps. How it works... If you have a number of tasks that do not have dependencies on each other, you can improve performance by executing the preceding tasks in parallel. This is most effective with partner links, where you know you are waiting on an external system to produce a response. The default behaviors of these flows is still to use a single thread to execute the branches if external systems are invoked. See the Using non-blocking service invocations in BPEL recipe to learn how to execute flows that contain partner links in parallel. There's more… It is possible to include a limited amount of synchronization between branches of a flow, so that tasks on one branch will wait for tasks on another branch to complete before proceeding. This is best used with caution, but it can provide benefits, and allow tasks that would not otherwise easily lend themselves to parallelization to be run in parallel. Using non-blocking service invocations in BPEL flows We can reduce the latency of forked external service invocations in a BPEL process to the longest flow's execution time if we assign a thread to each flow, making it multi-threaded. Getting ready You'll need a composite loaded in JDeveloper to execute this recipe. This composite will need a flow that makes calls to a partner link external service. How to do it... Follow these steps to use non-blocking service invocations: Right-click on each partner link that is being executed in your BPEL process flow, and select Edit. In the Property tab, select the green + icon and add nonBlockingInvoke as a property name. In the Value box at the bottom, enter true. How it works... This recipe causes flow branches to be executed in parallel, with a new thread to be used for each branch flow. For multiple service invocations that each have a high latency, this can greatly improve the total BPEL execution time. For example, assume we have a BPEL process that calls two web services, one that takes four seconds to execute, and one that takes six seconds to execute. Applying this change will prevent the BPEL process making the calls serially, which would take 10 seconds in total, and enforce parallel service calls in separate threads, reducing the execution time to just over six seconds, or the latency of the longest call plus time to collate the results in the main BPEL process execution thread. While it may sound like a silver bullet performance improvement, this recipe is actually not necessarily going to improve the execution time of our BPEL process! Consider that we may now be at the mercy of greater thread context switching in the CPU; for every invocation of our process, we now have a larger number of threads that will be spawned. If each service invocation has a low latency, the overhead of creating threads and collating callbacks might actually be greater than the cost of invoking the services in a single thread. Our example in this explanation is contrived, so ensure to test the response time of your composite and the profile of your application, when placed under operational load (which may result in lots of threads spawning), as these may well be different with the configuration applied. There's more… This recipe used an alternative way of setting property values to that which we've used elsewhere in the book. Previously, we've edited composite files directly; here, we used the JDeveloper BPEL graphical editor to achieve the same end result. If you check the composite.xml source, you'll see a property added with a name, such as partnerLink.[your service name].nonBlockingInvoke for each service added. Turning off payload validation and composite state monitoring Payload validation checks all inbound and outbound message data thus adding an overhead, especially for large message types. Composite state monitoring allows for administrators to view the results of all instance invocations. We can disable these to improve performance. Getting ready You will need to know the administration credentials for your Oracle SOA Suite WebLogic domain, and have access to the Oracle Enterprise Manager console. How to do it... By following these steps, we can turn off payload validation: Log in to Enterprise Manager. Open the SOA tab, and right-click on soa_infra , select SOA Administration and Common Properties . Un-tick the checkbox for Payload Validation to disable this feature. Un-tick the checkbox for Capture Composite Instance State. How it works... In this recipe, we globally disabled payload validation. This instructs SOA Suite to not check the inbound and outbound message payloads against the schemas associated with our services. This can be particularly useful, not only if the payload is coming from a trusted source, but even if the source is untrusted. A common alternative to payload validation is to add steps to manually validate the payloads at the point that we first receive the request, while not validating those that have come from internal or trusted sources. There are a number of levels of granularity for payload validation; it can be applied at the SOA Engine (BPEL) and composite levels to allow for fine-grained application of this property. You can access these properties via the enterprise manager console right-click menu on the SOA engines and deployed composites. For performance, I would recommend disabling this in all environments above development. Composite state management is responsible for tracking and representing the health of our running composites. This is a powerful administration feature, but costs a lot in terms of performance. Anecdotal testing shows that this can be responsible for up to 30 percent of processing time. As such, for high throughput applications, the value of this feature should be considered. There's more… See the recipes on audit logging to further control composite recording activities at runtime. Ensure that you check the payload validation at the Engine and Composite levels to ensure that they meet your performance requirements.

(For more resources related to this topic, see here.) Open Embedded Linux Entertainment Center (OpenELEC) is an open source embedded operating system developed specifically for the purpose of running complete and easy-to-use media center solutions on a wide variety of hardware. Built around the great open source media player and organizer, XBMC, OpenELEC is optimized to deliver a smooth, intuitive, and efficient user experience. It is developed with the goal of making the task of setting up and maintaining a Home Theater PC (HTPC) easy and straightforward for all users, regardless of their technical skills. This makes OpenELEC the obvious choice for anyone looking to enhance their home media experience with a fully functional media center. The OpenELEC project is open source, so anyone can download and use the operating system completely free of charge. This is a great enticement to get started with setting up a media center of your own, capable of providing many hours of digital entertainment for yourself and your friends and family to enjoy. Media center features Once installed and configured, OpenELEC incorporates XBMC to provide a wide variety of advanced features in a streamlined and straightforward interface. The following are the media types that can be indexed and/or browsed: Movies TV shows Music Pictures The experience of browsing indexed media is enhanced by automatic inclusion of media information and relevant images available from online databases. This provides easy access to ratings, resumes, artwork, and trailers, because everything is incorporated directly in the interface. Hardware requirements Because OpenELEC is very versatile in hardware compatibility, you will most likely be able to turn "that old PC, which has just been lying around" into a fully functional media center, just by installing OpenELEC on it. This approach is great for a fun hobby project, giving you the opportunity to experiment with OpenELEC at no cost. Alternatively, you can get brand new PCs small enough to fit behind a TV, making them the obvious choice for HTPC use. With a very small footprint and compatibility with small atom- or fusion-based platforms, this is where OpenELEC shows its full potential as a lightweight embedded operating system. Summary So, what is OpenELEC? finds out what OpenELEC actually is, what you can do with it, and why it’s so great. Resources for Article : Further resources on this subject: Creating a file server (Samba) [Article] Webcam and Video Wizardry [Article] Our First Project – A Basic Thermometer [Article]

(For more resources related to this topic, see here.) Concept of reusability The concept of reusability has its roots in the production process. Typically, most of us go about creating e-learning using a process similar to what is shown in the following screenshot. It works well for large teams and the one man band, except in the latter case, you become a specialist for all the stages of production. That's a heavy load. It's hard to be good at all things and it demands that you constantly stretch and improve your skills, and find ways to increase the efficiency of what you do. Reusability in Storyline is about leveraging the formatting, look and feel and interactions you create so that you can re-purpose your work and speed-up production. Not every project will be an original one-off, in fact most won't, so the concept is to approach development with a plan to repurpose 80 percent of the media, quizzes, interactions, and designs you create. As you do this, you begin to establish processes, templates, and libraries that can be used to rapidly assemble base courses. With a little tweaking and some minor customization, you'll have a new, original course in no time. Your client doesn't need to know that 80 percent was made from reusable elements with just 20 percent created as original, unique components, but you'll know the difference in terms of time and effort. Leveraging existing assets So how can you leverage existing assets with Storyline? The first things you'll want to look at are the courses you've built with other authoring programs, such as PowerPoint, QuizMaker Engage, Captivate, Flash, and Camtasia. If there are design themes, elements, or interactions within these courses that you might want to use for future Storyline courses, you should focus your efforts on importing what you can, and further adjusting within Storyline to create a new version of the asset that can be reused for future Storyline courses. If re-working the asset is too complex or if you don't expect to reuse it in multiple courses, then using Storyline's web object feature to embed the interaction without re-working it in any way may be the better approach. In both cases, you'll save time by reusing content you've already put a lot of time in developing. Importing external content Here are the steps to bring external content into Storyline: From the Articulate Startup screen or by choosing the Insert tab, and then New Slide within a project, select the Import option. There are options to import PowerPoint, Quizmaker, and Storyline. All of these will display the slides within the file to be imported. You can pick and choose which slides to import into a new or the current scene in Storyline. The Engage option displays the entire interaction that can be imported into a single slide in the current or a new scene. Click on Import to complete the process. Considerations when importing Keep the following points in mind when importing: PowerPoint and Quizmaker files can be imported directly into Storyline. Once imported, you can edit the content like you would any other Storyline slide. Master slides come along with the import making it simple to reuse previous designs. Note that 64-bit PowerPoint is not supported and you must have an installed, activated version of Quizmaker for the import to work. The PowerPoint to Storyline conversion is not one-to-one. You can expect some alignment issues with slide objects due to the fact that PowerPoint uses points and Storyline uses pixels. There are 2.66 pixels for each point which is why you'll need to tweak the imported slides just a bit. Same with Quizmaker though the reason why is slightly different; Quizmaker is 686 x 424 in size, whereas Storyline is 720 x 540 by default. Engage files can be imported into Storyline and they are completely functional, but cannot be edited within Storyline. Though the option to import Engage appears on the Import screen, what Storyline is really doing is creating a web object to contain the Engage interaction. Once imported into a new scene, clicking on the Engage interaction will display an Options menu where you can make minor adjustments to the behavior of the interaction as well as Preview and Edit in it Engage. You can also resize and position the interaction just as you would any web object. Remember that though web objects work in iPad and HTML5 outputs, Engage content is Flash, so it will not playback on an iPad or in an HTML5 browser. Like Quizmaker, you'll need an installed, activated version of Engage for the import to work. Flash, Captivate, and Camtasia files cannot be imported in Storyline and cannot be edited within Storyline. You can however, use web objects to embed these projects into Storyline or the Insert Flash option. In both cases, the imported elements appear seamless to the learner while retaining full functionality.   Build once, and reuse many times Quizzing is at the heart of many e-learning courses where often the quiz questions need to be randomized or even reused in different sections of a single course (that is, the same questions for a pre and post-test). The concept of building once and reusing many times works well with several aspects of Storyline. We'll start with quizzing and a feature called Question Banks as follows: Question Banks Question Bank offers a way to pool, reuse, and randomize questions within a project. Slides in a question bank are housed within the project file but are not visible until placed into the story. Question Banks can include groups of quiz slides and regular slides (that is, you might include a regular slide if you need to provide instructions for the quiz or would like to include a post-quiz summary). When you want to include questions from a Question Bank, you just need to insert a new Quizzing slide, and then choose Draw from Bank . You can then select one or more questions to include and randomize them if desired. Follow along… In this exercise we will be removing three questions from a scene and moving them into a question bank. This will allow you to draw one or more of those questions at any point in the project where the quiz questions are needed, as follows: From the Home tab, choose Question Banks , and then Create Question bank . Title this Identity Theft Questions . Notice that a new tab has opened in Normal View . The Question Bank appears in this tab. Click on the Import link and navigate to question slides 2, 3, and 4. From the Import drop-down menu at the top, select move questions into question bank . Click on the Story View tab and notice the three slides containing the quiz questions are no longer in the story. Click back on the Identity Theft tab and notice that they are located here. The questions will not become a part of the story until the next step, when you draw them from the bank. In Story View, click once on slide 1 to select it, and then from the Home tab, choose Question Banks and New Draw from Question Bank . From the Question Bank drop-down menu, select Identity Theft Questions . All questions will be selected by default and will be randomized after being placed into the story. This means that the learner will need to answer three questions before continuing onto the next slide in the story. Click on Insert . The Question Bank draw has been inserted as slide 2. To see how this works, Preview the scene. Save as Exercise 11 – Identity Theft Quiz.   There are multiple ways to get back to the questions that are in a question bank. You can do this by selecting the tab the questions are located in (in this case, Identity Theft ), you can view the question bank slide in Normal View or choose Question Banks from the Home tab and navigate to the name of the question bank you'd like to edit.

(For more resources related to this topic, see here.) Planning the Orchestrator deployment The installation of SCORCH is simple. You must plan the deployment appropriately according to your needs. This recipe discusses and provides steps on common planning tasks to be performed before inserting the DVD or mounting the ISO for organizations who have successfully deployed SCORCH. Getting ready The authors recommend you to review the latest information on SCORCH at http://technet.microsoft.com/en-us/library/hh420383.aspx as the requirements of the product and supported platforms are regularly updated by Microsoft. How to do it... There are three planning categories, people, process, and the technology (SCORCH product). Identify and agree on the roles and responsibilities of the SCORCH team. SCORCH deployments typically have three types of users; service accounts, Administrators, and operators. Services accounts: They perform actions for the specific components of SCORCH Administrators: They will typically perform all activities including, but not limited to, SCORCH installation, Runbook creation and management, and delegation of security to operators Operators: They will typically use the SCORCH console and the Runbook Designer to create and manage Runbooks Identify and document initial prototype processes to be used as the first candidate for automation and testing. The types of processes for this purpose should be simple repeatable tasks that fall into an organizations required standard service requests. Good candidates are service request which do not require authorization and approval. An additional example category is Windows operating system services that can be stopped and started as a part of trouble shooting. Plan for the following technology requirements areas for SCORCH: SCORCH deployment type Deployment type Description Single Server All SCORCH roles installed on one physical or virtual machine This scenario is typically implemented in test environments but is fully supported in production. This however becomes a single point of failure for highly automated environments. Multi-server The SCORCH roles are separated and installed on one or more machines Minimum hardware requirements for each SCORCH component Component Requirements Management Server Operating system: Windows Server 2008 R2 or Windows Server 2012* 1 gigabyte (GB) of RAM, 2 GB or more recommended 200 megabytes (MB) of available hard disk space Dual-core Intel microprocessor, 2.1 gigahertz (GHz) or better Microsoft .NET Framework 3.5 Service Pack 1 Orchestration database Database: Microsoft SQL Server 2008 R2 or SQL Server 2012 Collation: SQL_Latin1_General_CP1_CI_AS Local or Remote (Basic Engine only) Runbook Server Operating system: Windows Server 2008 R2 or Windows Server 2012* 1 gigabyte (GB) of RAM minimum, 2 GB or more recommended 200 megabytes (MB) of available hard disk space Dual-core Intel microprocessor, 2.1 gigabyte (GHz) or better Microsoft .NET Framework 3.5 Service Pack 1 Orchestrator Console/Web Service Operating system: Windows Server 2008 R2 or Windows Server 2012* 1 gigabyte (GB) of RAM minimum, 2 GB or more recommended 200 megabytes (MB) of available hard disk space Dual-core Intel microprocessor, 2.1 gigahertz (GHz) or better Microsoft .NET Framework 3.5 Service Pack 1 Web Service: Internet Information Services (IIS) 7.0 and enabled IIS role Microsoft .NET Framework 3.5 Service Pack 1 Microsoft .NET Framework 4 Microsoft Silverlight 4** Orchestrator Runbook Designer Operating system: Windows Server 2008 R2, Windows 7 (32/64 bit) or Windows Server 2012* 1 gigabyte (GB) of RAM minimum, 2 GB or more recommended 200 megabyte (MB) of available hard disk space Dual-core Intel microprocessor, 2.1 gigabyte (GHz) or better Microsoft .NET Framework 3.5 Service Pack 1 SCORCH 2012 SP1 It is required only for the computer running the console in its web browser but not the Web Service server. Services accounts and delegation groups Account/Group Type Notes Orchestrator management service Service account Create an Active Directory user account for this service. This is the main management server service account and it is granted log on as a service during the installation. Orchestrator Runbook monitor service Service account Typically this is the same account as the Orchestrator Management Service. Orchestrator Runbook service Service account Same user account as the Management and Runbook Server monitor service in a single deployment but can be different for multi-server deployments; Active Directory domain account recommended. Runbook authors (SCO_ADMINS) Group Create an Active Directory group. This group will have the equivalent access of full administration to the SCORCH deployment. Runbook operators (SCO_CON_USERS) Group Create an Active Directory group. This group will have the equivalent access of a Runbook operator to the SCORCH deployment. Installation user User The user with full administrative rights on the SCORCH servers is required to perform the installation and configuration of the SCORCH deployment. Network Communication Ports Source Targeted computer Default port Configurable Runbook Designer Management Server 135, 1024-65535 Yes. Management Server, Runbook Server, and Web Service Orchestration database 1433 Yes; specified during the installation on the SCORCH supported version of Microsoft SQL Server. This is the case where the SQL Server instance is not using the default port. Client browser Orchestrator Web Service 81 Yes; during the SCORCH installation. Client browser Orchestration Console 82 Yes; during the SCORCH installation. How it works... The planning activities discussed are the minimum activities the authors recommend. The tasks performed at this stage will ensure that you ask for and plan for all your requirements before investing time in the actual installation. An additional benefit is identifying any people or budgetary risks before the deployment. There's more... There are two additional planning areas which are typically ignored in technology focused deployments. These areas are communication strategies and stakeholder management. Communication strategy One of the inaccurate myths of SCORCH is that it would automate the IT professional. SCORCH when implemented right would improve efficiency but will not replace people. On the contrary you need to communicate with the people who perform the manual tasks as they hold the key to how to best automate their efforts. Early engagement with all IT team members should be one of your key planning tasks. Stakeholder management Stakeholders are all users affected by the SCORCH deployment. An important category of stakeholders are the management team responsible for policy creation and enforcement. Automation without organization buy in may lead to conflicts at the political level of your organization. An example of such a scenario is the ability to create Active Directory user accounts with rights to specific organization areas and restricted resources.

(For more resources related to this topic, see here.) Setting up a scan in Spiceworks The first thing Spiceworks tries to do to scan a network is contact Active Directory(AD); it also uses AD to populate the People portion of your Inventory. Let's set up AD first, as everything else we will be configuring is on the same page. We are all about saving your time and not going back and forth between pages. If you do not have AD in your environment, you can just skip to the Configuring IP range scans section. Scanning and Active Directory There is a wealth of information within AD that Spiceworks uses. We are going to need to configure Spiceworks to log into AD and get that information. OK, we need to get to the Active Directory Configuration screen in Spiceworks in order to do that. As with most things within the app, it is just a couple of clicks. From anywhere in the app, mouse over the Inventory link at the top of the page; a menu will open up. Click on Settings. This will take us to the Settings screen. You will be spending a lot of time here so you can either get very used to these clicks or just have a separate tab open with these settings already set up. The top section is called Getting Started and the first link is Active Directory Configuration. That is our destination for this section so click away. It will take you to the Active Directory Configuration page: There are three sections that are highlighted. Let's go over each and what they do: The area highlighted as 1 is where you are going to enter the credentials that allow Spiceworks to log into your AD and get information. You specify the Active Directory Server (Domain Controller), username and password. Usernames must be in either domain/username or username@domain.com. If you have SSL enabled for AD inquiries, check the Use SSL box. The area highlighted as 2 shows the frequency at which Spiceworks retrieves information from your AD environment. When Spiceworks queries AD, it does not cause a huge amount of traffic or load. Shortening these times should not cause undue stress on your AD servers. This is useful because when you add a user in AD, it will automatically get loaded into Spiceworks at the next scan. If you want any changes you make to users in Spiceworks to be uploaded into your AD environment, the section highlighted as 3 is for you. Just click on the box and any modifications you make in Spiceworks will automatically be synchronized with your AD. There is one more section that is not in the screenshot. This deals with your user portal and help desk. Setting up AD in your Spiceworks really makes a lot of difference with scans and filling in information. It is recommended that if you are running AD, hook this up. If you are wary about Spiceworks writing data into your AD environment, just set up the user that Spiceworks uses to connect as read-only and don't check the box that writes changes back to AD. Easy enough. Since you are convinced that you should connect your AD to Spiceworks, just fill in the ActiveDirectory server, User, and Password fields and click on Save. Spiceworks will automatically test the credentials and let you know immediately if it can connect. If you have some challenges with Spiceworks connecting to your Domain Controller with just the server name, another method is to put the IP address directly into that field. Let's move on to setting up an IP range scan and get some devices into your Spiceworks install. Configuring IP range scans Remember the Settings page that we have been to a couple of times? We are going back! In case you have forgotten, just mouse over to either Inventory or Help Desk and click on the Settings link at the bottom of the left column. Once on the Settings page, we are going to click on the Network Scan link. It is in the first section of links titled Getting Started. This takes us to the main Network Scan page. The first section is where we are going to set up our IP ranges. Since you will not have any ranges in here as you just installed Spiceworks, let's get one configured so you can get some information into the app. To do this, just click on the Add IP Range button and this window will pop up. There is a lot of flexibility that Spiceworks gives you regarding how it scans IP ranges. You can put a fill range (192.168.1.1-254) with or without exclusions, or just a single IP if you so wish. The next box is for exclusions, if you so choose. If you decide you want to scan a range that has both servers and desktops, you can exclude server IP addresses. This is handy. The last options are for scheduling this IP range scan. If you choose the Daily at… option as we have seen in the screenshot, you can also select the time of the day to run this scan. Other options in this drop-down list are every 4, 6, 8, or 12 hours. If you do decide that you want to scan on an hourly basis, the time of the day magically disappears. The bottom of the window lets you select what days of the week you want to run the scan. When Spiceworks runs an initial scan, it can take a bit of time as there is a ton of data that it is collecting. Spiceworks tries a multitude of credentials and reads all information from devices, which it then writes to the database. Once Spiceworks has scanned and written the data to the database, any subsequent scans just write delta data into it. Enter what range you want to scan, any exclusions you choose, and the scan frequency, and click on the Add button. Congratulations! You have just added an IP range scan! Scanning credentials As we have covered, Spiceworks uses a multitude of credentials to try and figure out what is on your network and put those devices into the inventory. This has been completely overhauled in Spiceworks. In this easy-to-use interface, you can enter all the credentials that you are going to need to have a successful scan. Here you can configure multiple usernames/passwords for the following protocols: WMI SSH SNMP Enable ESX/vSphere HTTP iLo SNMP v2c/v3 Telnet Intel vPro As you can see, if you need to put device-specific usernames and passwords into Spiceworks, you can do so using the format, Domainusername. So if you have a server that uses a unique username/password combination, it is easy to set all that up through this interface. The preceding screenshot shows an example of this. Something new in Spiceworks is the section where it shows devices that the credentials were successfully used on. This is really helpful for troubleshooting any scan errors! To add your own username/password combinations, just use these easy-to-follow directions: Click on the protocol you want to add credentials to on the left column (WMI, SNMP, and so on). Click on +Add Account in the middle column labeled Existing Accounts. Enter all the pertinent information on the left pane labeled Edit Account.For usernames that have passwords, there is a Show Password button as well, so you can make sure that you didn't fat finger it! That's it. Just fill in any credentials that will let Spiceworks access your devices on your network, and as far as permissions are concerned you should be good to go! Best practices and kicking off your first Spiceworks scan You have everything you need to start your first Spiceworks scan. It might be best to read the following best practices before you kick it off, though. They will guide you through some potential pitfalls. Scanning best practices For initial scans, be aware of the number of IP addresses you are scanning and the amount of information that Spiceworks is going to pull out of those devices. If you put in a full IP range on your first scan, do not expect Spiceworks to be completed in 10-15 minutes. The initial scan is the most network traffic intensive and will take the longest duration of time. Do full initial scans during nonbusiness hours. Though running an initial full scan shouldn't flood your network, depending on your network configuration, it is always best to run full initial scans during nonbusiness hours just in case. If you are running a 24 x 7 business, break up your IP ranges into smaller chunks and scan that way. Expect some unknown devices. Unless you are a super administrator with a team of hundreds behind you to make sure that every aspect of your network is 100 percent buttoned down, there will most likely be a few devices that Spiceworks cannot connect to. One of the biggest culprits is that WMI has been disabled, or that there is a firewall of some sort blocking Spiceworks from connecting to the machine. Don't get down on yourself if the scan doesn't work 100 percent the first time. If you are really worried about traffic that Spiceworks might cause, what information it collects, or how it will affect workstation performance, just set up a test environment and run a scan there. Whether it be 5 machines or 500, Spiceworks does the same to each one; so test away. Spiceworks is not designed to scan 10,000 devices at one time without a performance hit. If you have a very large network, break it up into smaller chunks for best performance. Spiceworks could get through a 10,000 device scan, but it would hurt performance until the scan is complete. If you have multiple sites linked either by WAN or VPN connections, drop a remote collector at these to run local scans and then send the data back to your main Spiceworks installation. You can find more information at http://community.spiceworks.com/help/Remote_Collectors OK, now that you have read the required best practices, you can set up your IP range on the Network Scan settings page, check the box associated with that range and click on Start Scan. Away you go! Depending on the IP range you set and the time of the day, your scan could take just a few minutes or several hours. If you are having some serious issues trying to get a successful scan, open a browser and hit this site: http://community.spiceworks.com/support. There are in-depth articles and even real-live support folks that can dive into the specifics of your environment, and they won't give up until you are successful. Let's assume that even if you did have an issue, it is resolved and you have got your first scan under your belt. Summary We were provided with details on how to set up a scan in Spiceworks. Also, we got to know how to run the scan we set up and the best practices. Resources for Article : Further resources on this subject: Using SpriteFonts in a Board-based Game with XNA 4.0 [Article] Why CoffeeScript?HTML5 Games Development: Using Local Storage to Store Game Data [Article] Making Money with Your Game [Article]

(For more resources related to this topic, see here.) Entering notes In order to enter notes into our score, we need to enter Note Entry mode. MuseScore has various modes that we can use to accomplish special tasks. You can enter Note Entry mode by clicking on the N button in the toolbar. You can tell whether you are in Note Entry mode at any given time by checking whether the N button is depressed. You may also enter/exit Note Entry mode by pressing the N key. After you enter Note Entry mode, the quarter note should be selected by default. If you hover over the staff, you should see a light blue outline of a note appear. Clicking here will cause a quarter note of that pitch to be inserted. In the toolbar, you will see several notes of different lengths, such as half notes, eighth notes, and whole notes. This area is called the Note Entry toolbar, and indicates which note will be inserted when you click on the staff. Right now, the quarter note should be selected. Click on the half note, and then click an area of the staff on top of the rest that is immediately after the quarter note we just inserted. A half note of the pitch you chose will be added. In MuseScore, whenever we add notes, we must overwrite other notes. First, we overwrote a whole rest with a quarter note, which caused three beats of rest to be added after the quarter note. Then, we overwrote a quarter rest with a half note. Since the half note was longer than the quarter rest, it also overwrote one beat from the half rest following it, and changed the rest to a quarter rest to accommodate the size of the half note. To add an accidental, simply insert the note without the accidental, and then press the appropriate accidental button in the toolbar. For example, let's insert an F eighth note. We click on the eighth note button, then on the F line of the staff, and finally on the sharp button in the toolbar. We can insert dotted notes in a similar fashion by using the dot button on the Note Entry toolbar. In the next measure, let's add a G dotted quarter note by clicking on the quarter note in the Note Entry toolbar, then clicking on the dot button, and then clicking on a G in the staff. The dot will stay selected after you insert the note. If you would like to deselect the dot, you can click on it again. It is also automatically deselected when you change the note duration. Thus, you should always select the dot after you select the value of the note you would like to be dotted. It is possible to notate more quickly using keyboard shortcuts. The number keys 1 through 9 will select different durations, and the letters A through G will insert the designated note. The 0 key inserts a rest. Inserting notes this way will always insert the closest note with the desired pitch. If you hold Ctrl (or on Mac) while pressing the up or down arrow keys, MuseScore will move the last note you inserted up or down an octave. So, inserting a C half note and moving it up an octave can be accomplished by pressing the sequence 6 C Ctrl + ↑. Notes can be adjusted by a half step by pressing the up or down arrows without holding the Ctrl key. Hitting the up arrow will always create sharps, and the down arrow creates flats. This allows us to insert an F eighth note with the keystroke sequence 4 f ↑. While at first the keyboard shortcuts may seem complicated, as you get the hang of MuseScore, it is worthwhile to learn them. They will allow you to notate music extremely quickly and make your overall experience with MuseScore much more pleasurable. Making chords is also very straightforward. We just click on top of our previously inserted note after selecting a note of the same value. Be careful! If a different note length is selected, it will overwrite the previous note. Chords can also be inserted rapidly with keyboard shortcuts. Just start by inserting the first note of the chord normally. If you would like to insert a note of the chord above the previous note, hold Alt and press the interval above the previous note you would like to insert. To insert it below, hold Shift and do the same. Notes are always inserted in the present key signature. So to insert a C first inversion chord, press the sequence E Alt + 3 Alt + 4, or to insert a C second inversion chord, press the sequence G Alt + 4 Alt + 3. Alternatively, after inserting the first note, you can hold Shift and type the letter names of the notes to add to the chord. So pressing the sequence G Shift+C Shift+E would insert the same C second inversion chord. If you ever make a mistake, you can always undo your latest changes by going to the Edit menu and selecting Undo. You can also use the keyboard shortcut Ctrl + Z (or + Z on Mac). Let's put some notes and chords in some measures for both the trombone and piano parts so that we have something to work with. Inserting triplets To insert a triplet, first enter Note Entry mode. Then, from the Note Entry toolbar, choose the total duration that you would like all three triplets to sum to. Next, insert the first note of the triplet in the position you would like the triplet to occupy. After this, exit Note Entry mode, and from the Notes menu, under the Tuplets submenu, click on the Triplet option. A triplet will be created with the selected note as the first note. MuseScore will automatically enter Note Entry mode for you again, and select the correct duration of note needed to complete the triplet. From here, you can replace the two rests with notes by inserting the correct notes on top of them, as we did when we entered notes previously. Also, there is a keyboard shortcut to make this process easier. While in Note Entry mode, select the proper duration you would like the entire triplet to be, as before, but then hit Ctrl + 3 (or + 3 on Mac). The triplet will be inserted, and the proper note duration to fill in the triplet will be selected. You can now enter the notes of the triplet as you would enter normal notes. For instance, to insert a triplet arpeggio of an F major triad totaling one beat, we would press the sequence 5 Ctrl + 3 F A C. For a B major triad totaling two beats, we would similarly press 6 Ctrl + 3 B D ↑ F ↑. Inserting ties Ties are very easy to create in MuseScore. The simplest way to insert a tie is to insert both of the notes that you want to be tied together, exit Note Entry mode, click on the first note, and then click on the tie button in the toolbar, or press the + key. Make sure the two notes you are trying to tie together have the same pitch, or no tie will be inserted. This method works for individual notes, and also for chords. In order to have flexibility when tying chords, you must tie each note of the chord individually if you want the full chord to be tied. An easy way to do this is to ensure that you are not in Note Entry mode, hold Shift, click on the first note of the first chord so that the whole chord is selected, and press the + key. Again, for this to work, you must have two chords with identical pitches next to each other. If you are working with keyboard shortcuts, then there is also a faster way to enter ties that does not require the use of the mouse. After you enter a note in Note Entry mode, the note you just entered will be selected, and the cursor will be located on the right-hand side of this note, as shown in the following screenshot: Then, using the appropriate keyboard shortcut, select the duration of note you would like this note to be tied to. Finally, press the + key. MuseScore will insert a note of the selected duration tied to the previous note. So, pressing the sequence 5 C 4 + will insert a quarter note C tied to an eighth note. While this method is extremely convenient for single notes, it does not work for chords. Often, it is necessary to flip the tie for visual appeal, especially when tying chords. This can be accomplished by ensuring that you are not in Note Entry mode, clicking on a tie, and then pressing the X key. Even though ties look very similar to slurs in many situations, they are created differently. Slurs will be discussed later. Copying and pasting Suppose that we would like to repeat a measure in the bass line, or that the next measure in the melody is very similar to the previous measure. As in a word processor, we can copy and paste measures and fragments of music. First, let's copy and paste a measure. Exit Note Entry mode by ensuring the N button in the toolbar is not selected. Then, click on a portion of the measure where no notes are present. The measure should be selected, as indicated by the blue box around it. Now, either go to the Edit menu and click on Copy, or press Ctrl + C ( + C on Mac). The measure will be copied to the clipboard. Now, click on a portion of the target measure without any notes, and either click on Paste from the Edit menu, or press Ctrl + V ( + V on Mac). The notes will be inserted, and the target measure will be overwritten. It is also possible to copy any portion of your score, even if it spans partial measures or multiple staves. First, click on the note at the top-left of the region you want to copy. In the following example, this would be the E♭ in the right hand. Then, press and hold the Shift key, and click on the note at the bottom right corner of the region you would like to copy. Here, that would be the D in the left hand. MuseScore will select all of the notes in between. Once you have selected the region, you can copy it in the same way you copied the measure before. To paste the region, click on the first note or rest in the uppermost stave where you would like to paste it, and paste as we did with a single measure using either Ctrl + V or Paste from the Edit menu. If your selection has different measure breaks or is in a different meter than the destination, the selection will be reflowed to fit the destination, and ties will be added as necessary. Inserting and deleting measures Often, it is helpful to insert or delete a measure in your score. Luckily, MuseScore makes this extremely easy. To insert a measure, select the measure (as we did when we copied a measure) immediately after the location where you would like to insert the measure. Then, go to the Create menu, and under the Measures submenu, select Insert Measure. A measure will be inserted. To insert multiple measures, select Insert Measures. A dialog box will prompt you for how many measures to insert. If you would like to add measures to the end of the score, you can select Append Measures from under the Measures submenu within the Create menu. There is no need to select any measures to perform this operation. To delete measures, simply select the measure by clicking any blank area within the measure, and then go to the Edit menu, and click on Delete Selected Measures. Doing so will delete this measure position within all staves, not just the selected staff. You can also select multiple measures (as we did earlier when we were copying by selecting one measure, holding the Shift key, and selecting additional measures), and use the same menu button to delete all of the measures that you have selected. Chord symbols In jazz and popular music, it is very common to give musicians chord symbols to read from. To create a chord symbol, make sure you are not in Note Entry mode, and click on a note that you would like to add a chord symbol to. Then, either go to the Create menu, go to the Text submenu, and select Chord Name, or press Ctrl + K ( + K on Mac). A text box should appear that looks exactly like the ones we saw before. Now, you can type the name of the chord in the same way you would write it on paper. (For example, D minor would be Dm, and a G7 chord would just be G7.) All lowercase b characters will be converted into flat signs, and all # characters will be converted into sharps. To move to the next location in the measure, press the space bar. If you press the space bar repeatedly, you will move forward without inserting any chords. Now that our chords are inserted, we can optionally make them look stylized. To do this, go to the Style menu and click on Edit General Style. Then, click on the Chordnames option on the left-hand side. You should see a textbox appear on the right-hand side containing the text stdchords.xml. Change this to jazzchords.xml, and then press OK. The chords you entered should be appropriately stylized. Many styles of notation, especially within jazz music, use chord symbols and slashes to indicate improvisation. To create these slashes in MuseScore, insert four quarter notes on the middle line of the staff. Then, after exiting Note Entry mode, right-click on each note and select Note Properties. Check the box that says Stemless. Also, find the option labeled velocity type and choose user, and then change the value of the box velocity (0-127) to 0. Now press OK. Then, locate the section of the palette labeled Note Heads, and drag the parallelogram slash shape on top of each note. This will create the slash notation. Beaming The proper beaming of notes is a key feature of quality engraved scores that often goes unappreciated. It is extremely easy to change the beaming patterns to enhance the readability of your score. There are several utilities in the palette that allow for this. To start, go to the section of the palette labeled Beam Properties. Hovering over each icon will tell you what it does. These properties can be applied to different notes. The Start beam option is for notes in the middle of an existing beam. It breaks the existing beam at the specified note, and starts a new beam on that note. The Middle of the beam option will ensure that the selected note is beamed to the notes on both sides of it, and the No beam option will break any beams going to the selected note. Let's learn how to use these with a simple use case scenario. Suppose you enter three eighth notes followed by an eighth rest. MuseScore will automatically choose the following beaming: However, to a musician who is sight-reading, it may be easy to confuse this with a triplet. To correct this, simply drag the No beam icon on top of the third eighth note in the passage. The note should highlight red as you hover over it, before you drop it. Once you let go of the mouse button, MuseScore will automatically adjust the beam according to what you specified. Similarly, choosing the beaming wisely can make difficult passages easier to read. Let's consider the case of two sixteenth notes followed by two eighth notes and two more sixteenth notes. Especially with the sharps and flats in this example, it would not be easy to sight-read such a passage. However, dragging the Start beam option on top of the B♮ makes this passage much cleaner and easier to read. To undo any of these changes, ensure that you are not in Note Entry mode, and click on the note that you have changed. Then, in the Beam Properties section of the palette, double-click the A icon to reset it back to default. Though MuseScore uses standard conventions for whether to put the beam above or below the notes, if you would like to change this, simply ensure that you are not in Note Entry mode, click on the beam, and press the X key. The beam will flip to the other side of the staff. Summary In this article, we learned the basics of creating notes including ties and triplets, copying and pasting measures, creating chord symbols, and also changing the beaming patterns to enhance the readability of our score. Resources for Article: Further resources on this subject: Importing and Adding Background Music with Audacity 1.3 [Article] New iPad Features in iOS 6 [Article] Quick start – media files and XBMC [Article]

(For more resources related to this topic, see here.) Understanding FMOD One of the main reasons why I chose FMOD for this book is that it contains two separate APIs—the FMOD Ex Programmer's API, for low-level audio playback, and FMOD Designer, for high-level data-driven audio. This will allow us to cover game audio programming at different levels of abstraction without having to use entirely different technologies. Besides that reason, FMOD is also an excellent piece of software, with several advantages to game developers: License: It is free for non-commercial use, and has reasonable licenses for commercial projects. Cross-platform: It works across an impressive number of platforms. You can run it on Windows, Mac, Linux, Android, iOS, and on most of the modern video game consoles by Sony, Microsoft, and Nintendo. Supported formats: It has native support for a huge range of audio file formats, which saves you the trouble of having to include other external libraries and decoders. Programming languages: Not only can you use FMOD with C and C++, there are also bindings available for other programming languages, such as C# and Python. Popularity: It is extremely popular, being widely considered as the industry standard nowadays. It was used in games such as BioShock, Crysis, Diablo 3, Guitar Hero, Start Craft II, and World of Warcraft. It is also used to power several popular game engines, such as Unity3D and CryEngine. Features: It is packed with features, covering everything from simple audio playback, streaming and 3D sound, to interactive music, DSP effects and low-level audio programming. Installing FMOD Ex Programmer's API Installing a C++ library can be a bit daunting at first. The good side is that once you have done it for the first time, the process is usually the same for every other library. Here are the steps that you should follow if you are using Microsoft Visual Studio: Download the FMOD Ex Programmer's API from http://www.fmod.org and install it to a folder that you can remember, such as C:FMOD. Create a new empty project, and add at least one .cpp file to it. Then, right-click on the project node on the Solution Explorer , and select Properties from the list. For all the steps that follow, make sure that the Configuration option is set to All Configurations . Navigate to C/C++ | General , and add C:FMODapiinc to the list of Additional Include Directories (entries are separated by semicolons). Navigate to Linker | General , and add C:FMODapilib to the list of Additional Library Directories . Navigate to Linker | Input , and add fmodex_vc.lib to the list of Additional Dependencies . Navigate to Build Events | Post-Build Event , and add xcopy /y "C:FMODapifmodex.dll" "$(OutDir)" to the Command Lin e list. Include the <fmod.hpp> header file from your code. Creating and managing the audio system Everything that happens inside FMOD is managed by a class named FMOD::System, which we must start by instantiating with the FMOD::Syste m_Create() function: FMOD::System* system; FMOD::System_Create(&system); Notice that the function returns the system object through a parameter. You will see this pattern every time one of the FMOD functions needs to return a value, because they all reserve the regular return value for an error code. We will discuss error checking in a bit, but for now let us get the audio engine up and running. Now that we have a system object instantiated, we also need to initialize it by calling the init() method: system->init(100, FMOD_INIT_NORMAL, 0); The first parameter specifies the maximum number of channels to allocate. This controls how many sounds you are able to play simultaneously. You can choose any number for this parameter because the system performs some clever priority management behind the scenes and distributes the channels using the available resources. The second and third parameters customize the initialization process, and you can usually leave them as shown in the example. Many features that we will use work properly only if we update the system object every frame. This is done by calling the update() method from inside your game loop: system->update(); You should also remember to shutdown the system object before your game ends, so that it can dispose of all resources. This is done by calling the release() method: system->release(); Loading and streaming audio files One of the greatest things about FMOD is that you can load virtually any audio file format with a single method call. To load an audio file into memory, use the createSound() method: FMOD::Sound* sound; system->createSound("sfx.wav", FMOD_DEFAULT, 0, &sound); To stream an audio file from disk without having to store it in memory, use the createStream() method: FMOD::Sound* stream; system->createStream("song.ogg", FMOD_DEFAULT, 0, &stream); Both methods take the path of the audio file as the first parameter, and return a pointer to an FMOD::Sound object through the fourth parameter, which you can use to play the sound. The paths in the previous examples are relative to the application path. If you are running these examples in Visual Studio, make sure that you copy the audio files into the output folder (for example, using a post-build event such as xcopy /y "$(ProjectDir)*.ogg" "$(OutDir)"). The choice between loading and streaming is mostly a tradeoff between memory and processing power. When you load an audio file, all of its data is uncompressed and stored in memory, which can take up a lot of space, but the computer can play it without much effort. Streaming, on the other hand, barely uses any memory, but the computer has to access the disk constantly, and decode the audio data on the fly. Another difference (in FMOD at least) is that when you stream a sound, you can only have one instance of it playing at any time. This limitation exists because there is only one decode buffer per stream. Therefore, for sound effects that have to be played multiple times simultaneously, you have to either load them into memory, or open multiple concurrent streams. As a rule of thumb, streaming is great for music tracks, voice cues, and ambient tracks, while most sound effects should be loaded into memory. The second and third parameters allow us to customize the behavior of the sound. There are many different options available, but the following list summarizes the ones we will be using the most. Using FMOD_DEFAULT is equivalent to combining the first option of each of these categories: FMOD_LOOP_OFF and FMOD_LOOP_NORMAL: These modes control whether the sound should only play once, or loop once it reaches the end FMOD_HARDWARE and FMOD_SOFTWARE: These modes control whether the sound should be mixed in hardware (better performance) or software (more features) FMOD_2D and FMOD_3D: These modes control whether to use 3D sound We can combine multiple modes using the bitwise OR operator (for instance, FMOD_DEFAULT | FMOD_LOOP_NORMAL | FMOD_SOFTWARE). We can also tell the system to stream a sound even when we are using the createSound() method, by setting the FMOD_CREATESTREAM flag. In fact, the createStream() method is simply a shortcut for this. When we do not need a sound anymore (or at the end of the game) we should dispose of it by calling the release() method of the sound object. We should always release the sounds we create, regardless of the audio system also being released. sound->release(); Playing sounds With the sounds loaded into memory or prepared for streaming, all that is left is telling the system to play them using the playSound() method: FMOD::Channel* channel; system->playSound(FMOD_CHANNEL_FREE, sound, false, &channel); The first parameter selects in which channel the sound will play. You should usually let FMOD handle it automatically, by passing FMOD_CHANNEL_FREE as the parameter. The second parameter is a pointer to the FMOD::Sound object that you want to play. The third parameter controls whether the sound should start in a paused state, giving you a chance to modify some of its properties without the changes being audible. If you set this to true, you will also need to use the next parameter so that you can unpause it later. The fourth parameter is an output parameter that returns a pointer to the FMOD::Channel object in which the sound will play. You can use this handle to control the sound in multiple ways, which will be the main topic of the next chapter. You can ignore this last parameter if you do not need any control over the sound, and simply pass in 0 in its place. This can be useful for non-lopping one-shot sounds. system->playSound(FMOD_CHANNEL_FREE, sound, false, 0); Checking for errors So far, we have assumed that every operation will always work without errors. However, in a real scenario, there is room for a lot to go wrong. For example, we could try to load an audio file that does not exist. In order to report errors, every function and method in FMOD has a return value of type FMOD_RESULT, which will only be equal to FMOD_OK if everything went right. It is up to the user to check this value and react accordingly: FMOD_RESULT result = system->init(100, FMOD_INIT_NORMAL, 0); if (result != FMOD_OK) { // There was an error, do something about it } For starters, it would be useful to know what the error was. However, since FMOD_RESULT is an enumeration, you will only see a number if you try to print it. Fortunately, there is a function called FMOD_ErrorString() inside the fmod_errors.h header file which will give you a complete description of the error. You might also want to create a helper function to simplify the error checking process. For instance, the following function will check for errors, print a description of the error to the standard output, and exit the application: #include <iostream> #include <fmod_errors.h> void ExitOnError(FMOD_RESULT result) { if (result != FMOD_OK) { std::cout << FMOD_ErrorString(result) << std::endl; exit(-1); } } You could then use that function to check for any critical errors that should cause the application to abort: ExitOnError(system->init(100, FMOD_INIT_NORMAL, 0)); The initialization process described earlier also assumes that everything will go as planned, but a real game should be prepared to deal with any errors. Fortunately, there is a template provided in the FMOD documentation which shows you how to write a robust initialization sequence. It is a bit long to cover here, so I urge you to refer to the file named Getting started with FMOD for Windows.pdf inside the documentation folder for more information. For clarity, all of the code examples will continue to be presented without error checking, but you should always check for errors in a real project. Project 1 building a simple audio manager In this project, we will be creating a SimpleAudioManager class that combines everything that was covered in this chapter. Creating a wrapper for an underlying system that only exposes the operations that we need is known as the façade design pattern , and is very useful in order to keep things nice and simple. Since we have not seen how to manipulate sound yet, do not expect this class to be powerful enough to be used in a complex game. Its main purpose will be to let you load and play one-shot sound effects with very little code (which could in fact be enough for very simple games). It will also free you from the responsibility of dealing with sound objects directly (and having to release them) by allowing you to refer to any loaded sound by its filename. The following is an example of how to use the class: SimpleAudioManager audio; audio.Load("explosion.wav"); audio.Play("explosion.wav"); From an educational point of view, what is perhaps even more important is that you use this exercise as a way to get some ideas on how to adapt the technology to your needs. It will also form the basis of the next chapters in the book, where we will build systems that are more complex. Class definition Let us start by examining the class definition: #include <string> #include <map> #include <fmod.hpp> typedef std::map<std::string, FMOD::Sound*> SoundMap; class SimpleAudioManager { public: SimpleAudioManager(); ~SimpleAudioManager(); void Update(float elapsed); void Load(const std::string& path); void Stream(const std::string& path); void Play(const std::string& path); private: void LoadOrStream(const std::string& path, bool stream); FMOD::System* system; SoundMap sounds; }; From browsing through the list of public class members, it should be easy to deduce what it is capable of doing: The class can load audio files (given a path) using the Load() method The class can stream audio files (given a path) using the Stream() method The class can play audio files (given a path) using the Play() method (granted that they have been previously loaded or streamed) There is also an Update() method and a constructor/destructor pair to manage the sound system The private class members, on the other hand, can tell us a lot about the inner workings of the class: At the core of the class is an instance of FMOD::System responsible for driving the entire sound engine. The class initializes the sound system on the constructor, and releases it on the destructor. Sounds are stored inside an associative container, which allows us to search for a sound given its file path. For this purpose, we will be relying on one of the C++ Standard Template Library (STL ) associative containers, the std::map class, as well as the std::string class for storing the keys. Looking up a string key is a bit inefficient (compared to an integer, for example), but it should be fast enough for our needs. An advantage of having all the sounds stored on a single container is that we can easily iterate over them and release them from the class destructor. Since the code for loading and streaming audio file is almost the same, the common functionality has been extracted into a private method called LoadOrStream(), to which Load() and Stream() delegate all of the work. This prevents us from repeating the code needlessly. Initialization and destruction Now, let us walk through the implementation of each of these methods. First we have the class constructor, which is extremely simple, as the only thing that it needs to do is initialize the system object. SimpleAudioManager::SimpleAudioManager() { FMOD::System_Create(&system); system->init(100, FMOD_INIT_NORMAL, 0); } Updating is even simpler, consisting of a single method call: void SimpleAudioManager::Update(float elapsed) { system->update(); } The destructor, on the other hand, needs to take care of releasing the system object, as well as all the sound objects that were created. This process is not that complicated though. First, we iterate over the map of sounds, releasing each one in turn, and clearing the map at the end. The syntax might seem a bit strange if you have never used an STL iterator before, but all that it means is to start at the beginning of the container, and keep advancing until we reach its end. Then we finish off by releasing the system object as usual. SimpleAudioManager::~SimpleAudioManager() { // Release every sound object and clear the map SoundMap::iterator iter; for (iter = sounds.begin(); iter != sounds.end(); ++iter) iter->second->release(); sounds.clear(); // Release the system object system->release(); system = 0; } Loading or streaming sounds Next in line are the Load() and Stream() methods, but let us examine the private LoadOrStream() method first. This method takes the path of the audio file as a parameter, and checks if it has already been loaded (by querying the sound map). If the sound has already been loaded there is no need to do it again, so the method returns. Otherwise, the file is loaded (or streamed, depending on the value of the second parameter) and stored in the sound map under the appropriate key. void SimpleAudioManager::LoadOrStream(const std::string& path, bool stream) { // Ignore call if sound is already loaded if (sounds.find(path) != sounds.end()) return; // Load (or stream) file into a sound object FMOD::Sound* sound; if (stream) system->createStream(path.c_str(), FMOD_DEFAULT, 0, &sound); else system->createSound(path.c_str(), FMOD_DEFAULT, 0, &sound); // Store the sound object in the map using the path as key sounds.insert(std::make_pair(path, sound)); } With the previous method in place, both the Load() and the Stream() methods can be trivially implemented as follows: void SimpleAudioManager::Load(const std::string& path) { LoadOrStream(path, false); } void SimpleAudioManager::Stream(const std::string& path) { LoadOrStream(path, true); } Playing sounds Finally, there is the Play() method, which works the other way around. It starts by checking if the sound has already been loaded, and does nothing if the sound is not found on the map. Otherwise, the sound is played using the default parameters. void SimpleAudioManager::Play(const std::string& path) { // Search for a matching sound in the map SoundMap::iterator sound = sounds.find(path); // Ignore call if no sound was found if (sound == sounds.end()) return; // Otherwise play the sound system->playSound(FMOD_CHANNEL_FREE, sound->second, false, 0); } We could have tried to automatically load the sound in the case when it was not found. In general, this is not a good idea, because loading a sound is a costly operation, and we do not want that happening during a critical gameplay section where it could slow the game down. Instead, we should stick to having separate load and play operations. A note about the code samples Although this is a book about audio, all the samples need an environment to run on. In order to keep the audio portion of the samples as clear as possible, we will also be using the Simple and Fast Multimedia Library 2.0 (SFML ) (http://www.sfml-dev.org). This library can very easily take care of all the miscellaneous tasks, such as window creation, timing, graphics, and user input, which you will find in any game. For example, here is a complete sample using SFML and the SimpleAudioManager class. It creates a new window, loads a sound, runs a game loop at 60 frames per second, and plays the sound whenever the user presses the space key. #include <SFML/Window.hpp> #include "SimpleAudioManager.h" int main() { sf::Window window(sf::VideoMode(320, 240), "AudioPlayback"); sf::Clock clock; // Place your initialization logic here SimpleAudioManager audio; audio.Load("explosion.wav"); // Start the game loop while (window.isOpen()) { // Only run approx 60 times per second float elapsed = clock.getElapsedTime().asSeconds(); if (elapsed < 1.0f / 60.0f) continue; clock.restart(); sf::Event event; while (window.pollEvent(event)) { // Handle window events if (event.type == sf::Event::Closed) window.close(); // Handle user input if (event.type == sf::Event::KeyPressed && event.key.code == sf::Keyboard::Space) audio.Play("explosion.wav"); } // Place your update and draw logic here audio.Update(elapsed); } // Place your shutdown logic here return 0; } Summary In this article, we have seen some of the advantages of using the FMOD audio engine. We saw how to install the FMOD Ex Programmer's API in Visual Studio, how to initialize, manage, and release the FMOD sound system, how to load or stream an audio file of any type from disk, how to play a sound that has been previously loaded by FMOD, how to check for errors in every FMOD function, and how to create a simple audio manager that encapsulates the act of loading and playing audio files behind a simple interface. Resources for Article : Further resources on this subject: Using SpriteFonts in a Board-based Game with XNA 4.0 [Article] HTML5 Games Development: Using Local Storage to Store Game Data [Article] Making Money with Your Game [Article]

(For more resources related to this topic, see here.) So we've got this thing for authentication and authorization. Let's see who is responsible and what for. There is an AccessDecisionManager, which, as the name suggests, is responsible for deciding whether we can access something or not; if not, an AccessDeniedException or InsufficientAuthenticationException is thrown. AuthenticationManager is another crucial interface. It is responsible for confirming who we are. Both are just interfaces, so we can swap our own implementations if we like. In a web application, the job of talking with these two components and the user is handled by a web filter called DelegatingFilterProxy, which is decomposed into several small filters. Each one is responsible for a different thing, so we can turn them on, off, or put our own filters in between and mess with them anyway we like. These are quite important, and we will dig into them later. For the big picture, all we need to know is that these filters take care of all the talking, redirect the user to the login page (or an access-denied page), and save the current user details in an HTTPSession. Well, the last part, while true, is a bit misleading. User details are kept in a SecurityContext object, which we can get a hold of by calling SecurityContextHolder.getContext(), and which in the end is stored in HTTPSession by our filters. But we had promised a big picture, not the gory details, so here it is: Quite simple, right? If we have an authentication protocol without login and password, it works in a similar way. We just switch one of the filters, or the authentication manager, to a different implementation. If we don't have a web application, we just need to do the talking ourselves. But this is all for web resources (URLs). What is much more interesting and useful is securing calls to methods. It looks, for example, like this: @PreAuthorize(["isAuthenticated() and hasRole('ROLE_ADMIN')"])public void somethingOnlyAdminCanDo() {} Here, we decided that somethingOnlyAdminCanDo will be protected by our AccessDecisionManager and that the user must be authenticated (not anonymous) and has to have an admin role. Can a user be anonymous and have an admin role at the same time? In theory, yes, but it would not make any sense. Because it's much cheaper to check if he is authenticated and stop right there. We see a bit of optimization in here. We could drop the isAuthenticated() method and the behavior wouldn't change. We can put this kind of annotation on any Java method, but our configuration and mechanism to fire up the security will depend on the type of objects we are trying to protect. For objects declared as Spring beans (which is a short name for anything defined in our Inversion of Control (IoC) configuration, either via XML or annotations), we don't need to do much. Spring will just create proxies (dynamic classes) that take over calls to our secured methods and fire up AccessDecisionManager before passing the call to the object we really wanted to call. For objects outside of the IoC container (anything created with new or just code not defined in Spring context), we can use the power of Aspect Oriented Programming (AOP) to get the same effect. If you don't know what AOP is, don't worry. It's just a bit of magic at the classloader and bytecode level. For now, the only important thing is that it works basically in the same way. This is depicted as follows: We can do much more than this, as we'll see next, but these are the basics. So, how does the AccessDecisionManager decide whether we can access something or not? Imagine a council of very old Jedi masters sitting around a fire. They decide whether or not you are permitted to call a secured method or access a web resource. Each of these masters makes a decision or abstains. Each of them can consult additional information (not only who you are and what you want to do, but every aspect of the situation). In Spring Security, those smart people are called AccessDecisionVoters, and each of them has one vote. The council can be organized in many different ways. It has one voice, and so it may make the decision based on a majority of votes. It may be veto-based, where everything is allowed unless someone disagrees. Or it may need everyone to agree to grant access, otherwise access is denied. The council is the AccessDecisionManager, and we have three implementations previously mentioned out of the box. We can also decide who's in the council and who is not. This is probably the most important decision we can make, because this will decide the security model that we will use in our application. Let's talk about the most popular counselors (implementations of AccessDecisionVoter). Model based on roles (RoleVoter): This guy makes his decision based on the role of the user and the required role for the resource/method. So if we write @PreAuthorize("hasRole('ROLE_ADMIN')"), you better be a damn admin or you'll get a no-no from this guy. Model based on entity access control permissions (AclEntryVoter): This guy doesn't worry about roles. He is much more than that. Acl stands for Access Control List, which represents a list of permissions. Every user has a list of permissions, possibly for every domain object (usually an object in the database), that you want to secure. So, for example, if we have a bank application, the supervisor can give Frank access to a single specific customer (say, ACME—A Company that Makes Everything), which is represented as an entity in the database and as an object in our system. No other employee will be able to do anything to that customer unless the supervisor grants that person the same permission as Frank. This is probably the most scrutinous voter we would ever use. Our customer can have a very detailed configuration with him/her. On the other hand, this is also the most cumbersome, as we need to create a usable graphical interface to set permissions for every user and every domain object. While we have done this a few times, most of our customers wanted a simpler approach, and even those who started with a graphical user interface to configure everything asked for a simplified version based on business rules, at the end of the project. If your customer describes his security needs in terms of rules such as "Frank can edit every customer he has created but he cannot do anything other than view other customers", it means it's time for PreInvocationAuthorizationAdviceVoter. Business rules model (PreInvocationAuthorizationAdviceVoter): This is usually used when you want to implement static business rules in the application. This goes like "if I've written a blog post, I can change it later, but others can only comment" and "if a friend asked me to help him write the blog post, I can do that, because I'm his friend". Most of these things are also possible to implement with ACLs, but would be very cumbersome. This is our favorite voter. With it, it's very easy to write, test, and change the security restrictions, because instead of writing every possible relation in the database (as with ACL voter) or having only dumb roles, we write our security logic in plain old Java classes. Great stuff and most useful, once you see how it works. Did we mention that this is a council? Yes we did. The result of this is that we can mix any voters we want and choose any council organization we like. We can have all three voters previously mentioned and allow access if any of them says "yes". There are even more voters. And we can write new ones ourselves. Do you feel the power of the Jedi council already? Do you feel the power of the Jedi council already? Summary This section provides an overview of authentication and authorization, which are the principles of Spring security. Resources for Article : Further resources on this subject: Migration to Spring Security 3 [Article] Getting Started with Spring Security [Article] So, what is Spring for Android? [Article]

(For more resources related to this topic, see here.) The particle systems in Motion 5 are a powerful engine by which we can take nearly any object, image, layer, or group and animate it using the parameters available to us in the HUD and Inspector. A particle system consists of two items—the emitter and the cell. The cell is referenced by the emitter and the emitter creates the animation over a lifespan specified by you. Let's say we had a PNG layer of an orange. If we created particles out of it, that orange would be put into a cell that is referenced by the emitter. You could use the emitter's parameters to duplicate that orange multiple times per second and have it animate in a particular direction until you decide it should end or die. On top of the ability to turn nearly anything your heart desires into particles, in Motion's library there are pre-animated particle emitters available to incorporate in all of your animations. Making particles and changing values in the HUD Let's take a look at how we can create a particle system using a shape from Motion's library and tweak a few of its parameters using the HUD. Getting ready From the exercise files of this article, double-click the 07_01 project. There is the Shape layer in the Layers tab, whose scale has been animated to repeat for the duration of the project. Our goal is to place the heart in a particle system so that we can have hundreds rain down onto the Canvas. How to do it... The following steps will take you through creating your first particle system: Make sure your playhead is at the beginning of your project. Select the Pink Heart layer and press E , or from the toolbar choose Create a particle emitter (the icon with the three bubbles rising up, as shown in the following screenshot): Play back the project. Several things just happened after you pressed that button. In the Layers tab, notice that the Pink Heart layer has been turned off. A cell and emitter have been created just above it. The cell holds information about the heart while the emitter is creating all the duplicate copies of the heart that are shooting out in a 360-degree circle across the screen at the same speed. You can see a screenshot of this next. Press F7 to bring up the HUD and see some of these parameters in more detail: Right now, thirty hearts are being born every second and live for a duration of five seconds where they pop off the screen. All the duplicate hearts also hold the original scale of the heart being referenced. Change the Birth Rate value to 5 and Life to 10 . Bring down the Scale parameter to 50 , as shown in the following screenshot: Instead of having the hearts come from the center of the screen, let's have them rain down from the top of the Canvas. Decrease the size of your Canvas by clicking on it and pressing Command + - a few times. Select the particle emitter and drag it up and offscreen. Use the Shift key to constrain the movement. Change the Emission Range slider from 360 to 180 and make sure the arrows point down. Play back the animation and tweak the Birth Rate and Scale sliders as desired. See the following screenshot for reference: There's more… If you're finding the HUD limited in terms of the options available to you for the emitter, don't worry, the Inspector has several additional parameters, including the option to add random values to your emitter that will add more realism to it! Take a peak at the following screenshot: Tweaking particle parameters in the Inspector In the previous recipe, we switched a few parameters in the HUD for our particle system of hearts, but if we're looking to fine-tune our animation, we need to go to the Inspector. Let's take a look at the several additional parameters available. Getting ready From the exercise files of this article, double-click the 07_02 project. Play back the project. There is a particle system in the center of the Canvas being emitted in the form of a rectangle. Currently, thirty hearts are being born every second and live for five seconds. We're going to tweak our particle system to change the flow of the animation so that our hearts emit out in a circle and away from the screen. How to do it... By following these steps, we'll gain a better understanding of how we can control the particle system: Select the Emitter in the Layers tab. Press F4 to open the Emitter tab of the Inspector. Change the Shape menu's option from rectangle to circle and the Arrangement option to Random Fill . Set the radius to 500. Increase Birth Rate to 200 and set Speed in Cell Controls to 0. If you play back the project, you'll see the hearts trying to form the shape of a circle. Since there is no speed value, the hearts stay put in the shape they originate from, as shown in this screenshot: Set Angle Value to 15 and Spin to 30. Rather than having the particles pop onscreen when they're born and pop offscreen when they die, we're going to add some tags to the Opacity Over Life parameter. The trick is to click the white line where we want to add a tag above it. Click the line once close to the beginning and twice close to the end. Use the following screenshot for reference: Click on the first tag and under the line, drag the Opacity slider to 0 (indicated in this screenshot). Repeat this step for the last slider. Now the hearts fade in at birth and fade out at death: As a final step, we're going to keyframe the speed of the particle system to start at 100 percent and eventually go down to 0 to reveal the shape of the circle we created. Go to the beginning of the Timeline and click the diamond icon next to Speed to add a keyframe. Change its value to 100. Move to three seconds and add another keyframe by clicking the diamond icon next to Speed again. Move to three seconds and set Speed to 0. Play back the animation and compare it to the following screenshot: There's more... You can create some amazing animations by changing the type of shape used to emit particles. For instance, if you change the shape of your particles to Image and use text as the source, the particles will spell out the word as long there is a substantial number of hearts being born and the Arrangement option is set to Random Fill . We'll see an example of this in a later exercise, but the following screenshot gives you a sneak peak on the end results you can get: Adding randomness values To give your particle systems just a little more, we're going to add a little randomness to some of the parameters. This randomness will help give your animations a more organic feel. Getting ready From the exercise files of this article, double-click the 07_03 project. Play back the project. The particle system consists of little demograms that rain down from a line. The Angle and Spin values cause the demograms to rotate while moving, and the Opacity Over Life parameter allows each particle to fade in and out during its lifespan. The particles also change color over their life as reflected in the Color Mode menu and the gradient outlining the cycle. We're going to add randomness values to tweak the animation. Whenever we add a randomness value, it's going to look at the original value above it and add and subtract to it randomly every second based on the number you specify. For example, say the Scale value is set to 50. If you add a Scale randomness of 10, every second the demograms will be somewhere between a scale of 40 and 60. How to do it... Let's start to randomize different values in this recipe: Select the emitter in the Layers tab and press F4 . In the Emitter tab of the Inspector, change the Birth Rate Randomness slider's value to 10. Now, every second, somewhere between 10 and 30 objects are born. Now, set Life Randomness to 3, Angle Randomness to 40, Spin Randomness to 50, and Scale Randomness to 80. Play back the project. Now we have our demograms growing at different sizes, being born at different angles and spinning at different rates! The following screenshot shows the Inspector with all of the mentioned changes and also a Color Over Life change (see the previous recipe to learn how to do it). The following is a screenshot showing a frame of the particle animation in the Canvas with random values applied: While playing back, click the Generate button next to Random Seed . You'll notice that your animation changes and a new number is created next to Generate . Random Seed takes all the random parameters and creates the animation based on this number. Change this number and change the randomness of the animation, as shown here: Working with particle behaviors While there are already a ton of parameters you can animate by keyframing their values in the particle and cell emitters, Motion offers you a few particle behaviors worth taking a look at from the library as well. Getting ready From the exercise files of this article, double-click the 07_04 project. Play back the project. The animation consists of a bouncing alarm clock from Motion's library in the foreground and a particle system referencing that alarm clock in the background. Unlike the other particle systems we've worked with in the previous recipes, the source of the particle system has been turned back on to preserve it. Our goal is to add two particle behaviors to have the clocks scale down and spin over its life cycle. By adjusting the speed and direction of our emitter, we will also have the clocks look like they're being pulled off into the distance. Also note, the anchor point of the clock was adjusted in advance in order to have the particles spin around the center of the clock. How to do it... Open the Particle group and select Emitter . Press Command + 2 to go to the Library tab. Under Behaviors , choose Particles | Spin Over Life . Click Apply . Press F7 to open the HUD. Set Increment Type to Birth and Death Values . Set Birth to 360 and Death to 0. If you're having trouble, try changing the values in the Behaviors tab of the Inspector. Now, play back the animation. Now, let's have the clocks in the particle system scale over life. Select the Emitter again. Press Command + 2 to go to the Library . Under Behaviors , choose Particles | Scale Over Life .Click Apply. In the HUD, set Increment Type to Birth and Death Values . Set Scale at Birth to 100 and Scale at Death to 0. If you're having trouble, try changing the values in the Behaviors tab of the Inspector. Play back the animation. The clocks now scale down gradually until they die, as shown in the following screenshot: Press F4 to go to the Emitter tab of the Inspector. Change the Emission Longitude value to 180 and Speed to 2000 , as shown in the following screenshot. Change the other parameters as desired. There's more... When you start building complex particle animations, keep in mind that you can save them to the Library as well as any of the behaviors you tweak. All particle presets can be found in the Library under Particle Emitters in a dedicated theme folder you create. The more time you spend automating your work, the more time you can dedicate to the creative process! Working with particle presets So far, we've created our own particle systems by selecting objects to place into cells referenced by the emitter. Motion ships with over 200 presets! One of the best ways to learn Motion is to dissect how some of the particle presets were created. Let's take a look at what it has to offer! Getting ready From the exercise files of this article, double-click on the 07_05 project. There are two particle presets in this project but one has been turned off. Play back the project and familiarize yourself with the Magic Dust preset. Stop the playback. Turn off the Magic Dust group and turn on the Buskerit group. Play back the project again. In this recipe, we'll take a look at each of these presets and change a few parameters to look at how the animations were created. How to do it... Let's now tweak these parameters of our presets in more detail: Reveal the content of the Buskerit group by clicking the disclosure triangle next to it. It contains a group called busker and seven still images that have been turned off. The images are being referenced by the emitter. Click the disclosure triangle next to the Busker group. It contains the content of the Busker emitter and the seven cells that hold information from those layers that have been shot off. The Emitter also has a Scale Over Life behavior in it. While the object continually changes scale over the animation, the Scale Over Life behavior was used to animate the "pop-up" intro of the instruments, whereas keyframes were used to change the scale over the project. Select the Scale Over Life behavior and press F2 to go to the Behaviors tab of the Inspector. Notice the Increment Type parameter set to Custom . Click the disclosure triangle next to Custom Scale to see the graph, as shown in the following screenshot: Select the Emitter in the Layers tab and press Command + 8 to open the Keyframe Editor . Notice that both the scale and rotation of the emitter are keyframed periodically throughout the entire animation, as shown in the following screenshot. In the Keyframe Editor , press Shift + K to move forward between keyframes. Change the Rotation and Scale values by double-clicking the values while on an existing keyframe. You'll know you're on one because the diamond shape will appear highlighted! Click the disclosure triangle to close the Buskerit group, turn it off, and turn on the Magic Dust group. Click the disclosure triangle to open it. A still image of a spark was used as the source for the Emitter. Several behaviors were used to animate the sparks. With the emitter selected, press F4 to go to the Emitter tab of the Inspector. Set Emission Range to 45 , Birth Rate to 200 , and Speed to 1000 . Play back the project to see the effect it has and compare it to the following: Select the Magic Dust group and go to the gear icon under the mini-Timeline. Choose Basic Motion | Motion Path . Tweak the path as desired. Play back the animation. There's more... Learning how particle presets work is a great way to get used to Motion. Particle presets are the best way to learn Motion! Looking at the library and seeing how something was created is the best way to get under the hood of the application and start creating your own animations. Don't be afraid to explore. Working with particle presets in 3D Particle presets are already powerful on their own but turn them into 3D and you'll find that a few of them will actually look like they have been extruded. Let's take a look! Getting ready From the exercise files of this article, double-click on the 07_06 project. Play back the project. There is a particle system of thin water being projected across the screen. Go to the Properties tab in the Inspector and click the rotation disclosure triangle to see the object rotate 90 degrees on its y axis. The water becomes invisible because it's flat the minute you change perspective. If we promote a particle system to 3D, we can get rid of this problem and have a few additional options available to us in terms of the way we emit the water. How to do it... Let's get rid of the flatness of our particle system by making it 3D: Select the Particle Emitter category and press F4 to go to the Inspector. Click on the 3D checkbox. As soon as you press the button, more options become available to you in the Emitter Controls section, including Emission Latitude and Emission Longitude , as seen here: Two additional selections—Box and Sphere —also became available under the Shape drop-down. Click on the word Rectangle and choose Box . Press F1 to go to the Properties tab of the Inspector. Click the disclosure triangle next to Scale and increase Scale Z to 200 . Rotate the water 90 degrees on the Y axis and notice it's no longer flat. Use the following screenshot of the Properties tab for reference: Go to the beginning of the Timeline and bring back Rotation Y to 0. Click the diamond shape next to Rotation Y to add a keyframe. Move to the end of the Timeline; and change the parameter to 720. Play back the project and see that, no matter what angle it is, the particle system never goes flat, as shown here:

(For more resources related to this topic, see here.) Node.js is an open source platform that allows you to build fast and scalable network applications using JavaScript. Node.js is built on top of V8, a modern JavaScript virtual machine that powers Google's Chrome web browser. Node.js uses an event-driven, non-blocking I/O model that makes it lightweight and efficient.Node.js can handle multiple concurrent network connections with little overhead, making it ideal for data-intensive, real-time applications. With Node.js, you can build many kinds of networked applications. For instance, you can use it to build a web application service, an HTTP proxy, a DNS server, an SMTP server, an IRC server, and basically any kind of process that is network intensive. You program Node.js using JavaScript, which is the language that powers the Web. JavaScript is a powerful language that, when mastered, makes writing networked, event-driven applications fun and easy. Node.js recognizes streams that are resistant to precarious network conditions and misbehaving clients. For instance, mobile clients are notoriously famous for having large latency network connections, which can put a big burden on servers by keeping around lots of connections and outstanding requests. By using streaming to handle data, you can use Node.js to control incoming and outgoing streams and enable your service to survive. Also, Node.js makes it easy for you to use third-party open source modules. By using Node Package Manager (NPM), you can easily install, manage, and use any of the several modules contained in a big and growing repository. NPM also allows you to manage the modules your application depends on in an isolated way, allowing different applications installed in the same machine to depend on different versions of the same module without originating a conflict, for instance. Given the way it's designed, NPM even allows different versions of the same module to coexist in the same application. Summary In this article, we learned that Node.js uses an event-driven, non-blocking I/O mode and can handle multiple concurrent network connections with little overhead. Resources for Article : Further resources on this subject: So, what is KineticJS? [Article] Cross-browser-distributed testing [Article] Accessing and using the RDF data in Stanbol [Article]

(For more resources related to this topic, see here.) System requirements Since XenConvert can only convert Windows-based hosts and installs on the same host, the requirements are pretty much the same, as follows: Operating system: Windows XP, Windows Vista, Windows 7, Windows Server 2003 (SP1 or later), Windows Server 2008 (R2) .Net Framework 4.0 Disk Space: 40 MB free disk space XenServer version 6.0 or 6.1 Converting a physical machine to a virtual machine Let's take a quick look at how to convert a physical machine to a virtual machine. First we need to install XenConvert on the source physical machine. We can download XenConvert from this link: http://www.citrix.com/downloads/xenserver/tools/conversion.html. Once the standard Windows installation process is complete, launch the XenConvert tool; but before that we need to prepare the host machine for the conversion. To know more about XenConvert, refer to the XenConvert guide at http://support.citrix.com/article/CTX135017. Preparing the host machine For best results, prepare the host machine as follows: Enable Windows Automount on Windows Server operating systems. Disable Windows Autoplay. Remove any virtualization software before performing a conversion. Ensure that adequate free space exists at the destination, which is approximately 101 percent of used space of all source volumes. Remove any network interface teams; they are not applicable to a virtual machine. We need to run the XenConvert tool on the host machine to start the physical-to-virtual conversion. We can convert the physical machine directly to our XenServer if this host machine is accessible. The other options are to convert to VHD, OVF, or vDisk, which can be imported later on to XenServer using some methods. These options are more useful if we don't have enough disk space or connectivity with XenServer. I chose XenServer and clicked on Next . We can select multiple partitions to be included in the conversion, or select none from the drop-down menu in Source Volume and those disks won't be included in the conversion. We can also increase or decrease the size of the new virtual partition to be allocated for this virtual machine. Click on Next . We'll be asked to provide the details of the XenServer host. The hostname needs either an IP address or a FQDN of the XenServer; a username and password are standard login requirements. In the Workspace field, enter the path to the folder to store the intermediate OVF package that XenConvert will use during the conversion process. XenConvert will store the OVF package in the path we give. Click on Next to select the storage repositories found with XenServer and continue to the last step, in which we'll be provided with the summary of the conversion. Soon after the conversion is completed, we'll be able to have this new machine in our XenCenter. We'll need to have XenServer Tools installed on this new virtual machine. Summary In this article we covered an advanced topic that explained the process of converting a physical Windows server to a virtual machine using XenConvert. Resources for Article : Further resources on this subject: Citrix XenApp Performance Essentials [Article] Defining alerts [Article] Publishing applications [Article]

In this article by Luca Milanesio, author of the book Learning Gerrit Code review, we will learn about Gerrit Code revew. GitHub is the world's largest platform for the free hosting of Git Projects, with over 4.5 million registered developers. We will now provide a step-by-step example of how to connect Gerrit to an external GitHub server so as to share the same set of repositories. Additionally, we will provide guidance on how to use the Gerrit Code Review workflow and GitHub concurrently. By the end of this article we will have our Gerrit installation fully integrated and ready to be used for both open source public projects and private projects on GitHub. (For more resources related to this topic, see here.) GitHub workflow GitHub has become the most popular website for open source projects, thanks to the migration of some major projects to Git (for example, Eclipse) and new projects adopting it, along with the introduction of the social aspect of software projects that piggybacks on the Facebook hype. The following diagram shows the GitHub collaboration model: The key aspects of the GitHub workflow are as follows: Each developer pushes to their own repository and pulls from others Developers who want to make a change to another repository, create a fork on GitHub and work on their own clone When forked repositories are ready to be merged, pull requests are sent to the original repository maintainer The pull requests include all of the proposed changes and their associated discussion threads Whenever a pull request is accepted, the change is merged by the maintainer and pushed to their repository on GitHub   GitHub controversy The preceding workflow works very effectively for most open source projects; however, when the projects gets bigger and more complex, the tools provided by GitHub are too unstructured, and a more defined review process with proper tools, additional security, and governance is needed. In May 2012 Linus Torvalds , the inventor of Git version control, openly criticized GitHub as a commit editing tool directly on the pull request discussion thread: " I consider GitHub useless for these kinds of things. It's fine for hosting, but the pull requests and the online commit editing, are just pure garbage " and additionally, " the way you can clone a (code repository), make changes on the web, and write total crap commit messages, without GitHub in any way making sure that the end result looks good. " See https://github.com/torvalds/linux/pull/17#issuecomment-5654674. Gerrit provides the additional value that Linus Torvalds claimed was missing in the GitHub workflow: Gerrit and GitHub together allows the open source development community to reuse the extended hosting reach and social integration of GitHub with the power of governance of the Gerrit review engine. GitHub authentication The list of authentication backends supported by Gerrit does not include GitHub and it cannot be used out of the box, as it does not support OpenID authentication. However, a GitHub plugin for Gerrit has been recently released in order to fill the gaps and allow a seamless integration. GitHub implements OAuth 2.0 for allowing external applications, such as Gerrit, to integrate using a three-step browser-based authentication. Using this scheme, a user can leverage their existing GitHub account without the need to provision and manage a separate one in Gerrit. Additionally, the Gerrit instance will be able to self-provision the SSH public keys needed for pushing changes for review. In order for us to use GitHub OAuth authentication with Gerrit, we need to do the following: Build the Gerrit GitHub plugin Install the GitHub OAuth filter into the Gerrit libraries (/lib under the Gerrit site directory) Reconfigure Gerrit to use the HTTP authentication type   Building the GitHub plugin The Gerrit GitHub plugin can be found under the Gerrit plugins/github repository on https://gerrit-review.googlesource.com/#/admin/projects/plugins/github. It is open source under the Apache 2.0 license and can be cloned and built using the Java 6 JDK and Maven. Refer to the following example: $ git clone https://gerrit.googlesource.com/plugins/github $ cd github $ mvn install […] [INFO] BUILD SUCCESS [INFO] ------------------------------------------------------- [INFO] Total time: 9.591s [INFO] Finished at: Wed Jun 19 18:38:44 BST 2013 [INFO] Final Memory: 12M/145M [INFO] ------------------------------------------------------- The Maven build should generate the following artifacts: github-oauth/target/github-oauth*.jar, the GitHub OAuth library for authenticating Gerrit users github-plugin/target/github-plugin*.jar, the Gerrit plugin for integrating with GitHub repositories and pull requests Installing GitHub OAuth library The GitHub OAuth JAR file needs to copied to the Gerrit /lib directory; this is required to allow Gerrit to use it for filtering all HTTP requests and enforcing the GitHub three-step authentication process: $ cp github-oauth/target/github-oauth-*.jar /opt/gerrit/lib/ Installing GitHub plugin The GitHub plugin includes the additional support for the overall configuration, the advanced GitHub repositories replication, and the integration of pull requests into the Code Review process. We now need to install the plugin before running the Gerrit init again so that we can benefit from the simplified automatic configuration steps: $ cp github-plugin/target/github-plugin-*.jar /opt/gerrit/plugins/github.jar Register Gerrit as a GitHub OAuth application Before going through the Gerrit init, we need to tell GitHub to trust Gerrit as a partner application. This is done through the generation of a ClientId/ClientSecret pair associated to the exact Gerrit URLs that will be used for initiating the 3-step OAuth authentication. We can register a new application in GitHub through the URL https://github.com/settings/applications/new, where the following three fields are requested: Application name : It is the logical name of the application authorized to access GitHub, for example, Gerrit. Main URL : The Gerrit canonical web URL used for redirecting to GitHub OAuth authentication, for example, https://myhost.mydomain:8443. Callback URL : The URL that GitHub should redirect to when the OAuth authentication is successfully completed, for example, https://myhost.mydomain:8443/oauth. GitHub will automatically generate a unique pair ClientId/ClientSecret that has to be provided to Gerrit identifying them as a trusted authentication partner. ClientId/ClientSecret are not GitHub credentials and cannot be used by an interactive user to access any GitHub data or information. They are only used for authorizing the integration between a Gerrit instance and GitHub. Running Gerrit init to configure GitHub OAuth We now need to stop Gerrit and go through the init steps again in order to reconfigure the Gerrit authentication. We need to enable HTTP authentication by choosing an HTTP header to be used to verify the user's credentials, and to go through the GitHub settings wizard to configure the OAuth authentication. $ /opt/gerrit/bin/gerrit.sh stop Stopping Gerrit Code Review: OK $ cd /opt/gerrit $ java -jar gerrit.war init [...] *** User Authentication *** Authentication method []: HTTP RETURN Get username from custom HTTP header [Y/n]? Y RETURN Username HTTP header []: GITHUB_USER RETURN SSO logout URL : /oauth/reset RETURN *** GitHub Integration *** GitHub URL [https://github.com]: RETURN Use GitHub for Gerrit login ? [Y/n]? Y RETURN ClientId []: 384cbe2e8d98192f9799 RETURN ClientSecret []: f82c3f9b3802666f2adcc4 RETURN Initialized /opt/gerrit $ /opt/gerrit/bin/gerrit.sh start Starting Gerrit Code Review: OK   Using GitHub login for Gerrit Gerrit is now fully configured to register and authenticate users through GitHub OAuth. When opening the browser to access any Gerrit web pages, we are automatically redirected to the GitHub for login. If we have already visited and authenticated with GitHub previously, the browser cookie will be automatically recognized and used for the authentication, instead of presenting the GitHub login page. Alternatively, if we do not yet have a GitHub account, we create a new GitHub profile by clicking on the SignUp button. Once the authentication process is successfully completed, GitHub requests the user's authorization to grant access to their public profile information. The following screenshot shows GitHub OAuth authorization for Gerrit: The authorization status is then stored under the user's GitHub applications preferences on https://github.com/settings/applications. Finally, GitHub redirects back to Gerrit propagating the user's profile securely using a one-time code which is used to retrieve the full data profile including username, full name, e-mail, and associated SSH public keys. Replication to GitHub The next steps in the Gerrit to GitHub integration is to share the same Git repositories and then keep them up-to-date; this can easily be achieved by using the Gerrit replication plugin. The standard Gerrit replication is a master-slave, where Gerrit always plays the role of the master node and pushes to remote slaves. We will refer to this scheme as push replication because the actual control of the action is given to Gerrit through a git push operation of new commits and branches. Configure Gerrit replication plugin In order to configure push replication we need to enable the Gerrit replication plugin through Gerrit init: $ /opt/gerrit/bin/gerrit.sh stop Stopping Gerrit Code Review: OK $ cd /opt/gerrit $ java -jar gerrit.war init [...] *** Plugins *** Prompt to install core plugins [y/N]? y RETURN Install plugin reviewnotes version 2.7-rc4 [y/N]? RETURN Install plugin commit-message-length-validator version 2.7-rc4 [y/N]? RETURN Install plugin replication version 2.6-rc3 [y/N]? y RETURN Initialized /opt/gerrit $ /opt/gerrit/bin/gerrit.sh start Starting Gerrit Code Review: OK The Gerrit replication plugin relies on the replication.config file under the /opt/gerrit/etc directory to identify the list of target Git repositories to push to. The configuration syntax is a standard .ini format where each group section represents a target replica slave. See the following simplest replication.config script for replicating to GitHub: [remote "github"] url = git@github.com:myorganisation/${name}.git The preceding configuration enables all of the repositories in Gerrit to be replicated to GitHub under the myorganisa tion GitHub Team account. Authorizing Gerrit to push to GitHub Now, that Gerrit knows where to push, we need GitHub to authorize the write operations to its repositories. To do so, we need to upload the SSH public key of the underlying OS user where Gerrit is running to one of the accounts in the GitHub myorganisation team, with the permissions to push to any of the GitHub repositories. Assuming that Gerrit runs under the OS user gerrit, we can copy and paste the SSH public key values from the ~gerrit/.ssh/id_rsa.pub (or ~gerrit/.ssh/id_dsa.pub) to the Add an SSH Key section of the GitHub account under target URL to be set to: https://github.com/settings/ssh Start working with Gerrit replication Everything is now ready to start playing with Gerrit to GitHub replication. Whenever a change to a repository is made on Gerrit, it will be automatically replicated to the corresponding GitHub repository. In reality there is one additional operation that is needed on the GitHub side: the actual creation of the empty repositories using https://github.com/new associated to the ones created in Gerrit. We need to make sure that we select the organization name and repository name, consistent with the ones defined in Gerrit and in the replication.config file. Never initialize the repository from GitHub with an empty commit or readme file; otherwise the first replication attempt from Gerrit will result in a conflict and will then fail. Now GitHub and Gerrit are fully connected and whenever a repository in GitHub matches one of the repositories in Gerrit, it will be linked and synchronized with the latest set of commits pushed in Gerrit. Thanks to the Gerrit-GitHub authentication previously configured, Gerrit and GitHub share the same set of users and the commits authors will be automatically recognized and formatted by GitHub. The following screenshot shows Gerrit commits replicated to GitHub: Reviewing and merging to GitHub branches The final goal of the Code Review process is to agree and merge changes to their branches. The merging strategies need to be aligned with real-life scenarios that may arise when using Gerrit and GitHub concurrently. During the Code Review process the alignment between Gerrit and GitHub was at the change level, not influenced by the evolution of their target branches. Gerrit changes and GitHub pull requests are isolated branches managed by their review lifecycle. When a change is merged, it needs to align with the latest status of its target branch using a fast-forward, merge, rebase, or cherry-pick strategy. Using the standard Gerrit merge functionality, we can apply the configured project merge strategy to the current status of the target branch on Gerrit. The situation on GitHub may have changed as well, so even if the Gerrit merge has succeeded there is no guarantee that the actual subsequent synchronization to GitHub will do the same! The GitHub plugin mitigates this risk by implementing a two-phase submit + merge operation for merging opened changes as follows: Phase-1 : The change target branch is checked against its remote peer on GitHub and fast forwarded if needed. If two branches diverge, the submit + merge is aborted and manual merge intervention is requested. Phase-2 : The change is merged on its target branch in Gerrit and an additional ad hoc replication is triggered. If the merge succeeds then the GitHub pull request is marked as completed. At the end of Phase-2 the Gerrit and GitHub statuses will be completely aligned. The pull request author will then receive the notification that his/her commit has been merged. Using Gerrit and GitHub on http://gerrithub.io When using Gerrit and GitHub on the web with public or private repositories, all of the commits are replicated from Gerrit to GitHub, and each one of them has a complete copy of the data. If we are using a Git and collaboration server on GitHub over the Internet, why can't we do the same for its Gerrit counterpart? Can we avoid installing a standalone instance of Gerrit just for the purpose of going through a formal Code Review? One hassle-free solution is to use the GerritHub service (http://gerrithub.io), which offers a free Gerrit instance on the cloud already configured and connected with GitHub through the github-plugin and github-oauth authentication library. All of the flows that we have covered in this article are completely automated, including the replication and automatic pull request to change automation. As accounts are shared with GitHub, we do not need to register or create another account to use GerritHub; we can just visit http://gerrithub.io and start using Gerrit Code Review with our existing GitHub projects without having to teach our existing community about a new tool. GerritHub also includes an initial setup Wizard for the configuration and automation of the Gerrit projects and the option to configure the Gerrit groups using the existing GitHub. Once Gerrit is configured, the Code Review and GitHub can be used seamlessly for achieving maximum control and social reach within your developer community. Summary We have now integrated our Gerrit installation with GitHub authentication for a seamless Single-Sign-On experience. Using an existing GitHub account we started using Gerrit replication to automatically mirror all the commits to GitHub repositories, allowing our projects to have an extended reach to external users, free to fork our repositories, and to contribute changes as pull requests. Finally, we have completed our Code Review in Gerrit and managed the merge to GitHub with a two-phase change submit + merge process to ensure that the target branches on both Gerrit and GitHub have been merged and aligned accordingly. Similarly to GitHub, this Gerrit setup can be leveraged for free on the web without having to manage a separate private instance, thanks to the free set target URL to http://gerrithub.io service available on the cloud. Resources for Article : Further resources on this subject: Getting Dynamics NAV 2013 on Your Computer – For (Almost) Free [Article] Building Your First Zend Framework Application [Article] Quick start - your first Sinatra application [Article]

(For more resources related to this topic, see here.) At the top of Burp Proxy, you will notice the following three tabs: intercept: HTTP requests and responses that are in transit can be inspected and modified from this window options: Proxy configurations and advanced preferences can be tuned from this window history: All intercepted traffic can be quickly analyzed from this window If you are not familiar with the HTTP protocol or you want to refresh your knowledge, HTTP Made Really Easy, A Practical Guide to Writing Clients and Servers, found at http://www.jmarshall.com/easy/http/, represents a compact reference. Step 1 – Intercepting web requests After firing up Burp and configuring the browser, let's intercept our first HTTP request. During this exercise, we will intercept a simple request to the publisher's website: In the intercept tab, make sure that Burp Proxy is properly stopping all requests in transit by checking the intercept button. This should be marked as intercept is on. In the browser, type http://www.packtpub.com/ in the URL bar and press Enter. Back in Burp Proxy, you should be able to see the HTTP request made by the browser. At this stage, the request is temporarily stopped in Burp Proxy waiting for the user to either forward or stop it. For instance, press forward and return to the browser. You should see the home page of Packt Publishing as you would normally interact with the website. Again, type http://www.packtpub.com/ in the URL bar and press Enter. Let's press drop this time. Back in the browser, the page will contain the warning Burp proxy error: message was dropped by user. We have dropped the request, thus Burp Proxy did not forward the request to the server. As a result, the browser received a temporary HTML page with the warning message generated by Burp, instead of the original HTML content. Let's try one more time. Type http://www.packtpub.com/ in the URL bar of the browser and press Enter. Once the request is properly captured by Burp Proxy, the action button becomes active. Click on it to display the contextual menu. This is an important functionality as it allows you to import the current web request in any of the other Burp tools. You can already imagine the potentialities of having a set of integrated tools that allow you to manipulate and analyze web requests so easily. For example, if we want to decode the request, we can simply click on send to decoder. Burp Proxy In Burp Proxy, we can also decide to automatically forward all requests without waiting for the user to either forward or drop the communication. By clicking on the intercept button, it is possible to switch from intercept is on to intercept is off. Nevertheless, the proxy will record all requests in transit. Also, Burp Proxy allows you to automatically intercept all responses matching specific characteristics. Take a look at the numerous options available in the intercept server response section from within the Burp Proxy options tab. For example, it is possible to intercept the server's response only if the client's request was intercepted. This is extremely helpful while testing input validation vulnerabilities as we are generally interested in evaluating the server's responses for all tampered requests. Or else, you may only want to intercept and inspect responses having a specific return code (for example, 200 OK). Step 2 – Inspecting web requests Once a request is properly intercepted, it is possible to inspect the entire content, headers, and parameters, using one of the four Burp Proxy message analysis tabs: raw: This view allows you to display the web request in raw format within a simple text editor. This is a very handy visualization as it enables maximum flexibility for further changing the content. params: In this view, the focus is on user-supplied parameters (GET/POST parameters, cookies). This is particularly important in case of complex requests as it allows to consider all entry points for potential vulnerabilities. Whenever applicable, Burp Proxy will also automatically perform URL decoding. In addition, Burp Proxy will attempt to parse commonly used formats, including JSON. headers: Similarly, this view displays the HTTP header names and values in tabular form. hex: In case of binary content, it is useful to inspect the hexadecimal representation of the resource. This view allows to display a request as in a traditional hex editor. The history tab enables you to analyze all web requests transited through the proxy: Click on the history tab. At the top, Burp Proxy shows all the requests in the bundle. At the bottom, it displays the content of the request and response corresponding to the specific selection. If you have previously modified the request, Burp Proxy history will also display the modified version. Displaying HTTP requests and responses intercepted by Burp Proxy By double-clicking on one of the requests, Burp will automatically open a new window with the specific content. From this window, it is possible to browse all the captured communication using the previous and next buttons Back in the history tab, Burp Proxy displays several details for each item including the request method, URL, response's code, and length. Each request is uniquely identified by a number, visible in the left-hand side column. Click on the request identifier. Burp Proxy allows you to set a color for that specific item. This is extremely helpful to highlight important requests or responses. For example, during the initial application enumeration, you may notice an interesting request; you can mark it and get back later for further testing. Burp Proxy history is also useful when you have to evaluate a sequence of requests in order to reproduce a specific application behavior. Click on the display filter, at the top of the history list to hide irrelevant content. If you want to analyze all HTTP requests containing at least one parameter, select the show only parameterised checkbox. If you want to display requests having a specific response, just select the appropriate response code in the filter by status code selection. At this point, you may have already understood the potentialities of the tool to filter and reveal interesting traffic. In addition, when using Burp Suite Professional, you can also use the filter by search term option. This feature is particularly important when you need to analyze hundreds of requests or responses as you can filter relevant traffic only by using regular expressions or simply matching particular strings. Using this feature, you may also be able to discover sensitive information (for example, credentials) embedded in the intercepted pages. Step 3 – Tampering web requests As part of a typical security assessment, you will need to modify HTTP requests and analyze the web application responses. For example, to identify SQL injection vulnerabilities, it is important to inject common attack vectors (for example, a single quote) in all user-supplied input, including HTTP headers, cookies, and GET/POST parameters. If you want to refresh your knowledge on common web application vulnerabilities, the OWASP Top Ten Project article at https://www. owasp.org/index.php/Category:OWASP_Top_Ten_Project is a good starting point. Tampering web requests with Burp is as easy as editing strings in a text editor: Intercept a request containing at least one HTTP parameter. For example, you can point your browser to http://www.packtpub.com/books/all?keys=ASP. Go to Burp Proxy | Intercept. At this point, you should see the corresponding HTTP request. From the raw view, you can simply edit any aspect of the web request in transit. For example, you can change the value of the the GET parameter's keys value from ASP to PHP. Edit the request to look like the following: GET /books/all?keys=PHP HTTP/1.1Host: www.packtpub.comUser-Agent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:15.0)Gecko/20100101 Firefox/15.0.1Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8Accept-Language: en-us,en;q=0.5Accept-Encoding: gzip, deflateProxy-Connection: keep-alive Click on forward and get back to the browser. This should result in a search query performed with the string PHP. You can verify it by simply checking the results in the HTML page. Although we have used the raw view to change the previous HTTP request, it is actually possible to use any of the Burp Proxy view. For example, in the params view, it is possible to add a new parameter by following these steps: Clicking on new (right side), from the Burp Proxy params view. Selecting the proper parameter type (URL, body, or cookie). URL should be used for GET parameters, whereas body denotes POST parameters. Typing the name and the value of the newly created parameter. Advanced features After practicing with the basic features provided by Burp Proxy, you are almost ready to experiment with more advanced configurations. Match and replace Let's imagine that you are testing an application designed for mobile devices using a standard browser from your computer. In most cases, the web server examines the user-agent provided by the browser to identify the specific platform and respond with customized resources that better fit mobile phones and tablets. Under these circumstances, you will particularly find the match and replace function, provided by Burp Proxy, very useful. Let's configure Burp Proxy in order to tamper the user-agent HTTP header field: In the options tab of Burp Proxy, scroll down to the match and replace section. Under the match and replace table, a drop-down list and two text fields allow to create a customized rule. Select request header from the drop-down list since we want to create a match condition pertaining to HTTP requests. Type ^User-Agent.*$ in the first text field. This field represents the match within the HTTP request. Burp Proxy's match and replace feature allows you to use simple strings as well as complex regular expressions. If you are not familiar with regular expressions, have a look at http://www.regular-expressions.info/quickstart. html. In the second text field, type Mozilla/5.0 (iPhone; U; CPU like Mac OS X; en) AppleWebKit/4h20+ (KHTML, like Gecko) Version/3.0 Mobile/1C25 Safari/419.3 or any other fake user-agent that you want to impersonate. Click add and verify that the new match has been added to the list; this button is shown here: Burp Proxy match and replace list Intercept a request, leave it to pass through the proxy, and verify that it has been automatically modified by the tool. Automatically modified HTTP header in Burp Proxy HTML modification Another interesting feature of Burp Proxy is the automatic HTML modification, that can be activated and configured in the appropriate section within Burp Proxy | options. By using this function, you can automatically remove JavaScript or modify HTML forms of all received HTTP responses. Some applications deploy client-side validation in the form of disabled HTML form fields or JavaScript code. If you want to verify the presence of server-side controls that enforce specific data formats, you would need to tamper the request with invalid data. In these situations, you can either manually tamper the request in the proxy or enable HTML modification to remove any client-side validation and use the browser in order to submit invalid data. This function can be also used to display hidden form fields. Let's see in practice how you can activate this feature: In Burp Proxy, go to options, scroll down to the HTML modification section. Numerous options are available in this section: unhide hidden form fields to display hidden HTML form fields, enable disabled form fields to submit all input forms present inside the HTML page, remove input field length limits to allow extra-long strings in the text fields, remove JavaScript form validation to make Burp Proxy all onsubmit handler JavaScript functions from HTML forms, remove all JavaScript to completely remove all JS scripts and remove object tags to remove embedded objects within the HTML document. Select the desired checkboxes to activate automatic HTML modification. Summary Using this feature, you will be able to understand whether the web application enforces server- side validation. For instance, some insecure applications use client-side validation only (for example, via JavaScript functions). You can activate the automatic HTML modification feature by selecting the remove JavaScript form validation checkbox in order to perform input validation testing directly from your browser. Resources for Article : Further resources on this subject: Visual Studio 2010 Test Types [Article] Ordered and Generic Tests in Visual Studio 2010 [Article] Manual, Generic, and Ordered Tests using Visual Studio 2008 [Article]  

Boost is a huge collection of libraries. Some of those libraries are small and meant for everyday use and others require a separate article to describe all of their features. This article is devoted to some of those big libraries and to give you some basics to start with. The first two recipes will explain the usage of Boost.Graph. It is a big library with an insane number of algorithms. We'll see some basics and probably the most important part of it visualization of graphs. We'll also see a very useful recipe for generating true random numbers. This is a very important requirement for writing secure cryptography systems. Some C++ standard libraries lack math functions. We'll see how that can be fixed using Boost. But the format of this article leaves no space to describe all of the functions. Writing test cases is described in the Writing test cases and Combining multiple test cases in one test module recipes. This is important for any production-quality system. The last recipe is about a library that helped me in many courses during my university days. Images can be created and modified using it. I personally used it to visualize different algorithms, hide data in images, sign images, and generate textures. Unfortunately, even this article cannot tell you about all of the Boost libraries. Maybe someday I'll write another book... and then a few more. Working with graphs Some tasks require a graphical representation of data. Boost.Graph is a library that was designed to provide a flexible way of constructing and representing graphs in memory. It also contains a lot of algorithms to work with graphs, such as topological sort, breadth first search, depth first search, and Dijkstra shortest paths. Well, let's perform some basic tasks with Boost.Graph! Getting ready Only basic knowledge of C++ and templates is required for this recipe. How to do it... In this recipe, we'll describe a graph type, create a graph of that type, add some vertexes and edges to the graph, and search for a specific vertex. That should be enough to start using Boost.Graph. We start with describing the graph type: #include <boost/graph/adjacency_list.hpp> #include <string> typedef std::string vertex_t; typedef boost::adjacency_list< boost::vecS , boost::vecS , boost::bidirectionalS , vertex_t > graph_type; Now we construct it: graph_type graph; Let's use a non portable trick that speeds up graph construction: static const std::size_t vertex_count = 5; graph.m_vertices.reserve(vertex_count); Now we are ready to add vertexes to the graph: typedef boost::graph_traits<graph_type> ::vertex_descriptor descriptor_t; descriptor_t cpp = boost::add_vertex(vertex_t("C++"), graph); descriptor_t stl = boost::add_vertex(vertex_t("STL"), graph); descriptor_t boost = boost::add_vertex(vertex_t("Boost"), graph); descriptor_t guru = boost::add_vertex(vertex_t("C++ guru"), graph); descriptor_t ansic = boost::add_vertex(vertex_t("C"), graph); It is time to connect vertexes with edges: boost::add_edge(cpp, stl, graph); boost::add_edge(stl, boost, graph); boost::add_edge(boost, guru, graph); boost::add_edge(ansic, guru, graph); We make a function that searches for a vertex: template <class GraphT> void find_and_print(const GraphT& g, boost::string_ref name) { Now we will write code that gets iterators to all vertexes: typedef typename boost::graph_traits<graph_type> ::vertex_iterator vert_it_t; vert_it_t it, end; boost::tie(it, end) = boost::vertices(g); It's time to run a search for the required vertex: typedef boost::graph_traits<graph_type>::vertex_descriptor desc_t; for (; it != end; ++ it) { desc_t desc = *it; if (boost::get(boost::vertex_bundle, g)[desc] == name.data()) { break; } } assert(it != end); std::cout << name << 'n'; } /* find_and_print */ How it works... In step 1, we are describing what our graph must look like and upon what types it must be based. boost::adjacency_list is a class that represents graphs as a two-dimensional structure, where the first dimension contains vertexes and the second dimension contains edges for that vertex. boost::adjacency_list must be the default choice for representing a graph; it suits most cases. The first template parameter, boost::adjacency_list, describes the structure used to represent the edge list for each of the vertexes; the second one describes a structure to store vertexes. We can choose different STL containers for those structures using specific selectors, as listed in the following table: Selector STL container boost::vecS std::vector boost::listS std::list boost::slistS std::slist boost::setS std::set boost::multisetS std::multiset boost::hash_setS std::hash_set The third template parameter is used to make an undirected, directed, or bidirectional graph. Use the boost::undirectedS, boost::directedS, and boost::bidirectionalS selectors respectively. The fifth template parameter describes the datatype that will be used as the vertex. In our example, we chose std::string. We can also support a datatype for edges and provide it as a template parameter. Steps 2 and 3 are trivial, but at step 4 you will see a non portable way to speed up graph construction. In our example, we use std::vector as a container for storing vertexes, so we can force it to reserve memory for the required amount of vertexes. This leads to less memory allocations/deallocations and copy operations during insertion of vertexes into the graph. This step is non-portable because it is highly dependent on the current implementation of boost::adjacency_list and on the chosen container type for storing vertexes. At step 4, we see how vertexes can be added to the graph. Note how boost::graph_traits<graph_type> has been used. The boost::graph_traits class is used to get types that are specific for a graph type. We'll see its usage and the description of some graph-specific types later in this article. Step 5 shows what we need do to connect vertexes with edges. If we had provided a datatype for the edges, adding an edge would look as follows: boost::add_edge(ansic, guru, edge_t(initialization_parameters), graph) Note that at step 6 the graph type is a template parameter. This is recommended to achieve better code reusability and make this function work with other graph types. At step 7, we see how to iterate over all of the vertexes of the graph. The type of vertex iterator is received from boost::graph_traits. The function boost::tie is a part of Boost.Tuple and is used for getting values from tuples to the variables. So calling boost::tie(it, end) = boost::vertices(g) will put the begin iterator into the it variable and the end iterator into the end variable. It may come as a surprise to you, but dereferencing a vertex iterator does not return vertex data. Instead, it returns the vertex descriptor desc, which can be used in boost::get(boost::vertex_bundle, g)[desc] to get vertex data, just as we have done in step 8. The vertex descriptor type is used in many of the Boost.Graph functions; we saw its use in the edge construction function in step 5. As already mentioned, the Boost.Graph library contains the implementation of many algorithms. You will find many search policies implemented, but we won't discuss them in this article. We will limit this recipe to only the basics of the graph library. There's more... The Boost.Graph library is not a part of C++11 and it won't be a part of C++1y. The current implementation does not support C++11 features. If we are using vertexes that are heavy to copy, we may gain speed using the following trick: vertex_descriptor desc = boost::add_vertex(graph); boost::get(boost::vertex_bundle, g_)[desc] = std::move(vertex_data); It avoids copy constructions of boost::add_vertex(vertex_data, graph) and uses the default construction with move assignment instead. The efficiency of Boost.Graph depends on multiple factors, such as the underlying containers types, graph representation, edge, and vertex datatypes. Visualizing graphs Making programs that manipulate graphs was never easy because of issues with visualization. When we work with STL containers such as std::map and std::vector, we can always print the container's contents and see what is going on inside. But when we work with complex graphs, it is hard to visualize the content in a clear way: too many vertexes and too many edges. In this recipe, we'll take a look at the visualization of Boost.Graph using the Graphviz tool. Getting ready To visualize graphs, you will need a Graphviz visualization tool. Knowledge of the preceding recipe is also required. How to do it... Visualization is done in two phases. In the first phase, we make our program output the graph's description in a text format; in the second phase, we import the output from the first step to some visualization tool. The numbered steps in this recipe are all about the first phase. Let's write the std::ostream operator for graph_type as done in the preceding recipe: #include <boost/graph/graphviz.hpp> std::ostream& operator<<(std::ostream& out, const graph_type& g) { detail::vertex_writer<graph_type> vw(g); boost::write_graphviz(out, g, vw); return out; } The detail::vertex_writer structure, used in the preceding step, must be defined as follows: namespace detail { template <class GraphT> class vertex_writer { const GraphT& g_; public: explicit vertex_writer(const GraphT& g) : g_(g) {} template <class VertexDescriptorT> void operator()(std::ostream& out, const VertexDescriptorT& d) const { out << " [label="" << boost::get(boost::vertex_bundle, g_)[d] << ""]"; } }; // vertex_writer } // namespace detail That's all. Now, if we visualize the graph from the previous recipe using the std::cout << graph; command, the output can be used to create graphical pictures using the dot command-line utility: $ dot -Tpng -o dot.png digraph G { 0 [label="C++"]; 1 [label="STL"]; 2 [label="Boost"]; 3 [label="C++ guru"]; 4 [label="C"]; 0->1 ; 1->2 ; 2->3 ; 4->3 ; }   The output of the preceding command is depicted in the following figure: We can also use the Gvedit or XDot programs for visualization if the command line frightens you. How it works... The Boost.Graph library contains function to output graphs in Graphviz (DOT) format. If we write boost::write_graphviz(out, g) with two parameters in step 1, the function will output a graph picture with vertexes numbered from 0. That's not very useful, so we provide an instance of the vertex_writer class that outputs vertex names. As we can see in step 2, the format of output must be DOT, which is understood by the Graphviz tool. You may need to read the Graphviz documentation for more info about the DOT format. If you wish to add some data to the edges during visualization, we need to provide an instance of the edge visualizer as a fourth parameter to boost::write_graphviz. There's more... C++11 does not contain Boost.Graph or the tools for graph visualization. But you do not need to worry—there are a lot of other graph formats and visualization tools and Boost. Graph can work with plenty of them. Using a true random number generator I know of many examples of commercial products that use incorrect methods for getting random numbers. It's a shame that some companies still use rand() in cryptography and banking software. Let's see how to get a fully random uniform distribution using Boost.Random that is suitable for banking software. Getting ready Basic knowledge of C++ is required for this recipe. Knowledge of different types of distributions will also be helpful. The code in this recipe requires linking against the boost_random library. How to do it... To create a true random number, we need some help from the operating system or processor. This is how it can be done using Boost: We'll need to include the following headers: #include <boost/config.hpp> #include <boost/random/random_device.hpp> #include <boost/random/uniform_int_distribution.hpp> Advanced random number providers have different names under different platforms: static const std::string provider = #ifdef BOOST_WINDOWS "Microsoft Strong Cryptographic Provider" #else "/dev/urandom" #endif ; Now we are ready to initialize the generator with Boost.Random: boost::random_device device(provider); Let's get a uniform distribution that returns a value between 1000 and 65535: boost::random::uniform_int_distribution<unsigned short> random(1000); That's it. Now we can get true random numbers using the random(device) call. How it works... Why does the rand() function not suit banking? Because it generates pseudo-random numbers, which means that the hacker could predict the next generated number. This is an issue with all pseudo-random number algorithms. Some algorithms are easier to predict and some harder, but it's still possible. That's why we are using boost::random_device in this example (see step 3). That device gathers information about random events from all around the operating system to construct an unpredictable hardware-generated number. The examples of such events are delays between pressed keys, delays between some of the hardware interruptions, and the internal CPU random number generator. Operating systems may have more than one such type of random number generators. In our example for POSIX systems, we used /dev/urandom instead of the more secure /dev/random because the latter remains in a blocked state until enough random events have been captured by the OS. Waiting for entropy could take seconds, which is usually unsuitable for applications. Use /dev/random to create long-lifetime GPG/SSL/SSH keys. Now that we are done with generators, it's time to move to step 4 and talk about distribution classes. If the generator just generates numbers (usually uniformly distributed), the distribution class maps one distribution to another. In step 4, we made a uniform distribution that returns a random number of unsigned short type. The parameter 1000 means that distribution must return numbers greater or equal to 1000. We can also provide the maximum number as a second parameter, which is by default equal to the maximum value storable in the return type. There's more... Boost.Random has a huge number of true/pseudo random generators and distributions for different needs. Avoid copying distributions and generators; this could turn out to be an expensive operation. C++11 has support for different distribution classes and generators. You will find all of the classes from this example in the <random> header in the std:: namespace. The Boost.Random libraries do not use C++11 features, and they are not really required for that library either. Should you use Boost implementation or STL? Boost provides better portability across systems; however, some STL implementations may have assembly-optimized implementations and might provide some useful extensions. Using portable math functions Some projects require specific trigonometric functions, a library for numerically solving ordinary differential equations, and working with distributions and constants. All of those parts of Boost.Math would be hard to fit into even a separate book. A single recipe definitely won't be enough. So let's focus on very basic everyday-use functions to work with float types. We'll write a portable function that checks an input value for infinity and not-a-number (NaN) values and changes the sign if the value is negative. Getting ready Basic knowledge of C++ is required for this recipe. Those who know C99 standard will find a lot in common in this recipe. How to do it... Perform the following steps to check the input value for infinity and NaN values and change the sign if the value is negative: We'll need the following headers: #include <boost/math/special_functions.hpp> #include <cassert> Asserting for infinity and NaN can be done like this: template <class T> void check_float_inputs(T value) { assert(!boost::math::isinf(value)); assert(!boost::math::isnan(value)); Use the following code to change the sign: if (boost::math::signbit(value)) { value = boost::math::changesign(value); } // ... } // check_float_inputs That's it! Now we can check that check_float_inputs(std::sqrt(-1.0)) and check_float_inputs(std::numeric_limits<double>::max() * 2.0) will cause asserts. How it works... Real types have specific values that cannot be checked using equality operators. For example, if the variable v contains NaN, assert(v!=v) may or may not pass depending on the compiler. For such cases, Boost.Math provides functions that can reliably check for infinity and NaN values. Step 3 contains the boost::math::signbit function, which requires clarification. This function returns a signed bit, which is 1 when the number is negative and 0 when the number is positive. In other words, it returns true if the value is negative. Looking at step 3 some readers might ask, "Why can't we just multiply by -1 instead of calling boost::math::changesign?". We can. But multiplication may work slower than boost::math::changesign and won't work for special values. For example, if your code can work with nan, the code in step 3 will be able to change the sign of -nan and write nan to the variable. The Boost.Math library maintainers recommend wrapping math functions from this example in round parenthesis to avoid collisions with C macros. It is better to write (boost::math::isinf)(value) instead of boost::math::isinf(value). There's more... C99 contains all of the functions described in this recipe. Why do we need them in Boost? Well, some compiler vendors think that programmers do not need them, so you won't find them in one very popular compiler. Another reason is that the Boost.Math functions can be used for classes that behave like numbers. Boost.Math is a very fast, portable, reliable library.

(For more resources related to this topic, see here.) Time handling features are an important part of every BI system. Programming languages, database systems, they all incorporate various time-related functions and Microsoft SQL Server Analysis Services (SSAS) is no exception there. In fact, that's one of its main strengths. The MDX language has various time-related functions designed to work with a special type of dimension called the Time and its typed attributes. While it's true that some of those functions work with any type of dimension, their usefulness is most obvious when applied to time-type dimensions. An additional prerequisite is the existence of multi-level hierarchies, also known as user hierarchies, in which types of levels must be set correctly or some of the time-related functions will either give false results or will not work at all. In this article we're dealing with typical operations, such as year-to-date calculations, running totals, and jumping from one period to another. We go into detail with each operation, explaining known and less known variants and pitfalls. We will discuss why some time calculations can create unnecessary data for the periods that should not have data at all, and why we should prevent it from happening. We will then show you how to prevent time calculations from having values after a certain point in time. In most BI projects, there are always reporting requirements to show measures for today, yesterday, month-to-date, quarter-to-date, year-to-date, and so on. We have three recipes to explore various ways to calculate today's date, and how to turn it into a set and use MDX's powerful set operations to calculate other related periods. Calculating date and time spans is also a common reporting requirement. Calculating the YTD (Year-To-Date) value In this recipe we will look at how to calculate the Year-To-Date value of a measure, that is, the accumulated value of all dates in a year up to the current member on the date dimension. An MDX function YTD() can be used to calculate the Year-To-Date value, but not without its constraints. In this recipe, we will discuss the constraints when using the YTD() function and also the alternative solutions. Getting ready Start SQL Server Management Studio and connect to your SSAS 2012 instance. Click on the New Query button and check that the target database is Adventure Works DW 2012. In order for this type of calculation to work, we need a dimension marked as Time in the Type property, in the Dimension Structure tab of SSDT. That should not be a problem because almost every database contains at least one such dimension and Adventure Works is no exception here. In this example, we're going to use the Date dimension. We can verify in SSDT that the Date dimension's Type property is set to Time. See the following screenshot from SSDT: Here's the query we'll start from: SELECT{ [Measures].[Reseller Sales Amount] } ON 0,{ [Date].[Calendar Weeks].[Calendar Week].MEMBERS } ON 1FROM[Adventure Works] Once executed, the preceding query returns reseller sales values for every week in the database. How to do it... We are going to use the YTD() function, which takes only one member expression, and returns all dates in the year up to the specified member. Then we will use the aggregation function SUM() to sum up the Reseller Sales Amount. Follow these steps to create a calculated measure with YTD calculation: Add the WITH block of the query. Create a new calculated measure within the WITH block and name it Reseller Sales YTD. The new measure should return the sum of the measure Reseller Sales Amount using the YTD() function and the current date member of the hierarchy of interest. Add the new measure on axis 0 and execute the complete query: WITH MEMBER [Measures].[Reseller Sales YTD] AS Sum( YTD( [Date].[Calendar Weeks].CurrentMember ), [Measures].[Reseller Sales Amount] ) SELECT { [Measures].[Reseller Sales Amount], [Measures].[Reseller Sales YTD] } ON 0, { [Date].[Calendar Weeks].[Calendar Week].MEMBERS } ON 1 FROM [Adventure Works] The result will include the second column, the one with the YTD values. Notice how the values in the second column increase over time: How it works... The YTD() function returns the set of members from the specified date hierarchy, starting from the first date of the year and ending with the specified member. The first date of the year is calculated according to the level [Calendar Year] marked as Years type in the hierarchy [Calendar Weeks]. In our example, the YTD() value for the member Week 9 CY 2008 is a set of members starting from Week 1 CY 2008 and going up to that member because the upper level containing years is of the Years type. The set is then summed up using the SUM() function and the Reseller Sales Amount measure. If we scroll down, we'll see that the cumulative sum resets every year, which means that YTD() works as expected. In this example we used the most common aggregation function, SUM(), in order to aggregate the values of the measure throughout the calculated set. SUM() was used because the aggregation type of the Reseller Sales Amount measure is Sum. Alternatively, we could have used the Aggregate() function instead. More information about that function can be found later in this recipe. There's more... Sometimes it is necessary to create a single calculation that will work for any user hierarchy of the date dimension. In that case, the solution is to prepare several YTD() functions, each using a different hierarchy, cross join them, and then aggregate that set using a proper aggregation function (Sum, Aggregate, and so on). However, bear in mind that this will only work if all user hierarchies used in the expression share the same year level. In other words, that there is no offset in years among them (such as exists between the fiscal and calendar hierarchies in Adventure Works cube in 2008 R2). Why does it have to be so? Because the cross join produces the set intersection of members on those hierarchies. Sets are generated relative to the position when the year starts. If there is offset in years, it is possible that sets won't have an intersection. In that case, the result will be an empty space. Now let's continue with a couple of working examples. Here's an example that works for both monthly and weekly hierarchies: WITHMEMBER [Measures].[Reseller Sales YTD] ASSum( YTD( [Date].[Calendar Weeks].CurrentMember ) *YTD( [Date].[Calendar].CurrentMember ),[Measures].[Reseller Sales Amount] )SELECT{ [Measures].[Reseller Sales Amount],[Measures].[Reseller Sales YTD] } ON 0,{ [Date].[Calendar Weeks].[Calendar Week].MEMBERS } ON 1FROM[Adventure Works] If we replace [Date].[Calendar Weeks].[Calendar Week].MEMBERS with [Date].[Calendar].[Month].MEMBERS, the calculation will continue to work. Without the cross join part, that wouldn't be the case. Try it in order to see for yourself! Just be aware that if you slice by additional attribute hierarchies, the calculation might become wrong. In short, there are many obstacles to getting the time-based calculation right. It partially depends on the design of the time dimension (which attributes exist, which are hidden, how the relations are defined, and so on), and partially on the complexity of the calculations provided and their ability to handle various scenarios. A better place to define time-based calculation is the MDX script. There, we can define scoped assignments, but that's a separate topic which will be covered later in the recipe, Using utility dimension to implement time-based calculations. In the meantime, here are some articles related to that topic: http://tinyurl.com/MoshaDateCalcs http://tinyurl.com/DateToolDim Inception-To-Date calculation A similar calculation is the Inception-To-Date calculation in which we're calculating the sum of all dates up to the current member, that is, we do not perform a reset at the beginning of every year. In that case, the YTD() part of the expression should be replaced with this: Null : [Date].[Calendar Weeks].CurrentMember Using the argument in the YTD() function The argument of the YTD() function is optional. When not specified, the first dimension of the Time type in the measure group is used. More precisely, the current member of the first user hierarchy with a level of type Years. This is quite convenient in the case of a simple Date dimension; a dimension with a single user hierarchy. In the case of multiple hierarchies or a role-playing dimension, the YTD() function might not work, if we forget to specify the hierarchy for which we expect it to work. This can be easily verified. Omit the [Date].[Calendar Weeks].CurrentMember part in the initial query and see that both columns return the same values. The YTD() function is not working anymore. Therefore, it is best to always use the argument in the YTD() function. Common problems and how to avoid them In our example we used the [Date].[Calendar Weeks] user hierarchy. That hierarchy has the level Calendar Year created from the same attribute. The type of attribute is Years, which can be verified in the Properties pane of SSDT: However, the Date dimension in the Adventure Works cube has fiscal attributes and user hierarchies built from them as well. The fiscal hierarchy equivalent to [Date].[Calendar Weeks] hierarchy is the [Date].[Fiscal Weeks] hierarchy. There, the top level is named Fiscal Year, created from the same attribute. This time, the type of the attribute is FiscalYear, not Year. If we exchange those two hierarchies in our example query, the YTD() function will not work on the new hierarchy. It will return an error: The name of the solution is the PeriodsToDate() function. YTD() is in fact a short version of the PeriodsToDate() function, which works only if the Year type level is specified in a user hierarchy. When it is not so (that is, some BI developers tend to forget to set it up correctly or in the case that the level is defined as, let's say, FiscalYear like in this test), we can use the PeriodsToDate() function as follows: MEMBER [Measures].[Reseller Sales YTD] ASSum( PeriodsToDate( [Date].[Fiscal Weeks].[Fiscal Year],[Date].[Fiscal Weeks].CurrentMember ),[Measures].[Reseller Sales Amount] ) PeriodsToDate() might therefore be used as a safer variant of the YTD() function. YTD() and future dates It's worth noting that the value returned by a SUM-YTD combination is never empty once a value is encountered in a particular year. Only the years with no values at all will remain completely blank for all their descendants. In our example with the [Calendar Weeks] hierarchy, scrolling down to the Week 23 CY 2008, you will see that this is the last week that has reseller sales. However, the Year-To-Date value is not empty for the rest of the weeks for year 2008, as shown in the following screenshot: This can cause problems for the descendants of the member that represents the current year (and future years as well). The NON EMPTY keyword will not be able to remove empty rows, meaning we'll get YTD values in the future. We might be tempted to use the NON_EMPTY_BEHAVIOR operator to solve this problem but it wouldn't help. Moreover, it would be completely wrong to use it, because it is only a hint to the engine which may or may not be used. It is not a mechanism for removing empty values. In short, we need to set some rows to null, those positioned after the member representing today's date. We'll cover the proper approach to this challenge in the recipe, Finding the last date with data. Calculating the YoY (Year-over-Year) growth (parallel periods) This recipe explains how to calculate the value in a parallel period, the value for the same period in a previous year, previous quarter, or some other level in the date dimension. We're going to cover the most common scenario – calculating the value for the same period in the previous year, because most businesses have yearly cycles. A ParallelPeriod() is a function that is closely related to time series. It returns a member from a prior period in the same relative position as a specified member. For example, if we specify June 2008 as the member, Year as the level, and 1 as the lag, the ParallelPeriod() function will return June 2007. Once we have the measure from the prior parallel period, we can calculate how much the measure in the current period has increased or decreased with respect to the parallel period's value. Getting ready Start SQL Server Management Studio and connect to your SSAS 2012 instance. Click on the New Query button, and check that the target database is Adventure Works DW 2012. In this example we're going to use the Date dimension. Here's the query we'll start from: SELECT{ [Measures].[Reseller Sales Amount] } ON 0,{ [Date].[Fiscal].[Month].MEMBERS } ON 1FROM[Adventure Works] Once executed, the previous query returns the value of Reseller Sales Amount for all fiscal months. How to do it... Follow these steps to create a calculated measure with YoY calculation: Add the WITH block of the query. Create a new calculated measure there and name it Reseller Sales PP. The new measure should return the value of the measure Reseller Sales Amount measure using the ParallelPeriod() function. In other words, the definition of the new measure should be as follows: MEMBER [Measures].[Reseller Sales PP] As( [Measures].[Reseller Sales Amount],ParallelPeriod( [Date].[Fiscal].[Fiscal Year], 1,[Date].[Fiscal].CurrentMember ) ) Specify the format string property of the new measure to match the format of the original measure. In this case that should be the currency format. Create the second calculated measure and name it Reseller Sales YoY %. The definition of that measure should be the ratio of the current member's value against the parallel period member's value. Be sure to handle potential division by zero errors (see the recipe Handling division by zero errors). Include both calculated measures on axis 0 and execute the query, which should look like: WITHMEMBER [Measures].[Reseller Sales PP] As( [Measures].[Reseller Sales Amount],ParallelPeriod( [Date].[Fiscal].[Fiscal Year], 1,[Date].[Fiscal].CurrentMember ) ), FORMAT_STRING = 'Currency'MEMBER [Measures].[Reseller Sales YoY %] Asiif( [Measures].[Reseller Sales PP] = 0, null,( [Measures].[Reseller Sales Amount] /[Measures].[Reseller Sales PP] ) ), FORMAT_STRING = 'Percent'SELECT{ [Measures].[Reseller Sales Amount],[Measures].[Reseller Sales PP],[Measures].[Reseller Sales YoY %] } ON 0,{ [Date].[Fiscal].[Month].MEMBERS } ON 1FROM[Adventure Works] The result will include two additional columns, one with the PP values and the other with the YoY change. Notice how the values in the second column repeat over time and that YoY % ratio shows the growth over time: How it works... The ParallelPeriod() function takes three arguments, a level expression, an index, and a member expression, and all three arguments are optional. The first argument indicates the level on which to look for that member's ancestor, typically the year level like in this example. The second argument indicates how many members to go back on the ancestor's level, typically one, as in this example. The last argument indicates the member for which the function is to be applied. Given the right combination of arguments, the function returns a member that is in the same relative position as a specified member, under a new ancestor. The value for the parallel period's member is obtained using a tuple which is formed with a measure and the new member. In our example, this represents the definition of the PP measure. The growth is calculated as the ratio of the current member's value over the parallel period member's value, in other words, as a ratio of two measures. In our example, that was YoY % measure. In our example we've also taken care of a small detail, setting the FORMAT_STRING to Percent. There's more... The ParallelPeriod() function is very closely related to time series, and typically used on date dimensions. However, it can be used on any type of dimension. For example, this query is perfectly valid: SELECT{ [Measures].[Reseller Sales Amount] } ON 0,{ ParallelPeriod( [Geography].[Geography].[Country],2,[Geography].[Geography].[State-Province].&[CA]&[US] ) } ON 1FROM[Adventure Works] The query returns Hamburg on rows, which is the third state-province in the alphabetical list of states-provinces under Germany. Germany is two countries back from the USA, whose member California, used in this query, is the third state-province underneath that country in the Geography.Geography user hierarchy. We can verify this by browsing the Geography user hierarchy in the Geography dimension in SQL Server Management Studio, as shown in the following screenshot. The UK one member back from the USA, has only one state-province: England. If we change the second argument to 1 instead, we'll get nothing on rows because there's no third state-province under the UK. Feel free to try it: All arguments of the ParallelPeriod() function are optional. When not specified, the first dimension of type Time in the measure group is used, more precisely, the previous member of the current member's parent. This can lead to unexpected results as discussed in the previous recipe. Therefore, it is recommended that you use all the arguments of the ParallelPeriod() function. ParallelPeriod is not a time-aware function The ParallelPeriod() function simply looks for the member from the prior period based on its relative position to its ancestor. For example, if your hierarchy is missing the first six months in the year 2005, for member January 2006, the function will find July 2005 as its parallel period (lagging by one year) because July is indeed the first month in the year 2005. This is exactly the case in Adventure Works DW SSAS prior to 2012. You can test the following scenario in Adventure Works DW SSAS 2008 R2. In our example we used the [Date].[Fiscal] user hierarchy. That hierarchy has all 12 months in every year which is not the case with the [Date].[Calendar] user hierarchy where there's only six months in the first year. This can lead to strange results. For example, if you search-replace the word "Fiscal" with the word "Calendar" in the query we used in this recipe, you'll get this as the result: Notice how the values are incorrect for the year 2006. That's because the ParallelPeriod() function is not a time-aware function, it merely does what it's designed for taking the member that is in the same relative position. Gaps in your time dimension are another potential problem. Therefore, always make the complete date dimensions, with all 12 months in every year and all dates in them, not just working days or similar shortcuts. Remember, Analysis Services isn't doing the date math. It's just navigating using the member's relative position. Therefore, make sure you have laid a good foundation for that. However, that's not always possible. There's an offset of six months between fiscal and calendar years, meaning if you want both of them as date hierarchies, you have a problem; one of them will not have all of the months in the first year. The solution is to test the current member in the calculation and to provide a special logic for the first year, fiscal or calendar; the one that doesn't have all months in it. This is most efficiently done with a scope statement in the MDX script. Another problem in calculating the YoY value is leap years. Calculating moving averages The moving average, also known as the rolling average, is a statistical technique often used in events with unpredictable short-term fluctuations in order to smooth their curve and to visualize the pattern of behavior. The key to get the moving average is to know how to construct a set of members up to and including a specified member, and to get the average value over the number of members in the set. In this recipe, we're going to look at two different ways to calculate moving averages in MDX. Getting ready Start SQL Server Management Studio and connect to your SSAS 2012 instance. Click on the New Query button and check that the target database is Adventure Works DW 2012. In this example we're going to use the Date hierarchy of the Date dimension. Here's the query we'll start from: SELECT{ [Measures].[Internet Order Count] } ON 0,{ [Date].[Date].[Date].MEMBERS} ON 1FROM[Adventure Works] Execute it. The result shows the count of Internet orders for each date in the Date.Date attribute hierarchy. Our task is to calculate the simple moving average (SMA) for dates in the year 2008 based on the count of orders in the previous 30 days. How to do it... We are going to use the LastPeriods() function with a 30 day moving window, and a member expression, [Date].[Date].CurrentMember, as two parameters, and also the AVG() function, to calculate the moving average of Internet order count in the last 30 days. Follow these steps to calculate moving averages: Add the WHERE part of the query and put the year 2006 inside using any available hierarchy. Add the WITH part and define a new calculated measure. Name it SMA 30. Define that measure using the AVG() and LastPeriods() functions. Test to see if you get a managed query similar to this. If so, execute it: WITHMEMBER [Measures].[SMA 30] ASAvg( LastPeriods( 30, [Date].[Date].CurrentMember ),[Measures].[Internet Order Count] )SELECT{ [Measures].[Internet Order Count],[Measures].[SMA 30] } ON 0,{ [Date].[Date].[Date].MEMBERS } ON 1FROM[Adventure Works]WHERE( [Date].[Calendar Year].&[2008] ) The second column in the result set will represent the simple moving average based on the last 30 days. Our final result will look like the following screenshot: How it works... The moving average is a calculation that uses the moving window of N items for which it calculates the statistical mean, that is, the average value. The window starts with the first item and then progressively shifts to the next one until the whole set of items is passed. The function that acts as the moving window is the LastPeriods() function. It returns N items, in this example, 30 dates. That set is then used to calculate the average orders using the AVG() function. Note that the number of members returned by the LastPeriods() function is equal to the span, 30, starting with the member that lags 30 - 1 from the specified member expression, and ending with the specified member. There's more... Another way of specifying what the LastPeriods() function does is to use a range of members with a range-based shortcut. The last member of the range is usually the current member of the hierarchy on an axis. The first member is the N-1th member moving backwards on the same level in that hierarchy, which can be constructed using the Lag(N-1) function. The following expression employing the Lag() function and a range-based shortcut is equivalent to the LastPeriods() in the preceding example: [Date].[Date].CurrentMember.Lag(29) : [Date].[Date].CurrentMember Note that the members returned from the range-based shortcut are inclusive of both the starting member and the ending member. We can easily modify the moving window scope to fit different requirements. For example, in case we need to calculate a 30-day moving average up to the previous member, we can use this syntax: [Date].[Date].CurrentMember.Lag(30) : [Date].[Date].PrevMember The LastPeriods() function is not on the list of optimized functions on this web page: http://tinyurl.com/Improved2008R2. However, tests show no difference in duration with respect to its range alternative. Still, if you come across a situation where the LastPeriods() function performs slowly, try its range alternative. Finally, in case we want to parameterize the expression (for example, to be used in SQL Server Reporting Services), these would be generic forms of the previous expressions: [Date].[Date].CurrentMember.Lag( @span - @offset ) :[Date].[Date].CurrentMember.Lag( @offset ) And LastPeriods( @span, [Date].[Date].CurrentMember.Lag( @offset ) ) The @span parameter is a positive value which determines the size of the window. The @offset parameter determines how much the right side of the window is moved from the current member's position. This shift can be either a positive or negative value. The value of zero means there is no shift at all, the most common scenario. Other ways to calculate the moving averages The simple moving average is just one of many variants of calculating the moving averages. A good overview of a possible variant can be found in Wikipedia: http://tinyurl.com/WikiMovingAvg MDX examples of other variants of moving averages can be found in Mosha Pasumansky's blog article: http://tinyurl.com/MoshaMovingAvg Moving averages and the future dates It's worth noting that the value returned by the moving average calculation is not empty for dates in future because the window is looking backwards, so that there will always be values for future dates. This can be easily verified by scrolling down in our example using the LastPeriods() function, as shown in the following screenshot: In this case the NON EMPTY keyword will not be able to remove empty rows. We might be tempted to use NON_EMPTY_BEHAVIOR to solve this problem but it wouldn't help. Moreover, it would be completely wrong. We don't want to set all the empty rows to null, but only those positioned after the member representing today's date. We'll cover the proper approach to this challenge in the following recipes. Summary This article presents various time-related functions in MDX language that are designed to work with a special type of dimension called the Time and its typed attributes. Resources for Article: Further resources on this subject: What are SSAS 2012 dimensions and cube? [Article] Creating an Analysis Services Cube with Visual Studio 2008 - Part 1 [Article] Terms and Concepts Related to MDX [Article]