How-To Tutorials

Reducing boilerplate code with annotations Mockito allows the use of annotations to reduce the lines of test code in order to increase the readability of tests. Let's take into consideration some of the tests that we have seen in previous examples. Removing boilerplate code by using the MockitoJUnitRunner The shouldCalculateTotalWaitingTimeAndAssertTheArgumentsOnMockUsingArgumentCaptor from Verifying behavior (including argument capturing, verifying call order and working with asynchronous code) section, can be rewritten as follows, using Mockito annotations, and the @RunWith(MockitoJUnitRunner.class) JUnit runner: @RunWith(MockitoJUnitRunner.class) public class _07ReduceBoilerplateCodeWithAnnotationsWithRunner { @Mock KitchenService kitchenServiceMock; @Captor ArgumentCaptor mealArgumentCaptor; @InjectMocks WaiterImpl objectUnderTest; @Test public void shouldCalculateTotalWaitingTimeAndAssert TheArgumentsOnMockUsingArgumentCaptor() throws Exception { //given final int mealPreparationTime = 10; when(kitchenServiceMock.calculate PreparationTime(any(Meal.class))).thenReturn(mealPreparationTime); //when int waitingTime = objectUnderTest.calculate TotalWaitingTime(createSampleMeals ContainingVegetarianFirstCourse()); //then assertThat(waitingTime, is(mealPreparationTime)); verify(kitchenServiceMock).calculatePreparation Time(mealArgumentCaptor.capture()); assertThat(mealArgumentCaptor.getValue(), is (VegetarianFirstCourse.class)); assertThat(mealArgumentCaptor.getAllValues().size(), is(1)); } private List createSampleMeals ContainingVegetarianFirstCourse() { List meals = new ArrayList(); meals.add(new VegetarianFirstCourse()); return meals; } } What happened here is that: All of the boilerplate code can be removed due to the fact that you are using the @RunWith(MockitoJUnitRunner.class) JUnit runner Mockito.mock(…) has been replaced with @Mock annotation You can provide additional parameters to the annotation, such as name, answer or extraInterfaces. The fieldname related to the annotated mock is referred to in any verification so it's easier to identify the mock ArgumentCaptor.forClass(…) is replaced with @Captor annotation. When using the @Captor annotation you avoid warnings related to complex generic types Thanks to the @InjectMocks annotation your object under test is initialized, proper constructor/setters are found and Mockito injects the appropriate mocks for you There is no explicit creation of the object under test You don't need to provide the mocks as arguments of the constructor Mockito @InjectMocksuses either constructor injection, property injection or setter injection Taking advantage of advanced mocks configuration Mockito gives you a possibility of providing different answers for your mocks. Let's focus more on that. Getting more information on NullPointerException Remember the Waiter's askTheCleaningServiceToCleanTheRestaurantMethod(): @Override public boolean askTheCleaningServiceToCleanTheRestaurant (TypeOfCleaningService typeOfCleaningService) { CleaningService cleaningService = cleaningServiceFactory.getMe ACleaningService(typeOfCleaningService); try{ cleaningService.cleanTheTables(); cleaningService.sendInformationAfterCleaning(); return SUCCESSFULLY_CLEANED_THE_RESTAURANT; }catch(CommunicationException communicationException){ System.err.println("An exception took place while trying to send info about cleaning the restaurant"); return FAILED_TO_CLEAN_THE_RESTAURANT; } } Let's assume that we want to test this function. We inject the CleaningServiceFactory as a mock but we forgot to stub the getMeACleaningService(…) method. Normally we would get a NullPointerException since, if the method is not stubbed, it will return null. But what will happen, if as an answer we would provide a RETURNS_SMART_NULLS answer? Let's take a look at the body of the following test: @Mock(answer = Answers.RETURNS_SMART_NULLS) CleaningServiceFactory cleaningServiceFactory; @InjectMocks WaiterImpl objectUnderTest; @Test public void shouldThrowSmartNullPointerExceptionWhenUsingUnstubbedMethod() { //given // Oops forgotten to stub the CleaningServiceFactory.getMeACle aningService(TypeOfCleaningService) method try { //when objectUnderTest.askTheCleaningServiceToCleanTheRestaurant( TypeOfCleaningService.VERY_FAST); fail(); } catch (SmartNullPointerException smartNullPointerException) { System.err.println("A SmartNullPointerException will be thrown here with a very precise information about the error [" + smartNullPointerException + "]"); } } What happened in the test is that: We create a mock with an answer RETURNS_SMART_NULLS of the CleaningServiceFactory The mock is injected to the WaiterImpl We do not stub the getMeACleaningService(…) of the CleaningServiceFactory The SmartNullPointerException will be thrown at the line containing the cleaningService.cleanTheTables() It will contain very detailed information about the reason for the exception to happen and where it happened In order to have the RETURNS_SMART_NULLS as the default answer (you wouldn't have to explicitly define the answer for your mock), you would have to create the class named MockitoConfiguration in a package org.mockito.configuration that either extends the DefaultMockitoConfiguration or implements the IMockitoConfiguration interface: package org.mockito.configuration; import org.mockito.internal.stubbing.defaultanswers.ReturnsSmartNulls; import org.mockito.stubbing.Answer; public class MockitoConfiguration extends DefaultMockitoConfiguration { public Answer<Object> getDefaultAnswer() { return new ReturnsSmartNulls(); } } Summary In this article we learned in detail about reducing the boilerplate code with annotations, and taking advantage of advanced mocks configuration, along with their code implementation. Resources for Article : Further resources on this subject: Testing your App [Article] Drools JBoss Rules 5.0 Flow (Part 2) [Article] Easily Writing SQL Queries with Spring Python [Article]

(For more resources related to this topic, see here.) Open Embedded Linux Entertainment Center (OpenELEC) is an open source embedded operating system developed specifically for the purpose of running complete and easy-to-use media center solutions on a wide variety of hardware. Built around the great open source media player and organizer, XBMC, OpenELEC is optimized to deliver a smooth, intuitive, and efficient user experience. It is developed with the goal of making the task of setting up and maintaining a Home Theater PC (HTPC) easy and straightforward for all users, regardless of their technical skills. This makes OpenELEC the obvious choice for anyone looking to enhance their home media experience with a fully functional media center. The OpenELEC project is open source, so anyone can download and use the operating system completely free of charge. This is a great enticement to get started with setting up a media center of your own, capable of providing many hours of digital entertainment for yourself and your friends and family to enjoy. Media center features Once installed and configured, OpenELEC incorporates XBMC to provide a wide variety of advanced features in a streamlined and straightforward interface. The following are the media types that can be indexed and/or browsed: Movies TV shows Music Pictures The experience of browsing indexed media is enhanced by automatic inclusion of media information and relevant images available from online databases. This provides easy access to ratings, resumes, artwork, and trailers, because everything is incorporated directly in the interface. Hardware requirements Because OpenELEC is very versatile in hardware compatibility, you will most likely be able to turn "that old PC, which has just been lying around" into a fully functional media center, just by installing OpenELEC on it. This approach is great for a fun hobby project, giving you the opportunity to experiment with OpenELEC at no cost. Alternatively, you can get brand new PCs small enough to fit behind a TV, making them the obvious choice for HTPC use. With a very small footprint and compatibility with small atom- or fusion-based platforms, this is where OpenELEC shows its full potential as a lightweight embedded operating system. Summary So, what is OpenELEC? finds out what OpenELEC actually is, what you can do with it, and why it’s so great. Resources for Article : Further resources on this subject: Creating a file server (Samba) [Article] Webcam and Video Wizardry [Article] Our First Project – A Basic Thermometer [Article]

(For more resources related to this topic, see here.) In this article, we'll focus on recipes for designing high performance SOA Suite 11g applications. These recipes look at how you can design your applications for high performance and scalability, where high performance is defined as providing low response times even under load, and scalability is defined as the ability to expand to cope with large numbers of requests. While many of the recipes in other articles can be applied after the application has been designed and written, those in this article need to be applied while the application is being written, and may require that your application is implemented in a certain way. Designing an application with performance as a requirement from the start is much easier than trying to add performance to an application that is already live. So, the recipes in this article provide some of the best value for money in terms of getting the most performance out of your SOA Suite infrastructure. However, while this book focuses on decisions that should be made during the design stages of a development process, this article is not a list of general SOA Suite design patterns. As for many of the recipes in other articles, a lot of the focus in this article is on reducing the amount of time your application spends waiting on external services and the SOA Suite database tables. There are many aspects to the performance of a SOA Suite application, and the design guidelines depend very much on the particular business problems that your application is designed to solve. Factors such as payload size, number of external systems being orchestrated, data transformation complexity, and persistence requirements, all have an impact on the performance of your application. Performance is a relative term, with each application and use-case having its own requirements, but there are a number of basic principles that can help ensure that your application will have a good chance of meeting its goals. Design for peak loads, not average loads. Average loads can be very misleading; there are many situations in which the average load of a system is not a good indicator of the expected load. A good example of this would be a tax return system, where the usage for most of the year is very low, building into a peak in 30 or so days before people's tax returns are due. Smaller payloads are faster. When designing your application, try and limit the amount of payload data that goes through your composites and processes. It is often better to store the data in a database and send the key and metadata through the processes, only going to retrieve data when required. Understand your transaction boundaries. Many applications suffer performance problems because their transactions boundaries are in the wrong places, causing work to be redone unnecessarily when failures happen, or leaving data in an inconsistent state. Understand what causes your application to access the database, and why. Much of the performance overhead of Oracle SOA Suite applications is in repeated trips to the database. These trips add value by persisting state between steps or within processes, but the overuse of steps that cause database persistence is a common cause of performance problems. Follow standard web service design patterns, such as using asynchronous callbacks and stateless invocations, where you are using web services. Using BPEL process parallelization By having your BPEL process execute steps in parallel when there are no dependencies, you can increase the performance by spending less time waiting for external systems to complete. Getting ready You will need JDeveloper installed, and have an open BPEL project. How to do it... Follow these steps to use BPEL process parallelization: Expand the BPEL Constructs section in the component palette. Drag Flow from the palette onto the process. Click on the + icon next to the flow to expand it. Populate the flow with the process steps. How it works... If you have a number of tasks that do not have dependencies on each other, you can improve performance by executing the preceding tasks in parallel. This is most effective with partner links, where you know you are waiting on an external system to produce a response. The default behaviors of these flows is still to use a single thread to execute the branches if external systems are invoked. See the Using non-blocking service invocations in BPEL recipe to learn how to execute flows that contain partner links in parallel. There's more… It is possible to include a limited amount of synchronization between branches of a flow, so that tasks on one branch will wait for tasks on another branch to complete before proceeding. This is best used with caution, but it can provide benefits, and allow tasks that would not otherwise easily lend themselves to parallelization to be run in parallel. Using non-blocking service invocations in BPEL flows We can reduce the latency of forked external service invocations in a BPEL process to the longest flow's execution time if we assign a thread to each flow, making it multi-threaded. Getting ready You'll need a composite loaded in JDeveloper to execute this recipe. This composite will need a flow that makes calls to a partner link external service. How to do it... Follow these steps to use non-blocking service invocations: Right-click on each partner link that is being executed in your BPEL process flow, and select Edit. In the Property tab, select the green + icon and add nonBlockingInvoke as a property name. In the Value box at the bottom, enter true. How it works... This recipe causes flow branches to be executed in parallel, with a new thread to be used for each branch flow. For multiple service invocations that each have a high latency, this can greatly improve the total BPEL execution time. For example, assume we have a BPEL process that calls two web services, one that takes four seconds to execute, and one that takes six seconds to execute. Applying this change will prevent the BPEL process making the calls serially, which would take 10 seconds in total, and enforce parallel service calls in separate threads, reducing the execution time to just over six seconds, or the latency of the longest call plus time to collate the results in the main BPEL process execution thread. While it may sound like a silver bullet performance improvement, this recipe is actually not necessarily going to improve the execution time of our BPEL process! Consider that we may now be at the mercy of greater thread context switching in the CPU; for every invocation of our process, we now have a larger number of threads that will be spawned. If each service invocation has a low latency, the overhead of creating threads and collating callbacks might actually be greater than the cost of invoking the services in a single thread. Our example in this explanation is contrived, so ensure to test the response time of your composite and the profile of your application, when placed under operational load (which may result in lots of threads spawning), as these may well be different with the configuration applied. There's more… This recipe used an alternative way of setting property values to that which we've used elsewhere in the book. Previously, we've edited composite files directly; here, we used the JDeveloper BPEL graphical editor to achieve the same end result. If you check the composite.xml source, you'll see a property added with a name, such as partnerLink.[your service name].nonBlockingInvoke for each service added. Turning off payload validation and composite state monitoring Payload validation checks all inbound and outbound message data thus adding an overhead, especially for large message types. Composite state monitoring allows for administrators to view the results of all instance invocations. We can disable these to improve performance. Getting ready You will need to know the administration credentials for your Oracle SOA Suite WebLogic domain, and have access to the Oracle Enterprise Manager console. How to do it... By following these steps, we can turn off payload validation: Log in to Enterprise Manager. Open the SOA tab, and right-click on soa_infra , select SOA Administration and Common Properties . Un-tick the checkbox for Payload Validation to disable this feature. Un-tick the checkbox for Capture Composite Instance State. How it works... In this recipe, we globally disabled payload validation. This instructs SOA Suite to not check the inbound and outbound message payloads against the schemas associated with our services. This can be particularly useful, not only if the payload is coming from a trusted source, but even if the source is untrusted. A common alternative to payload validation is to add steps to manually validate the payloads at the point that we first receive the request, while not validating those that have come from internal or trusted sources. There are a number of levels of granularity for payload validation; it can be applied at the SOA Engine (BPEL) and composite levels to allow for fine-grained application of this property. You can access these properties via the enterprise manager console right-click menu on the SOA engines and deployed composites. For performance, I would recommend disabling this in all environments above development. Composite state management is responsible for tracking and representing the health of our running composites. This is a powerful administration feature, but costs a lot in terms of performance. Anecdotal testing shows that this can be responsible for up to 30 percent of processing time. As such, for high throughput applications, the value of this feature should be considered. There's more… See the recipes on audit logging to further control composite recording activities at runtime. Ensure that you check the payload validation at the Engine and Composite levels to ensure that they meet your performance requirements.

(For more resources related to this topic, see here.) System requirements Since XenConvert can only convert Windows-based hosts and installs on the same host, the requirements are pretty much the same, as follows: Operating system: Windows XP, Windows Vista, Windows 7, Windows Server 2003 (SP1 or later), Windows Server 2008 (R2) .Net Framework 4.0 Disk Space: 40 MB free disk space XenServer version 6.0 or 6.1 Converting a physical machine to a virtual machine Let's take a quick look at how to convert a physical machine to a virtual machine. First we need to install XenConvert on the source physical machine. We can download XenConvert from this link: http://www.citrix.com/downloads/xenserver/tools/conversion.html. Once the standard Windows installation process is complete, launch the XenConvert tool; but before that we need to prepare the host machine for the conversion. To know more about XenConvert, refer to the XenConvert guide at http://support.citrix.com/article/CTX135017. Preparing the host machine For best results, prepare the host machine as follows: Enable Windows Automount on Windows Server operating systems. Disable Windows Autoplay. Remove any virtualization software before performing a conversion. Ensure that adequate free space exists at the destination, which is approximately 101 percent of used space of all source volumes. Remove any network interface teams; they are not applicable to a virtual machine. We need to run the XenConvert tool on the host machine to start the physical-to-virtual conversion. We can convert the physical machine directly to our XenServer if this host machine is accessible. The other options are to convert to VHD, OVF, or vDisk, which can be imported later on to XenServer using some methods. These options are more useful if we don't have enough disk space or connectivity with XenServer. I chose XenServer and clicked on Next . We can select multiple partitions to be included in the conversion, or select none from the drop-down menu in Source Volume and those disks won't be included in the conversion. We can also increase or decrease the size of the new virtual partition to be allocated for this virtual machine. Click on Next . We'll be asked to provide the details of the XenServer host. The hostname needs either an IP address or a FQDN of the XenServer; a username and password are standard login requirements. In the Workspace field, enter the path to the folder to store the intermediate OVF package that XenConvert will use during the conversion process. XenConvert will store the OVF package in the path we give. Click on Next to select the storage repositories found with XenServer and continue to the last step, in which we'll be provided with the summary of the conversion. Soon after the conversion is completed, we'll be able to have this new machine in our XenCenter. We'll need to have XenServer Tools installed on this new virtual machine. Summary In this article we covered an advanced topic that explained the process of converting a physical Windows server to a virtual machine using XenConvert. Resources for Article : Further resources on this subject: Citrix XenApp Performance Essentials [Article] Defining alerts [Article] Publishing applications [Article]

(For more resources related to this topic, see here.) So we've got this thing for authentication and authorization. Let's see who is responsible and what for. There is an AccessDecisionManager, which, as the name suggests, is responsible for deciding whether we can access something or not; if not, an AccessDeniedException or InsufficientAuthenticationException is thrown. AuthenticationManager is another crucial interface. It is responsible for confirming who we are. Both are just interfaces, so we can swap our own implementations if we like. In a web application, the job of talking with these two components and the user is handled by a web filter called DelegatingFilterProxy, which is decomposed into several small filters. Each one is responsible for a different thing, so we can turn them on, off, or put our own filters in between and mess with them anyway we like. These are quite important, and we will dig into them later. For the big picture, all we need to know is that these filters take care of all the talking, redirect the user to the login page (or an access-denied page), and save the current user details in an HTTPSession. Well, the last part, while true, is a bit misleading. User details are kept in a SecurityContext object, which we can get a hold of by calling SecurityContextHolder.getContext(), and which in the end is stored in HTTPSession by our filters. But we had promised a big picture, not the gory details, so here it is: Quite simple, right? If we have an authentication protocol without login and password, it works in a similar way. We just switch one of the filters, or the authentication manager, to a different implementation. If we don't have a web application, we just need to do the talking ourselves. But this is all for web resources (URLs). What is much more interesting and useful is securing calls to methods. It looks, for example, like this: @PreAuthorize(["isAuthenticated() and hasRole('ROLE_ADMIN')"])public void somethingOnlyAdminCanDo() {} Here, we decided that somethingOnlyAdminCanDo will be protected by our AccessDecisionManager and that the user must be authenticated (not anonymous) and has to have an admin role. Can a user be anonymous and have an admin role at the same time? In theory, yes, but it would not make any sense. Because it's much cheaper to check if he is authenticated and stop right there. We see a bit of optimization in here. We could drop the isAuthenticated() method and the behavior wouldn't change. We can put this kind of annotation on any Java method, but our configuration and mechanism to fire up the security will depend on the type of objects we are trying to protect. For objects declared as Spring beans (which is a short name for anything defined in our Inversion of Control (IoC) configuration, either via XML or annotations), we don't need to do much. Spring will just create proxies (dynamic classes) that take over calls to our secured methods and fire up AccessDecisionManager before passing the call to the object we really wanted to call. For objects outside of the IoC container (anything created with new or just code not defined in Spring context), we can use the power of Aspect Oriented Programming (AOP) to get the same effect. If you don't know what AOP is, don't worry. It's just a bit of magic at the classloader and bytecode level. For now, the only important thing is that it works basically in the same way. This is depicted as follows: We can do much more than this, as we'll see next, but these are the basics. So, how does the AccessDecisionManager decide whether we can access something or not? Imagine a council of very old Jedi masters sitting around a fire. They decide whether or not you are permitted to call a secured method or access a web resource. Each of these masters makes a decision or abstains. Each of them can consult additional information (not only who you are and what you want to do, but every aspect of the situation). In Spring Security, those smart people are called AccessDecisionVoters, and each of them has one vote. The council can be organized in many different ways. It has one voice, and so it may make the decision based on a majority of votes. It may be veto-based, where everything is allowed unless someone disagrees. Or it may need everyone to agree to grant access, otherwise access is denied. The council is the AccessDecisionManager, and we have three implementations previously mentioned out of the box. We can also decide who's in the council and who is not. This is probably the most important decision we can make, because this will decide the security model that we will use in our application. Let's talk about the most popular counselors (implementations of AccessDecisionVoter). Model based on roles (RoleVoter): This guy makes his decision based on the role of the user and the required role for the resource/method. So if we write @PreAuthorize("hasRole('ROLE_ADMIN')"), you better be a damn admin or you'll get a no-no from this guy. Model based on entity access control permissions (AclEntryVoter): This guy doesn't worry about roles. He is much more than that. Acl stands for Access Control List, which represents a list of permissions. Every user has a list of permissions, possibly for every domain object (usually an object in the database), that you want to secure. So, for example, if we have a bank application, the supervisor can give Frank access to a single specific customer (say, ACME—A Company that Makes Everything), which is represented as an entity in the database and as an object in our system. No other employee will be able to do anything to that customer unless the supervisor grants that person the same permission as Frank. This is probably the most scrutinous voter we would ever use. Our customer can have a very detailed configuration with him/her. On the other hand, this is also the most cumbersome, as we need to create a usable graphical interface to set permissions for every user and every domain object. While we have done this a few times, most of our customers wanted a simpler approach, and even those who started with a graphical user interface to configure everything asked for a simplified version based on business rules, at the end of the project. If your customer describes his security needs in terms of rules such as "Frank can edit every customer he has created but he cannot do anything other than view other customers", it means it's time for PreInvocationAuthorizationAdviceVoter. Business rules model (PreInvocationAuthorizationAdviceVoter): This is usually used when you want to implement static business rules in the application. This goes like "if I've written a blog post, I can change it later, but others can only comment" and "if a friend asked me to help him write the blog post, I can do that, because I'm his friend". Most of these things are also possible to implement with ACLs, but would be very cumbersome. This is our favorite voter. With it, it's very easy to write, test, and change the security restrictions, because instead of writing every possible relation in the database (as with ACL voter) or having only dumb roles, we write our security logic in plain old Java classes. Great stuff and most useful, once you see how it works. Did we mention that this is a council? Yes we did. The result of this is that we can mix any voters we want and choose any council organization we like. We can have all three voters previously mentioned and allow access if any of them says "yes". There are even more voters. And we can write new ones ourselves. Do you feel the power of the Jedi council already? Do you feel the power of the Jedi council already? Summary This section provides an overview of authentication and authorization, which are the principles of Spring security. Resources for Article : Further resources on this subject: Migration to Spring Security 3 [Article] Getting Started with Spring Security [Article] So, what is Spring for Android? [Article]

(For more resources related to this topic, see here.) Understanding the purpose of PITR PostgreSQL offers a tool called pg_dump to backup a database. Basically, pg_dump will connect to the database, read all the data in transaction isolation level "serializable" and return the data as text. As we are using "serializable", the dump is always consistent. So, if your pg_dump starts at midnight and finishes at 6 A.M, you will have created a backup, which contains all the data as of midnight but no further data. This kind of snapshot creation is highly convenient and perfectly feasible for small to medium amounts of data. A dump is always consistent. This means that all foreign keys are intact; new data added after starting the dump will be missing. It is most likely the most common way to perform standard backups. But, what if your data is so valuable and maybe so large in size that you want to backup it incrementally? Taking a snapshot from time to time might be enough for some applications; for highly critical data, it is clearly not. In addition to that, replaying 20 TB of data in textual form is not efficient either. Point-In-Time-Recovery has been designed to address this problem. How does it work? Based on a snapshot of the database, the XLOG will be replayed later on. This can happen indefinitely or up to a point chosen by you. This way, you can reach any point in time. This method opens the door to many different approaches and features: Restoring a database instance up to a given point in time Creating a standby database, which holds a copy of the original data Creating a history of all changes In this article, we will specifically feature on the incremental backup functionality and describe how you can make your data more secure by incrementally archiving changes to a medium of choice. Moving to the bigger picture The following picture provides an overview of the general architecture in use for Point-In-Time-Recovery: PostgreSQL produces 16 MB segments of transaction log. Every time one of those segments is filled up and ready, PostgreSQL will call the so called archive_command. The goal of archive_command is to transport the XLOG file from the database instance to an archive. In our image, the archive is represented as the pot on the bottom-right side of the image. The beauty of the design is that you can basically use an arbitrary shell script to archive the transaction log. Here are some ideas: Use some simple copy to transport data to an NFS share Run rsync to move a file Use a custom made script to checksum the XLOG file and move it to an FTP server Copy the XLOG file to a tape The possible options to manage XLOG are only limited by imagination. The restore_command is the exact counterpart of the archive_command. Its purpose is to fetch data from the archive and provide it to the instance, which is supposed to replay it (in our image, this is labeled as Restored Backup). As you have seen, replay might be used for replication or simply to restore a database to a given point in time as outlined in this article. Again, the restore_command is simply a shell script doing whatever you wish, file by file. It is important to mention that you, the almighty administrator, are in charge of the archive. You have to decide how much XLOG to keep and when to delete it. The importance of this task cannot be underestimated. Keep in mind, when then archive_command fails for some reason, PostgreSQL will keep the XLOG file and retry after a couple of seconds. If archiving fails constantly from a certain point on, it might happen that the master fills up. The sequence of XLOG files must not be interrupted; if a single file is missing, you cannot continue to replay XLOG. All XLOG files must be present because PostgreSQL needs an uninterrupted sequence of XLOG files; if a single file is missing, the recovery process will stop there at the very latest. Archiving the transaction log After taking a look at the big picture, we can take a look and see how things can be put to work. The first thing you have to do when it comes to Point-In-Time-Recovery is to archive the XLOG. PostgreSQL offers all the configuration options related to archiving through postgresql.conf. Let us see step by step what has to be done in postgresql.conf to start archiving: First of all, you should turn archive_mode on. In the second step, you should configure your archive command. The archive command is a simple shell command taking two parameters: %p: This is a placeholder representing the XLOG file that should be archived, including its full path (source). %f: This variable holds the name of the XLOG without the path pointing to it. Let us set up archiving now. To do so, we should create a place to put the XLOG. Ideally, the XLOG is not stored on the same hardware as the database instance you want to archive. For the sake of this example, we assume that we want to apply an archive to /archive. The following changes have to be made to postgresql.conf: wal_level = archive # minimal, archive, or hot_standby # (change requires restart) archive_mode = on # allows archiving to be done # (change requires restart) archive_command = 'cp %p /archive/%f' # command to use to archive a logfile segment # placeholders: %p = path of file to archive # %f = file name only Once those changes have been made, archiving is ready for action and you can simply restart the database to activate things. Before we restart the database instance, we want to focus your attention on wal_level. Currently three different wal_level settings are available: minimal archive hot_standby The amount of transaction log produced in the case of just a single node is by far not enough to synchronize an entire second instance. There are some optimizations in PostgreSQL, which allow XLOG-writing to be skipped in the case of single-node mode. The following instructions can benefit from wal_level being set to minimal: CREATE TABLE AS, CREATE INDEX, CLUSTER, and COPY (if the table was created or truncated within the same transaction). To replay the transaction log, at least archive is needed. The difference between archive and hot_standby is that archive does not have to know about currently running transactions. For streaming replication, however, this information is vital. Restarting can either be done through pg_ctl –D /data_directory –m fast restart directly or through a standard init script. The easiest way to check if our archiving works is to create some useless data inside the database. The following code snippets shows a million rows can be made easily: test=# CREATE TABLE t_test AS SELECT * FROM generate_series(1,1000000);SELECT 1000000test=# SELECT * FROM t_test LIMIT 3;generate_series----------------- 1 2 3(3 rows) We have simply created a list of numbers. The important thing is that 1 million rows will trigger a fair amount of XLOG traffic. You will see that a handful of files have made it to the archive: iMac:archivehs$ ls -ltotal 131072-rw------- 1 hs wheel 16777216 Mar 5 22:31000000010000000000000001-rw------- 1 hs wheel 16777216 Mar 5 22:31000000010000000000000002-rw------- 1 hs wheel 16777216 Mar 5 22:31000000010000000000000003-rw------- 1 hs wheel 16777216 Mar 5 22:31000000010000000000000004 Those files can be easily used for future replay operations. If you want to save storage, you can also compress those XLOG files. Just add gzip to your archive_command. Taking base backups In the previous section, you have seen that enabling archiving takes just a handful of lines and offers a great deal of flexibility. In this section, we will see how to create a so called base backup, which can be used to apply XLOG later on. A base backup is an initial copy of the data. Keep in mind that the XLOG itself is more or less worthless. It is only useful in combination with the initial base backup. In PostgreSQL, there are two main options to create an initial base backup: Using pg_basebackup Traditional copy/rsync based methods The following two sections will explain in detail how a base backup can be created: Using pg_basebackup The first and most common method to create a backup of an existing server is to run a command called pg_basebackup, which was introduced in PostgreSQL 9.1.0. Basically pg_basebackup is able to fetch a database base backup directly over a database connection. When executed on the slave, pg_basebackup will connect to the database server of your choice and copy all the data files in the data directory over to your machine. There is no need to log into the box anymore, and all it takes is one line of code to run it; pg_basebackup will do all the rest for you. In this example, we will assume that we want to take a base backup of a host called sample.postgresql-support.de. The following steps must be performed: Modify pg_hba.conf to allow replication Signal the master to take pg_hba.conf changes into account Call pg_basebackup Modifying pg_hba.conf To allow remote boxes to log into a PostgreSQL server and to stream XLOG, you have to explicitly allow replication. In PostgreSQL, there is a file called pg_hba.conf, which tells the server which boxes are allowed to connect using which type of credentials. Entire IP ranges can be allowed or simply discarded through pg_hba.conf. To enable replication, we have to add one line for each IP range we want to allow. The following listing contains an example of a valid configuration: # TYPE DATABASE USER ADDRESS METHODhost replication all 192.168.0.34/32 md5 In this case we allow replication connections from 192.168.0.34. The IP range is identified by 32 (which simply represents a single server in our case). We have decided to use MD5 as our authentication method. It means that the pg_basebackup has to supply a password to the server. If you are doing this in a non-security critical environment, using trust as authentication method might also be an option. What happens if you actually have a database called replication in your system? Basically, setting the database to replication will just configure your streaming behavior, if you want to put in rules dealing with the database called replication, you have to quote the database name as follows: "replication". However, we strongly advise not to do this kind of trickery to avoid confusion. Signaling the master server Once pg_hba.conf has been changed, we can tell PostgreSQL to reload the configuration. There is no need to restart the database completely. We have three options to make PostgreSQL reload pg_hba.conf: By running an SQL command: SELECT pg_reload_conf(); By sending a signal to the master: kill –HUP 4711 (with 4711 being the process ID of the master) By calling pg_ctl: pg_ctl –D $PGDATA reload (with $PGDATA being the home directory of your database instance) Once we have told the server acting as data source to accept streaming connections, we can move forward and run pg_basebackup. pg_basebackup – basic features pg_basebackup is a very simple-to-use command-line tool for PostgreSQL. It has to be called on the target system and will provide you with a ready-to-use base backup, which is ready to consume the transaction log for Point-In-Time-Recovery. The syntax of pg_basebackup is as follows: iMac:dbhs$ pg_basebackup --help pg_basebackup takes a base backup of a running PostgreSQL server. Usage: pg_basebackup [OPTION]... Options controlling the output: -D, --pgdata=DIRECTORY receive base backup into directory -F, --format=p|t output format (plain (default), tar) -x, --xlog include required WAL files in backup (fetch mode) -X, --xlog-method=fetch|stream include required WAL files with specified method -z, --gzip compress tar output -Z, --compress=0-9 compress tar output with given compression level General options: -c, --checkpoint=fast|spread set fast or spread checkpointing -l, --label=LABEL set backup label -P, --progress show progress information -v, --verbose output verbose messages -V, --version output version information, then exit -?, --help show this help, then exit Connection options: -h, --host=HOSTNAME database server host or socket directory -p, --port=PORT database server port number -s, --status-interval=INTERVAL time between status packets sent to server (in seconds) -U, --username=NAME connect as specified database user -w, --no-password never prompt for password -W, --password force password prompt (should happen automatically) A basic call to pg_basebackup would look like that: iMac:dbhs$ pg_basebackup -D /target_directory -h sample.postgresql-support.de In this example, we will fetch the base backup from sample.postgresql-support.de and put it into our local directory called /target_directory. It just takes this simple line to copy an entire database instance to the target system. When we create a base backup as shown in this section, pg_basebackup will connect to the server and wait for a checkpoint to happen before the actual copy process is started. In this mode, this is necessary because the replay process will start exactly at this point in the XLOG. The problem is that it might take a while until a checkpoint kicks in; pg_basebackup does not enforce a checkpoint on the source server straight away to make sure that normal operations are not disturbed. If you don't want to wait on a checkpoint, consider using --checkpoint=fast. It will enforce an instant checkpoint and pg_basebackup will start copying instantly. By default, a plain base backup will be created. It will consist of all the files in directories found on the source server. If the base backup should be stored on tape, we suggest to give –-format=t a try. It will automatically create a TAR archive (maybe on a tape). If you want to move data to a tape, you can save an intermediate step easily this way. When using TAR, it is usually quite beneficial to use it in combination with --gzip to reduce the size of the base backup on disk. There is also a way to see a progress bar while doing the base backup but we don't recommend to use this option (--progress) because it requires pg_basebackup to determine the size of the source instance first, which can be costly. pg_basebackup – self-sufficient backups Usually a base backup without XLOG is pretty worthless. This is because the base backup is taken while the master is fully operational. While the backup is taken, those storage files in the database instance might have been modified heavily. The purpose of the XLOG is to fix those potential issues in the data files reliably. But, what if we want to create a base backup, which can live without (explicitly archived) XLOG? In this case, we can use the --xlog-method=stream option. If this option has been chosen, pg_basebackup will not just copy the data as it is but it will also stream the XLOG being created during the base backup to our destination server. This will provide us with just enough XLOG to allow us to start a base backup made that way directly. It is self-sufficient and does not need additional XLOG files. This is not Point-In-Time-Recovery but it can come in handy in case of trouble. Having a base backup, which can be started right away, is usually a good thing and it comes at fairly low cost. Please note that --xlog-method=stream will require two database connections to the source server, not just one. You have to keep that in mind when adjusting max_wal_senders on the source server. If you are planning to use Point-In-Time-Recovery and if there is absolutely no need to start the backup as it is, you can safely skip the XLOG and save some space this way (default mode). Making use of traditional methods to create base backups These days pg_basebackup is the most common way to get an initial copy of a database server. This has not always been the case. Traditionally, a different method has been used which works as follows: Call SELECT pg_start_backup('some label'); Copy all data files to the remote box through rsync or any other means. Run SELECT pg_stop_backup(); The main advantage of this old method is that there is no need to open a database connection and no need to configure XLOG-streaming infrastructure on the source server. A main advantage is also that you can make use of features such as ZFS-snapshots or similar means, which can help to dramatically reduce the amount of I/O needed to create an initial backup. Once you have started pg_start_backup, there is no need to hurry. It is not necessary and not even especially desirable to leave the backup mode soon. Nothing will happen if you are in backup mode for days. PostgreSQL will archive the transaction log as usual and the user won't face any kind of downside. Of course, it is bad habit not to close backups soon and properly. However, the way PostgreSQL works internally does not change when a base backup is running. There is nothing filling up, no disk I/O delayed, or anything of this sort. Tablespace issues If you happen to use more than one tablespace, pg_basebackup will handle this just fine if the filesystem layout on the target box is identical to the filesystem layout on the master. However, if your target system does not use the same filesystem layout there is a bit more work to do. Using the traditional way of doing the base backup might be beneficial in this case. In case you are using --format=t (for TAR), you will be provided with one TAR file per tablespace. Keeping an eye on network bandwidth Let us imagine a simple scenario involving two servers. Each server might have just one disk (no SSDs). Our two boxes might be interconnected through a 1 Gbit link. What will happen to your applications if the second server starts to run a pg_basebackup? The second box will connect, start to stream data at full speed and easily kill your hard drive by using the full bandwidth of your network. An application running on the master might instantly face disk wait and offer bad response times. Therefore it is highly recommended to control the bandwidth used up by rsync to make sure that your business applications have enough spare capacity (mainly disk, CPU is usually not an issue). If you want to limit rsync to, say, 20 MB/sec, you can simply use rsync --bwlimit=20000. This will definitely make the creation of the base backup take longer but it will make sure that your client apps will not face problems. In general we recommend a dedicated network interconnect between master and slave to make sure that a base backup does not affect normal operations. Limiting bandwidth cannot be done with pg_basebackup onboard functionality.Of course, you can use any other tool to copy data and achieve similar results. If you are using gzip compression with –-gzip, it can work as an implicit speed brake. However, this is mainly a workaround. Replaying the transaction log Once we have created ourselves a shiny initial base backup, we can collect the XLOG files created by the database. When the time has come, we can take all those XLOG files and perform our desired recovery process. This works as described in this section. Performing a basic recovery In PostgreSQL, the entire recovery process is governed by a file named recovery.conf, which has to reside in the main directory of the base backup. It is read during startup and tells the database server where to find the XLOG archive, when to end replay, and so forth. To get you started, we have decided to include a simple recovery.conf sample file for performing a basic recovery process: restore_command = 'cp /archive/%f %p'recovery_target_time = '2013-10-10 13:43:12' The restore_command is essentially the exact counterpart of the archive_command you have seen before. While the archive_command is supposed to put data into the archive, the restore_command is supposed to provide the recovering instance with the data file by file. Again, it is a simple shell command or a simple shell script providing one chunk of XLOG after the other. The options you have here are only limited by imagination; all PostgreSQL does is to check for the return code of the code you have written, and replay the data provided by your script. Just like in postgresql.conf, we have used %p and %f as placeholders; the meaning of those two placeholders is exactly the same. To tell the system when to stop recovery, we can set the recovery_target_time. The variable is actually optional. If it has not been specified, PostgreSQL will recover until it runs out of XLOG. In many cases, simply consuming the entire XLOG is a highly desirable process; if something crashes, you want to restore as much data as possible. But, it is not always so. If you want to make PostgreSQL stop recovery at a specific point in time, you simply have to put the proper date in. The crucial part here is actually to know how far you want to replay XLOG; in a real work scenario this has proven to be the trickiest question to answer. If you happen to a recovery_target_time, which is in the future, don't worry, PostgreSQL will start at the very last transaction available in your XLOG and simply stop recovery. The database instance will still be consistent and ready for action. You cannot break PostgreSQL, but, you might break your applications in case data is lost because of missing XLOG. Before starting PostgreSQL, you have to run chmod 700 on the directory containing the base backup, otherwise, PostgreSQL will error out: iMac:target_directoryhs$ pg_ctl -D /target_directorystartserver startingFATAL: data directory "/target_directory" has group or world accessDETAIL: Permissions should be u=rwx (0700). This additional security check is supposed to make sure that your data directory cannot be read by some user accidentally. Therefore an explicit permission change is definitely an advantage from a security point of view (better safe than sorry). Now that we have all the pieces in place, we can start the replay process by starting PostgreSQL: iMac:target_directoryhs$ pg_ctl –D /target_directory startserver startingLOG: database system was interrupted; last known up at 2013-03-1018:04:29 CETLOG: creating missing WAL directory "pg_xlog/archive_status"LOG: starting point-in-time recovery to 2013-10-10 13:43:12+02LOG: restored log file "000000010000000000000006" from archiveLOG: redo starts at 0/6000020LOG: consistent recovery state reached at 0/60000B8LOG: restored log file "000000010000000000000007" from archiveLOG: restored log file "000000010000000000000008" from archiveLOG: restored log file "000000010000000000000009" from archiveLOG: restored log file "00000001000000000000000A" from archivecp: /tmp/archive/00000001000000000000000B: No such file ordirectoryLOG: could not open file "pg_xlog/00000001000000000000000B" (logfile 0, segment 11): No such file or directoryLOG: redo done at 0/AD5CE40LOG: last completed transaction was at log time 2013-03-1018:05:33.852992+01LOG: restored log file "00000001000000000000000A" from archivecp: /tmp/archive/00000002.history: No such file or directoryLOG: selected new timeline ID: 2cp: /tmp/archive/00000001.history: No such file or directoryLOG: archive recovery completeLOG: database system is ready to accept connectionsLOG: autovacuum launcher started The amount of log produced by the database tells us everything we need to know about the restore process and it is definitely worth investigating this information in detail. The first line indicates that PostgreSQL has found out that it has been interrupted and that it has to start recovery. From the database instance point of view, a base backup looks more or less like a crash needing some instant care by replaying XLOG; this is precisely what we want. The next couple of lines (restored log file ...) indicate that we are replaying one XLOG file after the other that have been created since the base backup. It is worth mentioning that the replay process starts at the sixth file. The base backup knows where to start, so PostgreSQL will automatically look for the right XLOG file. The message displayed after PostgreSQL reaches the sixth file (consistent recovery state reached at 0/60000B8) is of importance. PostgreSQL states that it has reached a consistent state. This is important. The reason is that the data files inside a base backup are actually broken by definition, but, the data files are not broken beyond repair. As long as we have enough XLOG to recover, we are very well off. If you cannot reach a consistent state, your database instance will not be usable and your recovery cannot work without providing additional XLOG. Practically speaking, not being able to reach a consistent state usually indicates a problem somewhere in your archiving process and your system setup. If everything up to now has been working properly, there is no reason not to reach a consistent state. Once we have reached a consistent state, one file after the other will be replayed successfully until the system finally looks for the 00000001000000000000000B file. The problem is that this file has not been created by the source database instance. Logically, an error pops up. Not finding the last file is absolutely normal; this type of error is expected if the recovery_target_time does not ask PostgreSQL to stop recovery before it reaches the end of the XLOG stream. Don't worry, your system is actually fine. You have successfully replayed everything to the file showing up exactly before the error message. As soon as all the XLOG has been consumed and the error message discussed earlier has been issued, PostgreSQL reports the last transaction it was able or supposed to replay, and starts up. You have a fully recovered database instance now and you can connect to the database instantly. As soon as the recovery has ended, recovery.conf will be renamed by PostgreSQL to recovery.done to make sure that it does not do any harm when the new instance is restarted later on at some point. More sophisticated positioning in the XLOG Up to now, we have recovered a database up to the very latest moment available in our 16 MB chunks of transaction log. We have also seen that you can define the desired recovery timestamp. But the question now is: How do you know which point in time to perform the recovery to? Just imagine somebody has deleted a table during the day. What if you cannot easily determine the recovery timestamp right away? What if you want to recover to a certain transaction? recovery.conf has all you need. If you want to replay until a certain transaction, you can refer to recovery_target_xid. Just specify the transaction you need and configure recovery_target_inclusive to include this very specific transaction or not. Using this setting is technically easy but as mentioned before, it is not easy by far to find the right position to replay to. In a typical setup, the best way to find a reasonable point to stop recovery is to use pause_at_recovery_target. If this is set to true, PostgreSQL will not automatically turn into a productive instance if the recovery point has been reached. Instead, it will wait for further instructions from the database administrator. This is especially useful if you don't know exactly how far to replay. You can replay, log in, see how far the database is, change to the next target time, and continue replaying in small steps. You have to set hot_standby = on in postgresql.conf to allow reading during recovery. Resuming recovery after PostgreSQL has paused can be done by calling a simple SQL statement: SELECT pg_xlog_replay_resume(). It will make the instance move to the next position you have set in recovery.conf. Once you have found the right place, you can set the pause_at_recovery_target back to false and call pg_xlog_replay_resume. Alternatively, you can simply utilize pg_ctl –D ... promote to stop recovery and make the instance operational. Was this explanation too complicated? Let us boil it down to a simple list: Add a restore_command to the recovery.conf file. Add recovery_target_time to the recovery.conf file. Set pause_at_recovery_target to true in the recovery.conf file. Set hot_standby to on in postgresql.conf file. Start the instance to be recovered. Connect to the instance once it has reached a consistent state and as soon as it stops recovering. Check if you are already where you want to be. If you are not: Change recovery_target_time. Run SELECT pg_xlog_replay_resume(). Check again and repeat this section if it is necessary. Keep in mind that once recovery has finished and once PostgreSQL has started up as a normal database instance, there is (as of 9.2) no way to replay XLOG later on. Instead of going through this process, you can of course always use filesystem snapshots. A filesystem snapshot will always work with PostgreSQL because when you restart a snapshotted database instance, it will simply believe that it had crashed before and recover normally. Cleaning up the XLOG on the way Once you have configured archiving, you have to store the XLOG being created by the source server. Logically, this cannot happen forever. At some point, you really have to get rid of this XLOG; it is essential to have a sane and sustainable cleanup policy for your files. Keep in mind, however, that you must keep enough XLOG so that you can always perform recovery from the latest base backup. But if you are certain that a specific base backup is not needed anymore, you can safely clean out all the XLOG that is older than the base backup you want to keep. How can an administrator figure out what to delete? The best method is to simply take a look at your archive directory: 000000010000000000000005000000010000000000000006000000010000000000000006.00000020.backup000000010000000000000007000000010000000000000008 Check out the filename in the middle of the listing. The .backup file has been created by the base backup. It contains some information about the way the base backup has been made and tells the system where to continue replaying the XLOG. If the backup file belongs to the oldest base backup you need to keep around, you can safely erase all the XLOG lower than file number 6; in this case, file number 5 could be safely deleted. In our case, 000000010000000000000006.00000020.backup contains the following information: START WAL LOCATION: 0/6000020 (file 000000010000000000000006)STOP WAL LOCATION: 0/60000E0 (file 000000010000000000000006)CHECKPOINT LOCATION: 0/6000058BACKUP METHOD: streamedBACKUP FROM: masterSTART TIME: 2013-03-10 18:04:29 CETLABEL: pg_basebackup base backupSTOP TIME: 2013-03-10 18:04:30 CET The .backup file will also provide you with relevant information such as the time the base backup has been made. It is plain there and so it should be easy for ordinary users to read this information. As an alternative to deleting all the XLOG files at one point, it is also possible to clean them up during replay. One way is to hide an rm command inside your restore_command. While this is technically possible, it is not necessarily wise to do so (what if you want to recover again?). Also, you can add the recovery_end_command command to your recovery.conf file. The goal of recovery_end_command is to allow you to automatically trigger some action as soon as the recovery ends. Again, PostgreSQL will call a script doing precisely what you want. You can easily abuse this setting to clean up the old XLOG when the database declares itself active. Switching the XLOG files If you are going for an XLOG file-based recovery, you have seen that one XLOG will be archived every 16 MB. What would happen if you never manage to create 16 MB of changes? What if you are a small supermarket, which just makes 50 sales a day? Your system will never manage to fill up 16 MB in time. However, if your system crashes, the potential data loss can be seen as the amount of data in your last unfinished XLOG file. Maybe this is not good enough for you. A postgresql.conf setting on the source database might help. The archive_timeout tells PostgreSQL to create a new XLOG file at least every x seconds. So, if you are this little supermarket, you can ask the database to create a new XLOG file every day shortly before you are heading for home. In this case, you can be sure that the data of the day will safely be on your backup device already. It is also possible to make PostgreSQL switch to the next XLOG file by hand. A procedure named pg_switch_xlog() is provided by the server to do the job: test=# SELECT pg_switch_xlog();pg_switch_xlog----------------0/17C0EF8(1 row) You might want to call this procedure when some important patch job has finished or if you want to make sure that a certain chunk of data is safely in your XLOG archive. Summary In this article, you have learned about Point-In-Time-Recovery, which is a safe and easy way to restore your PostgreSQL database to any desired point in time. PITR will help you to implement better backup policies and make your setups more robust. Resources for Article: Further resources on this subject: Introduction to PostgreSQL 9 [Article] PostgreSQL: Tips and Tricks [Article] PostgreSQL 9: Reliable Controller and Disk Setup [Article]

(For more resources related to this topic, see here.) Concept of reusability The concept of reusability has its roots in the production process. Typically, most of us go about creating e-learning using a process similar to what is shown in the following screenshot. It works well for large teams and the one man band, except in the latter case, you become a specialist for all the stages of production. That's a heavy load. It's hard to be good at all things and it demands that you constantly stretch and improve your skills, and find ways to increase the efficiency of what you do. Reusability in Storyline is about leveraging the formatting, look and feel and interactions you create so that you can re-purpose your work and speed-up production. Not every project will be an original one-off, in fact most won't, so the concept is to approach development with a plan to repurpose 80 percent of the media, quizzes, interactions, and designs you create. As you do this, you begin to establish processes, templates, and libraries that can be used to rapidly assemble base courses. With a little tweaking and some minor customization, you'll have a new, original course in no time. Your client doesn't need to know that 80 percent was made from reusable elements with just 20 percent created as original, unique components, but you'll know the difference in terms of time and effort. Leveraging existing assets So how can you leverage existing assets with Storyline? The first things you'll want to look at are the courses you've built with other authoring programs, such as PowerPoint, QuizMaker Engage, Captivate, Flash, and Camtasia. If there are design themes, elements, or interactions within these courses that you might want to use for future Storyline courses, you should focus your efforts on importing what you can, and further adjusting within Storyline to create a new version of the asset that can be reused for future Storyline courses. If re-working the asset is too complex or if you don't expect to reuse it in multiple courses, then using Storyline's web object feature to embed the interaction without re-working it in any way may be the better approach. In both cases, you'll save time by reusing content you've already put a lot of time in developing. Importing external content Here are the steps to bring external content into Storyline: From the Articulate Startup screen or by choosing the Insert tab, and then New Slide within a project, select the Import option. There are options to import PowerPoint, Quizmaker, and Storyline. All of these will display the slides within the file to be imported. You can pick and choose which slides to import into a new or the current scene in Storyline. The Engage option displays the entire interaction that can be imported into a single slide in the current or a new scene. Click on Import to complete the process. Considerations when importing Keep the following points in mind when importing: PowerPoint and Quizmaker files can be imported directly into Storyline. Once imported, you can edit the content like you would any other Storyline slide. Master slides come along with the import making it simple to reuse previous designs. Note that 64-bit PowerPoint is not supported and you must have an installed, activated version of Quizmaker for the import to work. The PowerPoint to Storyline conversion is not one-to-one. You can expect some alignment issues with slide objects due to the fact that PowerPoint uses points and Storyline uses pixels. There are 2.66 pixels for each point which is why you'll need to tweak the imported slides just a bit. Same with Quizmaker though the reason why is slightly different; Quizmaker is 686 x 424 in size, whereas Storyline is 720 x 540 by default. Engage files can be imported into Storyline and they are completely functional, but cannot be edited within Storyline. Though the option to import Engage appears on the Import screen, what Storyline is really doing is creating a web object to contain the Engage interaction. Once imported into a new scene, clicking on the Engage interaction will display an Options menu where you can make minor adjustments to the behavior of the interaction as well as Preview and Edit in it Engage. You can also resize and position the interaction just as you would any web object. Remember that though web objects work in iPad and HTML5 outputs, Engage content is Flash, so it will not playback on an iPad or in an HTML5 browser. Like Quizmaker, you'll need an installed, activated version of Engage for the import to work. Flash, Captivate, and Camtasia files cannot be imported in Storyline and cannot be edited within Storyline. You can however, use web objects to embed these projects into Storyline or the Insert Flash option. In both cases, the imported elements appear seamless to the learner while retaining full functionality.   Build once, and reuse many times Quizzing is at the heart of many e-learning courses where often the quiz questions need to be randomized or even reused in different sections of a single course (that is, the same questions for a pre and post-test). The concept of building once and reusing many times works well with several aspects of Storyline. We'll start with quizzing and a feature called Question Banks as follows: Question Banks Question Bank offers a way to pool, reuse, and randomize questions within a project. Slides in a question bank are housed within the project file but are not visible until placed into the story. Question Banks can include groups of quiz slides and regular slides (that is, you might include a regular slide if you need to provide instructions for the quiz or would like to include a post-quiz summary). When you want to include questions from a Question Bank, you just need to insert a new Quizzing slide, and then choose Draw from Bank . You can then select one or more questions to include and randomize them if desired. Follow along… In this exercise we will be removing three questions from a scene and moving them into a question bank. This will allow you to draw one or more of those questions at any point in the project where the quiz questions are needed, as follows: From the Home tab, choose Question Banks , and then Create Question bank . Title this Identity Theft Questions . Notice that a new tab has opened in Normal View . The Question Bank appears in this tab. Click on the Import link and navigate to question slides 2, 3, and 4. From the Import drop-down menu at the top, select move questions into question bank . Click on the Story View tab and notice the three slides containing the quiz questions are no longer in the story. Click back on the Identity Theft tab and notice that they are located here. The questions will not become a part of the story until the next step, when you draw them from the bank. In Story View, click once on slide 1 to select it, and then from the Home tab, choose Question Banks and New Draw from Question Bank . From the Question Bank drop-down menu, select Identity Theft Questions . All questions will be selected by default and will be randomized after being placed into the story. This means that the learner will need to answer three questions before continuing onto the next slide in the story. Click on Insert . The Question Bank draw has been inserted as slide 2. To see how this works, Preview the scene. Save as Exercise 11 – Identity Theft Quiz.   There are multiple ways to get back to the questions that are in a question bank. You can do this by selecting the tab the questions are located in (in this case, Identity Theft ), you can view the question bank slide in Normal View or choose Question Banks from the Home tab and navigate to the name of the question bank you'd like to edit.

(For more resources related to this topic, see here.) Planning the Orchestrator deployment The installation of SCORCH is simple. You must plan the deployment appropriately according to your needs. This recipe discusses and provides steps on common planning tasks to be performed before inserting the DVD or mounting the ISO for organizations who have successfully deployed SCORCH. Getting ready The authors recommend you to review the latest information on SCORCH at http://technet.microsoft.com/en-us/library/hh420383.aspx as the requirements of the product and supported platforms are regularly updated by Microsoft. How to do it... There are three planning categories, people, process, and the technology (SCORCH product). Identify and agree on the roles and responsibilities of the SCORCH team. SCORCH deployments typically have three types of users; service accounts, Administrators, and operators. Services accounts: They perform actions for the specific components of SCORCH Administrators: They will typically perform all activities including, but not limited to, SCORCH installation, Runbook creation and management, and delegation of security to operators Operators: They will typically use the SCORCH console and the Runbook Designer to create and manage Runbooks Identify and document initial prototype processes to be used as the first candidate for automation and testing. The types of processes for this purpose should be simple repeatable tasks that fall into an organizations required standard service requests. Good candidates are service request which do not require authorization and approval. An additional example category is Windows operating system services that can be stopped and started as a part of trouble shooting. Plan for the following technology requirements areas for SCORCH: SCORCH deployment type Deployment type Description Single Server All SCORCH roles installed on one physical or virtual machine This scenario is typically implemented in test environments but is fully supported in production. This however becomes a single point of failure for highly automated environments. Multi-server The SCORCH roles are separated and installed on one or more machines Minimum hardware requirements for each SCORCH component Component Requirements Management Server Operating system: Windows Server 2008 R2 or Windows Server 2012* 1 gigabyte (GB) of RAM, 2 GB or more recommended 200 megabytes (MB) of available hard disk space Dual-core Intel microprocessor, 2.1 gigahertz (GHz) or better Microsoft .NET Framework 3.5 Service Pack 1 Orchestration database Database: Microsoft SQL Server 2008 R2 or SQL Server 2012 Collation: SQL_Latin1_General_CP1_CI_AS Local or Remote (Basic Engine only) Runbook Server Operating system: Windows Server 2008 R2 or Windows Server 2012* 1 gigabyte (GB) of RAM minimum, 2 GB or more recommended 200 megabytes (MB) of available hard disk space Dual-core Intel microprocessor, 2.1 gigabyte (GHz) or better Microsoft .NET Framework 3.5 Service Pack 1 Orchestrator Console/Web Service Operating system: Windows Server 2008 R2 or Windows Server 2012* 1 gigabyte (GB) of RAM minimum, 2 GB or more recommended 200 megabytes (MB) of available hard disk space Dual-core Intel microprocessor, 2.1 gigahertz (GHz) or better Microsoft .NET Framework 3.5 Service Pack 1 Web Service: Internet Information Services (IIS) 7.0 and enabled IIS role Microsoft .NET Framework 3.5 Service Pack 1 Microsoft .NET Framework 4 Microsoft Silverlight 4** Orchestrator Runbook Designer Operating system: Windows Server 2008 R2, Windows 7 (32/64 bit) or Windows Server 2012* 1 gigabyte (GB) of RAM minimum, 2 GB or more recommended 200 megabyte (MB) of available hard disk space Dual-core Intel microprocessor, 2.1 gigabyte (GHz) or better Microsoft .NET Framework 3.5 Service Pack 1 SCORCH 2012 SP1 It is required only for the computer running the console in its web browser but not the Web Service server. Services accounts and delegation groups Account/Group Type Notes Orchestrator management service Service account Create an Active Directory user account for this service. This is the main management server service account and it is granted log on as a service during the installation. Orchestrator Runbook monitor service Service account Typically this is the same account as the Orchestrator Management Service. Orchestrator Runbook service Service account Same user account as the Management and Runbook Server monitor service in a single deployment but can be different for multi-server deployments; Active Directory domain account recommended. Runbook authors (SCO_ADMINS) Group Create an Active Directory group. This group will have the equivalent access of full administration to the SCORCH deployment. Runbook operators (SCO_CON_USERS) Group Create an Active Directory group. This group will have the equivalent access of a Runbook operator to the SCORCH deployment. Installation user User The user with full administrative rights on the SCORCH servers is required to perform the installation and configuration of the SCORCH deployment. Network Communication Ports Source Targeted computer Default port Configurable Runbook Designer Management Server 135, 1024-65535 Yes. Management Server, Runbook Server, and Web Service Orchestration database 1433 Yes; specified during the installation on the SCORCH supported version of Microsoft SQL Server. This is the case where the SQL Server instance is not using the default port. Client browser Orchestrator Web Service 81 Yes; during the SCORCH installation. Client browser Orchestration Console 82 Yes; during the SCORCH installation. How it works... The planning activities discussed are the minimum activities the authors recommend. The tasks performed at this stage will ensure that you ask for and plan for all your requirements before investing time in the actual installation. An additional benefit is identifying any people or budgetary risks before the deployment. There's more... There are two additional planning areas which are typically ignored in technology focused deployments. These areas are communication strategies and stakeholder management. Communication strategy One of the inaccurate myths of SCORCH is that it would automate the IT professional. SCORCH when implemented right would improve efficiency but will not replace people. On the contrary you need to communicate with the people who perform the manual tasks as they hold the key to how to best automate their efforts. Early engagement with all IT team members should be one of your key planning tasks. Stakeholder management Stakeholders are all users affected by the SCORCH deployment. An important category of stakeholders are the management team responsible for policy creation and enforcement. Automation without organization buy in may lead to conflicts at the political level of your organization. An example of such a scenario is the ability to create Active Directory user accounts with rights to specific organization areas and restricted resources.

(For more resources related to this topic, see here.) Step 1 – open the SQL Server Management Studio and connect to the cube The Microsoft SQL Server Management Studio (SSMS) is a client application used by the administrators to manage instances and by developers to create object and write queries. We will use SSMS to connect on the cube and write our first MDX query. Here's a screenshot of SSMS with a connection on a SSAS server: Click on the Windows button, click on All Programs, click on Microsoft SQL Server 2012, and then click on SQL Server Management Studio. In the Connect to Server window, in the Server type box, select Analysis Services. In the Server name box, type the name of your Analysis Services server. Click on Connect. In the SQL Server Management Studio window, click on the File menu, click on New, and then click on Analysis Services MDX Query. In the Connect to Analysis Services window, in the Server name box, type the name of you Analysis Services server, and then click on Connect. SELECT FROM WHERE If you have already written SQL queries, you might have already made connections with the T-SQL language. Here's my tip for you: don't, you will only hurt yourself. Some words are the same, but it is better to think MDX when writing MDX rather than to think SQL when writing MDX. Step 2 – SELECT The SELECT clause is the main part of the MDX query. You will define what are the measure and dimension members that you want to display. You also have to define on which axis of your result set you want to display the measure and dimension members. Axes Axes are the columns and rows of the result set. With SQL Server Analysis Services, upto 128 axes can be specified. The axes have a number which is zero-based. The first axe is 0, the second on is 1, and so on. So, if you want to use two axes, the first one will be 0 and the second will be 1. You cannot use axe 0 and axe 2, if you don't define axe 1. For the first five axes, you can use the axis alias instead. After the axe 4, you will have to revert to the number because no other aliases are available. Axe Number Alias 0 Columns 1 Rows 2 Pages 3 Sections 4 Chapters Even if SSAS supports 128 axes, if you try to use more than two axes in SSMS in your query, you will get this error when you execute your MDX query: Results cannot be displayed for cellsets with more than two axes. So, always write your MDX queries using only two axes in SSMS and separate them with a comma. Tuples A tuple is a specific point in the cube where dimensions meet. A tuple can contain one or more members from the cube's dimensions, but you cannot have two members from the same dimension. If you want to display only the calendar year 2008, you will have to write [Date].[CY 2008]. If you want to have more than one dimension, you have to enclose them using parenthesis () and separate them with a comma. Calendar year for United States will look like ([Date].[CY 2008], [Geography].[United States]). Even if you are writing a tuple with only a single member from a single dimension, it is good practice to enclose it in parenthesis. Sets If you want to display the year 2005 to 2008, you will write four single-dimension tuples which composes a set. When writing the set, you separate the tuples with commas and wrap it all with curly braces {} and separate the tuples with commas such as {[Date].[CY 2005], [Date].[CY 2006] , [Date].[CY 2007] , [Date].[CY 2008]} to have the calendar years from 2005 to 2008. Since all the tuples are from the same dimension, you can also write it using a colon (:), such as {[Date].[CY 2005]: [Date].[CY 2008]} which will give you the years 2005 to 2008. With SSAS 2012, you can write {[Date].[CY 2008]: [Date].[CY 2005]} and the result will still be from 2005 to 2008. What about the calendar year 2008 for both Canada and the United States? You will write two tuples. A set can be composed of one or more tuples. The tuples must have the same dimensionality; otherwise, an error will occur. Meaning that the first member is from the Date dimension and the second from the Geography dimension. You cannot have the first tuple with Date-Geography and the second being Geography-Date; you will encounter an error. So the calendar year 2008 with Canada and United States will look such as {([Date].[CY 2008], [Geography].[Canada]), ([Date].[CY 2008], [Geography].[United States])}. When writing tuples, always use the form [Dimension].[Level].[MemberName]. So, [Geography].[Canada] should be written as [Geography].[Country].[Canada]. You could also use the member key instead of the member name. In SSAS, use the ampersand (&) when using the key; [Geography].[State-Province].[Quebec] with the name becomes [Geography].[State-Province].&[QC]&[CA] using the keys. What happens when you want to write bigger sets such as for the bikes and components product category in Canada and the United States from 2005 to 2008? Enter the Crossjoin function. Crossjoin takes two or more sets for arguments and returns you a set with the cross products or the specified sets. Crossjoin ({[Product].[Category].[Bikes], [Product].[Category].[Components]}, {[Geography].[Country].[Canada], [Geography].[Country].[United States]}, {[Date].[CY 2005] : [Date].[CY 2008]}) The MDX queries can be written using line-break to add visibility to the code. So each time we write a new set and even tuples, we write it on a new line and add some indentation: Crossjoin ({[Product].[Category].[Bikes], [Product].[Category].[Components]},{[Geography].[Country].[Canada], [Geography].[Country].[United States]}, {[Date].[CY 2005] : [Date].[CY 2008]}) Step 3 – FROM The FROM clause defines where the query will get the data. It can be one of the following four things: A cube. A perspective (a subset of dimensions and measures). A subcube (a MDX query inside a MDX query). A dimension (a dimension inside your SSAS database, you must use the dollar sign ($) before the name of the dimension). Step 4 – WHERE The WHERE clause is used to filter the dimensions and members out of the MDX query. The set used in the WHERE clause won't be displayed in your result set. Step 5 – comments Comment your code. You never know when somebody else will take a look on your queries and trying to understand what has been written could be harsh. There are three ways to use delimit comments inside the query: /* and */ // -- (pair of dashes) The /* and */ symbols can be used to comment multiple lines of text in your query. Everything between the /* and the */ symbols will be ignored when the MDX query is parsed. Use // or -- to begin a comment on a single line. Step 6 – your first MDX query So if you want to display the Resellers Sales Amount and Reseller Order Quantity measures on the columns, the years from 2006 to 2008 with the bikes and components product categories for Canada. First, identify what will go where. Start with the two axes, continue with the FROM clause, and finish with the WHERE clause. SELECT{[Measures].[Reseller Sales Amount], [Measures].[Reseller Order Quantity]} on columns,Crossjoin({[Date].[CY 2006] : [Date].[CY 2008]}, {[Product].[Category].[Bikes], [Product].[Category].[Components]}) on rowsFROM [Adventure Works]WHERE {[Geography].[Country].[Canada]} This query will return the following result set:     Reseller Sales Amount Reseller Order Quantity CY 2006 Bikes $3,938,283.99 4,563 CY 2006 Components $746,576.15 2,954 CY 2007 Bikes $4,417,665.71 5,395 CY 2007 Components $997,617.89 4,412 CY 2008 Bikes $1,909,709.62 2,209 CY 2008 Components $370,698.68 1,672 Summary In this article, we saw how to write the MDX queries in various steps. We used the FROM, WHERE, and SELECT clauses in writing the queries. This article was a quick start guide for starting to query and it will help you write more complex queries. Happy querying! Resources for Article : Further resources on this subject: Connecting to Microsoft SQL Server Compact 3.5 with Visual Studio [Article] MySQL Linked Server on SQL Server 2008 [Article] Microsoft SQL Azure Tools [Article]

(For more resources related to this topic, see here.) Node.js is an open source platform that allows you to build fast and scalable network applications using JavaScript. Node.js is built on top of V8, a modern JavaScript virtual machine that powers Google's Chrome web browser. Node.js uses an event-driven, non-blocking I/O model that makes it lightweight and efficient.Node.js can handle multiple concurrent network connections with little overhead, making it ideal for data-intensive, real-time applications. With Node.js, you can build many kinds of networked applications. For instance, you can use it to build a web application service, an HTTP proxy, a DNS server, an SMTP server, an IRC server, and basically any kind of process that is network intensive. You program Node.js using JavaScript, which is the language that powers the Web. JavaScript is a powerful language that, when mastered, makes writing networked, event-driven applications fun and easy. Node.js recognizes streams that are resistant to precarious network conditions and misbehaving clients. For instance, mobile clients are notoriously famous for having large latency network connections, which can put a big burden on servers by keeping around lots of connections and outstanding requests. By using streaming to handle data, you can use Node.js to control incoming and outgoing streams and enable your service to survive. Also, Node.js makes it easy for you to use third-party open source modules. By using Node Package Manager (NPM), you can easily install, manage, and use any of the several modules contained in a big and growing repository. NPM also allows you to manage the modules your application depends on in an isolated way, allowing different applications installed in the same machine to depend on different versions of the same module without originating a conflict, for instance. Given the way it's designed, NPM even allows different versions of the same module to coexist in the same application. Summary In this article, we learned that Node.js uses an event-driven, non-blocking I/O mode and can handle multiple concurrent network connections with little overhead. Resources for Article : Further resources on this subject: So, what is KineticJS? [Article] Cross-browser-distributed testing [Article] Accessing and using the RDF data in Stanbol [Article]

(For more resources related to this topic, see here.) Understanding FMOD One of the main reasons why I chose FMOD for this book is that it contains two separate APIs—the FMOD Ex Programmer's API, for low-level audio playback, and FMOD Designer, for high-level data-driven audio. This will allow us to cover game audio programming at different levels of abstraction without having to use entirely different technologies. Besides that reason, FMOD is also an excellent piece of software, with several advantages to game developers: License: It is free for non-commercial use, and has reasonable licenses for commercial projects. Cross-platform: It works across an impressive number of platforms. You can run it on Windows, Mac, Linux, Android, iOS, and on most of the modern video game consoles by Sony, Microsoft, and Nintendo. Supported formats: It has native support for a huge range of audio file formats, which saves you the trouble of having to include other external libraries and decoders. Programming languages: Not only can you use FMOD with C and C++, there are also bindings available for other programming languages, such as C# and Python. Popularity: It is extremely popular, being widely considered as the industry standard nowadays. It was used in games such as BioShock, Crysis, Diablo 3, Guitar Hero, Start Craft II, and World of Warcraft. It is also used to power several popular game engines, such as Unity3D and CryEngine. Features: It is packed with features, covering everything from simple audio playback, streaming and 3D sound, to interactive music, DSP effects and low-level audio programming. Installing FMOD Ex Programmer's API Installing a C++ library can be a bit daunting at first. The good side is that once you have done it for the first time, the process is usually the same for every other library. Here are the steps that you should follow if you are using Microsoft Visual Studio: Download the FMOD Ex Programmer's API from http://www.fmod.org and install it to a folder that you can remember, such as C:FMOD. Create a new empty project, and add at least one .cpp file to it. Then, right-click on the project node on the Solution Explorer , and select Properties from the list. For all the steps that follow, make sure that the Configuration option is set to All Configurations . Navigate to C/C++ | General , and add C:FMODapiinc to the list of Additional Include Directories (entries are separated by semicolons). Navigate to Linker | General , and add C:FMODapilib to the list of Additional Library Directories . Navigate to Linker | Input , and add fmodex_vc.lib to the list of Additional Dependencies . Navigate to Build Events | Post-Build Event , and add xcopy /y "C:FMODapifmodex.dll" "$(OutDir)" to the Command Lin e list. Include the <fmod.hpp> header file from your code. Creating and managing the audio system Everything that happens inside FMOD is managed by a class named FMOD::System, which we must start by instantiating with the FMOD::Syste m_Create() function: FMOD::System* system; FMOD::System_Create(&system); Notice that the function returns the system object through a parameter. You will see this pattern every time one of the FMOD functions needs to return a value, because they all reserve the regular return value for an error code. We will discuss error checking in a bit, but for now let us get the audio engine up and running. Now that we have a system object instantiated, we also need to initialize it by calling the init() method: system->init(100, FMOD_INIT_NORMAL, 0); The first parameter specifies the maximum number of channels to allocate. This controls how many sounds you are able to play simultaneously. You can choose any number for this parameter because the system performs some clever priority management behind the scenes and distributes the channels using the available resources. The second and third parameters customize the initialization process, and you can usually leave them as shown in the example. Many features that we will use work properly only if we update the system object every frame. This is done by calling the update() method from inside your game loop: system->update(); You should also remember to shutdown the system object before your game ends, so that it can dispose of all resources. This is done by calling the release() method: system->release(); Loading and streaming audio files One of the greatest things about FMOD is that you can load virtually any audio file format with a single method call. To load an audio file into memory, use the createSound() method: FMOD::Sound* sound; system->createSound("sfx.wav", FMOD_DEFAULT, 0, &sound); To stream an audio file from disk without having to store it in memory, use the createStream() method: FMOD::Sound* stream; system->createStream("song.ogg", FMOD_DEFAULT, 0, &stream); Both methods take the path of the audio file as the first parameter, and return a pointer to an FMOD::Sound object through the fourth parameter, which you can use to play the sound. The paths in the previous examples are relative to the application path. If you are running these examples in Visual Studio, make sure that you copy the audio files into the output folder (for example, using a post-build event such as xcopy /y "$(ProjectDir)*.ogg" "$(OutDir)"). The choice between loading and streaming is mostly a tradeoff between memory and processing power. When you load an audio file, all of its data is uncompressed and stored in memory, which can take up a lot of space, but the computer can play it without much effort. Streaming, on the other hand, barely uses any memory, but the computer has to access the disk constantly, and decode the audio data on the fly. Another difference (in FMOD at least) is that when you stream a sound, you can only have one instance of it playing at any time. This limitation exists because there is only one decode buffer per stream. Therefore, for sound effects that have to be played multiple times simultaneously, you have to either load them into memory, or open multiple concurrent streams. As a rule of thumb, streaming is great for music tracks, voice cues, and ambient tracks, while most sound effects should be loaded into memory. The second and third parameters allow us to customize the behavior of the sound. There are many different options available, but the following list summarizes the ones we will be using the most. Using FMOD_DEFAULT is equivalent to combining the first option of each of these categories: FMOD_LOOP_OFF and FMOD_LOOP_NORMAL: These modes control whether the sound should only play once, or loop once it reaches the end FMOD_HARDWARE and FMOD_SOFTWARE: These modes control whether the sound should be mixed in hardware (better performance) or software (more features) FMOD_2D and FMOD_3D: These modes control whether to use 3D sound We can combine multiple modes using the bitwise OR operator (for instance, FMOD_DEFAULT | FMOD_LOOP_NORMAL | FMOD_SOFTWARE). We can also tell the system to stream a sound even when we are using the createSound() method, by setting the FMOD_CREATESTREAM flag. In fact, the createStream() method is simply a shortcut for this. When we do not need a sound anymore (or at the end of the game) we should dispose of it by calling the release() method of the sound object. We should always release the sounds we create, regardless of the audio system also being released. sound->release(); Playing sounds With the sounds loaded into memory or prepared for streaming, all that is left is telling the system to play them using the playSound() method: FMOD::Channel* channel; system->playSound(FMOD_CHANNEL_FREE, sound, false, &channel); The first parameter selects in which channel the sound will play. You should usually let FMOD handle it automatically, by passing FMOD_CHANNEL_FREE as the parameter. The second parameter is a pointer to the FMOD::Sound object that you want to play. The third parameter controls whether the sound should start in a paused state, giving you a chance to modify some of its properties without the changes being audible. If you set this to true, you will also need to use the next parameter so that you can unpause it later. The fourth parameter is an output parameter that returns a pointer to the FMOD::Channel object in which the sound will play. You can use this handle to control the sound in multiple ways, which will be the main topic of the next chapter. You can ignore this last parameter if you do not need any control over the sound, and simply pass in 0 in its place. This can be useful for non-lopping one-shot sounds. system->playSound(FMOD_CHANNEL_FREE, sound, false, 0); Checking for errors So far, we have assumed that every operation will always work without errors. However, in a real scenario, there is room for a lot to go wrong. For example, we could try to load an audio file that does not exist. In order to report errors, every function and method in FMOD has a return value of type FMOD_RESULT, which will only be equal to FMOD_OK if everything went right. It is up to the user to check this value and react accordingly: FMOD_RESULT result = system->init(100, FMOD_INIT_NORMAL, 0); if (result != FMOD_OK) { // There was an error, do something about it } For starters, it would be useful to know what the error was. However, since FMOD_RESULT is an enumeration, you will only see a number if you try to print it. Fortunately, there is a function called FMOD_ErrorString() inside the fmod_errors.h header file which will give you a complete description of the error. You might also want to create a helper function to simplify the error checking process. For instance, the following function will check for errors, print a description of the error to the standard output, and exit the application: #include <iostream> #include <fmod_errors.h> void ExitOnError(FMOD_RESULT result) { if (result != FMOD_OK) { std::cout << FMOD_ErrorString(result) << std::endl; exit(-1); } } You could then use that function to check for any critical errors that should cause the application to abort: ExitOnError(system->init(100, FMOD_INIT_NORMAL, 0)); The initialization process described earlier also assumes that everything will go as planned, but a real game should be prepared to deal with any errors. Fortunately, there is a template provided in the FMOD documentation which shows you how to write a robust initialization sequence. It is a bit long to cover here, so I urge you to refer to the file named Getting started with FMOD for Windows.pdf inside the documentation folder for more information. For clarity, all of the code examples will continue to be presented without error checking, but you should always check for errors in a real project. Project 1 building a simple audio manager In this project, we will be creating a SimpleAudioManager class that combines everything that was covered in this chapter. Creating a wrapper for an underlying system that only exposes the operations that we need is known as the façade design pattern , and is very useful in order to keep things nice and simple. Since we have not seen how to manipulate sound yet, do not expect this class to be powerful enough to be used in a complex game. Its main purpose will be to let you load and play one-shot sound effects with very little code (which could in fact be enough for very simple games). It will also free you from the responsibility of dealing with sound objects directly (and having to release them) by allowing you to refer to any loaded sound by its filename. The following is an example of how to use the class: SimpleAudioManager audio; audio.Load("explosion.wav"); audio.Play("explosion.wav"); From an educational point of view, what is perhaps even more important is that you use this exercise as a way to get some ideas on how to adapt the technology to your needs. It will also form the basis of the next chapters in the book, where we will build systems that are more complex. Class definition Let us start by examining the class definition: #include <string> #include <map> #include <fmod.hpp> typedef std::map<std::string, FMOD::Sound*> SoundMap; class SimpleAudioManager { public: SimpleAudioManager(); ~SimpleAudioManager(); void Update(float elapsed); void Load(const std::string& path); void Stream(const std::string& path); void Play(const std::string& path); private: void LoadOrStream(const std::string& path, bool stream); FMOD::System* system; SoundMap sounds; }; From browsing through the list of public class members, it should be easy to deduce what it is capable of doing: The class can load audio files (given a path) using the Load() method The class can stream audio files (given a path) using the Stream() method The class can play audio files (given a path) using the Play() method (granted that they have been previously loaded or streamed) There is also an Update() method and a constructor/destructor pair to manage the sound system The private class members, on the other hand, can tell us a lot about the inner workings of the class: At the core of the class is an instance of FMOD::System responsible for driving the entire sound engine. The class initializes the sound system on the constructor, and releases it on the destructor. Sounds are stored inside an associative container, which allows us to search for a sound given its file path. For this purpose, we will be relying on one of the C++ Standard Template Library (STL ) associative containers, the std::map class, as well as the std::string class for storing the keys. Looking up a string key is a bit inefficient (compared to an integer, for example), but it should be fast enough for our needs. An advantage of having all the sounds stored on a single container is that we can easily iterate over them and release them from the class destructor. Since the code for loading and streaming audio file is almost the same, the common functionality has been extracted into a private method called LoadOrStream(), to which Load() and Stream() delegate all of the work. This prevents us from repeating the code needlessly. Initialization and destruction Now, let us walk through the implementation of each of these methods. First we have the class constructor, which is extremely simple, as the only thing that it needs to do is initialize the system object. SimpleAudioManager::SimpleAudioManager() { FMOD::System_Create(&system); system->init(100, FMOD_INIT_NORMAL, 0); } Updating is even simpler, consisting of a single method call: void SimpleAudioManager::Update(float elapsed) { system->update(); } The destructor, on the other hand, needs to take care of releasing the system object, as well as all the sound objects that were created. This process is not that complicated though. First, we iterate over the map of sounds, releasing each one in turn, and clearing the map at the end. The syntax might seem a bit strange if you have never used an STL iterator before, but all that it means is to start at the beginning of the container, and keep advancing until we reach its end. Then we finish off by releasing the system object as usual. SimpleAudioManager::~SimpleAudioManager() { // Release every sound object and clear the map SoundMap::iterator iter; for (iter = sounds.begin(); iter != sounds.end(); ++iter) iter->second->release(); sounds.clear(); // Release the system object system->release(); system = 0; } Loading or streaming sounds Next in line are the Load() and Stream() methods, but let us examine the private LoadOrStream() method first. This method takes the path of the audio file as a parameter, and checks if it has already been loaded (by querying the sound map). If the sound has already been loaded there is no need to do it again, so the method returns. Otherwise, the file is loaded (or streamed, depending on the value of the second parameter) and stored in the sound map under the appropriate key. void SimpleAudioManager::LoadOrStream(const std::string& path, bool stream) { // Ignore call if sound is already loaded if (sounds.find(path) != sounds.end()) return; // Load (or stream) file into a sound object FMOD::Sound* sound; if (stream) system->createStream(path.c_str(), FMOD_DEFAULT, 0, &sound); else system->createSound(path.c_str(), FMOD_DEFAULT, 0, &sound); // Store the sound object in the map using the path as key sounds.insert(std::make_pair(path, sound)); } With the previous method in place, both the Load() and the Stream() methods can be trivially implemented as follows: void SimpleAudioManager::Load(const std::string& path) { LoadOrStream(path, false); } void SimpleAudioManager::Stream(const std::string& path) { LoadOrStream(path, true); } Playing sounds Finally, there is the Play() method, which works the other way around. It starts by checking if the sound has already been loaded, and does nothing if the sound is not found on the map. Otherwise, the sound is played using the default parameters. void SimpleAudioManager::Play(const std::string& path) { // Search for a matching sound in the map SoundMap::iterator sound = sounds.find(path); // Ignore call if no sound was found if (sound == sounds.end()) return; // Otherwise play the sound system->playSound(FMOD_CHANNEL_FREE, sound->second, false, 0); } We could have tried to automatically load the sound in the case when it was not found. In general, this is not a good idea, because loading a sound is a costly operation, and we do not want that happening during a critical gameplay section where it could slow the game down. Instead, we should stick to having separate load and play operations. A note about the code samples Although this is a book about audio, all the samples need an environment to run on. In order to keep the audio portion of the samples as clear as possible, we will also be using the Simple and Fast Multimedia Library 2.0 (SFML ) (http://www.sfml-dev.org). This library can very easily take care of all the miscellaneous tasks, such as window creation, timing, graphics, and user input, which you will find in any game. For example, here is a complete sample using SFML and the SimpleAudioManager class. It creates a new window, loads a sound, runs a game loop at 60 frames per second, and plays the sound whenever the user presses the space key. #include <SFML/Window.hpp> #include "SimpleAudioManager.h" int main() { sf::Window window(sf::VideoMode(320, 240), "AudioPlayback"); sf::Clock clock; // Place your initialization logic here SimpleAudioManager audio; audio.Load("explosion.wav"); // Start the game loop while (window.isOpen()) { // Only run approx 60 times per second float elapsed = clock.getElapsedTime().asSeconds(); if (elapsed < 1.0f / 60.0f) continue; clock.restart(); sf::Event event; while (window.pollEvent(event)) { // Handle window events if (event.type == sf::Event::Closed) window.close(); // Handle user input if (event.type == sf::Event::KeyPressed && event.key.code == sf::Keyboard::Space) audio.Play("explosion.wav"); } // Place your update and draw logic here audio.Update(elapsed); } // Place your shutdown logic here return 0; } Summary In this article, we have seen some of the advantages of using the FMOD audio engine. We saw how to install the FMOD Ex Programmer's API in Visual Studio, how to initialize, manage, and release the FMOD sound system, how to load or stream an audio file of any type from disk, how to play a sound that has been previously loaded by FMOD, how to check for errors in every FMOD function, and how to create a simple audio manager that encapsulates the act of loading and playing audio files behind a simple interface. Resources for Article : Further resources on this subject: Using SpriteFonts in a Board-based Game with XNA 4.0 [Article] HTML5 Games Development: Using Local Storage to Store Game Data [Article] Making Money with Your Game [Article]

(For more resources related to this topic, see here.) Step 1 – Planning the workflow When you'll be writing a real application, you should start with the requirements to application functionality. For the blog example, they described in the Getting Started: Requirements Analysis section, at the very beginning of the tutorial. Direct URL is http://www.yiiframework.com/doc/blog/1.1/en/start.requirements. After you have written all the desired features, you basically start implementing them one by one. Of course, in serious software development there's a lot of gotchas included but overall it's the same. Blog example is a database driven application, so we need to prepare a database schema beforehand. Here's what they came up with for the blog demo. This image is a verbatim copy from the blog example demo. Note that there are two links missing. The posts table have tags field which is the storage area for tags written in raw and is not a foreign key to tags table. Also author field in comment should really be the foreign key to user table. Anyways, we'll not cover the actual database generation, and suggest you can do it yourself. The blog tutorial at the Yii website has all the relevant instructions addressed to total newbies. Next in this article we will see how easy it is with Yii to get a working user interface by which one will be able to manipulate our database. Step 2 – Linking to the database from your app Once you design and physically create, the database in some database management system like MySQL or maybe SQLite, you are ready to configure your app to point to this database. The skeleton app generated by the ./yiic webapp command needs to be configured to point to this database. To do this, you need to set a db component in the main config file located at protected/config/main.php. There is a section that contains an array of components. Below is the setup for a MySQL database located at the same server as the web application itself. You will find a commented-out template for this already present when you generate your app. /protected/config/main.php'components'=>array( /* other components */ 'db'=>array( 'connectionString' => 'mysql:host=localhost;dbname=DB_NAME, 'emulatePrepare' => true, 'username' => YOUR_USERNAME, 'password' => YOUR_PASSWORD, 'charset' => 'utf8', ), /* other components */), This is a default component having a class CDbConnection and is used by all of our ActiveRecord design patterns which we will create later. As with all application components, all configuration parameters corresponds to the public properties of the component's class, so, you can check the API documentation for details. By the way, you really want to understand more about the main application config. Read about it in the Definitive Guide to Yii at the official website, at Fundamentals | Application | Application Configuration. Direct URL is http://www.yiiframework.com/doc/guide/1.1/en/basics.application#application-configuration. Just remember that all configuration parameters are just properties of CWebApplication object, which you can read about it the API documentation, direct URL is http://www.yiiframework.com/doc/api/1.1/CWebApplication. Step 3 – Generating code automatically Now that we have our app linked up to a fully built database, we can start using one of Yii's greatest features: automatic code generation. To get started, there are two types of code generation that are necessary: Generate a model classes based on the tables in your database Run the CRUD generator that takes a model and sets up a corresponding controller and set of views for basic listing, creating, viewing, updating and deleting from the table Console way There are two ways to go about automatic code generating. Originally, there was only the yiic tool used earlier to create the skeleton app. For the automatic code generation features, you would use yiic shell index.php command, which would bring up a command-line interface where you could run subcommands for modeling and scaffolding. $ /usr/local/yii/framework/yiic shell index.phpYii Interactive Tool v1.1 (based on Yiiv1.1.13)Please type 'help' for help. Type 'exit' to quit.>> model Post tbl_post generate models/Post.php unchanged fixtures/tbl_post.php generate unit/PostTest.phpThe following model classes are successfully generated: PostIf you have a 'db' database connection, you can test these models nowwith: $model=Post::model()->find(); print_r($model);>> crud Post generate PostController.php generate PostTest.phpmkdir /var/www/app/protected/views/post generate create.php generate update.php generate index.php generate view.php As you can see, this is a quick and easy way to perform the model and crud actions. The model command produces just two files: For your actual model class For unit tests The crud command creates your controller and view files. Gii Console tools may be the preferred option for some, but for developers who like to use graphical tools, there is now solution for this, called Gii. To use Gii, it is necessary to turn it on in the main config file: protected/config/main.php. You will find the template for it already present, but it is commented out by default. Simply uncomment it, set your password, and decide from what hosts it may be accessed. The configuration looks like this: 'gii'=>array( 'class'=>'system.gii.GiiModule', 'password'=>'giiPassword', // If removed, Gii defaults to localhost only. // Edit carefully to taste. 'ipFilters'=>array('127.0.0.1','::1'), // For development purposes, // a wildcard will allow access from anywhere. // 'ipFilters'=>array('*'),), Once Gii is configured, it can be accessed by navigating to the app URL with ?r=gii after it. For example, http://www.example.com/index.php?r=gii. It will begin with a prompt asking for the password set in the config file. Once entered, it will display a list of generators. If the database is not set in the config file, you will see an error when you attempt to use one. The first most basic generator in Gii is the model generator. It asks for a table name from the database and a name to be used for the PHP class. Note that we can specify a table name prefix which will be ignored when generating the model class name. For instance, the blog demo's user table is tbl_user, where the tbl_ is a prefix. This feature exists to support some setups, especially common in shared hosting environments, where a single database holds tables for several distinct applications. In such an environment, it's a common practice to prefix something to names of tables to avoid getting into naming conflict and easily find tables relevant to some specific application. So, as this prefixes don't mean anything in the application itself, Gii offers a way to automatically ignore them. Model class names are being constructed from the remaining table names by the obvious rules: Underscores are converted to uppercasing the next letter The first letter of the class name is being uppercased as well. The first step in getting your application off the ground is to generate models for all the entity tables in your database. Things like bridge tables will not need models, as they simply relate two entities to one another, rather than actually being a distinct thing. Bridge tables are being used for generating relations between models, expressed in the relations method in model class. For the blog demo, basic models are User, Post, Comment, Tag, and Lookup. The second phase of scaffolding is to generate the CRUD code for each of these models. This will create a controller and a series of view templates. The controller (for example. PostController) will handle routing to actions related to the given model. The view files represent everything needed to list and view entities, as well as the forms needed to create and update individual entities. Summary In this article we created an application by following a series of steps such as planning the workflow, linking to the database from your app, and generating code automatically. Resources for Article : Further resources on this subject: Database, Active Record, and Model Tricks [Article] Building multipage forms (Intermediate) [Article] Creating a Recent Comments Widget in Agile [Article]

(For more resources related to this topic, see here.) Introduction to Ember.js Ember.js is a frontend MVC JavaScript framework that runs in the browser. It is for developers who are looking to build ambitious and large web applications that rival native applications. Ember.js was created from concepts introduced by native application frameworks, such as Cocoa. Ember.js helps you to create great experiences for the user. It will help you to organize all the direct interactions a user may perform on your website. A common use case for Ember.js is when you believe your JavaScript code will become complex; when the code base becomes complex, problems about maintaining and refactoring the code base will arise. MVC stands for model-view-controller. This kind of structure makes it easy to make modifications or refactor changes to any part of your code. It will also allow you to adhere to Don't Repeat Yourself (DRY) principles. The model is responsible for notifying associated views and controllers when there has been a change in the state of the application. The controller sends CRUD requests to the model to notify it of a change in state. It can also send requests to the view to change how the view is representing the current state of the model. The view will then receive information from the model to create a graphical rendering. If you are still unclear on how the three parts interact with each other, the following is a simple diagram illustrating this: Ember.js decouples the problematic areas of your frontend, enabling you to focus on one area at a time without worrying about affecting other parts of your application. To give you an example of some of these areas of Ember.js, take a look at the following list: Navigation : Ember's router takes care of your application's navigation Auto-updating templates : Ember view expressions are binding-aware, meaning they will update automatically if the underlying data ever changes Data handling : Each object you create will be an Ember object, thus inheriting all Ember.object methods Asynchronous behavior : Bindings and computed properties within Ember help manage asynchronous behavior Ember.js is more of a framework than a library. Ember.js expects you to build a good portion of your frontend around its methodologies and architecture, creating a solid application architecture once you are finished with it. This is the main difference between Ember and a framework like Angular.js. Angular allows itself to be incorporated into an existing application, whereas an Ember application would have had to have been planned out with its specific architecture in mind. Backbone.js would be another example of a library that can easily be inserted into existing JavaScript projects. Ember.js is a great framework for handling complex interactions performed by users in your application. You may have been led to believe that Ember.js is a difficult framework to learn, but this is false. The only difficulty for developers lies in understanding the concepts that Ember.js tries to implement. How to set up Ember.js The js folder contains a subfolder named libs and the app.js file. libs is for storing any external libraries that you will want to include into your application. app.js is the JavaScript file that contains your Ember application structure. index.html is a basic HTML index file that will display information in the user's browser. We will be using this file as the index page of the sample application that we will be creating. We create a namespace called MovieTracker where we can access any necessary Ember.js components. Initialize() will instantiate all the controllers currently available with the namespace. After that is done, it injects all the controllers onto a router. We then set ApplicationController as the rendering context of our views. Your application must have ApplicationController, otherwise your application will not be capable of rendering dynamic templates. Router in Ember is a subclass of the Ember StateManager. The Ember StateManager tracks the current active state and triggers callbacks when states have changed. This router will help you match the URL to an application state and detects the browser URL at application load time. The router is responsible for updating the URL as the application's state changes. When Ember parses the URL to determine the state, it attempts to find Ember.Route that matches this state. Our router must contain root and index. You can think of root as a general container for routes. It is a set of routes. An Ember view is responsible for structuring the page through the view's associated template. The view is also responsible for registering and responding to user events. ApplicationView we are creating is required for any Ember application. The view we created is associated with our ApplicationController as well. The templateName variable is the name we use in our index.html file. The templateName variable can be changed to anything you wish. Creating an Ember Object An object or a model is a way to manage data in a structured way. In other words, they are a way of representing persistent states in your application. In Ember.js, almost every object is derived from the Ember.Object class. Since most objects will be derived from the same base object, they will end up sharing properties with each other. This allows the observation and binding to properties of other objects.

(For more resources related to this topic, see here.) Step 1 – preparing for the deployment OpenNMS is a large and complex piece of software and its deployment can be intimidating. Making sure we are prepared will save us a lot of trouble along the road. Here are some of the things needed before getting started: You should have a good internet connection. There are a lot of software packages to download, install and configure. You should have a clean (newly installed) and up-to-date system. OpenNMS can be deployed on any platform where the JVM runs such as Linux, Unix, Solaris, Mac, and Windows. You should read the official documentation or at least skim through the quick start guide at http://www.opennms.org/wiki/QuickStart, the tutorial at http://www.opennms.org/wiki/Tutorial, and the install guide at http://www.opennms.org/documentation/installguide.html. Step 2 – setting up OpenNMS software repositories OpenNMS conveniently makes available its software to several operating systems through their native software repository applications (for example, APT, YUM, and fink). The procedure for this will depend on your platform; instructions can be found at http://www.opennms.org/documentation/installguide.html#before-you-begin. Users of Windows can safely ignore this step as there is a standalone package providing a more Windows-like installation procedure. Step 3 – installing the Java Development Kit (JDK) Normally, Java software only needs the Java Runtime Environment (JRE) to be installed to run. But this will not be sufficient, you must install the JDK, either the OpenJDK implementation or Oracle's. The installation is straight forward and detailed at http://www.opennms.org/documentation/installguide.html#java. The default options are fine to get started. If installing on Windows, both the 3 2-bit and 64-bit JDK are available on Oracle's website. Bear in mind that the JDK must match the OpenNMS standalone setup executable that you will run. Step 4 – installing and configure PostgreSQL PostgreSQL is usually available in Linux with excellent support from the operating systems' repositories. Otherwise, there are available binaries for other common operating systems. Here, instructions will diverge a little from OpenNMS's available documentation at http://www.opennms.org/documentation/installguide.html#postgresql, which are meant to get you started with as little fuss as possible at the expense of security. Access controls will be configured using encrypted passwords (not using trust authentication as in the online tutorials). This can be achieved by editing the pg_hba.conf file; its location will vary depending on your platform. On CentOS 6 it is located in /var/lib/pgsql/data/ and on Debian 6 in /etc/postgresql/8.4/main/ (refer to your OS documentation). Locate the following configuration lines and edit the authentication method to use md5 encrypted passwords as shown in the following code. You will need to reload or restart the service for changes to take effect: # "local" is for Unix domain socket connections onlylocal all postgres md5local all all md5# IPv4 local connections:host all all 127.0.0.1/32 md5# IPv6 local connections:host all all ::1/128 md5 The PostgreSQL defaults in postgresql.conf should be fine, but to take advantage of OpenNMS' ability to scale you will need to tune PostgreSQL further. A good starting point is going through http://www.opennms.org/documentation/installguide.html#postgresql-configure and http://www.opennms.org/documentation/installguide.html#performance-tuning. To complete the database setup we will execute some SQL commands from the command line to do a number of initial tasks. The following code shows how to set a strong password for the PostgreSQL user postgres , create a database user called opennms with restricted privileges and a password of your choice, and create a database called opennms owned by our new opennms user (creating the database manually is not necessarily and would also be taken care of by the installer): # su - postgres$ psql -c "ALTER USER postgres WITH PASSWORD 'newpassword'" -d template1$ psql -c "CREATE USER opennms WITH LOGIN ENCRYPTED PASSWORD 'opennmspassword';"$ psql -c "CREATE DATABASE opennms WITH OWNER=opennms ENCODING 'UNICODE';" With a new database at our disposal we can now start importing functions and data into it. The next step is to install the iplike package that contains an optimized function to do lookups based on IP addresses. It is usually available in your OS software repository. If you are on Windows, you do not have to do this explicitly. It is taken care automatically by the standalone OpenNMS package. Once installed, you may have to run a script as shown in the following code to install the function in the opennms database: # install_iplike.sh If you are not using opennms (the default) as the database name you will have to edit the iplike script to change the database name manually. If you are using PostgreSQL 9.0 or later the procedural language is already installed by default, otherwise it needs to be installed in the opennms database with the following command: # createlang -U postgres plpgsql opennms If you are on Windows, you will be installing the OpenNMS standalone package. You do not need to worry about installing iplike or the plpgsql procedural language as both of them will be installed automatically. Step 5 – installing OpenNMS Before we go ahead with the OpenNMS installation, it is a good time to install the remaining optional dependencies jicmp and jrrd. On Windows those dependencies come with the standalone package and you do not need to do anything. On other OS it should be pulled in as a dependency when installing using APT, YUM, or fink. We are now ready to install OpenNMS as detailed at http://www.opennms.org/documentation/installguide.html#installing; instructions are included for various platforms. Once installation of OpenNMS software is complete, either through the software repositories or using the standalone package for Windows, you must take care to properly configure it for database access. Locate the file $OPENNMS_HOME/etc/opennms-datasources.xml and edit the data sources as shown in the following code: <jdbc-data-source name="opennms" database-name="opennms" class-name="org.postgresql.Driver" url="jdbc:postgresql://localhost:5432/opennms" user-name="opennms" password="opennmspassword" /><jdbc-data-source name="opennms-admin" database-name="template1" class-name="org.postgresql.Driver" url="jdbc:postgresql://localhost:5432/template1" user-name="postgres" password="newpassword" /> If everything was done correctly up to this point the OpenNMS installation can now be completed. First, optionally tell OpenNMS to find a suitable JRE (from your JDK) and then finalize the OpenNMS deployment using the install tool that comes with it. Essentially, the following two commands should be executed: # $OPENNMS_HOME/bin/runjava -s# $OPENNMS_HOME/bin/install -dis If the installation was successful you should be able to start the OpenNMS service. The command for doing this will depend on your platform and method of installation: On debian it is as simple as the following command: # service opennms start Red Hat systems like Fedora moved to the systemctl service manager, it would be something more like the following command: # systemctl start opennms.service On Windows you can do the following: # cd C:Program FilesOpenNMSbin# opennms.bat start And that's it Once started we can log in to OpenNMS with the credentials admin/admin using a browser pointed at http : //localhost:8980/opennms. If this is your first time installing a large Java system, there is a good chance it will not work the first time around. Don't give up, the scary errors are simply likely the result of one or two minor mistakes along the way. Repeat and verify each step all over again. Summary Installation helps you learn how to download and install OpenNMS with the minimum fuss and then set it up so that you can use it as soon as possible. Resources for Article: Further resources on this subject: Geronimo Architecture: Part 2 [Article] Moodle 2.0 Multimedia: Working with 2D and 3D Maps [Article] Network Monitoring Essentials [Article]

(For more resources related to this topic, see here.) The objective of the first chapter is to quickly introduce you to the technology and then dive deep into the understanding the fabric of the tool. The chapter also helps you set the environment, which will be used throughout the book. Chapter 1, Know Your Horse Before You Ride It, starts with discussing the various features of APEX. This is to give heads up to the readers about the features offered by the tool and to inform them about some of the strengths of the tool. In order to understand the technology better, we discuss the various web server combinations possible with APEX, namely the Internal mod_plsql, External mod_plsql, and Listener configuration. While talking about the Internal mod_plsql configuration, we see the steps to enable the XMLDB HTTP server. In the Internal mod_plsql configuration, Oracle uses a DAD defined in EPG to talk to the database and the web server. So, we try to create a miniature APEX of our own, by creating our DAD using it to talk to the database and the web server. We then move on to learn about the External mod_plsql configuration. We discuss the architecture and the roles of the configuration files, such as dads.conf and httpd.conf. We also have a look at a typical dads.conf file and draw correlations between the configurations in the Internal and External mod_plsql configuration. We then move on to talk about the wwv_flow_epg_include_mod_local procedure that can help us use the DAD of APEX to call our own stored PL/SQL procedures. We then move on to talk about APEX Listener which is a JEE alternative to mod_plsql and is Oracle’s direction for the future. Once we are through with understanding the possible configurations, we see the steps to set up our environment. We use the APEX Listener configuration and see the steps to install the APEX engine, create a Weblogic domain, set the listener in the domain, and create an APEX workspace. With our environment in place, we straight away get into understanding the anatomy of APEX by analyzing various parts of its URL. This discussion includes a natter on the session management, request handling, debugging, error handling, use of TKPROF for tracing an APEX page execution, cache management and navigation, and value passing in APEX. We also try to understand the design behind the zero session ID in this section. Our discussions till now would have given you a brief idea about the technology, so we try to dig in a little deeper and understand the mechanism used by APEX to send web requests to its PL/SQL engine in the database by decoding the APEX page submission. We see the use of the wwv_flow.accept procedure and understand the role of page submission. We try to draw an analogy of an APEX form with a simple HTML to get a thorough understanding about the concept. The next logical thing after page submission is to see the SQL and PL/SQL queries and blocks reaching the database. We turn off the database auditing and see the OWA web toolkit requests flowing to the database as soon as we open an APEX page. We then broaden our vision by quickly knowing about some of the lesser known alternatives of mod_plsql. We end the chapter with a note of caution and try to understand the most valid criticisms of the technology, by understanding the SQL injection and Cross-site Scripting (XSS). After going through the architecture we straight away spring into action and begin the process of learning how to build the reports in APEX. The objective of this chapter is to help you understand and implement the most common reporting requirements along with introducing some interesting ways to frame the analytical queries in Oracle. The chapter also hugely focuses on the methods to implement different kinds of formatting in the APEX classic reports. We start Chapter 2, Reports, by creating the objects that will be used throughout the book and installing the reference application that contains the supporting code for the topics discussed in the second chapter. We start the chapter by setting up an authentication mechanism. We discuss external table authentication in this chapter. We then move on to see the mechanism of capturing the environment variables in APEX. These variables can help us set some logic related to a user’s session and environment. The variables also help us capture some properties of the underlying database session of an APEX session. We capture the variables using the USERENV namespace, DBMS_SESSION package, and owa_util package. After having a good idea of the ways and means to capture the environment variables, we build our understanding of developing the search functionality in a classic APEX report. This is mostly a talk about the APEX classic report features, we also use this opportunity to see the process to enable sorting in the report columns, and to create a link that helps us download the report in the CSV format. We then discuss various ways to implement the group reports in APEX. The discussion shows a way to implement this purely, by using the APEX’s feature, and then talks about getting the similar results by using the Oracle database feature. We talk about the APEX’s internal grouping feature and the Oracle grouping sets. The section also shows the first use of JavaScript to manipulate the report output. It also talks about a method to use the SQL query to create the necessary HTML, which can be used to display a data column in a classic report. We take the formatting discussion further, by talking about a few advanced ways of highlighting the report data using the APEX’s classic report features, and by editing the APEX templates. After trying our hand at formatting, we try to understand the mechanism to implement matrix reports in APEX. We use matrix reports to understand the use of the database features, such as the with clause, the pivot operator, and a number of string aggregation techniques in Oracle. The discussion on the string aggregation techniques includes the talk on the LISTAGG function, the wm_concat function, and the use of the hierarchical queries for this purpose. We also see the first use of the APEX items as the substitution variables in this book. We will see more use of the APEX items as the substitution variables to solve the vexed problems in other parts of the book as well. We do justice with the frontend as well, by talking about the use of jQuery, CSS, and APEX Dynamic Actions for making the important parts of data stand out. Then, we see the implementation of the handlers, such as this.affectedElements and the use of the jQuery functions, such as this.css() in this section. We also see the advanced formatting methods using the APEX templates. We then end this part of the discussion, by creating a matrix report using the APEX Dynamic Query report region. The hierarchical reports are always intriguing, because of the enormous ability to present the relations among different rows of data, and because of the use of the hierarchical queries in a number of unrelated places to answer the business queries. We reserve a discussion on the Oracle’s hierarchical queries for a later part of the section and start it, by understanding the implementation of the hierarchical reports by linking the values in APEX using drilldowns. We see the use of the APEX items as the substitution variable to create the dynamic messages. Since we have devised a mechanism to drill down, we should also build a ladder for the user to climb up the hierarchical chain. Now, the hierarchical chain for every person will be different depending on his position in the organization, so we build a mechanism to build the dynamic bread crumbs using the PL/SQL region in APEX. We then talk about two different methods of implementing the hierarchical queries in APEX. We talk about the connect, by clause first, and then continue our discussion by learning the use of the recursive with clause for the hierarchical reporting. We end our discussion on the hierarchical reporting, by talking about creating the APEX’s Tree region that displays the hierarchical data in the form of a tree. The reports are often associated with the supporting files that give more information about the business query. A typical user might want to upload a bunch of files while executing his piece of the task and the end user of his action might want to check out the files uploaded by him/her. To understand the implementation of this requirement, we check out the various ways to implement uploading and downloading the files in APEX. We start our discussion, by devising the process of uploading the files for the employees listed in the oehr_employees table. The solution of uploading the files involves the implementation of the dynamic action to capture the ID of the employee on whose row the user has clicked. It shows the use of the APEX items as the substitution variables for creating the dynamic labels and the use of JavaScript to feed one APEX item based on the another. We extensively talk about the implementation of jQuery in Dynamic Actions in this section as well. Finally, we check out the use of the APEX’s file browse item along with the WWV_FLOW_FILES table to capture the file uploaded by the user. Discussion on the methods to upload the files is immediately followed by talking about the ways to download these files in APEX. We nest the use of the functions, such as HTF.ANCHOR and APEX_UTIL.GET_BLOB_FILE_SRC for one of the ways to download a file, and also talk about the use of dbms_lob.getlength along with the APEX format mask for downloading the files. We then engineer our own stored procedure that can download a blob stored in the database as a file. We end this discussion, by having a look at the APEX’s p process, which can also be used for downloading. AJAX is the mantra of the new age and we try our hand at it, by implementing the soft deletion in APEX. We see the mechanism of refreshing just the report and not reloading the page as soon as the user clicks to delete a row. We use JavaScript along with the APEX page process and the APEX templates to achieve this objective. Slicing and dicing along with auditing are two of the most common requirements in the reporting world. We see the implementation of both of these using both the traditional method JavaScript with the page processes and the new method of using Dynamic Actions. We extend our use case a little further and learn about a two way interaction between the JavaScript function and the page process. We learn to pass the values back and forth between the two. While most business intelligence and reporting solutions are a one way road and are focused on the data presentation, Oracle APEX can go a step further, and can give an interface to the user for the data manipulations as well. To understand this strength of APEX, we have a look at the process of creating the tabular forms in APEX. We extend our understanding of the tabular forms to see a magical use of jQuery to convert a certain sections of a report from display-only to editable textboxes. We move our focus from implementing the interesting and tricky frontend requirements to framing the queries to display the complex data types. We use the string aggregation methods to display data in a column containing a varray. Time dimension is one of the most widely used dimension in reporting circles and comparing current performance with the past records is a favorite requirement of most businesses. With this in mind, we shift our focus to understand and implement the time series reports in APEX. We start our discussion by understanding the method to implement a report that shows the contribution of each business line in every quarter. We implement this by using partitioning dimensions on the fly. We also use the analytical functions, such as ratio_to_report, lead, and lag in the process of creating the time series reports. We use our understanding of time dimension to build a report that helps a user compare one time period to the other. The report gives the user the freedom to select the time segments, which he wishes to compare. We then outwit a limitation of this report, by using the query partition clause for the data densification. We bring our discussion on reports based on time dimension, by presenting a report based on the modal clause to you. The report serves as an example to show the enormous possibilities to code, by using the modal clause in Oracle. We bring our discussion on reports based on time dimension, by presenting a report based on the modal clause to you. The report serves as an example to show the enormous possibilities to code, by using the modal clause in Oracle. Chapter 3, In the APEX Mansion – Interactive Reports is all about Interactive Reports and the dynamic reporting. While the second chapter was more about the data presentation using the complex queries and the presentation methods, this chapter is about taking the presentation part a step ahead, by creating more visually appealing Interactive Reports. We start the discussion of this chapter, by talking about the ins and outs of Interactive Reports. We postmortem this feature of APEX to learn about every possible way of using Interactive Reports for making more sense of data. The chapter has a reference application of its own, which shows the code in action. We start our discussion, by exploring at various features of the Actions menu in Interactive Reports. The discussion is on search functionality, Select Columns feature, filtering, linking and filtering Interactive Reports using URLs, customizing Rows per page feature of an IR, using Control Break, creating computations in IR, creating charts in IR, using the Flashback feature and a method to see the back end flashback query, configuring the e-mail functionality for downloading a report, and for subscription of reports and methods to understand the download of reports in HTML, CSV, and PDF formats. Once we are through with understanding the Actions menu, we move on to understand the various configuration options in an IR. We talk about the Link section, the Icon View section, the Detail section, the Column Group section, and the Advanced section of the Report Attributes page of an IR. While discussing these sections, we understand the process of setting different default views of an IR for different users. Once we are through with our dissection of an IR, we put our knowledge to action, by inserting our own item in the Actions menu of an IR using Dynamic Actions and jQuery. We continue our quest for finding the newer formatting methods, by using different combinations of SQL, CSS, APEX templates, and jQuery to achieve unfathomable results. The objectives attained in this section include formatting a column of an IR based on another column, using CSS in the page header to format the APEX data, changing the font color of the alternate rows in APEX, using a user-defined CSS class in APEX, conditionally highlighting a column in IR using CSS and jQuery, and formatting an IR using the region query. After going through a number of examples on the use of CSS and jQuery in APEX, we lay down a process to use any kind of changes in an IR. We present this process by an example that changes one of the icons used in an IR to a different icon. APEX also has a number of views for IR, which can be used for intelligent programming. We talk about an example that uses the apex_application_page_ir_rpt view that show different IR reports on the user request. After a series of discussion on Interactive Reports, we move on to find a solution to an incurable problem of IR. We see a method to put multiple IR on the same page and link them as the master-child reports. We had seen an authentication mechanism (external table authentication) and data level authorization in Chapter 2, Reports. We use this chapter to see the object level authorization in APEX. We see the method to give different kinds of rights on the data of a column in a report to different user groups. After solving a number of problems and learning a number of things, we create a visual treat for ourselves. We create an Interactive report dashboard using Dynamic Actions. This dashboard presents different views of an IR as gadgets on a separate APEX page. We conclude this chapter, by looking at advanced ways of creating the dynamic reports in APEX. We look at the use of the table function in both native and interface approach, and we also look at the method to use the APEX collections for creating the dynamic IRs in APEX. Chapter 4, The Fairy Tale Begins – Advanced Reporting, is all about advanced reporting and pretty graphs. Since we are talking about advanced reporting, we see the process of setting the LDAP authentication in APEX. We also see the use of JXplorer to help us get the necessary DN for setting up the LDAP authentication. We also see the means to authenticate an LDAP user using PL/SQL in this section. This chapter also has a reference application of its own that shows the code in action. We start the reporting building in this chapter, by creating the Sparkline reports. This report uses jQuery for producing the charts. We then move on to use another jQuery library to create a report with slider. This report lets the user set the value of the salary of any employee using a slider. We then get into the world of the HTML charts. We start our talk, by looking at the various features of creating the HTML chart regions in APEX. We understand a method to implement the Top N and Bottom N chart reports in the HTML charts. We understand the APEX’s method to implement the HTML charts and use it to create an HTML chart on our own. We extend this technique of generating HTML from region source a little further, by using XMLTYPE to create the necessary HTML for displaying a report. We take our spaceship into a different galaxy of the charts world and see the use of Google Visualizations for creating the charts in APEX. We then switch back to AnyChart, which is a flash charting solution, and has been tightly integrated with APEX. It works on an XML and we talk about customizing this XML to produce different results. We put our knowledge to action, by creating a logarithmic chart and changing the style of a few display labels in APEX. We continue our discussion on AnyChart, and use the example of Doughnut Chart to understand advanced ways of using AnyChart in APEX. We use our knowledge to create Scatter chart, 3D stacked chart, Gauge chart, Gantt chart, Candlestick chart, Flash image maps, and SQL calendars. We move out of the pretty heaven of AnyChart only to get into another beautiful space of understanding the methods of displaying the reports with the images in APEX. We implement the reports with the images using the APEX’s format masks and also using HTF.IMG with the APEX_UTIL.GET_BLOB_FILE_SRC function. We then divert our attention to advanced jQuery uses such as, the creation of the Dialog box and the Context menu in APEX. We create a master-detail report using dialog boxes, where the child report is shown in the Dialog box. We close our discussion in this chapter, by talking about creating wizards in APEX and a method to show different kinds of customized error messages for problems appearing at different points of a page process. Chapter 5, Flight to Space Station: Advanced APEX, is all about advanced APEX. The topics discussed in this chapter fall into a niche category of reporting the implementation. This chapter has two reference applications of its own that shows the code in action. We start this chapter, by creating both the client-side and server-side image maps APEX. These maps are often used where regular shapes are involved. We then see the process of creating the PL/SQL Server Pages (PSPs). PSPs are similar to JSP and are used for putting the PL/SQL and HTML code in a single file. Loadpsp then converts this file into a stored procedure with the necessary calls to owa web toolkit functions. The next utility we check out is loadjava. This utility helps us load a Java class as a database object. This utility will be helpful, if some of the Java classes are required for code processing. We had seen the use of AnyChart in the previous chapter, and this chapter introduces you to FusionChart. We create a Funnel Chart using FusionChart in this chapter. We also use our knowledge of the PL/SQL region in APEX to create a Tag Cloud. We then stroll into the world of the APEX plugins. We understand the interface functions, which have to be created for plugins. We discuss the concepts and then use them to develop an Item type plugin and a Dynamic Action plugin. We understand the process of defining a Custom Attribute in a plugin, and then see the process of using it in APEX. We move on to learn about Websheets and their various features. We acquaint ourselves with the interface of the Websheet applications and understand the concept of sharing Websheets. We also create a few reports in a Websheet application and see the process of importing and using an image in Websheets. We then spend some time to learn about data grids in Websheets and the method to create them. We have a look at the Administration and View dropdowns in a Websheet application. We get into the administration mode and understand the process of configuring the sending of mails to Gmail server from APEX. We extend our administration skills further, by understanding the ways to download an APEX application using utilities, such as oracle.apex.APEXExport. Reporting and OLAP go hand in hand, so we see the method of using the OLAP cubes in APEX. We see the process of modeling a cube and understand the mechanism to use its powerful features. We then have a talk about Oracle Advanced Queues that can enable us to do reliable communication among different systems with different workloads in the enterprise, and gives improved performance. We spend a brief time to understand some of the other features of APEX, which might not be directly related to reporting, but are good to know. Some of these features include locking and unlocking of pages in APEX, the Database Object Dependencies report, Shortcuts, the Dataloading wizard, and the APEX views. We bring this exclusive chapter to an end, by discussing about the various packages that enable us to schedule the background jobs in APEX, and by discussing various other APEX and Database API, which can help us in the development process.