How-To Tutorials

(For more resources related to this topic, see here.) Step 1 – Recording and adding commands in a test In this section we will show you how to record a test on a demo e-commerce application. We will test the product search feature of the application using the following steps: Launch the Firefox browser. Open the website for testing in the Firefox browser. For this example we will use http://demo.magentocommerce.com/. Open Selenium IDE from the Tools menu. Selenium IDE by default sets the recording mode on. If it's not pressed, you can start recording by pressing the (record) button in the top-right corner. Now switch back to the Firefox browser window and type Nokia in the search textbox and click on the Search button as shown: Check if the link Nokia 2610 Phone is present in the search results. We can do that by selecting the link and opening the context menu (right-click) and selecting Show All Available Commands | assertElementPresent link=Nokia 2610 Phone. Next, we will click on the Nokia 2610 Phone link to open the product page and check if the Nokia 2610 Phone text is displayed on the product page. To do this, select the Nokia 2610 Phone text and open the context menu (right-click) and select Show All Available Commands | assertTextPresent link=Nokia 2610 Phone: Go back to Selenium IDE. All the previous steps are recorded by Selenium IDE in the Command-Target-Value format as shown in the following screenshot. Stop the recording session by clicking on the Recording button: Step 2 – Saving the recorded test Before we play back the recorded test, let's save it in Selenium IDE: Select File | Save Test Case from the Selenium IDE main menu: In the Save As dialog box, enter the test case name as SearchTest.html and click on the Save button. The test will be saved with the name SearchTest. Step 3 – Saving the test suite In Selenium IDE, we can group multiple tests in a test suite. Let's create a test suite and Selenium IDE will automatically add SearchTest to this suite: Select File | Save Test Suite from the Selenium IDE main menu. In the Save As dialog box, enter the test case name as SearchFeatureTests.html and click on the Save button. You can create and record more than one test case in a test suite. Step 4 – Running the recorded test Selenium IDE provides multiple ways to execute the tests: Option 1 – running a single test case Select the test which you want to execute from the test suite pane and click on the (play current test case) button . Selenium IDE will start the playback of the test and you can see the steps that we recorded earlier are being played automatically in the browser window. At end of execution, Selenium IDE will display results as per the following screenshot: Option 2 – running all tests from a test suite If you have multiple tests in a test suite, you can use the (play the entire test suite) button to play all the test cases. After the test is executed in Selenium IDE, you can see the results in Log tab. All the steps which are successfully completed will be heighted in green and checks in dark green. If there are any failures in the test, those will be highlighted in red. This is how Selenium IDE helps you testing your web application. Summary We learned how to record a test, save a test case, enhance a test by adding commands, and run a test with Selenium IDE. This article also got you started on with programming with Selenium WebDriver. Resources for Article : Further resources on this subject: Python Testing: Installing the Robot Framework [Article] First Steps with Selenium RC [Article] User Extensions and Add-ons in Selenium 1.0 Testing Tools [Article]

(For more resources related to this topic, see here.) What is vCenter Operations Manager? vCenter Operations Manager, also known as vC Ops, is a VMware product that allows IT administrators to monitor their virtual environments in the most efficient way. It also aids in design and capacity planning. vC Ops allows us administrators and IT managers to have visibility into our entire virtual infrastructure and goes beyond the simple alarms and performance charts offered in vCenter Server alone. It offers dashboards, alerts, and several detailed reports to help us better assess our environments. We can even monitor several different vCenter environments by simply configuring vC Ops to connect with any vCenter Server that we have in our environment. vC Ops is a vApp consisting of two virtual appliances that can be downloaded from the VMware website. It comes with a management plugin that's easily installed on the vSphere client. Alternatively we can browse the management site directly, if preferable. With vSphere Version 5.1 and above, we can also use vSphere Web Client to manage vC Ops and we can find embedded metrics within the summary pages for most of the objects in our virtual infrastructure. As we can see from the following screenshot, the default dashboard offered in all the licensed versions of vC Ops above the Foundation edition holds a lot of information. We get an idea of the three major metrics, or badges, that vC Ops tells us about: Health, Risk, and Efficiency. Dashboards such as the one shown in the following screenshot can quickly give us an insight into the things that are happening in our environment and visually point out any errors or issues inside our environment that may have cause for more investigation. Although vCenter Server comes with its own alarms and performance charts, vC Ops actually learns about our environment and reports alerts based on that. In fact, it's recommended that we let vC Ops run for a month after the initial installation before we start looking at the metrics and going through the reports. vCenter Server has several alarms that will show up on our vSphere Client, but we need to set these alarms with hard triggers. For example, in the following screenshot, we can see the vCenter 5.1 alarm triggers for Host memory usage in vSphere Web Client. It shows that if the memory usage for a physical host is above 90 percent for five minutes or longer, it will give us a warning. If the host memory usage is above 95 percent for five minutes or longer, it will give us a critical error. For an alarm like this, a rigid or static trigger threshold may be appropriate. Memory should really not be at more than 95 percent utilization for too long. In that situation, we would want to add more memory to that host or perhaps vMotion VMs to another host if we have that option. However, what if we have an alarm triggered for CPU usage of a virtual machine? If this virtual machine consistently runs with high CPU usage because it's supposed to, vCenter will still tell us there is a critical error. Since vC Ops actually learns our environment, it will tell us that this is not an anomalous behavior, and we may not need to worry about it. Another example of when this is useful would be if a VM routinely runs scheduled tasks that cause CPU or memory utilization to be high for a brief time during the day. vCenter alarms would trigger everyday or every time this happens. vC Ops will learn this behavior, thereby reducing the barrage of alerts admins receive everyday. vC Ops will still tell us that CPU usage runs high via badge scores, such as Workload or Stress, so we don't have to worry about missing information either. Other than immediate notifications and alarms found throughout the dashboards and reports, vC Ops also gives us visibility into longer-term issues and helps us know whether it's safe to add or remove VMs as well as physical resources to our environment. Benefits of troubleshooting with vC Ops As mentioned in the previous section, vC Ops actually understands our environment and reports anomalous behavior. This is not to say that if a VM is always using 100 percent of its storage, vC Ops will let you know that. It will also let you know how long it's been happening and the normal range for the VM. If we zoom in on the Workload badge, as shown in the following screenshot, we can see that it shows CPU usage, memory usage, disk I/O, and network I/O. The blue bar above each graph shows the normal range for each metric. If it were outside of that normal range, that would indicate anomalous behavior. This can be very helpful for troubleshooting because now we can dive in and see what's changed. From vCenter Server alone, we can see some historical data, and we can see real-time metrics, but without doing some pretty intense math, we won't know the normal range. Another benefit you get with vC Ops, which you wouldn't necessarily see in the vCenter Server performance data, is that you can check whether VMs are undersized. An undersized VM is a virtual machine with less compute resources than it actually needs to perform properly. Again, this is not based solely on random peaks or bursts, but rather on historical and present data that has been run through algorithms, and vC Ops lets us know how much compute the VM should be assigned for it to work efficiently. So, for instance, if an application is running slow, or even slow at particular times in a day consistently, we would be able to open vC Ops, highlight the affected machine, and then go to the Planning tab. From here, we can see how much time this VM has been running without enough memory or CPU, for example, and it will also tell us how much additional resources it recommends. One of the most interesting benefits is when you pair vC Ops with vCenter Configuration Manager. Again, let's say we have an application that's all of a sudden running slow or sporadically. If we open vC Ops and highlight the problematic VM again, we can find the recent events and tasks that have been performed on that VM. On this page, we can also see a graph with the performance of the VM. We can see where the performance spiked and also the events that correlated with that timeframe. Perhaps the event would be something similar to "VM RAM was changed from 10 GB to 2 GB". Even before we use vCenter Configuration Manager, we've narrowed down the issue to being lack of memory. Now, if we check with vCenter Configuration Manager, we may be able to see which administrator made that change, if that change was made from vCenter Configuration Manager. The last benefit I'm going to bring up here, though there are certainly more, is the easy way to find relationships between VMs and other vSphere inventory objects. So, why is finding the relationship between a VM and what it's connected to important? Let's look at our previous two scenarios, where we had an application running slow on a VM. If we were to look at the VM, it could show that things are running slow, but we may be unable to find the reason immediately. However, if we look at the relationships between them, there may be a common denominator, such as a datastore or host, that is actually causing the issue. We may see that all the VMs on that particular datastore or host are running poorly, but if we correct the error at the root of the problem, we'll correct the issues on all the VMs connected to it. Using vCenter alone, we might have taken a lot longer to figure out this correlation, but because we can see all the relationships mapped out for us, it's easier to do a root cause analysis. See the following two screenshots for illustrations of how this might look. The following example is of the Overview section of the Environment tab. It shows all of the elements across the environment from the top down but highlights those related to, for example, a selected VM. The following example, found under the Relationships section of the Environment tab, gives you a different view. It shows only the components that are in a direct relationship with the component you have selected in the left-hand side pane. Benefits of capacity planning with vC Ops VMware used to have a solution called vCenter Capacity IQ. In early 2012, VMware stopped selling Capacity IQ and put all of the capacity planning features from Capacity IQ into vC Ops. We can upgrade vCenter Capacity IQ licenses to vC Ops licenses. vCenter itself doesn't really offer much by way of capacity planning. Obviously, we can look at how much space we have free on our datastores as well as how many compute resources we have free, but it would still be an estimate. With vC Ops, we take the guesswork out of it with the use of oversized VM reporting and what-if scenarios. The great thing about vC Ops is that it actually has a lot of capacity management features in every view and dashboard. No matter where we've drilled in, we'll be able to see information about how much storage or compute resources are left. However, most of the capacity planning features can be easily found under the Planning and Analysis tabs. Here, we can find information, not only on how much storage or compute a VM is using, but also what the trends have been for future planning purposes. For instance, you might be able to see that a datastore has been losing about 2 GB of free space every week. If the datastore is 1 TB, vC Ops can estimate when we might run out of space. Much like the undersized VM analysis discussed in the troubleshooting section, we also have data on which VMs are oversized. Many times applications and/or application owners will ask for outrageous amounts of CPU or memory. There are also cases where we perform physical to virtual migrations to convert a physical server to a virtual server and we just leave the original amount of compute resources even if it's not necessary. vCenter Server is never going to tell us that we've over allocated memory or CPU, for example, to a VM. vC Ops, however, will show us a full report on the VMs on which we can reclaim compute resources and how much can be reclaimed. This may seem like a small reclamation of resources, but let's say we have 100 VMs, and 25 of them are using an extra two vCPUs, then we can essentially reclaim 50 vCPUs as well as reduce CPU contention within our environment. Probably the most interesting benefit of using vC Ops to do our capacity planning is the what-if scenarios. We can actually click on a link under the Planning tab to pull up a what-if scenario wizard. For example, with this wizard, we can manually input the number of VMs we want to add, and it will output how that will affect the current environment. We can also have it automatically input the variables using trending analytics for our environment. vC Ops will take a look at the average size of our VMs, analyze any pertinent historical data, and then tell us if we have enough resources to add a certain amount of VMs. This is an incredibly powerful tool that most normal admins would not be able to replicate through the use of a simple script with simple mathematics. Feature comparison of versions A number of versions of vC Ops are available. The following is a quick overview of the various versions. What is vCenter Operations Manager Suite? The main focus of this article is vC Ops although we do get into some of the other components of the suite. VMware has historically sold the current components of the suite separately. Recently though, they've decided it would be beneficial to bundle them in a suite. The components of the suite are as follows: vCenter Operations Manager vCenter Configuration Manager vFabric Hyperic vCenter Infrastructure Navigator vCenter Chargeback Manager For more information on these components, check the documentation given on VMware.com. Licensing versions Not every component comes in the vC Ops suite. It depends on which version of the license has been purchased. The versions of the suite are as follows: Foundation Standard Advanced Enterprise The versions of the vC Ops suite should not be confused with the versions of the vCloud suite. They are separate items. As of Version 5.7, vC Ops licensing works in the following way. This chart was taken directly from the VMware.com website. By looking at the chart, Foundation and Standard look the same, as do Advanced and Enterprise. However, that's not actually the case. Currently the Foundation license is included in every version of vCenter. It does not give us insights using historical data, it only reports in real time. However, it will store the historical data if we decide to upgrade the license at a later date. Foundation doesn't really offer much by way of capacity planning either. It's really more of just an extension of our current alerts that we get from vCenter. As shown in the chart, Standard gives us all that is included in Foundation, as well as the capacity features. It is basically a full version of vCenter Operations Manager. The Advanced and Enterprise versions give us what is really more of a suite with the components mentioned earlier. The difference between these being that Enterprise gives us more OS- and application-level monitoring and Advanced really just gives us the VM-level monitoring. Using vC Ops with other solutions There are several plugins or adapters to vC Ops that will extend our monitoring capabilities even further. One plugin that came out in the last year or so is vCenter Operations Manager for Horizon View. Through the use of vC Ops adapters, we can now monitor our Horizon View virtual desktop implementations. This gives us custom dashboards that give us a lot more insight than we've had before. If you've ever run a VDI environment, you must be well aware that the ability to pinpoint our problem areas quickly is a necessity. Other VMware solutions it will connect to are vCloud Director 1.5.0 and above. There are several storage adapters for vC Ops as well. Among these are the EMC Smarts Adapter, EMC Symmetrix Adapter, EMC VNX Adapter, and NetApp Adapter. There are also several monitoring solutions that vC Ops can connect with to get a view of your whole environment, such as HP BAC Adapter, HP SiteScope Adapter, IBM Tivoli Monitoring Adapter, and Microsoft SCOM Adapter. It will also plug into Oracle Enterprise Manager to help you monitor your Oracle databases. In most cases, after connecting vC Ops to these adapters, we need to browse to a custom site other than our vC Ops management site. These sites will have custom dashboards set up with information from the product(s) we are connected to. At present, this information does not really affect the regular vC Ops dashboards. You will still see the same information there that you would see if you did not have the adapters installed and collecting data. Summary In this article, we discussed what vCenter Operations Manager is. It's a solution that allows us to monitor our virtual infrastructure as well as plan for future capacity issues that may arise. We then discussed some of the benefits of troubleshooting our virtual environments as well as doing some capacity planning with the help of vCenter Operations Manager. It's also important to keep in mind that there are several different versions of vC Ops, and it is possible to get it in a bundle with vCenter Configuration Manager, vFabric Hyperic, vCenter Infrastructure Navigator, and vCenter Chargeback Manager. Finally, we went over some of the other hardware and software solutions that we can integrate into our vCenter Operations Manager and/or suite, such as storage arrays and other third-party monitoring solutions. Resources for Article: Further resources on this subject: Securing vCloud Using the vCloud Networking and Security App Firewall [article] Cloning and Snapshots in VMware Workstation [article] Network Virtualization and vSphere [article]

Alfresco 3 Cookbook Over 70 recipes for implementing the most important functionalities of Alfresco The reader can benefit from the previous article on Alfresco 3: Writing and Executing Scripts.   Add/Change contents of a document Let’s explore some example JavaScript. In the following example scripts, you will be able to witness the APIs and functionalities. Getting ready We will store the JavaScript files in the Company Home>Data Dictionary>Scripts>Cookbook folder (this folder does not exist in your repository and create this folder). And will run the sample scripts against a document – Test_JS_API.txt in the folder Company Home>InfoAxon>Chapter 8. I have uploaded this text file with a simple line of text: A sample Document created to investigate in JavaScript API. and used our custom content type iabook:Product. if (document.hasPermission("Write")) { if (document.mimetype == "text/plain") { if (!document.hasAspect("cm:versionable")) document.addAspect("cm:versionable"); var wcopy = document.checkout(); var cnt = wcopy.content; cnt += "rnThis line is added using the JavaScript."; wcopy.content = cnt; wcopy.checkin("Sample Line added via JS"); } } How to do it... Create a new script file in the Company Home>Data Dictionary>Scripts>Cookbook folder and save this code; let’s say the file is named changecontent.js Execute the script using Run Action on the document Test_JS_API.txt in the Chapter 8 folder. After running the script, a new version of the document will be created and a new line will be added in the document. Thus each time you run the script for this document, a line will be appended at the end of the content and a new version will be created. How it works... The document object here automatically refers to the current document, in our case, it is Test_JS_API.txt, since we have executed the script against this document. First we have checked whether we have proper permission to perform the write operation on the document. If the permission is there, we check the mimetype of the document, since the textual content writing operation is possible only for a few mimetypes such as text, html, and so on. After that, we check whether the document is versionable or not, by default, any content you upload in the repository is not versionable. So we add the cm:versionable aspect in case it is not there already. Then we checkout the document and append the line of text we want in the working copy. After updating the content, we checking the working copy with a commit comment. This comment is visible in the Version History of the document. Though it is not always mandatory to check for the required permissions, it is a good practice to confirm for the relevant permissions, otherwise Alfresco may throw runtime errors in case the required permissions are not available.   Creating a backup copy of a document In this recipe, we will write a script to create a backup copy of a particular document. How to do it... Create a new script file in the Company Home>Data Dictionary>Scripts>Cookbook folder and add the following code. Let’s say the file is named createbackup.js var back = space.childByNamePath("Backup"); if (back == null && space.hasPermission("CreateChildren")) { back = space.createFolder("Backup"); } if (back != null && back.hasPermission("CreateChildren")) { var copied = document.copy(back); if (copied != null) { var backName = "Backup of " + copied.name; copied.name = backName; copied.properties.description = "This is a Backup copy created by JS"; copied.save(); } } Execute the script using Run Action on the document Test_JS_API.txt in the Chapter 8 folder. After executing the script, a new folder named Backup will be created (if it does not exist already) and a copy of this document (named Backup of Test_JS_API.txt) will be created in the backup folder. (Move the mouse over the image to enlarge.) How it works... The space object here automatically refers to the current space. In our case, it is Chapter 8, since we have executed the script against a document from this folder. The document object here automatically refers to the current document. In our case, it is Test_JS_API.txt, since we have executed the script against this document. First we have checked whether a space already exists there with the name Backup under Chapter 8. If not, we create the space. This is the space where we intend to create our backup copy. After that, we check whether we have the proper permission to create a new document in the backup folder. We do this by checking the CreateChildren permission. If we have the proper required permission, we create a copy of the document in the backup folder. Then we change a few properties of the copied document – we change the name and description, for instance. After changing the properties, we save the changes. Note that you do not need to save after changing the content of a document. However, you need to do this in case you change any property of the content item.   Adding a tag to a document In this recipe, we will write a script that can be used to tag a document. How to do it… Create a new script file in the Company Home>Data Dictionary>Scripts>Cookbook folder and add the following code; let’s say the file is named addtag.js if (!document.hasAspect("cm:taggable")) document.addAspect("cm:taggable"); document.addTag("test"); Execute the script using Run Action on the document Test_JS_API.txt in the Chapter 8 folder. The document will not be taggable, and a new tag has been added with the document – test. This is reflected in the property sheet of the document. Now, you can also add more tags using the property editor dialog. How it works... The code we presented is rather simple in this case. As usual, the document object here automatically refers to the current document. In our case, it is Test_JS_API.txt, since we have executed the script against this document. First we have checked whether the document already has the cm:taggable aspect associated with it, if not we add this aspect. Then it is just about adding a tag – we added a tag test. You can also add multiple tags at a time using the addTags method (we have used the addTag method to add a single tag in our example).  

(For more resources related to this topic, see here.) In order to provide the instructions for this article to work without modification, you will need administrative access to a system running Ubuntu 12.04 LTS with SSH service running and accessible via a network. Additionally, the system will need to be able to access the Chef Server and the public Internet for package updates and gem installation. Bootstrapping servers Bootstrapping is the process of setting up something without external intervention. Chef uses bootstrap scripts that are executed (over SSH) on a remote server to perform any initial configuration that you desire. These scripts are written using ERB (a Ruby template language) and serve as a launching pad to setting up a new server. Typically these scripts would be run on a brand new server, but can be applied to any server that you can SSH into. Additionally, bootstrap files are Linux-distribution and Ruby-distribution dependent because they have commands that are specific to particular distributions. For example, an Ubuntu Linux server with Ruby 1.9 installed from source, or a RedHat Enterprise Linux server with Ruby 1.8.7 installed from EPEL. In addition to any initial configuration, bootstrapping registers the node with the Chef Server so that it becomes a member of the infrastructure and can have configurations and roles applied to it. How the process works The bootstrapping begins with developing a bootstrap script that targets the distribution and version of that distribution that is running on the server you are looking to provision. Once the script is written, the knife tool is used to remotely log into the new system and run the script to perform the initial configuration. Knife does some interpolation locally of the bootstrap script before it is run on the server. This means that you can leverage Chef's data and configuration during the bootstrap process. Common uses here would include setting up initial firewall rules, routes, users, or other mandatory initial provisioning. Chef provides some pre-written bootstrap scripts for the following platforms, making it easy to get started: centos5-gems fedora13-gems ubuntu10.04-gems ubuntu10.04-apt ubuntu12.04-gems Examining the bootstrap script If you are looking for an example of what a bootstrap file contains, you can find the ones provided with chef in lib/chef/knife/bootstrap inside the directory containing your Chef gem (which, if you used the Opscode packages on a Debian system, would reside in /usr/lib/ruby/vendor_ruby/chef). I strongly suggest reading over the bootstrap script you will be using so that you have a good idea of what you're running on your system as root before doing so. Performing the bootstrap Bootstrapping a server is quite simple and involves a single invocation of knife (once your bootstrap script is complete). Knife looks for bootstrap files in a directory called bootstrap in your local Chef directory. Good names for bootstrap files would include the distribution name as well as the version and type of Ruby installation you are performing (that is, centos5-ruby19, ubuntu11.10-rvm-ruby19, and so on). In our case, we will be using the pre-supplied bootstrap script that configures Ubuntu 12.04 with gems to bootstrap our new system. For example: export SERVER_IP="11.22.33.44" export USER="ubuntu_user" knife bootstrap -x $USER --sudo $SERVER_IP -d ubuntu12.04-gems This is where the environment variable, $SERVER_IP, is set to the IP address of your newly setup server and $USER is set to the user you created on your Ubuntu server that can execute sudo. When executed, this command tells knife to execute the bootstrap command, which is responsible for loading the bootstrap script specified by the -d flag (-d is for distribution) over SSH logging into the remote server specified by the environment variable, $SERVER_IP using the remote user specified by the -x flag. In this case, it would run the following steps: SSH as ubuntu_user to 11.22.33.44 Execute the contents of ubuntu12.04-gems.erb on the remote server using sudo Once that is complete, the server will be bootstrapped according to the steps in the bootstrap file, which, if you were to look at the provided bootstrap script, you would see the following: Update the APT repository Install Ruby 1.8 and some development dependencies from APT Install the latest version of RubyGems Update the local Ruby gems Use gem to install ohai, the system-reporting agent Use gem to install Chef Copy the validation certificate file to the server Copy the encrypted data bag secret (if applicable) Generate any ohai hints needed Copy the Chef Client configuration to the server Copy the first-run JSON data to the server Execute the Chef Client in order to: Register the node Run any initially configured run list data Verifying the registration Once this has been completed, we can verify that the node has been registered with the Chef Server in either of two ways: using knife, or using the web console. Using knife To verify the node was registered with the Chef Server, we will be using the client command provided by knife. This can be accomplished with the list subtask like this: user@server:$> knife client list host1 host2 new-host-name Where new-host-name would be the hostname of the node you just bootstrapped (Chef Client will automatically determine the hostname when it registers itself using the bootstrapped machine's FQDN as you set it up). Via the web console To verify that the node was registered using the web console, we must first log into our Chef Server using the administrative credentials that were configured during setup. Once logged in, there will be a set of tabs along the top where you can switch between the different data collections you can manage. The URL of the web console will vary between installations, but if you followed the instructions earlier, it would be accessible at http://chefserver.yourdomain.com:4040. As you can see, there are tabs for managing environments, roles, nodes, cookbooks, data bags, clients, and users. Additionally, there is a tab for performing searches, which is useful for validating search queries to be used in recipes. Once you are on the Nodes tab, you will see a list of nodes that Chef knows about, which will look like the following screenshot: In our case, chef-server is the Chef Server itself that we registered during the installation phase, and monitoring-production is the hostname of the newly registered server. Summary In this Article, we have discussed the process of getting started with Chef. Resources for Article : Further resources on this subject: OpenAM: Backup, Recovery, and Logging [Article] Starting Up Tomcat 6: Part 1 [Article] SQL Server 2008 R2: Multiserver Management Using Utility Explorer [Article]

In this article by William Smith, author of Learning Xamarin Studio, we will configure our operating system. (For more resources related to this topic, see here.) Configuring your Mac To configure your Mac, perform the following steps: From the Apple menu, open System Preferences. Open the Personal group. Select the Security and Privacy item. Open the Firewall tab, and ensure the Firewall is turned off. Configuring your Windows machine To configure your Windows machine, download and install the Xamarin Unified Installer. This installer includes a tool called Xamarin Bonjour Service, which runs Apple's network discovery protocol. Xamarin Bonjour Service requires administrator rights, so you may want to just run the installer as an administrator. Configuring a Windows VM within Mac There is really no difference between using the Visual Studio plugin from a Windows machine or from a VM using software, such as Parallels or VMware. However, if you are running Xamarin Studio on a Retina Macbook Pro, it is advisable to adjust the hardware video settings. Otherwise, some of the elements within Xamarin Studio will render poorly making them difficult to use. The following screenshot contains the recommended video settings: To adjust the settings in Parallels, follow these steps: If your Windows VM is running, shut it down. With your VM shut down, go to Virtual Machine | Configure…. Choose the Hardware tab. Select the Video group. Under Resolution, choose Scaled. Final installation steps Now that the necessary tools are installed and the settings have been enabled, you still need to link to your Xamarin account in Visual Studio, as well as connect Visual Studio to your Mac build machine. To connect to your Xamarin account, follow these steps: In Visual Studio, go to Tools | Xamarin Account…. Click Login to your Xamarin Account and enter your credentials. Once your credentials are verified, you will receive a confirmation message. To connect to your Mac build machine, follow these steps: On your Mac, open Spotlight and type Xamarin build host. Choose Xamarin.iOS Build Host under the Applications results group. After the Build Host utility dialog opens click the Pair button to continue. You will be provided with a PIN. Write this down. On your PC, open Xamarin Studio. Go to Tools | Options | Xamarin | iOS Settings. After the Build Host utility opens, click the Continue button. If your Mac and network are correctly configured, you will see your Mac in the list of available build machines. Choose your build machine and click the Continue button. You will be prompted to enter the PIN. Do so, then click the Pair button. Once the machines are paired, you can build, test, and deploy applications using the networked Mac. If for whatever reason you want to unpair these two machines, open the Xamarin.iOS Build Host on your Mac again, and click the Invalidate PIN button. When prompted, complete the process by clicking the Unpair button. Summary In this article, we learned how to configure our operating system. We also learned how to connect to your Mac build machine. Resources for Article: Further resources on this subject: Updating data in the background [Article] Gesture [Article] Making POIApp Location Aware [Article]

  (For more resources on Cocos2d, see here.)   Box2D setup and debug drawing In our first physics recipe, we will explore the basics of creating a Box2D project and setting up a Box2D world. The example creates a scene that allows the user to create realistic 2D blocks. Getting ready Please refer to the project RecipeCollection02 for full working code of this recipe. How to do it... The first thing we need to do is create a Box2D project using the built-in Box2D project template: Go to File | New Project. Under User Templates click on Cocos2d. Now, right click on Cocos2d Box2d Application. Click Choose, name your project, and hit Save. Now, execute the following code: #import "Box2D.h"#import "GLES-Render.h"//32 pixels = 1 meter#define PTM_RATIO 32@implementation Ch4_BasicSetup-(CCLayer*) runRecipe { [super runRecipe]; /* Box2D Initialization *///Set gravity b2Vec2 gravity; gravity.Set(0.0f, -10.0f); //Initialize world bool doSleep = YES; world = new b2World(gravity, doSleep); world->SetContinuousPhysics(YES); //Initialize debug drawing m_debugDraw = new GLESDebugDraw( PTM_RATIO ); world->SetDebugDraw(m_debugDraw); uint32 flags = 0; flags += b2DebugDraw::e_shapeBit; m_debugDraw->SetFlags(flags); //Create level boundaries [self addLevelBoundaries]; //Add batch node for block creation CCSpriteBatchNode *batch = [CCSpriteBatchNodebatchNodeWithFile:@"blocks.png" capacity:150]; [self addChild:batch z:0 tag:0]; //Add a new block CGSize screenSize = [CCDirector sharedDirector].winSize; [self addNewSpriteWithCoords:ccp(screenSize.width/2, screenSize.height/2)]; //Schedule step method [self schedule:@selector(step:)]; return self;}/* Adds a polygonal box around the screen */-(void) addLevelBoundaries { CGSize screenSize = [CCDirector sharedDirector].winSize; //Create the body b2BodyDef groundBodyDef; groundBodyDef.position.Set(0, 0); b2Body *body = world->CreateBody(&groundBodyDef); //Create a polygon shape b2PolygonShape groundBox; //Add four fixtures each with a single edge groundBox.SetAsEdge(b2Vec2(0,0), b2Vec2(screenSize.width/PTM_RATIO,0)); body->CreateFixture(&groundBox,0); groundBox.SetAsEdge(b2Vec2(0,screenSize.height/PTM_RATIO),b2Vec2(screenSize.width/PTM_RATIO,screenSize.height/PTM_RATIO)); body->CreateFixture(&groundBox,0); groundBox.SetAsEdge(b2Vec2(0,screenSize.height/PTM_RATIO),b2Vec2(0,0)); body->CreateFixture(&groundBox,0); groundBox.SetAsEdge(b2Vec2(screenSize.width/PTM_RATIO,screenSize.height/PTM_RATIO), b2Vec2(screenSize.width/PTM_RATIO,0)); body->CreateFixture(&groundBox,0);}/* Adds a textured block */-(void) addNewSpriteWithCoords:(CGPoint)p { CCSpriteBatchNode *batch = (CCSpriteBatchNode*) [selfgetChildByTag:0]; //Add randomly textured block int idx = (CCRANDOM_0_1() > .5 ? 0:1); int idy = (CCRANDOM_0_1() > .5 ? 0:1); CCSprite *sprite = [CCSprite spriteWithBatchNode:batchrect:CGRectMake(32 * idx,32 * idy,32,32)]; [batch addChild:sprite]; sprite.position = ccp( p.x, p.y); //Define body definition and create body b2BodyDef bodyDef; bodyDef.type = b2_dynamicBody; bodyDef.position.Set(p.x/PTM_RATIO, p.y/PTM_RATIO); bodyDef.userData = sprite; b2Body *body = world->CreateBody(&bodyDef); //Define another box shape for our dynamic body. b2PolygonShape dynamicBox; dynamicBox.SetAsBox(.5f, .5f);//These are mid points for our 1m box //Define the dynamic body fixture. b2FixtureDef fixtureDef; fixtureDef.shape = &dynamicBox; fixtureDef.density = 1.0f; fixtureDef.friction = 0.3f; body->CreateFixture(&fixtureDef);}/* Draw debug data */-(void) draw { //Disable textures glDisable(GL_TEXTURE_2D); glDisableClientState(GL_COLOR_ARRAY); glDisableClientState(GL_TEXTURE_COORD_ARRAY); //Draw debug data world->DrawDebugData(); //Re-enable textures glEnable(GL_TEXTURE_2D); glEnableClientState(GL_COLOR_ARRAY); glEnableClientState(GL_TEXTURE_COORD_ARRAY);}/* Update graphical positions using physical positions */-(void) step: (ccTime) dt { //Set velocity and position iterations int32 velocityIterations = 8; int32 positionIterations = 3; //Steo the Box2D world world->Step(dt, velocityIterations, positionIterations); //Update sprite position and rotation to fit physical bodies for (b2Body* b = world->GetBodyList(); b; b = b->GetNext()) { if (b->GetUserData() != NULL) { CCSprite *obj = (CCSprite*)b->GetUserData(); obj.position = CGPointMake( b->GetPosition().x * PTM_RATIO,b->GetPosition().y * PTM_RATIO); obj.rotation = -1 * CC_RADIANS_TO_DEGREES(b->GetAngle()); } }}/* Tap to add a block */- (void)ccTouchesEnded:(NSSet *)touches withEvent:(UIEvent *)event { for( UITouch *touch in touches ) { CGPoint location = [touch locationInView: [touch view]]; location = [[CCDirector sharedDirector] convertToGL: location]; [self addNewSpriteWithCoords: location]; }}@end How it works... The Box2D sample project is a simple way to understand what a physics system looks like. Initialization: Upon initialization of the b2World object, we set a few things including gravity, object sleeping, and continuous physics. Sleeping allows bodies that are at rest to take up less system resources. Gravity is typically set to a negative number in the Y direction but can be reset at any time using the following method on b2World: void SetGravity(const b2Vec2& gravity); In addition to storing a pointer to the main b2World instance, we also usually store a pointer to an instance of GLESDebugDraw. Debug drawing: Debug drawing is handled by the GLESDebugDraw class as defined in GLESRender.h. Debug drawing encompasses drawing five different elements onscreen. These include shapes, joint connections, AABBs (axis-aligned bounding boxes), broad-phase pairs, and a center of mass bit. Visual to physical drawing ratio: We define the constant PTM_RATIO at 32, to allow consistent conversion between the physical world and the visual world. PTM stands for pixel to meter. Box2D measures bodies in meters and is built and optimized to work with bodies between the sizes of 0.1 to 10.0 meters. Setting this ratio to 32 is a common convention for optimal shapes to appear between 3.2 to 320 pixels on screen. Optimization aside, there is no upper or lower limit to Box2D body size. Level boundaries: In this and many future examples, we add a level boundary roughly encompassing the entire screen. This is handled with the creation of a b2Body object with four fixtures. Each fixture has a b2Polygon shape that defines a single edge. Creating an edge typically involves the following: b2BodyDef bodyDef;bodyDef.position.Set(0, 0);b2Body *body = world->CreateBody(&bodyDef);b2PolygonShape poly;poly.SetAsEdge(b2Vec2(0,0), b2Vec2(480/PTM_RATIO,0));body->CreateFixture(&poly,0); Because these edges have no corresponding visual components (they are invisible), we do not need to set the bodyDef.userData pointer. Creating the blocks: Blocks are created much in the same way that the level boundaries are created. Instead of calling SetAsEdge, we call SetAsBox to create a box-shaped polygon. We then set the density and friction attributes of the fixture. We also set bodyDef.userData to point to the CCSprite we created. This links the visual and the physical, and allows our step: method to reposition sprites as necessary. Scheduling the world step: Finally, we schedule our step method. In this method, we run one discrete b2World step using the following code: int32 velocityIterations = 8;int32 positionIterations = 3;world->Step(dt, velocityIterations, positionIterations); The Box2D world Step method moves the physics engine forward one step. The Box2D constraint solver runs in two phases: the velocity phase and position phase. These determine how fast the bodies move and where they are in the game world. Setting these variables higher results in a more accurate simulation at the cost of speed. Setting velocityIterations to 8 and positionIterations to 3 is the suggested baseline in the Box2D manual. Using the dt variable syncs the logical timing of the application with the physical timing. If a game step takes an inordinate amount of time, the physics system will move forward quickly to compensate. This is referred to as a variable time step. An alternative to this would be a fixed time step set to 1/60th of a second. In addition to the physical step, we also reposition and re-orientate all CCSprites according to their respective b2Body positions and rotations: for (b2Body* b = world->GetBodyList(); b; b = b->GetNext()) { if (b->GetUserData() != NULL) { CCSprite *obj = (CCSprite*)b->GetUserData(); obj.position = CGPointMake( b->GetPosition().x * PTM_RATIO,b->GetPosition().y * PTM_RATIO); obj.rotation = -1 * CC_RADIANS_TO_DEGREES(b->GetAngle()); }} Taken together, these pieces of code sync the physical world with the visual.  

  Joomla! 1.5 Cookbook Over 60 quick and direct recipes to help you overcome common Joomla! queries. Find quick solutions to common Joomla! problems Part of Packt's Cookbook series: Each recipe is a carefully organized sequence of instructions to complete the task as efficiently as possible Look at recipes that cover the portions of Joomla! 1.6 that are brand new Over 60 practical recipes covering a range of site management and core Joomla! activities Generally speaking, a basic article is a simple matter to create and manage. When a site gets large with a lot of articles, it can be unwieldy. This need is filled by a Content Construction Kit or CCK. Several good CCK's exist in the Joomlasphere and each has its own unique means to accomplish the task of content management and creation. For our purposes we'll look at the CCK known as K2. K2 provides you an integrated solution that features rich content forms - extending the basic article. This is an article with additional fields that can contain the article images, videos, image galleries or even attachments. Add nested-level categories and tags to that, and you have a very powerful system. In this article we'll look at the installation and use of K2. Installation and introduction of K2 In this recipe you will go through the steps to install and configure K2 in your Joomla! site. Getting ready You will want to download K2 from the following URL: http://getk2.org Additionally, you may want to have a development site to install and learn this on rather than your production site. How to do it... Installation of K2 works like any other Joomla! extension. Be sure and backup your files and database before beginning - this way you can easily roll back should something go wrong. Download K2. Log in to your Joomla! site as the Super Administrator. Click Extensions | Install / Uninstall. Browse, locate, and upload the K2 package. Install.Installation of K2 should be complete. If you are not running the Joom!Fish language translation extension on your site, you will see an informational message stating that K2 did not install the content elements. Joom!Fish is used to translate your site into other languages. If you have no plans on using this, then ignore the message. Now when you go to Components, you'll see the K2 icon. Clicking it will show you this screen: The next step is to bring all of our current content (articles, media, and so on) into K2. While in the K2 Dashboard, look at the upper right-hand side for the Import Joomla! content button. You will see this message (or a similar one depending on your K2 version): Click Ok to start the process. Once complete, you'll see all your content now shows up in the K2 Items menu. Here are the sample Joomla! data items that have been imported. Keep in mind that all of your pre-existing Joomla! sections, categories, and articles are still in the core Joomla! areas. All these items are simply duplicated in K2 now. If you have existing content take a few minutes and go into the Article Manager of Joomla! and unpublish them. You may need to update menu links to your new K2 categories and articles as necessary. Additionally, you'll note that the main Joomla! administrator page has been updated to include the K2 dashboard. Further configuration of K2 K2 has a powerful, nested-category system. It offers you the ability to configure in detail, each category, allowing any category to inherit from another category's parameter settings. Let's use this feature of K2 to configure a master category that will allow other categories to inherit the settings. This means you can change one category and affect them all at the same time – which is quite time-saving. A word of caution, this could impact your search engine-friendly URLs depending on the extension you use to create them. Open the K2 Dashboard. Select Categories. Click New in the upper right-hand side. Fill in the following details: Title is the title of your category Title Alias is the internal reference Parent category should be left as –None-- - all other categories will inherit from this category Inherit parameter options from category should be left as –None-- Associated "Extra Fields" Group should be left as None. Published is Yes Access Level is Public I have filled mine out and the following screenshot is the completed Details section. For now, this is all you'll need to do. Later recipes will utilize this: This is my master category that all other categories will seek out their parameters from. The reason for this will be clear shortly. When you have finished, save the category. How it works... K2 is nearly a content management system, like Joomla!, in itself. It actually substitutes many of the native Joomla! functions like article management, section, and category as well as providing some access control functions. When using K2, you will no longer use many of the native Joomla! menus, but rather you will use the K2 version of the tools. If you install this component, it has the ability to pull in all the data (users, articles) into K2 through the import feature. There's more... In the previous recipe, we set up a master category; now we'll configure it here.

In this article by Anand Balachandran Pillai, the author of the book Software Architecture with Python, learn to build complex software architecture for a software application using Python. (For more resources related to this topic, see here.) Imagine the checkout counter of a supermarket on a Saturday evening, the usual rush hour time. It is common to see long queues of people waiting to check out with their purchases. What would a store manager do to reduce the rush and waiting time? A typical manager would try a few approaches includingtelling those manning the checkout counters to pick up their speed and try and redistribute people to different queues so that each queue roughly has the same wait time. In other words, the manager would manage the current load with available resources by optimizing performance of existing resources. However, if the store has existing counters which are not in operation and enough people at hand to manage them, the manager could enable those counters and move people to these new counters,in other words, add resources to the store to scalethe operation. Software systems too scale in a similar way. An existing software application can be scaled by adding compute resources to it. When the system scales by either adding or making better use of resources inside a compute node, such as CPU or RAM, it is said to scale verticallyor scale up. On the contrary, when a system scales by adding more compute nodes to it, such as a creating a load balanced cluster, it is said to scale horizontallyor scale out. The degree to which a software system is able to scale when compute resources are added is called its scalability. Scalability is measured in terms of how much the systems performance characteristics, such as throughput or latency, improve with respect to the addition of resources. For example, if a system doubles its capacity by doubling the number of servers, it is scaling linearly. Increasing the concurrency of a system often increases its scalability. In the preceding supermarket example, the manager is able to scale out his operations by opening additional counters. In other words, he increases the amount of concurrent processing done in his store. Concurrency is the amount of work that gets done simultaneously in a system. We look at different techniques of scaling a software application with Python. We start with concurrency techniques within a machine, such as multithreading and multiprocessing, and go on to discuss asynchronous execution. We also look at how to scale outan application across multiple servers and also some theoretical aspects of scalability and its relation to availability. Scalability andperformance How do we measure the scalability of a system? Let's take an example and see how this could be done. Let's say our application is a simple report generation system for employees. It is able to load employee data from a database and generate a variety of reports at bulk, such as payslips, tax deduction reports, employee leave reports, and so on. The system is able to generate 120 reports per minute––this is the throughputor capacityof the system expressed as the number of successfully completed operations in a given unit of time. Let's say the time it takes to generate a report at the server side (latency) is roughly 2seconds. Let's say, the architect decides to scale up the system by doubling the RAM on its server ––scaling upthe system. Once this is done, a test shows that the system is able to increase its throughput to 180 reports per minute. The latency remains the same at 2 seconds. So at this point, the system has scaled close to linearin terms of the memory added. The scalability of the system expressed in terms of throughput increase is as follows: Scalability (throughput) = 180/120 = 1.5X As the second step, the architect decides to double the number of servers on the backend all with the same memory. After this step, he finds that the system's performance throughput has now increased to 350 reports per minute. The scalability achieved by this step is as follows: Scalability (throughput) = 350/180 = 1.9X The system has now responded much better, with a close to linear increase in scalability. After further analysis, the architect finds that by rewriting the code that was processing reports on the server to run in multiple processes instead of a single process, he is able to reduce the processing time at the server and hence the latency of each request by roughly 1 second per request at peak time. The latency has now gone down from 2seconds to 1 second. The system's performance with respect to latency has become better,as follows: Performance (latency) X = 2/1 = 2X How does this affect thescalability ? Since the latency per request has come down, the system overall would be able to respond to similar loads at a faster rate (since processing time per request is lesser now) than what it was able to earlier. In other words, with the exact same resources, the system's throughput performance, and hence scalability, would have increased, assuming other factors remain the same. Let's summarize what we discussed so far in the following lists: First, the architect increased the throughput of a single system by scaling it up by adding extra memory as a resource, which increased the overall scalability of the system. In other words, he scaled the performance of a single system by scaling up which boosted overall performance of the whole system. Next, he added more nodes to the system, and hence its ability to perform work concurrently, and found that the system responded well by rewarding him with a near linear scalability factor. Simply put, he increased the throughput of the system by scaling its resource capacity. In other words, he increased scalability of the system by scaling out by adding more compute nodes. Finally, he made a critical fix by running a computation in more than one process. In other words, he increased the concurrency of a single system by dividing the computation to more than one part. He found that this increased the performance characteristic of the application by reducing its latency, potentially setting up the application to handle workloads better at high stress. We find that there is a relation between scalability, performance, concurrency, and latency as follows: When performance of a single system goes up, the scalability of the total system goes up When an application scales in a single machine by increasing its concurrency, it has the potential to improve performance and hence the net scalability of the system in deployment When a system reduces its performance time at server or its latency,it positively contributes to scalability We have captured the relationships between concurrency, latency, performance and scalability in the following table: Concurrency Latency Performance Scalability High Low High High High High Variable Variable Low High Poor Poor An ideal system is one which has good concurrency and low latency––a system that has high performance and would respond better to scaling up and/or scaling out. A system with high concurrency but also high latency would have variable characteristics,its performance, and hence scalability, would be potentially very sensitive to other factors such as network load, current system load, geographical distribution of compute resources and requests,and so on. A system with low concurrency and high latency is the worst case––it would be difficult to scale such a system as it has poor performance characteristics. The latency and concurrency issues should be addressed before the architect decides to scale the system either horizontally or vertically. Scalability is always described in terms of variation in performance throughput. Concurrency A system's concurrency is the degree to which the system is able to perform work simultaneously instead of sequentially. An application written to be concurrent in general can execute more units of work in a given time than one which is written to be sequential or serial. When wemake a serial application concurrent, we make the application make better use of existing compute resources in the system––CPU and/or RAM at a given time. Concurrency in other words is the cheapest way of making an application scale inside a machinein terms of the cost of compute resources. Concurrency can be achieved using different techniques. The common techniquesare as follows: Multithreading: The simplest form of concurrency is to rewrite the application to perform parallel tasks in different threads. A thread is the simplest sequence of programming instructions that can be performance by a CPU. A program can consist of any number of threads. By distributing tasks to multiple threads, a program can execute more work simultaneously. All threads run inside the same process. Multiprocessing: The next step of concurrency is to scale the program to run in multiple processes instead of a single process. Multiprocessing involves more overhead than multithreading in terms of message passing and shared memory. However, programs that perform a lot of latent operations such as disk reads and those which perform lot of CPU heavy computation can benefit more from multiple processes than multiple threads. Asynchronous Processing: In this technique, operations are performed asynchronously,in other words, there is no ordering of concurrent tasks with respect to time. Asynchronous processing picks tasks usually from a queue of tasks and schedules them to execute at a future time, often receiving the results in callback functions or special future objects. Typically, operations are performed in a single thread. There are other forms of concurrent computing, but in this article, we will focus our attention to only these three, hence we are not introducing any other types of concurrent computing here. Python, especially Python3, has built-in support for all these types of concurrent computing techniques in its standard library. For example, it supports multithreading via its threading module and multiple processes via its multiprocessing module. Asynchronous execution support is available via the asynciomodule. A form of concurrent processing which combines asynchronous execution with threads and processes is available via the concurrent.futuresmodule. Concurrency versusparallelism We will take a brief look at the concept of concurrency and its close cousin, namely parallelism. Both concurrency and parallelism are about executing work simultaneously than sequentially. However, in concurrency, the two tasks need not be executing at the exact same time. Instead, they just need to be scheduled to be executed simultaneously. Parallelism, on the other hand, requires that both the tasks execute together at a given pointin time. To take a real-life example, let's say you are painting two exterior walls of your house. You have employed just one painter and you find that he is taking a lot more time than you thought. You can solve the problem in two ways. Instruct the painter to paint a few coats on one wall before switching to the next wall and doing the same there. Assuming he is efficient, he will work on both walls simultaneously (though not at the same time) and achieve same degree completion on both walls for a given time. This is a concurrent solution. Employ one more painter, and instruct first painter to paint the first wall and second painter to paint the second wall. This is a truly parallel solution. For example, two threads performing byte code computations in a single core CPU are not exactly performing parallel computation as the CPU can accommodate only one thread at a time. However, from a programmer's perspective, they are concurrent, since the CPU scheduler performs fast switching in and out of the threads, so theylook parallel in all appearances and purposes. But they are not truly parallel. However on a multi-core CPU, two threads can perform parallel computations at any given time in its different cores. This is true parallelism. Parallel computation requires computation resources to increase at least linearly with respect to its scale. Concurrent computation can be achieved using techniques of multi-tasking where work is scheduled and executed in batches, making better use of existing resources. In this article, we will use the term concurrent nearly uniformly to indicate both types of execution. In some places, it may indicate concurrent processing in the traditional way, and in some other, it may indicate true parallel processing. Kindly use the context to disambiguate. Concurrency in Python –multithreading We will start our discussion on concurrent techniques in Python with multithreading. Python supports multiple threads in programming via its threading module. The threading module exposes a Threadclass thatencapsulates a thread of execution. Along with it, it also exposes the following synchronization primitives: Lock object:This is useful for synchronized, protected access to share resources and its cousin RLock Condition object:This is useful for threads to synchronize while waiting for arbitrary conditions Event object: This provides a basic signaling mechanism between threads Semaphore object:This allows synchronized access to limited resources Barrier object:This allows a set of fixed number of threads to wait for each other and synchronize to a particular state and proceed The thread objects in Python can be combined with the synchronized Queueclass in the queue module for implementing thread-safe producer/consumer workflows. Thumbnail generator Let's start our discussion of multithreading in Python with the example of a program which is used to generate thumbnails of image URLs. We use the Python Imaging Library (PIL) for performing the following operation: # thumbnail_converter.py from PIL import Image import urllib.request def thumbnail_image(url, size=(64, 64), format='.png'): """ Save thumbnail of an image URL """ im = Image.open(urllib.request.urlopen(url)) # filename is last part of the URL minus extension + '.format' pieces = url.split('/') filename = ''.join((pieces[-2],'_',pieces[-1].split('.')[0],'_thumb',format)) im.thumbnail(size, Image.ANTIALIAS) im.save(filename) print('Saved',filename) This works very well for single URLs. Let's say we want to convert five image URLs to their thumbnails as shown in the following code snippet: img_urls = ['https://dummyimage.com/256x256/000/fff.jpg', 'https://dummyimage.com/320x240/fff/00.jpg', 'https://dummyimage.com/640x480/ccc/aaa.jpg', 'https://dummyimage.com/128x128/ddd/eee.jpg', 'https://dummyimage.com/720x720/111/222.jpg'] The code forusing the preceding function would be as follows: for url in img_urls: thumbnail_image(urls) Let's see how such a function performs with respect to the time taken: Let's now scale the program to multiple threads so that we can perform the conversions concurrently. Here is the rewritten code to run each conversion in its own thread (not showing the function itself as it hasn't changed): import threading for url in img_urls: t=threading.Thread(target=thumbnail_image,args=(url,)) t.start() Take a look at the response time of the threaded thumbnail convertor for five URLs as shown in the following screenshot: With this change, the program returns in 1.76 seconds, almost equal to the time taken by a single URL in the serial execution we saw earlier. In other words, the program has now linearly scaled with respect to the number of threads. Note that we had to make no change to the function itself to get this scalability boost. Summary In this article, you learned the importance of writing Scalable applications. We also saw the relationships between concurrency, latency, performance, and scalability and the techniques we can use to achieve concurrency. You also learned how to generate the thumbnail of image URLs using PIL. Resources for Article: Further resources on this subject: Putting the Fun in Functional Python [article] Basics of Jupyter Notebook and Python [article] Jupyter and Python Scripting [article]

Here is what you missed this week on Packt Hub - Tech news, tutorials and insights. Tutorials Implementing GANs, OpenCV, BFS algorithms and more tutorials for machine learning folks this week. Web developers can learn to implement Dockers with Microservices, JS micro-optimizations and Selenium tutorials from this week’s tutorials. Data Tutorials Predicting Bitcoin price from historical and live data Creating a reference generator for a job portal using Breadth First Search (BFS) algorithm Datasets and deep learning methodologies to extend image-based applications to videos Generative Models in action: How to create a Van Gogh with Neural Artistic Style... [Editor's Pick] 3 ways to deploy a QT and OpenCV application Web Development Tutorials How to build Dockers with microservices 6 JavaScript micro-optimizations you need to know How to work with the Selenium IntelliJ IDEA plugin How to handle exceptions and synchronization methods with Selenium WebDriver API [Editor's Pick] Other Tutorials Creating a custom layout implementation for your Android app Insights & Opinions See why Jupyter notebooks are hot and Oracle databases are not on Data insights this week. Learn why DDD (domain driven design) is trending and why programmers dread threads on insights for web developers this week. Data Insights & Opinions Why Oracle is losing the Database Race Top 10 Tools for Computer Vision Paper in Two minutes: Attention Is All You Need 10 reasons why data scientists love Jupyter notebooks [Editor's Pick] Top 5 programming languages for crunching Big Data effectively Why DeepMind made Sonnet open source Top 5 free Business Intelligence tools Paper in Two minutes: i-RevNet, a deep invertible convolutional network Web Development Insights & Opinions Concurrency programming 101: Why do programmers hang by a thread? What is domain driven design? [Editor's Pick] Other Insights and Opinions The key differences between Kubernetes and Docker Swarm News Data News Polaris GPS: Rubrik’s new SaaS platform for data management applications Google Employees Protest against the use of Artificial Intelligence in Military CockroachDB 2.0 is out! Huawei launches HiAI Apple steals AI chief from Google Emoji Scavenger Hunt showcases TensorFlow.js D3 5.0 is out! The 5 biggest announcements from TensorFlow Developer Summit 2018 [Editor's Pick] Web Development News SurveyJS leaves beta Sails.js 1.0 has arrived on the shores Other News Microsoft commits $5 billion to IoT projects Netflix releases FlameScope Introducing MapD Cloud, the first Analytics Platform with GPU Acceleration on Cloud Coinbase Commerce API launches AWS Sydney Summit 2018 is all about IoT Kali Linux 2018.1 released [Editor's Pick]

Introducing Microsoft Windows Mobile There exist several types of smart devices in the market including Smart Phones, Pocket PCs, Pocket PC Phones, Tablet PCs, etc. Every smart device is installed with a mobile‑based operating system with respect to the features of the device. One of such operating systems is Microsoft Windows CE. Microsoft Windows CE is a small, embedded operating system (runs from ROM) that has a look and feel similar to Microsoft Windows 95/98. It includes scaled down versions of Microsoft Excel, Microsoft Word, Microsoft Internet Explorer, etc. Microsoft Windows Mobile (Windows Mobile in short) is a complete software platform built on Windows CE. Unlike Windows CE, the Windows Mobile for Smart Phone or Pocket PC operating systems is specifically designed for devices that require a specialized hardware configuration. The software includes standardized interfaces and applications that ensure compatibility across hardware designs. The Pocket PC is the best example device that gets equipped with Microsoft Windows Mobile operating system. The Pocket PC runs Windows CE as its core operating system. Pocket PCs come with mobile versions of Microsoft Office applications in addition to Microsoft Outlook Mobile. Though there are different Pocket PCs, many come with Wi-Fi to enable you to connect to the Internet when you are near to a wireless hotspot. You can compose email messages and send them wirelessly or by synchronizing with your desktop computer. A Pocket PC Phone is a bit different from an ordinary Pocket PC. You can do everything with a Pocket PC Phone that you can do with a Pocket PC, but with the addition of cellular phone capabilities. If you have a Pocket PC Phone, you can access the Internet through the GPRS service. A Smart Phone has phone capabilities and comes with a smaller set of applications. Though you can add third-party software titles to your Smart Phone, the smaller keypad and screen are designed to give you quick one-handed access to important data. A Smart Phone is a good choice for business users who need to check email, keep track of their calendars, and take voice notes. Microsoft.NET enables us to develop and deploy .NET applications on Microsoft Windows Mobile-enabled smart devices like Smart Phones, Pocket PCs, Tablet PCs, etc. To develop for either Smart Phones or Pocket PCs, we need not really buy those devices. We simply need to have smart device client extensions installed as a part of Visual Studio 2005 (which automatically installs .NET Compact Edition). When the extensions are installed, we are provided with few device emulators for developing and testing .NET-based mobile applications. However, for testing and production, it is recommended to have physical smart devices. The next section focuses on developing a simple Pocket PC application, which consumes the web service developed previously. Consuming a Web Service from Pocket PC Now, let us make use a web service for the Pocket PC. You need not have a physical Pocket PC in your hands to test it. We can simply use existing emulators available as part of Visual Studio 2005. The following are the steps: Open Visual Studio 2005 Environment    Go to File | New | Project.    Select and provide information as shown in the following figure:      Add a Web Reference for the web service you created.    Drag and drop a DataGrid on to the Pocket PC emulator as shown below:      Modify the existing code as follows: Public Class Form1 Private Sub Form1_Load(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles MyBase.Load Me.DataGrid1.DataSource = (New EmpService.Service).getList.Tables(0) End SubEnd Class     Press F5, and select any Emulator for deployment. The output should look like the following: Conclusion We have seen the deployment of .NET-based application on to smart devices enabled with Microsoft Windows Mobile operating system.

  Moodle 2.0 Multimedia Cookbook Add images, videos, music, and much more to make your Moodle course interactive and fun         Read more about this book       (For more resources on Moodle 2.0, see here.) Introduction Whenever you think of a map, you may either think of the traditional planisphere or the terrestrial globe. There are several types of maps apart from those previously mentioned. We can work with maps of the moon, Mars, constellations, and even the universe! Thus, we are not only going to focus on our planet, but we are going to travel even further! The topic of this article is going to deal with Traveling Around the World and Watching the Universe. After reading this article, you can focus on your next holiday! We explain how to work with different types of maps. We are going to be as creative as possible. We should try to work with maps in an unconventional way. That is to say, the idea is to use a map for a Geography class, but we can use maps as a resource for any type of activity. Thus, we can work with the Geography teacher and he/she could work on another geographical feature of the place that we are working with. Therefore, in that way, we are adding more information to the place we are exploring. Maps are very attractive and they may become quite appealing to our students as long as we find a way to develop a rich activity using them. We should encourage the use of maps and the available resources that we have on the Web so that they can insert them in their homework by themselves as well. Thus, we can develop the activities in such a way that we can either provide the map or ask them to design a map. We can also work with maps in the case of Literature. We can ask students to draw a map of a place that has never existed in the real world, though it did in a story. Thus, another bit of homework that could prove helpful would be for students to design and carry out the map of such a place using the tools that we are going to explore in the following recipes. An example of this could be to draw the map of the country Ruritania and locate the cities of Zenda and Strealsau. These places do not exist in the real world, but they exist in the book The Prisoner of Zenda by Anthony Hope. So, many things can be done with maps. Creating maps with sceneries In this activity, we are going to create a map with sceneries. Therefore, we could either browse our files for pictures from our trips or holidays, or we can search for sceneries on the Web. After selecting the pictures, we create a new folder in Windows Explorer, for example C:Images_Traveling. In this folder, we save all the pictures so as to organize our work. We will use the following well-known website: http://earth.google.com/ to design a map using the pictures we have saved in the folder that we have just created. Let's get ready! Getting ready In this activity, we will work with the previously mentioned website. Therefore, we need to open the web browser and enter it. Click on Download Google Earth 6. Read the Google Maps/Earth Terms of Service and if you agree, click on Agree and Download. The icon of Google Earth will appear on your desktop, as shown in the following screenshot: How to do it... We have already carried out the first steps for this activity. Now, we have to design the maps with the pictures that we want to add. There are also some pictures that are available in the maps; you can also work with them, though the aim of this activity is to upload images in the map. Follow these steps in order to create a folder and find images for the activity: Click on the icon on your desktop and open Google Earth. Bring the Earth closer with the icons on the right. Locate a remote city in the southern hemisphere, as shown in the following screenshot: In the Fly to block, write "Mar del Plata", or any other remote city. Then press Enter or click on the magnifying glass next to the block. You will travel virtually to the desired city. Bring the map forward and locate the place where the picture to be uploaded was taken. Click on Add | Photo. Complete the Name block. Click on Browse. Search for the picture that you want to upload and click on it. Complete the other blocks: Description | View | Photo. Click on OK. The picture will appear, as shown in the following screenshot: You can repeat the same process as many times as the number of pictures you want to upload. After uploading all the pictures, click on File | Save | Save Image, as shown in the following screenshot: Complete the File name block and click on Save. How it works... After uploading the desired pictures to the map, we can create an activity. We could start this course with a little social interaction. We ask our students to think about what element they shouldn't forget if they happen to go to this place. They may not know this city, for sure, unless they live nearby. This is the most interesting part of inserting a remote city that they may want to know more about it! Therefore, a Chat is a good idea to have where all the students will be invited in order to discuss the city. We upload the map that we have created with the images to our activity within the Moodle course. Choose the weekly outline section where you want to insert this activity and follow these steps: Click on Add an activity | Chat. Complete the Name of this chat room and Introduction text blocks. Click on the Insert/edit image icon | Find or upload an image | Browse and look for the image that we have just saved. Click on Upload this file. Complete the Image description block and click on Insert. Click on Save and return to course. The activity looks as shown in the following screenshot: Drawing regions within a map In this activity, we are going to use an interactive website in which we choose a map to work with. It is a very simple one, but we could enhance it by adding interesting ingredients to the recipe. We will use a software for drawing a region on the map, and highlight a region for our students to work with. As it was pointed out before, we are not going to focus on geographical features, though you can add this ingredient yourself when designing the activity. Getting ready We open our default web browser and work with the following website: http://www.fusioncharts.com/maps/Default.asp. We click on Map Gallery and choose a map to work with. In this case, we choose a map of the world and highlight five regions, one for each continent. You can modify it and work with different regions within a continent or a country too. How to do it... We look for the desired map. We can find different types of maps to work with. Everything depends on what type of activity we have in mind. In this case, as the topic of this article has to do with traveling, we circle five regions and ask our students to choose where they would like to go. First of all, we have to find the map and save it as an image so that we can draw the regions and upload it to our Moodle course. Therefore, follow these steps: Click on click here | World Map with countries on the aforementioned site. Another pop-up window appears, displaying a map of the world with the countries. There appears a Map Configuration block where you can customize some features, as shown in the next screenshot. Click on Save Map as Image, as shown in the following screenshot: Another pop-up window will appear. Click on Save. Complete the File name block. Click on Save. Click on Open. A pop-up window displaying the map will appear. Click on File | Copy. Paste the map in Paint or Inkscape. Click on Edit | Paste from and browse for the name of the file. Select the file and click on Open. Use the resources available to draw the regions that you want students to work with, as shown in the following screenshot: Click on File | Save as and write a name for the file. Click on Save. How it works... We have already drawn the regions that we want our students to work with. We have chosen one country from every continent; you can choose another or design it in a different way. We can add a writing activity in which students choose where they would like to travel using the previous map. Select the weekly outline section where you want to add the activity and follow these steps: Click on Add an activity | Upload a single file within Assignments. Complete the Assignment name and Description blocks. Click on the Insert/edit image icon | Find or upload an image | Browse. When you find the image that you want to upload, click on Open | Upload this file. Complete the Image description block. Click on Insert. Click on Save and return to course. The activity is ready! When students click on the activity, it looks as shown in the following screenshot: Labeling a map with pins In this recipe, we will learn how to insert a map in our Moodle course labeled with pins, because we pin all the cities that we are going to work with. Therefore, we insert the map as a resource. After that, we design activities for our students to use the interactive map that we have just added. It is another way to use a resource, making our Moodle course more appealing to the eyes of our students. Getting ready We are going to work with Google Earth, as we did in the first recipe, so we have already installed it. We should think of the cities to insert in our course because we need to pin them all! How to do it... Click on the Google Earth icon that you have on your desktop. This is a way to enrich our traveling course by enhancing its appearance. So, these are the steps that you have to follow: Complete the Fly to block with the place that you want to pin. Click on the yellow pin, as shown in the following screenshot: A pop-up window will appear. Complete the Name block by writing the name of the city. Check the Latitude and Longitude, so that you place the pin correctly. You may complete the Description block. You can change the appearance of the pin by clicking on the pin itself. Another pop-up window will appear showing different sorts of icons, as shown in the following screenshot: You can choose the desired icon by clicking on it | OK. The icon that you have selected will appear in the map. Pin as many cities as you are going to work with and repeat steps 1-7. After pinning all the cities, save the file. Click on File | Save | Save Place as. Complete the File name block (remember to save the file in the folder which was created for this course) | Save. You have already saved the pinned map. How it works... We have to insert the map in our Moodle course. In this case, we are going to Add a resource, because we are introducing all the activities that are to come. So, choose the weekly outline section where you want to save the resource. These are the steps that you have to follow: Click on Add a resource | File. Complete the Name and Description blocks. Click on Add | Browse. Click on the file that you are going to upload | Open | Upload this file | Save and return to course. Although we have added a file, students can work with the map interactively! There's more We can embed the map in an HTML block in our Moodle course. Click on the downwards arrow next to Add... in Add a block, as shown in the following screenshot: Choose HTML and a new block will appear in our Moodle course. Embedding a map in an HTML block Open Google Earth and follow these steps in order to embed the map in the block that we have already added: Click on the View in Google Maps icon, as shown in the following screenshot: Another window appears. Click on Link | Customize and preview embedded map, as shown in the following screenshot: Click on Custom and adjust the Width and Height. In the Preview section, click on the minus sign and adjust the map to fit the window. Copy the HTML code to embed in our Moodle course. Go back to the Moodle course and click on the configuration icon to embed the map. Complete the Block title. In the Content block, click on the HTML icon, paste the HTML code which was copied, and click on Update. Click on Save changes. The map will look as shown in the following screenshot:

  ASP.NET MVC 2 Cookbook A fast-paced cookbook with recipes covering all that you wanted to know about developing with ASP.NET MVC Solutions to the most common problems encountered with ASP.NET MVC development Build and maintain large applications with ease using ASP.NET MVC Recipes to enhance the look, feel, and user experience of your web applications Expand your MVC toolbox with an introduction to lots of open source tools Part of Packt's Cookbook series: Each recipe is a carefully organized sequence of instructions to complete the task as efficiently as possible         Introduction ASP.NET MVC provides a simple, but powerful, framework for validating forms. In this article, we'll start by creating a simple form, and then incrementally extend the functionality of our project to include client-side validation, custom validators, and remote validation. Basic input validation The moment you create an action to consume a form post, you're validating. Or at least the framework is. Whether it is a textbox validating to a DateTime, or checkbox to a Boolean, we can start making assumptions on what should be received and making provisions for what shouldn't. Let's create a form. How to do it... Create an empty ASP.NET MVC 2 project and add a master page called Site.Master to Views/Shared. In the models folder, create a new model called Person. This model is just an extended version of the Person class.Models/Person.cs: public class Person { [DisplayName("First Name")] public string FirstName { get; set; } [DisplayName("Middle Name")] public string MiddleName { get; set; } [DisplayName("Last Name")] public string LastName { get; set; } [DisplayName("Birth Date")] public DateTime BirthDate { get; set; } public string Email { get; set; } public string Phone { get; set; } public string Postcode { get; set; } public string Notes { get; set; } } Create a controller called HomeController and amend the Index action to return a new instance of Person as the view model.Controllers/HomeController.cs: public ActionResult Index() { return View(new Person()); } Build and then right-click on the action to create an Index view. Make it an empty view that strongly types to our Person class. Create a basic form in the Index view.Views/Home/Index.aspx: <% using (Html.BeginForm()) {%> <%: Html.EditorForModel() %> <input type="submit" name="submit" value="Submit" /> <% } %> We'll go back to the home controller now to capture the form submission. Create a second action called Index, which accepts only POSTs.Controllers/HomeController.cs: [HttpPost] public ActionResult Index(... At this point, we have options. We can consume our form in a few different ways, let's have a look at a couple of them now:Controllers/HomeController.cs (Example): // Individual Parameters public ActionResult Index(string firstName, DateTime birthdate... // Model Public ActionResult Index(Person person) { Whatever technique you choose, the resolution of the parameters is roughly the same. The technique that I'm going to demonstrate relies on a method called UpdateModel. But first we need to differentiate our POST action from our first catch-all action. Remember, actions are just methods, and overrides need to take sufficiently different parameters to prevent ambiguity. We will do this by taking a single parameter of type FormCollection, though we won't necessarily make use of it.Controllers/HomeController.cs: [HttpPost] public ActionResult Index(FormCollection form) { var person = new Person(); UpdateModel(person); return View(person); } The UpdateModel technique is a touch more long-winded, but comes with advantages. The first is that if you add a breakpoint on the UpdateModel line, you can see the exact point when an empty model becomes populated with the form collection, which is great for demonstration purposes. The main reason I go back to UpdateModel time and time again, is the optional second parameter, includeProperties. This parameter allows you to selectively update the model, thereby bypassing validation on certain properties that you might want to handle independently. Build, run, and submit your form. If your page validates, your info should be returned back to you. However, add your birth date in an unrecognized format and watch it bomb. UpdateModel is a temperamental beast. Switch your UpdateModel for TryUpdateModel and see what happens. TryUpdateModel will return a Boolean indicating the success or failure of the submission. However, the most interesting thing is happening in the browser. How it works... With ASP.NET MVC, it sometimes feels like you're stripping the development process back to basics. I think this is a good thing; more control to render the page you want is good. But there is a lot of clever stuff going on in the background, starting off with Model Binders. When you send a request (GET, POST, and so on) to an ASP.NET MVC application, the query string, route values and the form collection are passed through model binding classes, which result in usable structures (for example, your action's input parameters). These model binders can be overridden and extended to deal with more complex scenarios, but since ASP.NET MVC2, I've rarely made use of this. A good starting point for further investigation would be with DefaultModelBinder and IModelBinder. What about that validation message in the last screenshot, where did it come from? Apart from LableFor and EditorFor, but we also have ValidationMessageFor. If the model binders fail at any point to build our input parameters, the model binder will add an error message to the model state. The model state is picked up and displayed by the ValidationMessageFor method, but more on that later.

  CakePHP 1.3 Application Development Cookbook Over 70 great recipes for developing, maintaining, and deploying web applications Introduction This article deals with one of the most important aspects of a CakePHP application: the relationship between models, also known as model bindings or associations. Being an integral part of any application's logic, it is of crucial importance that we master all aspects of how model bindings can be manipulated to get the data we need, when we need it. In order to do so, we will go through a series of recipes that will show us how to change the way bindings are fetched, what bindings and what information from a binding is returned, how to create new bindings, and how to build hierarchical data structures. Adding Containable to all models The Containable behavior is a part of the CakePHP core, and is probably one of the most important behaviors we have to help us deal with model bindings. Almost all CakePHP applications will benefit from its functionalities, so in this recipe we see how to enable it for all models. How to do it... Create a file named app_model.php and place it in your app/ folder, with the following contents. If you already have one, make sure that either you add the actsAs property shown as follows, or that your actsAs property includes Containable. <?php class AppModel extends Model { public $actsAs = array('Containable'); } ?> How it works... The Containable behavior is nothing more and nothing less than a wrapper around the bindModel() and unbindModel() methods, defined in the CakePHP's Model class. It is there to help us deal with the management of associations without having to go through a lengthy process of redefining all the associations when calling one of these methods, thus making our code much more readable and maintainable. This is a very important point, because a common mistake CakePHP users make is to think that Containable is involved in the query-making process, that is, during the stage where CakePHP creates actual SQL queries to fetch data. Containable saves us some unneeded queries, and optimizes the information that is fetched for each related model, but it will not serve as a way to change how queries are built in CakePHP. Limiting the bindings returned in a find This recipe shows how to use Containable to specify what related models are returned as a result of a find operation. It also shows us how to limit which fields are obtained for each association. Getting ready To go through this recipe we need some sample tables to work with. Create a table named families, using the following SQL statement: CREATE TABLE `families`( `id` INT UNSIGNED AUTO_INCREMENT NOT NULL, `name` VARCHAR(255) NOT NULL, PRIMARY KEY(`id`) ); Create a table named people, using the following SQL statement: CREATE TABLE `people`( `id` INT UNSIGNED AUTO_INCREMENT NOT NULL, `family_id` INT UNSIGNED NOT NULL, `name` VARCHAR(255) NOT NULL, `email` VARCHAR(255) NOT NULL, PRIMARY KEY(`id`), KEY `family_id`(`family_id`), CONSTRAINT `people__families` FOREIGN KEY(`family_id`) REFERENCES `families`(`id`) ); Create a table named profiles, using the following SQL statement: CREATE TABLE `profiles`( `id` INT UNSIGNED AUTO_INCREMENT NOT NULL, `person_id` INT UNSIGNED NOT NULL, `website` VARCHAR(255) default NULL, `birthdate` DATE default NULL, PRIMARY KEY(`id`), KEY `person_id`(`person_id`), CONSTRAINT `profiles__people` FOREIGN KEY(`person_id`) REFERENCES `people`(`id`) ); Create a table named posts, using the following SQL statement: CREATE TABLE `posts`( `id` INT UNSIGNED AUTO_INCREMENT NOT NULL, `person_id` INT UNSIGNED NOT NULL, `title` VARCHAR(255) NOT NULL, `body` TEXT NOT NULL, `created` DATETIME NOT NULL, `modified` DATETIME NOT NULL, PRIMARY KEY(`id`), KEY `person_id`(`person_id`), CONSTRAINT `posts__people` FOREIGN KEY(`person_id`) REFERENCES `people`(`id`) ); Even if you do not want to add foreign key constraints to your tables, make sure you use KEYs for each field that is a reference to a record in another table. By doing so, you will significantly improve the speed of your SQL queries when the referenced tables are joined. Add some sample data, using the following SQL statements: INSERT INTO `families`(`id`, `name`) VALUES (1, 'The Does'); INSERT INTO `people`(`id`, `family_id`, `name`, `email`) VALUES (1, 1, 'John Doe', 'john.doe@example.com'), (2, 1, 'Jane Doe', 'jane.doe@example.com'); INSERT INTO `profiles`(`person_id`,`website`,`birthdate`) VALUES (1, 'http://john.example.com', '1978-07-13'), (2, NULL, '1981-09-18'); INSERT INTO `posts`(`person_id`, `title`, `body`, `created`, `modified`) VALUES (1, 'John's Post 1', 'Body for John's Post 1', NOW(), NOW()), (1, 'John's Post 2', 'Body for John's Post 2', NOW(), NOW()); We need Containable added to all our models. We proceed now to create the main model. Create a file named person.php and place it in your app/models folder with the following contents: <?php class Person extends AppModel { public $belongsTo = array('Family'); public $hasOne = array('Profile'); public $hasMany = array('Post'); } ?> Create the model Family in a file named family.php and place it in your app/models folder with the following contents: <?php class Family extends AppModel { public $hasMany = array('Person'); } ?> How to do it... When Containable is available for our models, we can add a setting to the find operation called contain. In that setting we specify, in an array-based hierarchy, the associated data we want returned. A special value contain can receive is false, or an empty array, which tells Containable not to return any associated data. For example, to get the first Person record without associated data, we simply do: $person = $this->Person->find('first', array( 'contain' => false )); Another way to tell CakePHP not to obtain related data is through the use of the recursive find setting. Setting recursive to -1 will have exactly the same effect as setting contain to false. If we want to obtain the first Person record together with the Family they belong to, we do: $person = $this->Person->find('first', array( 'contain' => array('Family') )); Using our sample data, the above query will result in the following array structure: array( 'Person' => array( 'id' => '1', 'family_id' => '1', 'name' => 'John Doe', 'email' => 'john.doe@example.com' ), 'Family' => array( 'id' => '1', 'name' => 'The Does' ) ) Let's say that now we also want to obtain all Post records for the person and all members in the family that Person belongs to. We would then have to do: $person = $this->Person->find('first', array( 'contain' => array( 'Family.Person' 'Post' ) )); The above would result in the following array structure (the created and modified fields have been removed for readability): array( 'Person' => array( 'id' => '1', 'family_id' => '1', 'name' => 'John Doe', 'email' => 'john.doe@example.com' ), 'Family' => array( 'id' => '1', 'name' => 'The Does', 'Person' => array( array( 'id' => '1', 'family_id' => '1', 'name' => 'John Doe', 'email' => 'john.doe@example.com' ), array( 'id' => '2', 'family_id' => '1', 'name' => 'Jane Doe', 'email' => 'jane.doe@example.com' ) ) ), 'Post' => array( array( 'id' => '1', 'person_id' => '1', 'title' => 'John's Post 1', 'body' => 'Body for John's Post 1' ), array( 'id' => '2', 'person_id' => '1', 'title' => 'John's Post 2', 'body' => 'Body for John's Post 2' ) ) ) We can also use Containable to specify which fields from a related model we want to fetch. Using the preceding sample, let's limit the Post fields so we only return the title and the Person records for the person's Family, so we only return the name field. We do so by adding the name of the field to the associated model hierarchy: $person = $this->Person->find('first', array( 'contain' => array( 'Family.Person.name', 'Post.title' ) )); The returned data structure will then look like this: array( 'Person' => array( 'id' => '1', 'family_id' => '1', 'name' => 'John Doe', 'email' => 'john.doe@example.com' ), 'Family' => array( 'id' => '1', 'name' => 'The Does', 'Person' => array( array( 'name' => 'John Doe', 'family_id' => '1', 'id' => '1' ), array( 'name' => 'Jane Doe', 'family_id' => '1', 'id' => '2' ) ) ), 'Post' => array( array( 'title' => 'John's Post 1', 'id' => '1', 'person_id' => '1' ), array( 'title' => 'John's Post 2', 'id' => '2', 'person_id' => '1' ) ) ) You may notice that even when we indicated specific fields for the Family => Person binding, and for the Post binding, there are some extra fields being returned. Those fields (such as family_id) are needed by CakePHP, and known as foreign key fields, to fetch the associated data, so Containable is smart enough to include them in the query. Let us say that we also want a person's e-mail. As there is more than a field needed, we will need to use the array notation, using the fields setting to specify the list of fields: $person = $this->Person->find('first', array( 'contain' => array( 'Family' => array( 'Person' => array( 'fields' => array('email', 'name') ) ), 'Post.title' ) )); How it works... We use the contain find setting to specify what type of containment we want to use for the find operation. That containment is given as an array, where the array hierarchy mimics that of the model relationships. As the hierarchy can get deep enough to make array notation complex to deal with, the dot notation used throughout this recipe serves as an useful and more readable alternative. If we want to refer to the model Person that belongs to the model Family, the proper contain syntax for that is Person => Family (we can also use Person.Family, which is more concise.) We also use the fields setting to specify which fields we want fetched for a binding. We do that by specifying an array of field names as part of the binding Containable setting. Containable looks for the contain find setting right before we issue a find operation on a model. If it finds one, it alters the model bindings to be returned by issuing unbindModel() calls on the appropriate models to unbind those relationships that are not specified in the contain find setting. It then sets the recursive find setting to the minimum value required to fetch the associated data. Let us use a practical example to further understand this wrapping process. Using our Person model (which has a belongsTo relationship to Family, a hasOne relationship to Profile, and a hasMany relationship to Post), the following Containable based query: $person = $this->Person->find('first', array( 'contain' => array('Family.Person') )); or the same query using array notation: $person = $this->Person->find('first', array( 'contain' => array('Family' => 'Person') )); is equivalent to the following set of instructions, which do not use Containable, but the built in unbindModel() method available in CakePHP's Model class: $this->Person->unbindModel(array( 'hasOne' => array('Profile'), 'hasMany' => array('Post') )); $person = $this->Person->find('first', array( 'recursive' => 2 )); Not using Containable is not only much more complicated, but can also pose a problem if we decide to alter some of our relationships. In the preceding example, if we decide to remove the Profile binding, or change its relationship type, we would have to modify the unbindModel() call. However, if we are using Containable, the same code applies, without us having to worry about such changes. Format of the contain find parameter We have seen how to use the contain find parameter to limit which bindings are returned after a find operation. Even when its format seems self-explanatory, let us go through another example to have a deeper understanding of Containable's array notation. Assume that we have the models and relationships shown in the following diagram: Transforming that diagram to something the Containable behavior understands is as simple as writing it using an array structure. For example, if we are issuing a find operation on the User model and we want to refer to the Profile relationship, a simple array('Profile') expression would suffice, as the Profile model is directly related to the User model. If we want to refer to the Comment relationship for the Article records the User is an owner of, which belongs to an Article that itself belongs to our User model, then we add another dimension to the structure, which is now represented as array('Article' => 'Comment'). We can already deduce how the next example will look like. Assume we want to obtain the Comment together with the Profile of the User that commented on each Article. The structure will then look like: array('Article' => array('Comment' => array('User' => 'Profile'))). Sometimes we want to simplify the readability, and fortunately the Containable behavior allows the above expression to be rewritten as array('Article.Comment.User.Profile'), which is known as dot notation. However, if you want to change other parameters to the binding, then this syntax would have to be changed to the full array-based expression. Reset of binding changes When you issue a find operation that uses the Containable behavior to change some of its bindings, CakePHP will reset all bindings' changes to their original states, once the find is completed. This is what is normally wanted on most cases, but there are some scenarios where you want to keep your changes until you manually reset them, such as when you need to issue more than one find operation and have all those finds use the modified bindings. To force our binding changes to be kept, we use the reset option in the contain find parameter, setting it to false. When we are ready to reset them, we issue a call to the resetBindings() method added by the Containable behavior to our model. The following sample code shows this procedure: $person = $this->Person->find('first', array( 'contain' => array( 'reset' => false, 'Family' ) )); // ... $this->Person->resetBindings(); Another way to achieve the same result is by calling the contain() method (setting its first argument to the contained bindings, and its second argument to false to indicate that we wish to keep these containments), available to all models that use Containable, issue the find (without, need to use the contain setting), and then reset the bindings: $this->Person->contain(array('Family'), false); $person = $this->Person->find('first'); // ... $this->Person->resetBindings();  

(For more resources related to this topic, see here.) What kind of game will we be making? We are going to make one of the classics, a Tower Defense game. (http://old.casualcollective.com/#games/FETD2.) Our game won't be as polished as the example, but it gives you a solid base to work with and develop further. Mission briefing We will use the cloning tools again to create hordes of enemies to fight. We will also use these tools to create cannons and cannonballs. It's easy to re-use assets from other projects in Scratch 2.0. The new Backpack feature allows you to easily exchange assets between projects. How this works will be demonstrated in this article. Why is it awesome? This example is a lot more involved than the previous one. The final result will be a much more finished game which still leaves plenty of room to adapt and continue building on. While making this game, you will learn how to draw a background and how to make and use different costumes for a single sprite. We will make full use of the cloning technique to create many copies of similar objects. We will also use more variables and another type of variable called List to keep track of all the things going on in the game. You will also learn about a simple way to create movement patterns for computer-controlled objects. Your Hotshot objectives We will divide the articlein to the following tasks based primarily on the game sprites and their behavior: Creating a background Creating enemies Creating cannons Fighting back Mission checklist Click on the Create button to start a new project. Remove the Scratch cat by right-clicking on it and selecting delete. Creating a background Because the placement and the route to walk is important in this kind of game, we will start with the creation of the background. To the left of the Sprites window, you will see a separate picture. Underneath is the word Stage and another word, the name of the picture that's being shown. This picture is white when you start a new project because nothing is drawn on it yet. The following is an example with our background image already drawn in:   Engage thrusters We will draw a grassy field with a winding road running through it when looked at from the top, by going through the following steps: Click on the white image. Next click on the Backdrops tab to get to the drawing tool. This is similar to the Costumes tab for sprites, but the size of the drawing canvas is clearly limited to the size of the stage. Choose a green color and draw a rectangle from the top-left to the bottom-right of the canvas. Then click on the Fill tool and fill the rectangle with the same color to create a grassy background. On top of the field, we will draw a path which the enemies will use to walk on. Switch the Fill tool to a brown color. Draw rectangles to form a curving path as shown in the following screenshot. The background is now done. Let's save our work before moving on. Objective complete – mini debriefing The background is just a pretty picture with no direct functionality in the game. It tells the player what to expect in the game. It will be logical that enemies are going to follow the road that was drawn. We will also use this road as a guideline when scripting the movement path of the enemies. The open spaces between the path make it obvious where the player could place the cannons. Creating enemies We will quickly create an enemy sprite to make use of the background we just drew. These enemies will follow the path drawn in the background. Because the background image is fixed, we can determine exactly where the turns are. We will use a simple movement script that sends the enemies along the path from one end of the stage to the other. Like with the targets in the previous project, we will use a base object that creates clones of itself that will actually show up on stage. Prepare for lift off We will first draw an enemy sprite. Let's keep this simple for now. We can always add to the visual design later. The steps to draw it are as follows: Click on the paintbrush icon to create a new sprite. Choose a red color and draw a circle. Make sure the circle is of proper size compared to the path in the background. Fill the circle with the same color. We name the new sprite enemy1. That's all for now! We will add more to the appearance of the enemy sprite later. The enemy sprite appears as a red circle large enough to fit the path.   Engage thrusters Let's make it functional first with a script. We will place the base enemy sprite at the start of the path and have it create clones. Then we will program the clones to follow the path as shown in the following steps: The script will start when the when <green flag> clicked block is clicked. Place the sprite at the start of the path with a go to x: -240 y: 0 block. Wait for three seconds by using the wait ... secs block to allow the player to get ready for the game. Add a repeat … block. Fill in 5 to create five clones per wave. Insert a create clone of <myself> block. Then wait for two seconds by using the wait ... secs block so the enemy clones won't be spawned too quickly. Before we start moving the clones, we have to determine what path they will follow. The key information here are the points where the path bends in a new direction. We can move the enemies from one bend to another in an orderly manner. Be warned that it may take some time to complete this step. You will probably need to test and change the numbers you are going to use to move the sprites correctly. If you don't have the time to figure it all out, you can check and copy the image with the script blocks at the end of this step to get a quick result. Do you remember how the xy-coordinate system of the stage worked from the last project? Get a piece of paper (or you can use the text editor on your computer) and get ready to take some notes. Examine the background you drew on the stage, and write down all the xy-coordinates that the path follows in order. These points will serve as waypoints. Look at the screenshot to see the coordinates that I came up with. But remember that the numbers for your game could be different if you drew the path differently. To move the enemy sprites, we will use the glide … secs to x: … y: ... blocks. With this block, a sprite will move fluidly to the given point in the given amount of time as shown in the following steps: Start the clone script with a when I start as a clone block. Beyond the starting point, there will be seven points to move to. So stack together seven glide … blocks. In the coordinate slots, fill in the coordinates you just wrote down in the correct order. Double-check this since filling in a wrong number will cause the enemies to leave the path. Deciding how long a sprite should take to complete a segment depends on the length of that segment. This requires a bit of guesswork since we didn't use an exact drawing method. Your most accurate information is the differences between the coordinates you used from point to point. Between the starting point (-240,0) and the first waypoint (-190,0), the enemy sprite will have moved 50 pixels. Let's say we want to move 10 pixels per second. That means the sprite should move to it's new position in 5 seconds. The difference between the first (-190,0) and the second (-190,125) waypoint is 125. So according to the same formula, the sprite should move along this segment of the path in 12.5 seconds. Continue calculating the glide speeds like this for the other blocks. These are the numbers I came up with : 5, 12.5, 17, 26.5, 15.5, 14, and 10.5, but remember that yours may be different You can use the formula new position – old position / 10 = result to figure out the numbers you need to use. To finish off, delete the clone when it reaches the end of the path. Test your script and see the enemies moving along the path. You might notice they are very slow and bunched together because they don't travel enough distances between spawns. Let's fix that by adding a variable speed multiplier. Not only can we easily tweak the speed of the sprites but we can also use this later to have other enemy sprites move at different speeds, as shown in the following steps: Create a variable and make sure it is for this sprite only. Name it multiplier_R. The R stands for red, the color of this enemy. Place set <multiplier_R> to … at the start of the <green flag> script. Fill in 0.3 as a number for the basic enemy. Take the speed numbers you filled in previously and multiply them with the multiplier. Use a ...*... operator block. Place the multiplier_R variable in one slot. Type the correct number in the other slot. Place the calculation in the glide block instead of the fixed number.The completed scripts for enemy movement will look as follows: Objective complete – mini debriefing Test the game again and see how the enemies move much faster, about three times as fast if you have used 0.3 for the multiplier. You can play with the variable number a bit to see the effect. If you decrease the multiplier, the enemies will move even faster. If you increase the number, the enemies will become slower.

Data transformation with External Tables One of the main uses of the External Tables is their support of the ETL process, allowing the user to perform a data load that is transformed to the target format without an intermediate stage table. Let's read an External Table whose contents are: This data can be loaded in a single command to multiple tables. Let's create several tables with the same structure: SQL> desc amount_jan Name Null? Type ----------------- -------- ------------ REGION VARCHAR2(16) AMOUNT NUMBER(3) Now issue the command to send the data from the external table to the different tables. INSERT ALL INTO AMOUNT_JAN (REGION, AMOUNT) VALUES(COUNTRY, JAN) INTO AMOUNT_FEB (REGION, AMOUNT) VALUES(COUNTRY, FEB) INTO AMOUNT_MAR (REGION, AMOUNT) VALUES(COUNTRY, JAN) INTO AMOUNT_APR (REGION, AMOUNT) VALUES(COUNTRY, JAN) INTO AMOUNT_MAY (REGION, AMOUNT) VALUES(COUNTRY, JAN) INTO AMOUNT_JUN (REGION, AMOUNT) VALUES(COUNTRY, JAN) INTO AMOUNT_JUL (REGION, AMOUNT) VALUES(COUNTRY, JAN) INTO AMOUNT_AUG (REGION, AMOUNT) VALUES(COUNTRY, JAN) INTO AMOUNT_SEP (REGION, AMOUNT) VALUES(COUNTRY, JAN) INTO AMOUNT_OCT (REGION, AMOUNT) VALUES(COUNTRY, JAN) INTO AMOUNT_NOV (REGION, AMOUNT) VALUES(COUNTRY, JAN) INTO AMOUNT_DEC (REGION, AMOUNT) VALUES(COUNTRY, JAN)SELECT COUNTRY, JAN, FEB, MAR, APR, MAY, JUN, JUL, AUG, SEP, OCT, NOV, DECFROM REGION_REVENUE; In this example, we will perform a conditional insert to different tables depending on the value of the amount column. We will first create three tables, one for low, another for average, and a third for high amounts: SQL> create table low_amount( 2 region varchar2(16), 3 month number(2), 4 amount number(3));Table created.SQL> create table high_amount as select * from low_amount;Table created. Now we can read the External Table and have the data inserted conditionally to one of three mutually exclusive targets. INSERT ALL WHEN ( JAN <= 500 ) THEN INTO LOW_AMOUNT( REGION, MONTH, AMOUNT) VALUES ( COUNTRY, '01', JAN ) WHEN ( FEB <= 500 ) THEN INTO LOW_AMOUNT( REGION, MONTH, AMOUNT) VALUES ( COUNTRY, '02', FEB ) WHEN ( MAR <= 500 ) THEN INTO LOW_AMOUNT( REGION, MONTH, AMOUNT) VALUES ( COUNTRY, '03', MAR ) WHEN ( APR <= 500 ) THEN INTO LOW_AMOUNT( REGION, MONTH, AMOUNT) VALUES ( COUNTRY, '04', APR ) WHEN ( MAY <= 500 ) THEN INTO LOW_AMOUNT( REGION, MONTH, AMOUNT) VALUES ( COUNTRY, '05', MAY ) WHEN ( JUN <= 500 ) THEN INTO LOW_AMOUNT( REGION, MONTH, AMOUNT) VALUES ( COUNTRY, '06', JUN ) WHEN ( JUL <= 500 ) THEN INTO LOW_AMOUNT( REGION, MONTH, AMOUNT) VALUES ( COUNTRY, '07', JUL ) WHEN ( AUG <= 500 ) THEN INTO LOW_AMOUNT( REGION, MONTH, AMOUNT) VALUES ( COUNTRY, '08', AUG ) WHEN ( SEP <= 500 ) THEN INTO LOW_AMOUNT( REGION, MONTH, AMOUNT) VALUES ( COUNTRY, '09', SEP ) WHEN ( OCT <= 500 ) THEN INTO LOW_AMOUNT( REGION, MONTH, AMOUNT) VALUES ( COUNTRY, '10', OCT ) WHEN ( NOV <= 500 ) THEN INTO LOW_AMOUNT( REGION, MONTH, AMOUNT) VALUES ( COUNTRY, '11', NOV ) WHEN ( DEC <= 500 ) THEN INTO LOW_AMOUNT( REGION, MONTH, AMOUNT) VALUES ( COUNTRY, '12', DEC ) WHEN ( JAN > 500 ) THEN INTO HIGH_AMOUNT( REGION, MONTH, AMOUNT) VALUES ( COUNTRY, '01', JAN ) WHEN ( FEB > 500 ) THEN INTO HIGH_AMOUNT( REGION, MONTH, AMOUNT) VALUES ( COUNTRY, '02', FEB ) WHEN ( MAR > 500 ) THEN INTO HIGH_AMOUNT( REGION, MONTH, AMOUNT) VALUES ( COUNTRY, '03', MAR ) WHEN ( APR > 500 ) THEN INTO HIGH_AMOUNT( REGION, MONTH, AMOUNT) VALUES ( COUNTRY, '04', APR ) WHEN ( MAY > 500 ) THEN INTO HIGH_AMOUNT( REGION, MONTH, AMOUNT) VALUES ( COUNTRY, '05', MAY ) WHEN ( JUN > 500 ) THEN INTO HIGH_AMOUNT( REGION, MONTH, AMOUNT) VALUES ( COUNTRY, '06', JUN ) WHEN ( JUL > 500 ) THEN INTO HIGH_AMOUNT( REGION, MONTH, AMOUNT) VALUES ( COUNTRY, '07', JUL ) WHEN ( AUG > 500 ) THEN INTO HIGH_AMOUNT( REGION, MONTH, AMOUNT) VALUES ( COUNTRY, '08', AUG ) WHEN ( SEP > 500 ) THEN INTO HIGH_AMOUNT( REGION, MONTH, AMOUNT) VALUES ( COUNTRY, '09', SEP ) WHEN ( OCT > 500 ) THEN INTO HIGH_AMOUNT( REGION, MONTH, AMOUNT) VALUES ( COUNTRY, '10', OCT ) WHEN ( NOV > 500 ) THEN INTO HIGH_AMOUNT( REGION, MONTH, AMOUNT) VALUES ( COUNTRY, '11', NOV ) WHEN ( DEC > 500 ) THEN INTO HIGH_AMOUNT( REGION, MONTH, AMOUNT) VALUES ( COUNTRY, '12', DEC )SELECT COUNTRY, JAN, FEB, MAR, APR, MAY, JUN, JUL, AUG, SEP, OCT, NOV, DECFROM REGION_REVENUE; Extending the alert.log analysis with External Tables Reading the alert.log from the database is a useful feature which can help you to find any outstanding error messages reported in this file. create table ALERT_LOG ( text_line varchar2(512)) organization external ( type ORACLE_LOADER default directory BACKGROUND_DUMP_DEST access parameters( records delimited by newline nobadfile nodiscardfile nologfile ) location( 'alert_beta.log') ); Once the External Table has been created, the alert.log file can be queried just like any other regular table. SQL> select text_line from alert_log 2 where text_line like 'ORA-%';TEXT_LINE-----------------------------------------------------------------ORA-1109 signalled during: ALTER DATABASE CLOSE NORMAL...ORA-00313: open failed for members of log group 1 of thread 1ORA-00312: online log 1 thread 1: '/u01/oracle/oradata/beta/redo01.log'ORA-27037: unable to obtain file statusORA-00313: open failed for members of log group 2 of thread 1ORA-00312: online log 2 thread 1: '/u01/oracle/oradata/beta/redo02.log'ORA-27037: unable to obtain file statusORA-00313: open failed for members of log group 3 of thread 1ORA-00312: online log 3 thread 1: '/u01/oracle/oradata/beta/redo03.log'ORA-27037: unable to obtain file status Querying the alert.log file up to this phase is useful just to see the contents of the file and look for basic ORA-% strings. This could also be achieved by using the alert.log link in the Enterprise Manager (EM). The alert.log file can be queried by means of the EM, but as this can only be viewed from the EM in an interactive mode, you can only rely on the preset alerts. If further automatic work needs to be done, then it is useful to do some more work with the alert analysis tool. A temporary table can be used to store the contents of the ALERT_LOG table, along with an extra TIMESTAMP column, so it can be queried in detail in an EM-like manner. create global temporary table TMP_ALERT_LOG ( LINE_NO NUMBER(6), TIMESTAMP DATE, TEXT_LINE VARCHAR2(512))on commit preserve rows; A bit of PLSQL programming is necessary so the ALERT_LOG file can be modified and inserted into the TMP_ALERT_LOG, (enabling further queries can be done). declarecursor alertLogCur is select ROWNUM, TEXT_LINE from ALERT_LOG;currentDate date;altertLogRec ALERT_LOG.TEXT_LINE%TYPE;testDay varchar2(10);begincurrentDate := sysdate;for alertLogInst in alertLogCur loop -- fetch row and determine if this is a date row testDay := substr(alertLogInst.text_line, 1, 3); if testDay = 'Sun' or testDay = 'Mon' or testDay = 'Tue' or testDay = 'Wed' or testDay = 'Thu' or testDay = 'Fri' or testDay = 'Sat' then -- if this is a date row, it sets the current logical record date currentDate := to_date( alertlogInst.text_line, 'Dy Mon DD HH24:MI:SS YYYY'); end if; insert into TMP_ALERT_LOG values( alertLogInst.rownum, currentDate, alertLogInst.text_line );end loop;end;/ As the contents of the alert.log end up in a temporary table, more than one DBA can query it at the same time, or restrict the DBA's accessibilities. There is no need to manage the purge and maintenance of the table after the session has ended, it can be indexed and there is little overhead by means of this procedure. Moreover, as this is a temporary object, minimum redo log information is generated. Once the external ALERT_LOG and the temporary ALERT_LOG tables have been created, it is possible to perform, not only filters by date (provided by Enterprise Manager) but also any query against the alert.log file. SELECT TIMESTAMP, TEXT_LINEFROM TMP_ALERT_LOGWHERE TIMESTAMP IN ( SELECT TIMESTAMP FROM TMP_ALERT_LOG WHERE TEXT_LINE LIKE 'ORA-%')AND TIMESTAMP BETWEEN SYSDATE-30 AND SYSDATEORDER BY LINE_NO; Further treatment can be done on this concept to look for specific error messages, analyze specific time frames and perform drill down analysis. This procedure can be extended to read the trace files or any other text file from the database. Reading the listener.log from the database One particular extension of the above procedure is to read the listener.log file. This file has a specific star-delimited field file format which can be advantageous, and eases the read by means of the Loader driver. The file format is as follows: 21-JUL-2008 00:39:50 * (CONNECT_DATA=(SID=beta)(CID=(PROGRAM=perl)(HOST=alpha.us.oracle.com)(USER=oracle))) * (ADDRESS=(PROTOCOL=tcp)(HOST=192.168.2.10)(PORT=8392)) * establish * beta * 021-JUL-2008 00:39:56 * (CONNECT_DATA=(SID=beta)(CID=(PROGRAM=perl)(HOST=alpha.us.oracle.com)(USER=oracle))) * (ADDRESS=(PROTOCOL=tcp)(HOST=192.168.2.10)(PORT=8398)) * establish * beta * 021-JUL-2008 00:40:16 * service_update * beta * 021-JUL-2008 00:41:19 * service_update * beta * 021-JUL-2008 00:44:43 * ping * 0 The file has a format that can be deduced from the above data sample: TIMESTAMP * CONNECT DATA [* PROTOCOL INFO] * EVENT [* SID] * RETURN CODE As you can see this format, even though it is structured, it may have a different number of fields, so at loading time this issue must be considered. In order for us to map this table to the database, we should consider the variable number of fields to have the External Table created. We'll create a temporary table so that this doesn't create an additional transactional overhead. Now, let's create an External Table based on this format that points to $ORACLE_HOME/network/log: create directory NETWORK_LOG_DIRas '$ORACLE_HOME/network/log'; Now, let's create the external table: create table LISTENER_LOG ( TIMESTAMP date, CONNECT_DATA varchar2(2048), PROTOCOL_INFO varchar2(64), EVENT varchar2(64), SID varchar2(64), RETURN_CODE number(5))organization external ( type ORACLE_LOADER default directory NETWORK_LOG_DIR access parameters ( records delimited by NEWLINE nobadfile nodiscardfile nologfile fields terminated by "*" LDRTRIM reject rows with all null fields ( "TIMESTAMP" char date_format DATE mask "DD-MON-YYYY HH24:MI:SS ", "CONNECT_DATA", "PROTOCOL_INFO", "EVENT", "SID", "RETURN_CODE" ) ) location ('listener.log'))reject limit unlimited; The structure of interest is specified above, so there will be several rows rejected. Seeing as this file is not fully structured, you will find some non formatted information; the bad file and the log file are not meaningful in this context. Another application of the LISTENER_LOG External Table is usage trend analysis. This query can be issued to detect usage peak hours. SQL> select to_char(round(TIMESTAMP, 'HH'), 'HH24:MI') HOUR, 2 lpad('#', count(*), '#') CX 3 from listener_log 4 group by round(TIMESTAMP, 'HH') 5 order by 1;HOUR CX----- ------------------------------------------------14:00 ###15:00 ##########################16:00 ######################17:00 #####################18:00 #####################19:00 ############### Reading the listener.log file this way allows the DBA not only to keep track of the listener behavior, but also it allows a security administrator to easily spot hacking attempts. Let's find out who is trying to access the database with sqlplus.exe. SQL> select timestamp, protocol_info 2 from listener_log 3 where connect_data like '%sqlplus.exe%' 4 /TIMESTAMP PROTOCOL_INFO-------------------- --------------------------------------------------------01-SEP-2008 14:30:37 (ADDRESS=(PROTOCOL=tcp)(HOST=192.168.2.101)(PORT=3651))01-SEP-2008 14:31:08 (ADDRESS=(PROTOCOL=tcp)(HOST=192.168.2.101)(PORT=3666))01-SEP-2008 14:31:35 (ADDRESS=(PROTOCOL=tcp)(HOST=192.168.2.101)(PORT=3681)) The use of External Tables to analyze the listener.log can be used not only to have an in-database version of the listener.log perform periodic and programmatic analysis of the listener behavior, but also to determine usage trends and correlate information with the audit team so that unauthorized connection programs can be easily and quickly spotted. Further useful applications can be found by reading the listener.log file. There are two fields that must be further parsed to get information out of them, but parsing those fields goes beyond the scope of this article. The structure that the analysis should consider is detailed next: Connect String SID: The Database Oracle SID, which is populated if the connection was performed by SID, otherwise it is NULL. CID: It contains two subfields, PROGRAM and HOST. SERVER: This field indicates the connection type, either dedicated or shared. SERVICE_NAME: This field is populated when the connection is performed by a Service instead of SID. COMMAND: The command issued by the user. SERVICE: Present only when listener commands are issued. FAILOVER_MODE: In Real Application Clusters (RAC) environments this field is used if the client performed a connection due to a failover. It shows the failover mode used. Protocol PROTOCOL: Indicates the used to perform the connection; this will be TCP most of the times. HOST: This is the client's IP Address. PORT: The port number of the oracle server used to establish the connection. Mapping XML files as External Tables XML has become a de facto information exchange format, which is why oracle has included the XML Database (XDB) feature from 9.2.0. However, it requires the data to be actually loaded into the database before it can be processed. An External Table allows the user to take a quick look at the contents of the external file prior to performing any further processing. In this example an External Table is created out of an XML file. This file is read by means of a CLOB field, and some further XDB commands can be issued against the external XML file to extract and view data. Let's create the external XML file first: create table EMPLOYEES_XML (xmlFile CLOB)organization external ( type ORACLE_LOADER default directory EXTTABDIR access parameters ( fields (xmllob char terminated by ',') column transforms (xmlFile from lobfile(xmllob)) ) location('employees.dat'))reject limit unlimited; The employees.dat file contains the file name of the XML file to load as an external CLOB file. This file, for the purpose of the demo, contains the file name: employees.xml. Now the file can be queried from the database as if it was a regular table with a single XML column. Dynamically changing the external reference When managing External Tables, there should be an easy way to redefine the external source file. It is enough to change the External Table properties by means of an ALTER TABLE command. Let's create a stored procedure that performs this task by means of a dynamically generated DDL command. This procedure, named Change_External_Table redefines the location property. Using a stored program unit is a flexible way to perform this task. create procedure change_external_table( p_table_name in varchar2, p_file_name in varchar2) isbeginexecute immediate 'alter table '|| p_table_name|| ' location ('''|| p_file_name|| ''')' ;exceptionwhen othersthenraise_application_error(sqlcode,sqlerrm) ;end ;/ Oracle 11g External Table enhancements External Tables work the same in 10g and in 11g, so there are no differences when working with these two versions. When working with Data Pump External Tables, and one single row proves defective, the data set reading operation is aborted. An enhancement in this 11g release prevents the data load aborting, thus saving reprocessing time. Summary Managing data with External Tables is a means not only for mapping external flat files as regular (but limited) tables inside the database, but also a tool to more efficiently perform administrative tasks such as programmatically processing database log files such as the alert.log or the listener.log files. It can be used to easily view external XML formatted files from inside the database without actually loading the file to the database. It can also be used as a means of unloading data in temporary external storage to exchange data among different Oracle versions. This particular feature allows the user to easily build an Oracle Datamart that allows the pre-formatting and summarization of data from the source, enabling it to be directly inserted into the target data warehouse.