How-To Tutorials

Capabilities and Permissions So far, we have given users existing roles in different Moodle contexts. In the following few pages, we want to have a look at the inside of a role that is called capabilities and permissions. Once we have understood them, we will be able to modify existing roles and create entirely new custom ones. Role Definitions Existing roles are accessed via Users | Permissions | Define Roles in the Site Administration block. The screen that will be shown is similar to the familiar roles assignment screen, but has a very different purpose: When you click on a role name, its composition is shown. Each role contains a unique Name, a unique Short name (used when uploading users), and an optional Description. The Legacy role type has been introduced for backward compatibility, to allow old legacy code that has not been fully ported to work with the new system comprising new roles and capabilities. It is expected that this facility will disappear in the future (this might be for some time since a lot of core code depends on it), and should be ignored in due course unless you are working with legacy code or third-party add-ons. In addition to these four fields, each role consists of a large number of capabilities. Currently, Moodle's roles system contains approximately 200 capabilities. A capability is a description of a particular Moodle feature (for example) to grade assignments or to edit a Wiki page. Each capability represents a permissible Moodle action: Permission is a capability and its value, taken together. So each row of the table in the screen shot represents permission. The left column is the capability name and the radio buttons specify the value. So now permission has a description, a unique name, a value, and up to four associated risks. The description, for example, Approve course creation provides a short explanation of the capability. On clicking, the description or the online Moodle documentation is opened in a separate browser. The name, for instance moodle /site: approvecourse, follows a strict naming convention that identifies the capability in the overall role system: level/type: function. The level states to which part of Moodle the capability belongs (such as moodle, mod, block, gradereport, or enroll). The type is the class of the capability and the function identifies the actual functionality. The permission of each capability has to have one of the four values: Permission Description Not Set By default, all permissions for a new role are set to this value. The value in the context where it will be assigned will be inherited from the parent-context. To determine what this value is, Moodle searches upward through each context, until it 'finds' an explicit value (Allow, Prevent or Prohibit) for this capability, i.e. the search terminates when an explicit permission is found. For example, if a role is assigned to a user in a Course context, and a capability has a value of 'Not set,' then the actual permission will be whatever the user has at the category level, or, failing to find an explicit permission at the category level, at the site level. If no explicit permission is found, then the value in the current context becomes Prevent. Allow To grant permission for a capability choose Allow. It applies in the context in which the role will be assigned and all contexts which are below it (children, grand-children, etc). For example, when assigned in the course context, students will be able to start new discussions in all forums in that course, unless some forum contains an override or a new assignment with a Prevent or Prohibit value for this capability. Prevent To remove permission for a capability choose Prevent. If it has been granted in a higher context (no matter at what level), it will be overridden. The value can be overridden again in a lower context. Prohibit This is the same as Prevent, but the value cannot be overridden again in a lower context. The value is rarely needed, but useful when an admin wants to prohibit a user from certain functionality throughout the entire site, in which case the capability is set to Prohibit and then assigned in the site context.   Principally, permissions at lower contexts override permissions at higher contexts. The exception is "Prohibit", which by definition cannot be overridden at lower levels. Resolving Permission Conflicts There is a possibility of conflict if two users are assigned the same role in the same context, where one role allows a capability and the other prevents it. In this case, Moodle will look upwards in higher contexts for a decider. This does not apply to Guest accounts, where "Prevent" will be used by default. For example, a user has two roles in the Course context, one that allows functionality and one that prevents it. In this case, Moodle checks the Category and the System contexts respectively, looking for another defined permission. If none is found, then the permission is set to "Prevent". Permission Risks Additionally, Moodle displays the risks associated with each capability, that is, the risks that each capability can potentially raise. They can be any combination of the following four risk types: Risk Icon Description Configuration Users can change site configuration and behavior. XSS Users can add files and texts that allow cross-site scripting (potentially malicious scripts which are embedded in web pages and executed on the user's computer). Privacy Users can gain access to private information of other users. Spam Users can send spam to site users or others. Risks are only displayed. It is not possible to change these settings, since they only act as warnings. When you click on a risk icon, the "Risks" documentation page is opened in a separate browser window. Moodle's default roles have been designed with the following capability risks in mind:

Luigi's Pizza On The Run mobile shop is working well now. And he wants to adapt it to different mobile devices. Let's look at the following: Understanding the Lowest Common Denominator method Finding and comparing features of different mobile devices Deciding to adapt or not Adapting and progressively enhancing POTR application using Wireless Abstraction Library Detecting device capabilities Evaluating tools that can aid in adaptation Moving your blog to the mobile web By the end of this article, you will have a strong foundation in adapting to different devices. What is Adaptation? Adaptation, sometimes called multiserving, means delivering content as per each user device's capabilities. If the visiting device is an old phone supporting only WML, you will show a WML page with Wireless Bitmap (wbmp) images. If it is a newer XHTML MP-compliant device, you will deliver an XHTML MP version, customized according to the screen size of the device. If the user is on iMode in Japan, you will show a Compact HTML (cHTML) version that's more forgiving than XHTML. This way, users get the best experience possible on their device. Do I Need Adaptation? I am sure most of you are wondering why you would want to create somany different versions of your mobile site? Isn't following the XHTML MPstandard enough? On the Web, you could make sure that you followed XHTML and the site will work in all browsers. The browser-specific quirks are limited and fixes are easy. However, in the mobile world, you have thousands of devices using hundreds of different browsers. You need adaptation precisely for that reason! If you want to serve all users well, you need to worry about adaptation. WML devices will give up if they encounter a <b> tag within an <a> tag. Some XHTML MP browsers will not be able to process a form if it is within a table. But a table within a form will work just fine. If your target audience is limited, and you know that they are going to use a limited range of browsers, you can live without adaptation. Can't I just Use Common Capabilities and Ignore the Rest? You can. Finding the Lowest Common Denominator (LCD) of the capabilities of target devices, you can design a site that will work reasonably well in all devices. Devices with better capabilities than LCD will see a version that may not be very beautiful but things will work just as well. How to Determine the LCD? If you are looking for something more than the W3C DDC guidelines, you may be interested in finding out the capabilities of different devices to decide on your own what features you want to use in your application. There is a nice tool that allows you to search on device capabilities and compare them side by side. Take a look at the following screenshot showing mDevInf (http://mdevinf.sourceforge.net/) in action, showing image formats supported on a generic iMode device. You can search for devices and compare them, and then come to a conclusion about features you want to use. This is all good. But when you want to cater to wider mobile audience, you must consider adaptation. You don't want to fight with browser quirks and silly compatibility issues. You want to focus on delivering a good solution. Adaptation can help you there. OK, So How do I Adapt? You have three options to adapt: Design alternative CSS: this will control the display of elements and images. This is the easiest method. You can detect the device and link an appropriate CSS file. Create multiple versions of pages: redirect the user to a device-specific version. This is called "alteration". This way you get the most control over what is shown to each device. Automatic Adaptation: create content in one format and use a tool to generate device-specific versions. This is the most elegant method. Let us rebuild the pizza selection page on POTR to learn how we can detect the device and implement automatic adaptation. Fancy Pizza Selection Luigi has been asking to put up photographs of his delicious pizzas on the mobile site, but we didn't do that so far to save bandwidth for users. Let us now go ahead and add images to the pizza selection page. We want to show larger images to devices that can support them. Review the code shown below. It's an abridged version of the actual code. <?php include_once("wall_prepend.php"); ?> <wall:document><wall:xmlpidtd /> <wall:head> <wall:title>Pizza On The Run</wall:title> <link href="assets/mobile.css" type="text/css" rel="stylesheet" /> </wall:head> <wall:body> <?php echo '<wall:h2>Customize Your Pizza #'.$currentPizza.':</wall:h2> <wall:form enable_wml="false" action="index.php" method="POST"> <fieldset> <wall:input type="hidden" name="action" value="order" />'; // If we did not get the total number of pizzas to order, // let the user select if ($_REQUEST["numPizza"] == -1) { echo 'Pizzas to Order: <wall:select name="numPizza">'; for($i=1; $i<=9; $i++) { echo '<wall:option value="'.$i.'">'.$i.'</wall:option>'; } echo '</wall:select><wall:br/>'; } else { echo '<wall:input type="hidden" name="numPizza" value="'.$_REQUEST["numPizza"].'" />'; } echo '<wall:h3>Select the pizza</wall:h3>'; // Select the pizza $checked = 'checked="checked"'; foreach($products as $product) { // Show a product image based on the device size echo '<wall:img src="assets/pizza_'.$product[ "id"].'_120x80.jpg" alt="'.$product["name"].'"> <wall:alternate_img src="assets/pizza_'.$product[ "id"].'_300x200.jpg" test="'.($wall->getCapa( 'resolution_width') >= 200).'" /> <wall:alternate_img nopicture="true" test="'.( !$wall->getCapa('jpg')).'" /> </wall:img><wall:br />'; echo '<wall:input type="radio" name="pizza[ '.$currentPizza.']" value="'.$product["id"].'" '.$checked.'/>'; echo '<strong>'.$product["name"].' ($'.$product[ "price"].')</strong> - '; echo $product["description"].'<wall:br/>'; $checked = ''; } echo '<wall:input type="submit" class="button" name= "option" value="Next" /> </fieldset></wall:form>'; ?> <p><wall:a href="?action=home">Home</wall:a> - <wall:caller tel="+18007687669"> +1-800-POTRNOW</wall:caller></p> </wall:body> </wall:html> What are Those <wall:*> Tags? All those <wall:*> tags are at the heart of adaptation. Wireless Abstraction Library (WALL) is an open-source tag library that transforms the WALL tags into WML, XHTML, or cHTML code. E.g. iMode devices use <br> tag and simply ignore <br />. WALL will ensure that cHTML devices get a <br> tag and XHTML devices get a <br /> tag. You can find a very good tutorial and extensive reference material on WALL from: http://wurfl.sourceforge.net/java/wall.php. You can download WALL and many other tools too from that site. WALL4PHP—a PHP port of WALL is available from http://wall.laacz.lv/. That's what we are using for POTR. Let's Make Sense of This Code! What are the critical elements of this code? Most of it is very similar to standard XHTML MP. The biggest difference is that tags have a "wall:" prefix. Let us look at some important pieces: The wall_prepend.php file at the beginning loads the WALL class, detects the user's browser, and loads its capabilities. You can use the $wall object in your code later to check device capabilities etc. <wall:document> tells the WALL parser to start the document code. <wall:xmlpidtd /> will insert the XHTML/WML/CHTML prolog as required. This solves part of the headache in adaptation. The next few lines define the page title and meta tags. Code that is not in <wall:*> tags is sent to the browser as is. The heading tag will render as a bold text on a WML device. You can use many standard tags with WALL. Just prefix them with "wall:". We do not want to enable WML support in the form. It requires a few more changes in the document structure, and we don't want it to get complex for this example! If you want to support forms on WML devices, you can enable it in the <wall:form> tag. The img and alternate_img tags are a cool feature of WALL. You can specify the default image in the img tag, and then specify alternative images based on any condition you like. One of these images will be picked up at run time. WALL can even skip displaying the image all together if the nopicture test evaluates to true. In our code, we show a 120x100 pixels images by default, and show a larger image if the device resolution is more than 200 pixels. As the image is a JPG, we skip showing the image if the device cannot support JPG images. The alternate_img tag also supports showing some icons available natively on the phone. You can refer to the WALL reference for more on this. Adapting the phone call link is dead simple. Just use the <wall:caller> tag. Specify the number to call in the tel attribute, and you are done. You can also specify what to display if the phone does not support phone links in alt attribute. When you load the URL in your browser, WALL will do all the heavy liftingand show a mouth-watering pizza—a larger mouth-watering pizza if you have a large screen! Can I Use All XHTML Tags? WALL supports many XHTML tags. It has some additional tags to ease menu display and invoke phone calls. You can use <wall:block> instead of code <p> or <div> tags because it will degrade well, and yet allow you to specify CSS class and id. WALL does not have tags for tables, though it can use tables to generate menus. Here's a list of WALL tags you can use: a, alternate_img, b, block, body, br, caller, cell, cool_menu, cool_menu_css, document, font, form, h1, h2, h3, h4, h5, h6, head, hr, i, img, input, load_capabilities, marquee, menu, menu_css, option, select, title, wurfl_device_id, xmlpidtd. Complete listings of the attributes available with each tag, and their meanings are available from: http://wurfl.sourceforge.net/java/refguide.php. Complete listings of the attributes available with each tag, and their meanings are available from: http://wurfl.sourceforge.net/java/refguide.php. Will This Work Well for WML? WALL can generate WML. WML itself has limited capabilities so you will be restricted in the markup that you can use. You have to enclose content in <wall:block> tags and test rigorously to ensure full WML support. WML handles user input in a different way and we can't use radio buttons or checkboxes in forms. A workaround is to change radio buttons to a menu and pass values using the GET method. Another is to convert them to a select drop down. We are not building WML capability in POTR yet. WALL is still useful for us as it can support cHTML devices and will automatically take care of XHTML implementation variations in different browsers. It can even generate some cool menus for us! Take a look at the following screenshot.

Choosing an Editor If this is your first skin, you really should be thinking about what editor you will be using. If you don't already have an editor or the development environment for other coding you may be working with, the immediate choice that may come to mind is Microsoft Notepad, but there's no need to put yourself through that type of abuse. As we're working with Microsoft technologies while working with DotNetNuke, the natural choice will be Microsoft Visual Web Developer (VWD) which is free. There are other choices for editors here, but VWD will be the one used by most in this context, so we'll move on with it in our examples. If you are using Microsoft's VisualStudio .NET (Microsoft's premier development environment), you will notice that the screens and menus are virtually the same. Installing Visual Web Developer Before we can do anything, we'll need VWD installed. If you have already installed VWD, feel free to skip this section. These are the steps for getting VWD installed: Be sure you have version 2.0 of the .net framework. This can be downloaded from http://www.asp.net or with Windows Updates. Download the VWD install file from http://www.asp.net from the Downloads section. The file will be about three megabytes in size. Once on your local drive, double-click on the fi le to run the installation. You will encounter several common wizard screens. One wizard screen to notein particular is for installing SQL Server 2005 Express Edition. If you do not already have a version of SQL Server 2005 installed, be sure to select to install this. DotNetNuke will have to have an edition of this to run off for it's data store. This is a screen shot of the recommended installation options to choose. Stepping through the wizard, you will start the installation. The installation process may take a while depending upon what options you chose. For example, if you chose to install the MSDN library (documentation & helpfiles), it will take much longer. It will only download the items it needs. At the end of the installation, it will prompt you to register the software. If you do not register VWD within thirty days, it will stop working. If you encounter problems in the installation of VWD, you can find additional assistance at the http://forums.asp.net/discussion website. Installing the DotNetNuke Starter Kits E ven though we now have VWD and SQL Server, we'll need the DotNetNuke files to set up before we can start skinning portals. Do so by using the following steps: Navigate to http://www.dotnetnuke.com. If you haven't already registered on this site, do so now. If you are not already logged in, do so now. Click on Downloads and download the latest version of the starter kit. Right-click on the zip file you downloaded and extract the contents. Double-click on the vscontent file that was extracted. This will start theVisual Studio Content Installer. Select all the components, and click Next. Click Finish to install the starter kit. There are a few components that will be installed. See that in the next screenshot one of the components did not get installed. This is fine as long as the first one, DotNetNuke Web Application(the one we'll be using) installed successfully. The following is what you should see so far: If you encounter problems in the installation of the DotNetNuke starter kits, you can find additional assistance at the http://www.dotnetnuke.com website by clicking on the Forums link and then drilling-down to the Install It! link. Setting Up Your Development Environment In almost any programming project, you will have two environments: the development environment and the post-deployed environment. While skinning, this is no different. Most likely, you will have a local computer where you work on your skin. When you are done with it and are ready to package and deploy it, itwill be installed on the target or live DotNetNuke website which will be your post-deployed environment. To set up our development environment, fire up VWD. We'll now create a new DotNetNuke install: Click on File, and then click New Web Site. A dialog box appears. Click on DotNetNuke Web Application Framework. For Location, pick File System (should be the default item), then type the following location beside it: C:DotNetNukeSkinning. This is the screenshot of what you should see so far: Click OK. It will take a few moments to copy over all the needed web files. You will then be presented with a welcome screen. As the welcome page directs, press Ctrl plus F5 to run your DotNetNuke application. After a few moments, a DotNetNuke install should open in a web browser. If you are presented with the following message, right-click on the information bar at the top and enable the intranet settings in the Internet Explorer.This is what you should see at this point: You are presented with a choice of installation methods. Select Auto andthen select Next. You should then see a web page with a log of installation of the application.Click on the link at the bottom that says Click Here To Access Your Portal. If you encounter problems in the installation of the DotNetNuke, you can find additional assistance at the http://www.dotnetnuke.com website by clicking on the Forums link and then drilling-down to the Install It! link. Congratulations! You now have DotNetNuke up and running. Click Login in the upper-right corner of the screen with the username as host and a password as dnnhost. You should be on the Home page with several modules on it. To make the home page easier to work with, delete all the modules on it, and add a blank Text/HTML module. (In case you have never deleted a module from a page before, you will find the delete menu item if you hover over the downward-pointing triangles to the left of each of the titles.) Depending on the version of DNN you downloaded, you may experienced system message from DotNetNuke on the Home page titled Insecureaccount details. Although changing the default password as it instructs is always a good idea, it is not necessary on a development computer or a non-production implementation of DotNetNuke. However, if you don't want it to nag you about it go ahead and change it. This is our DotNetNuke portal that we will use to test the skins we will create. Move back over to VWD. Close the welcome page. The skins for DotNetNuke will be found in ~Portals_defaultSkins. Go to that directory now as shown here: Congratulations! You have now set up your development environment, and we are now ready for skinning. Creating Your First Skin We will now create a skin and record time. You may be impressed by how fast and easy it is for you to create a skin. Remember when we downloaded the starter kits from DotNetNuke.com? One template is for creating a skin. As of the time of this writing, the current download's template will produce a skin that looks just like the default skin. If this is what you're looking for, you can achieve the same result by copying the DNN-Blue folder and renaming it to something else. Rather than doing this, however, we are starting from scratch. Creat e a folder in your development environment. Name it as FirstSkin. InVWD, to create a new folder, right-click on the folder you want to create it in—in this case Skins—and select New Folder. Next, create an htm file inside the FirstSkin folder called Skin.htm. Use the File menu to create a New File. This will bring up a dialog box where you will pick what type of file you wish to create. Pick HTML Page and name the file as Skin.htm. Now, open our new Skin.htm file. A typical htm document will have tags like , , and . A DotNetNuke skin has none of these. Delete any content so you have clean slate to start from. Once we have a blank htm page to work from, type in the following and save: [LOGIN][MENU]<div id="ContentPane" runat="server"></div> Go to the Skins menu item on your Admin menu. You will now see two drop-down boxes, one for Skins and one for Containers. In the drop-down for Skins, pick the skin you have created. You should see something like this: Click on the link in the lower-middle portion of the screen that says ParseSkin Package. You should see your skin now: Now that our skin has been parsed, let's apply it to our current DotNetNuke portal by clicking on the Apply link. Keep in mind that we only have one pane, the ContentPane. If this was a live site with modules on other panes, the positions may have been changed. Now, go to the home page by clicking on your menu bar at the top. What Do We Have Here? I know what you're thinking: This has got to be the world's simplest DotNetNuke skin. And you're right. You may not be rushing to install this skin on your production portals, but you have created your very first operational skin! Let's go over what just happened, from creating our skin to seeing it in action. Skinsstart out as a simple HTML file. Just as with any website, an HTML file will have some degree of markup. Of course, we have not added much markup to our skin yet. If you're wondering from where DotNetNuke gets all the HTML structure such as the html, head, and body tags, take a look at Default.aspx in the root of your DNN install. This is the page used essentially everytime a page is served up. You can look in that file and find an ASP.NET element called SkinPlaceHolder. This is where our skin will be injected into each DotNetNuke page. Everything before and after this place holder is what will be served to any DNN page request no matter what skin is applied. The code we entered for our skin is: [LOGIN][MENU]<div id="ContentPane" runat="server"></div> Of the code we typed, [LOGIN] and [MENU] are special keywords to DotNetNuke,called tokens. The [Login] token will turn into the login link you're used to seeing and the [Menu] token will serve as our DotNetNuke menu. Adding the [login] token will ensure that we're not locked out of our portal after applying this skin. The <div> tag we added will be a simple ContentPane for now. Notice the two attributes we added to this tag <div><em>—id and runat. These are attributes required by ASP.NET. The id is a unique identifier in the page and the value given to it (ContentPane) is recognized as name by DotNetNuke. The runat attribute tells the ASP.NET engine that it needs to be processed by it. Why Parse? Recall when we clicked on a link to parse our skin. What DotNetNuke does at this point is take our HTM file and replace those tokens with ASP.NET user controlsthat have been predefined in DotNetNuke. At the end of this parsing process, the result is an ASCX file that becomes the real skin file, which is loaded into the Default.aspx at the runtime event of a page request. Anytime after parsing the skin for the first time, you may go in and look at the ASCX file with a text editor, and even modify and see immediate changes without doing a parse. As tempting as editing the ASCX file may be (especially if you're an ASP.NET developer and understand editing ASCX files), you really should not be doing that. This ASCX file is regenerated and is replaced each time a HTM skin file is re-parsed.We will also want to create our skins in a way that would be compatible with the future versions of DotNetNuke. Starting off with an HTM skin file puts us on the path to achieve this goal. Finishing Touches The next thing you will want to do is add more tokens and a little HTML to make yourself a little more proud of your DNN skin. To do this, go back to your HTM file and add two or three items from the list of tokens shown as follows: [LOGO][BANNER][SEARCH][LANGUAGE][CURRENTDATE][BREADCRUMB][USER][COPYRIGHT][TERMS][PRIVACY][DOTNETNUKE] For a complete list of all DotNetNuke tokens, please refer to the DotNetNuke Skinning Guide document by Shaun Walker. You candownload it from http://www.dotnetnuke.com/LinkClick.aspx?fileticket=2ptHepzmuFA%3d&tabid=478&mid=857. Now add in some HTML. You may want to add in a few <hr>(horizontal rule) or <br>(vertical break) tags to separate things out. When you make changes and want to see them, remember to go to the Admin menu and then to the Skins page and re-parse the skin, then go to the Home page to see the changes. Summary The title for this article was Creating Your First Skin and that's exactly what we did.There are many reasons why you couldn't or wouldn't use this skin for a live site. Ofcourse, any website needs a good design, and some graphics, but if you've managed a DNN site, before you know you'll need some more panes and some precise positioning.

On Databases Databases come in many forms. The simplest definition of a database is any system of storing, organizing, and retrieving data. With this definition, things like memory, hard drives, file systems, files on those file systems (stored in plain text, tab-delimited, XML, JSON, or even BDB formats), and even applications like MySQL, PostgreSQL, and Oracle are considered databases. Databases allow users to: Store Data Organize Data Retrieve Data It is important to keep a broad perspective on what data and databases really are so that you can always choose the best solution for your particular problem. The SQL databases (MySQL, PostgreSQL, Oracle, and others) are remarkable because of the flexibility and performance they provide. In my work, I look to them first when developing an application, with an eye towards getting the data model right before optimization. Once the application is solid, and once I fully understand what parts of the data system are too slow or fast enough, then I can start building my own database on top of the file system or other existing technologies that will give me the kind of performance I need. PostgreSQL: Free, BSD -licensed popular database. http://postgresql.org/ MySQL: Free, GPL-licensed popular database. http://mysql.org Oracle: Commercial industrial database. http://oracle.com SQL Server: Microsoft's commercial database. http://www.microsoft.com/SQL/default.mspx Among the SQL databases, which one is best? There are many criteria I use to evaluate SQL databases, and the one I pay attention to most is how they comply (if at all) with the ACID model. And given the technical merits of the various SQL databases, I consistently choose PostgreSQL above all other SQL databases when given a choice. Allow me to explain why. The ACID Model ACID is an acronym, standing for the four words Atomicity, Consistency, Isolation, and Durability. These are fancy words for some very basic and essential concepts. Atomicity means that you either do all of the changes you want, or none of them, without leaving the database in some weird in-between state. When you take into account catastrophes like power failures or corruption, atomicity isn't as simple as it first seems. Consistency means that any state of the database will be internally consistent with the rules that constrain the data. That is, if you have a table with a primary key, then that table will not contain any violations of the primary key constraints after any transaction. Isolation means that you can be modifying many different parts of the database at the same time without affecting each other. (As a higher feature, there is Serialization, which requires that transactions occur one after the other, or at least the results of transactions.) Durability means that once a transaction completes, it is never lost, ever. Atomicity: All or nothing Consistency: Rules kept Isolation: No partials seen Durability: Doesn't disappear ACID compliance isn't rocket science, but it isn't trivial either. These requirements form a minimum standard absolutely necessary to provide a database for a reasonable application. That is, if you can't guarantee these things, then the users of your application are going to be frustrated since they assume, naturally, that the ACID model is followed. And if the users of the application get frustrated, then the developers of the application will get frustrated as they try to comply with the user's expectations. A lot of frustration can be avoided if the database simply complies with the principles of the ACID model. If the database gets it right, then the rest of the application will have no problem getting it right as well. Our users will be happy since their expectations of ACID compliance will be met. Remember: Users expect ACID! What Violating the ACID Model Looks Like To consider the importance of the ACID model, let's examine, briefly, what happens when the model is violated. When Atomicity isn't adhered to, users will see their data partially committed. For instance, they might find their online profile only partially modified, or their bank transfer partially transferred. This is, of course, devastating to the unwary user. When Consistency is violated, the rules that the data should follow aren't adhered to. Perhaps the number of friends shown doesn't match the friends they actually have in a social networking application. Or perhaps they see their bank balance doesn't match what the numbers add up to. Or worse, perhaps your order system is counting orders that don't even exist and not counting orders that do. When Isolation isn't guaranteed, they will either have to use a system where only one person can change something at a time, locking out all others, or they will see inconsistencies throughout the world of data, inconsistencies resulting from transactions that are in progress elsewhere. This will make the data unreliable just like violating Atomicity or Consistency. A bank user, for instance, will believe their transfer of funds was successful when in reality their money was simultaneously being withdrawn by another transaction. When Durability is lost, then users will never know if their transaction really went through, and won't mysteriously disappear down the road with all the trouble that entails. I am sure we have all had experiences dealing with data systems that didn't follow the ACID model. I remember the days when you had to save your files frequently, and even then you still weren't ensured that all of your data would be properly saved. I also recall applications that would make partial changes, or incomplete changes, and expose these inconsistent states to the user. In today's world, writing applications with faults like the above is simply inexcusable. There are too many tools out there that are readily available that make writing ACID compliant systems easy. One of those tools, probably the most popular of all, is the SQL database. Satisfying ACID with Transactions The principle way that databases comply with ACID requirements is through the concept of transactions. Ideally, each transaction would occur in an instant, updating the database according to the state of the database at that moment. In reality, this isn't possible. It takes time to accumulate the data and apply the changes. Typical transaction SQL commands: BEGIN: Start a new transaction COMMIT: Commit the transaction ROLLBACK: Roll back the transaction in progress Since multiple sessions can each be creating and applying a transaction simultaneously, special precautions have to be taken to ensure that the data that each transaction “sees” is consistent, and that the effects of each transaction appear all together or not at all. Special care is also taken to ensure that when a transaction is committed, the database will be put in a state where catastrophic events will not leave the transaction partially committed. Contrary to popular belief, there are a variety of ways that databases support transactions. It is well worth the time to read and understand PostgreSQL's two levels of transaction isolation and the four possible isolation levels in Section 12.2 of the PostgreSQL documentation. Note that some of the inferior levels of transaction isolation violate some extreme cases of ACID compliance for the sake of performance. These edge cases can be properly handled with appropriate use of row-locking techniques. Row-locking is an issue beyond this article. Keep in mind that the levels of transaction isolation are only what appear to users of the database. Inside the database, there is a remarkable variety of methods on actually implementing transactions. Consider that while you are in a transaction, making changes to the database, every other transaction has to see one version of the database while you see another. In effect, you have to have copies of some of the data lying around somewhere. Queries to that data have to know which version of the data to retrieve the copy, the original, or the modified version (and which modified version?) Changes to the data have to go somewhere the original, a copy, or some modified version (again, which?) Answering these questions leads to the various implementations of transactions in ACID compliant databases. For the purposes of this article, I will examine only two: Oracle's and PostgreSQL's implementations. If you are only familiar with Oracle, then hopefully you will learn something new and fascinating as you investigate PostgreSQL's method.

Password Strength Checker Visual cues are great way to inform the user of the status of things in the application. Message boxes and alerts are used much too often for this purpose, but they usually end up irritating the user. A much smoother and enjoyable user experience is provided by subtly indicating to the user the status as an application is used. In this section, we are going to create an application that indicates the strength of a typed password to the user by the use of colors and checkboxes. We are going to use check-boxes very differently than their normal usage. This is an example of using GWT widgets in new and different ways, and mixing and matching them to provide a great user experience. Time for Action—Creating the Checker In the current day and age, passwords are required for almost everything, and choosing secure passwords is very important. There are numerous criteria suggested for creating a password that is secure from most common password cracking exploits. These criteria run the gamut from creating 15 letter passwords with a certain number of lower case and numeric digits to creating passwords using random password generators. In our example application, we are going to create a password strength checker that is very simple, and only checks the number of letters in the password. A password string that contains less than five letters will be considered weak, while a password that contains between five and seven letters will be considered to be of medium strength. Any password containing more than seven letters will be considered as strong. The criteria were deliberately kept simple so that we can focus on creating the application without getting all tangled up in the actual password strength criteria. This will help us to understand the concepts and then you can extend it to use any password strength criteria that your application warrants. This example uses a service to get the password strength, but this could also be done all on the client without needing to use a server. 1. Create a new Java file named PasswordStrengthService.java in the com.packtpub.gwtbook.samples.client package. Define a PasswordStrengthService interface with one method to retrieve the strength of a password string provided as a parameter to the method: public interface PasswordStrengthService extends RemoteService{public int checkStrength(String password);} 2. Create the asynchronous version of this service definition interface in a new Java file named PasswordStrengthServiceAsync.java in the com.packtpub.gwtbook.samples.client package : public interface PasswordStrengthServiceAsync{public void checkStrength(String password, AsyncCallback callback);} 3. Create the implementation of our password strength service in a new Java file named PasswordStrengthServiceImpl.java in the com.packtpub.gwtbook.samples.server package. public class PasswordStrengthServiceImpl extendsRemoteServiceServlet implements PasswordStrengthService{private int STRONG = 9;private int MEDIUM = 6;private int WEAK = 3;public int checkStrength(String password){if (password.length() <= 4){return WEAK;}else if (password.length() < 8){return MEDIUM;}else{return STRONG;}}} 4. Now let's create the user interface for this application. Create a new Java file named PasswordStrengthPanel.java in the com.packtpub.gwtbook.samples.client.panels package that extends the com.packtpub.gwtbook.samples.client.panels.SamplePanel class. Create a text box for entering the password string an ArrayList named strengthPanel for holding the checkboxes that we will use for displaying the strength of the password. Also create the PasswordStrengthService object. public TextBox passwordText = new TextBox();final PasswordStrengthServiceAsync pwStrengthService =(PasswordStrengthServiceAsync)GWT.create(PasswordStrengthService.class);public ArrayList strength = new ArrayList(); 5. Add a private method for clearing all the checkboxes by setting their style to the default style. private void clearStrengthPanel(){for (Iterator iter = strength.iterator(); iter.hasNext();){((CheckBox) iter.next()).setStyleName(getPasswordStrengthStyle(0));}} 6. Add a private method that will return the CSS name, based on the password strength. This is a nice way for us to dynamically set the style for the checkbox, based on the strength. private String getPasswordStrengthStyle(int passwordStrength){if (passwordStrength == 3){return "pwStrength-Weak";}else if (passwordStrength == 6){return "pwStrength-Medium";}else if (passwordStrength == 9){return "pwStrength-Strong";}else{return "";}} 7. In the constructor for the PasswordStrengthPanel class, create a HorizontalPanel named strengthPanel, add nine checkboxes to it, and set its style. As mentioned before, the styles that we are using in the sample applications in this book are available in the file Samples.css, which is part of the source code distribution for this book. We also add these same checkboxes to the strength object, so that we can retrieve them later to set their state. These checkboxes will be used for displaying the password strength visually. Create a new VerticalPanel that we will use as the container for the widgets that we are adding to the user interface. Finally, create the service target and set its entry point. HorizontalPanel strengthPanel = new HorizontalPanel();strengthPanel.setStyleName("pwStrength-Panel");for (int i = 0; i < 9; i++){CheckBox singleBox = new CheckBox();strengthPanel.add(singleBox);strength.add(singleBox);}VerticalPanel workPanel = new VerticalPanel();ServiceDefTarget endpoint=(ServiceDefTarget) pwStrengthService;endpoint.setServiceEntryPoint(GWT.getModuleBaseURL() +"pwstrength"); 8. In the same constructor, set the style for the password text box, and add an event handler to listen for changes to the password box. passwordText.setStyleName("pwStrength-Textbox");passwordText.addKeyboardListener(new KeyboardListener(){public void onKeyDown(Widget sender, char keyCode, int modifiers){}public void onKeyPress(Widget sender, char keyCode, int modifiers){}public void onKeyUp(Widget sender, char keyCode,int modifiers){if (passwordText.getText().length() > 0){AsyncCallback callback = new AsyncCallback(){public void onSuccess(Object result){clearStrengthPanel();int checkedStrength = ((Integer) result).intValue();for (int i = 0; i < checkedStrength; i++){((CheckBox) strength.get(i)).setStyleName(getPasswordStrengthStyle(checkedStrength));}}public void onFailure(Throwable caught){Window.alert("Error calling the password strengthservice." + caught.getMessage());}};pwStrengthService.checkStrength(passwordText.getText(), callback);}else{clearStrengthPanel();}}}); 9. Finally, in the constructor, add the password text box and the strength panel to the work panel. Create a little info panel that displays descriptive text about this application, so that we can display this text when this sample is selected in the list of available samples in our Samples application. Add the info panel and the work panel to a dock panel, and initialize the widget. HorizontalPanel infoPanel = new HorizontalPanel();infoPanel.add(new HTML("<div class='infoProse'>Start typing a passwordstring. The strength of the password will bechecked and displayed below. Red indicates that thepassword is Weak, Orange indicates a Mediumstrength password and Green indicates a Strongpassword. The algorithm for checking the strengthis very basic and checks the length of the passwordstring.</div>"));workPanel.add(passwordText);workPanel.add(infoPanel);workPanel.add(strengthPanel);DockPanel workPane = new DockPanel();workPane.add(infoPanel, DockPanel.NORTH);workPane.add(workPanel, DockPanel.CENTER);workPane.setCellHeight(workPanel, "100%");workPane.setCellWidth(workPanel, "100%");initWidget(workPane); 10. Add the service to the module file for the Samples application—Samples.gwt.xml in the com.packtpub.gwtbook.samples package. <servlet path="/pwstrength" class="com.packtpub.gwtbook.samples.server.PasswordStrengthServiceImpl"/> Here is the user interface for the password strength checking application: Now start typing a password string to check its strength. Here is the password strength when you type a password string that is less than five characters: What Just Happened? The password strength service checks the size of the provided string and returns an integer value of three, six, or nine based on whether it is weak, medium, or strong. It makes this determination by using the criteria that if the password string is less than five characters in length, it is weak, and if it is more than five characters but not greater than seven characters, it is considered a medium strength password. Anything over seven characters is considered to be a strong password. The user interface consists of a text box for entering a password string and a panel containing nine checkboxes that visually displays the strength of the typed string as a password. An event handler is registered to listen for keyboard events generated by the password text box. Whenever the password text changes, which happens when we type into the field or change a character in the field, we communicate asynchronously with the password strength service and retrieve the strength of the given string as a password. The returned strength is displayed to the user in a visual fashion by the use of colors to symbolize the three different password strengths. The password strength is displayed in a compound widget that is created by adding nine checkboxes to a HorizontalPanel. The color of the checkboxes is changed using CSS depending on the strength of the password string. This process of combining the basic widgets provided by GWT into more complex widgets to build user interfaces is a common pattern in building GWT applications. It is possible to build quite intricate user interfaces in this way by utilizing the power of the GWT framework. Summary In the current day and age, passwords are required for almost everything, and choosing secure passwords is very important. In this article, we implemented a password strength checker in Google Web Toolkit (GWT) and AJAX. By going through the article, the reader can also get a general idea of implementing other interactive user forms.

In this article, we will discuss the following topics: Need for Java Business Integration (JBI) Enterprise Service Bus Normalized Message Router Service Engines in NetBeans Need for Java Business Integration (JBI) To have a good understanding of Service Engines (a specific type of JBI component), we need to first understand the reason for Java Business Integration. In the business world, not all systems talk the same language. They use different protocols and different forms of communications. Legacy systems in particular can use proprietary protocols for external communication. The advent and acceptance of XML has been greatly beneficial in allowing systems to be easily integrated, but XML itself is not the complete solution. When some systems were first developed, they were not envisioned to be able to communicate with many other systems; they were developed with closed interfaces using closed protocols. This, of course, is fine for the system developer, but makes system integration very difficult. This closed and proprietary nature of enterprise systems makes integration between enterprise applications very difficult. To allow enterprise systems to effectively communicate between each other, system integrators would use vendor-supplied APIs and data formats or agree on common exchange mechanisms between their systems. This is fine for small short term integration, but quickly becomes unproductive as the number of enterprise applications to integrate gets larger. The following figure shows the problems with traditional integration. As we can see in the figure, each third party system that we want to integrate with uses a different protocol. As a system integrator, we potentially have to learn new technologies and new APIs for each system we wish to integrate with. If there are only two or three systems to integrate with, this is not really too much of a problem. However, the more systems we wish to integrate with, the more proprietary code we have to learn and integration with other systems quickly becomes a large problem. To try and overcome these problems, the Enterprise Application Integration (EAI) server was introduced. This concept has an integration server acting as a central hub. The EAI server traditionally has proprietary links to third party systems, so the application integrator only has to learn one API (the EAI server vendors). With this architecture however, there are still several drawbacks. The central hub can quickly become a bottleneck, and because of the hub-and-spoke architecture, any problems at the hub are rapidly manifested at all the clients. Enterprise Service Bus To help solve this problem, leading companies in the integration community (led by Sun Microsystems) proposed the Java Business Integration Specification Request (JSR 208) (Full details of the JSR can be found at http://jcp.org/en/jsr/detail?id=208). JSR 208 proposed a standard framework for business integration by providing a standard set of service provider interfaces (SPIs) to help alleviate the problems experienced with Enterprise Application Integration. The standard framework described in JSR 208 allows pluggable components to be added into a standard architecture and provides a standard common mechanism for each of these components to communicate with each other based upon WSDL. The pluggable nature of the framework described by JSR 208 is depicted in the following figure. It shows us the concept of an Enterprise Service Bus and introduces us to the Service Engine (SE) component: JSR 208 describes a service engine as a component, which provides business logic and transformation services to other components, as well as consuming such services. SEs can integrate Java-based applications (and other resources), or applications with available Java APIs. Service Engine is a component which provides (and consumes) business logic and transformation services to other components. There are various Service Engines available, such as the BPEL service engine for orchestrating business processes, or the Java EE service engine for consuming Java EE Web Services. The Normalized Message Router As we can see from the previous figure, SE's don't communicate directly with each other or with the clients, instead they communicate via the NMR. This is one of the key concepts of JBI, in that it promotes loose coupling of services. So, what is NMR and what is its purpose? NMR is responsible for taking messages from clients and routing them to the appropriate Service Engines for processing. (This is not strictly true as there is another standard JBI component called the Binding Component responsible for receiving client messages. Again, this further enhances the support for loose coupling within JBI, as Service Engines are decoupled from their transport infrastructure). NMR is responsible for passing normalized (that is based upon WSDL) messages between JBI components. Messages typically consist of a payload and a message header which contains any other message data required for the Service Engine to understand and process the message (for example, security information). Again, we can see that this provides a loosely coupled model in which Service Engines have no prior knowledge of other Service Engines. This therefore allows the JBI architecture to be flexible, and allows different component vendors to develop standard based components. Normalized Message Router enables technology for allowing messages to be passed between loosely coupled services such as Service Engines. The figure below gives an overview of the message routing between a client application and two service engines, in this case the EE and SQL service engines. In this figure, a request is made from the client to the JBI Container. This request is passed via NMR to the EE Service Engine. The EE Service Engine then makes a request to the SQL Service Engine via NMR. The SQL Service Engine returns a message to the EE Service Engine again via NMR. Finally, the message is routed back to the client through NMR and JBI framework. The important concept here is that NMR is a message routing hub not only between clients and service engines, but also for intra-communication between different service engines. The entire architecture we have discussed is typically referred to as an Enterprise Service Bus. Enterprise Service Bus (ESB) is a standard-based middleware architecture that allows pluggable components to communicate with each other via a messaging subsystem.

Introduction When you are integrating different systems together, it can be very easy to use your vendor’s APIs and program directly against them. Using that approach, developers can easily integrate applications. Supporting these applications however can become problematical. If we have a few systems integrated together in this approach, everything is fine, but the more systems we integrate together, the more integration code we have and it rapidly becomes unfeasible to support this point-to-point integration. To overcome this problem, the integration hub was developed. In this scenario, developers would write against the API of the integration vendor and only had to learn one API. This is a much better approach than the point-to-point integration method however it still has its limitations. There is still a proprietary API to learn (albeit only one this time), but if the integration hub goes down for any reason, then entire Enterprise can become unavailable. The Enterprise Service Bus (ESB) overcomes these problems by providing a scalable, standards based integration architecture. The NetBeans SOA pack includes a copy of OpenESB, which follows this architecture promoted by the Java Business Integration Specification JSR 208. Workings of an ESB At the heart of the ESB is the Normalized Message Router (NMR) - a pluggable framework that allows Java Business Integration (JBI) components to be plugged into it as required.  The NMR is responsible for passing messages between all of the different JBI components that are plugged into it. The two main JBI components that are plugged into the NMR are Binding Components and Service Engines.  Binding Components are responsible for handling all protocol specific transport such as HTTP, SOAP, JMS, File system access, etc.  Service Engines on the other hand execute business logic as BPEL processes, SQL statements, invoking external Java EE web services, etc.   There is a clear separation between Binding Components and Service Engines with protocol specific transactions being handled by the former and business logic being performed by the latter. This architecture promotes loose coupling in that service engines do not communicate directly with each other.  All communication between different JBI components is performed through Binding Components by use of normalized messages as shown in the sequence chart below. In the case of OpenESB, all of these normalized messages are based upon WSDL.  If, for example, a BPEL process needs to invoke a web service or send an email, it does not need to know about SOAP or SMTP that is the responsibility of the Binding Components.  For one Service Engine to invoke another Service Engine all that is required is a WSDL based message to be constructed, which can then be routed via the NMR and Binding Components to the destination Service Engine. OpenESB provides many different Binding Components and Service Engines enabling integration with many varied different systems. So, we can see that OpenESB provides us with a standard based architecture that promotes loose coupling between components.  NetBeans 6 provides tight integration with OpenESB allowing developers to take full advantage of its facilities. Integrating Netbeans6 IDE with OpenESB Integration with NetBeans comes in two parts.  First, NetBeans allows the different JBI components to be managed from within the IDE.  Binding Components and Service Engines can be installed into OpenESB from within NetBeans and from thereon the full lifecycle of the components (start, stop, restart, uninstall) can be controlled directly from within the IDE. Secondly, and more interestingly, the NetBeans IDE provides full editing support for developing Composite Applications¬ applications that bring together business logic and data from different sources.  One of the main features of Composite Applications is probably the BPEL editor.  This allows BPL process to be built up graphically allowing interaction with different data sources via different partner links, which may be web services, different BPEL processes, or SQL statements. Once a BPEL process or composite application has been developed, the NetBeans SOA pack provides tools to allow different bindings to be added onto the application depending on the Binding Components installed into OpenESB.  So, for example, a file binding could be added to a project that could poll the file system periodically looking for input messages to start a BPEL process, the output of which could be saved into a different file or sent directly to an FTP site. In addition to support for developing Composite Applications, the NetBeans SOA pack provides support for some features many Java developers would find useful, namely XML and WSDL editing and validation.  XML and WSDL files can be edited within the IDE as either raw text, or via graphical editors.  If changes are made in the raw text, the graphical editors update accordingly and vice versa.  

To keep things simple though, the article doesn’t discuss how to implement database-backed web pages with Python, concentrating only on how to connect Python with MySQL. Sample Application The best way to learn new programming techniques is to write an application that exercises them. This article will walk you through the process of building a simple Python application that interacts with a MySQL database. In a nutshell, the application picks up some live data from a web site and then persists it to an underlying MySQL database. For the sake of simplicity, it doesn’t deal with a large dataset. Rather, it picks up a small subset of data, storing it as a few rows in the underlying database. In particular, the application gets the latest post from the Packt Book Feed page available at http://feeds.feedburner.com/packtpub/sDsa?format=xml. Then, it analyzes the post’s title, finding appropriate tags for the article associated with the post, and finally inserts information about the post into the posts and posttags underlying database tables. As you might guess, a single post may be associated with more than one tag, meaning a record in the posts table may be related to several records in the posttags table. Diagrammatically, the sample application components and their interactions might look like this: Note the use of appsample.py. This script file will contain all the application code written in Python. In particular, it will contain the list of tags, as well as several Python functions packaging application logic. Software Components To build the sample discussed in the article you’re going to need the following software components installed on your computer: Python 2.5.x MySQLdb 1.2.x MySQL 5.1 All these software components can be downloaded and used for free. Although you may already have these pieces of software installed on your computer, here’s a brief overview of where you can obtain them. You can download an appropriate Python release from the Downloads page at Python’s web site at http://python.org/download/. You may be tempted to download the most recent release. Before you choose the release, however, it is recommended that you visit the Python for MySQL page at http://sourceforge.net/projects/mysql-python/ to check what Python releases are supported by the current MySQLdb module that will be used to connect your Python installation with MySQL. MySQLdb is the Python DB API-2.0 interface for MySQL. You can pick up the latest MySQLdb package (version 1.2.2 at the time of writing) from the sourceforge.net’s Python for MySQL page at http://sourceforge.net/projects/mysql-python/. Before you can install it, though, make sure you have Python installed in your system. You can obtain the MySQL 5.1 distribution from the mysql.com web site at http://dev.mysql.com/downloads/mysql/5.1.html, picking up the package designed for your operating system. Setting up the Database Assuming you have all the software components that were outlined in the preceding section installed in your system, you can now start building the sample application. The first step is to create the posts and posttags tables in your underlying MySQL database. As mentioned earlier, a single post may be associated with more than one tag. What this means in practice is that the posts and posttags tables should have a foreign key relationship. In particular, you might create these tables as follows: CREATE TABLE posts ( title VARCHAR(256) PRIMARY KEY, guid VARCHAR(1000), pubDate VARCHAR(50) ) ENGINE = InnoDB; CREATE TABLE posttags ( title VARCHAR(256), tag VARCHAR(20), PRIMARY KEY(title,tag), FOREIGN KEY(title) REFERENCES posts(title) ) ENGINE = InnoDB; As you might guess, you don’t need to populate above tables with data now. This will be automatically done later when you launch the sample. Developing the Script Now that you have the underlying database ready, you can move on and develop the Python code to complete the sample. In particular, you’re going to need to write the following components in Python: tags nested list of tags that will be used to describe the posts obtained from the Packt Book Feed page. obtainPost function that will be used to obtain the information about the latest post from the Packt Book Feed page. determineTags function that will determine appropriate tags to be applied to the latest post obtained from the Packt Book Feed page. insertPost function that will insert the information about the post obtained into the underlying database tables: posts and posttags. execPr function that will make calls to the other, described above functions. You will call this function to launch the application. All the above components will reside in a single file, say, appsample.py that you can create in your favorite text editor, such as vi or Notepad. First, add the following import declarations to appsample.py: import MySQLdb import urllib2 import xml.dom.minidom As you might guess, the first module is required to connect Python with MySQL, providing the Python DB API-2.0 interface for MySQL. The other two are needed to obtain and then parse the Packt Book Feed page’s data. You will see them in action in the obtainPost function in a moment. But first let’s create a nested list of tags that will be used by the determineTags function that determines the tags appropriate for the post being analyzed. To save space here, the following list contains just a few tags. You may and should include more tags to this list, of course. tags=["Python","Java","Drupal","MySQL","Oracle","Open Source"] The next step is to add the obtainPost function responsible for getting the data from the Packt Book Feed page and generating the post dictionary that will be utilized in further processing: def obtainPost(): addr = "http://feeds.feedburner.com/packtpub/sDsa?format=xml" xmldoc = xml.dom.minidom.parseString(urllib2.urlopen(addr).read()) item = xmldoc.getElementsByTagName("item")[0] title = item.getElementsByTagName("title")[0].firstChild.data guid = item.getElementsByTagName("guid")[0].firstChild.data pubDate = item.getElementsByTagName("pubDate")[0].firstChild.data post ={"title": title, "guid": guid, "pubDate": pubDate} return post Now that you have obtained all the required information about the latest post on the Packt Book Feed page, you can analyze the post’s title to determine appropriate tags. For that, add the determineTags function to appsample.py: def determineTags(title, tagslist): curtags=[] for curtag in tagslist: if title.find(curtag)>-1:curtags.append(curtag) return curtags By now, you have both the post and tags to be persisted to the database. So, add the insertPost function that will handle this task (don’t forget to change the parameters specified to the MySQLdb.connect function for the actual ones): def insertPost(title, guid, pubDate, curtags): db=MySQLdb.connect(host="localhost",user="usrsample",passwd="pswd",db="dbsample") c=db.cursor() c.execute("""INSERT INTO posts (title, guid, pubDate) VALUES(%s, %s,%s)""", (title, guid, pubDate)) db.commit() for tag in curtags: c.execute("""INSERT INTO posttags (title, tag) VALUES(%s,%s)""", (title, tag)) db.commit() db.close() All that is left to do is add the execPr function that brings all the pieces together, calling the above functions in the proper order: def execPr(): p = obtainPost() t = determineTags(p["title"],tags) insertPost(p["title"], p["guid"], p["pubDate"], t) Now let’s test the code we just wrote. The simplest way to do this is through Python’s interactive command line. To start an interactive Python session, you can type python at your system shell prompt. It’s important to realize that since the sample discussed here is going to obtain some data from the web, you must connect to the Internet before you launch the application. Once you’re connected, you can launch the execPr function in your Python session, as follows: >>>import appsample >>>appsample.execPr() If everything is okay, you should see no messages. To make sure that everything really went as planned, you can check the posts and posttags tables. To do this, you might connect to the database with the MySQL command-line tool and then issue the following SQL commands: SELECT * FROM posts; The above should generate the output that might look like this: |title |guid |pubDate ------------------------------------------------------------------ Open Source CMS Award Voting Now Closed | http://www.packtpub.com/ article/2008-award-voting-closed | Tue, 21 Oct 2008 09:29:54 +0100 Then, you might want to check out the posttags table: SELECT * FROM posttags; This might generate the following output: |title |tag Open Source CMS Award Voting Now Closed | Open Source Please note that you may see different results since you are working with live data. Another thing to note here is that if you want to re-run the sample, you first need to empty the posts and posttags tables. Otherwise, you will encounter the problem related to the primary key constraints. However, that won’t be a problem at all if you re-run the sample in a few days, when a new post or posts appear on the Packt Book Feed page. Conclusion In this article you looked at a simple Python application persisting data to an underlying MySQL database. Although, for the sake of simplicity, the sample discussed here doesn’t offer a web interface, it illustrates how you can obtain data from the Internet, and then utilize it within your application, and finally store that data in the database.

The Core Database Much of the data we see in Joomla! is stored in the database. A base installation has over thirty tables. Some of these are related to core extensions and others to the inner workings of Joomla!. There is an official database schema, which describes the tables created during the installation. For more information, please refer to: http://dev.joomla.org/ component/option,com_jd-wiki/Itemid,31/id,guidelines:database/. A tabular description is available at: http://dev.joomla.org/downloads/Joomla15_DB-Schema.htm. We access the Joomla! database using the global JDatabase object. The JDatabase class is an abstract class, which is extended by different database drivers. There are currently only two database drivers included in the Joomla! core, MySQL and MySQLi. We access the global JDatabase object using JFactory: $db =& JFactory::getDBO(); Extending the Database When we create extensions, we generally want to store data in some form. If we are using the database, it is important to extend it in the correct way. Table Prefix All database tables have a prefix, normally jos_, which helps in using a single database for multiple Joomla! installations. When we write SQL queries, to accommodate the variable table prefix, we use a symbolic prefix that is substituted with the actual prefix at run time. Normally the symbolic prefix is #__, but we can specify an alternative prefix if we want to. Schema Conventions When we create tables for our extensions, we must follow some standard conventions. The most important of these is the name of the table. All tables must use the table prefix and should start with name of the extension. If the table is storing a specific entity, add the plural of the entity name to the end of the table name separated by an underscore. For example, an items table for the extension 'My Extension' would be called #__myExtension_items. Table field names should all be lowercase and use underscore word separators; you should avoid using underscores if they are not necessary. For example, you can name an email address field as email. If you had a primary and a secondary email field, you could call them email and email_secondary; there is no reason to name the primary email address email_primary. If you are using a primary key record ID, you should call the field id, make it of type integer auto_increment, and disallow null. Doing this will allow you to use the Joomla! framework more effectively. Common Fields We may use some common fields in our tables. Using these fields will enable us to take advantage of the Joomla! framework. Publishing We use publishing to determine whether to display data. Joomla! uses a special field called published, of type tinyint(1); 0 = not published, 1 = published. Hits If we want to keep track of the number of times a record has been viewed, we canuse the special field hits, of type integer and with the default value 0. Checking Out To prevent more than one user trying to edit one record at a time we can check out records (a form of software record locking). We use two fields to do this, checked_out and checked_out_time. checked_out, of type integer, holds the ID of the user that has checked out the record. checked_out_time, of type datetime, holds the date and time when the record was checked out. A null date and a user ID of 0 is recorded if the record is not checked out. Ordering We often want to allow administrators the ability to choose the order in which items appear. The ordering field, of type integer, can be used to number records sequentially to determine the order in which they are displayed. This field does not need to be unique and can be used in conjunction with WHERE clauses to form ordering groups. Parameter Fields We use a parameter field, a TEXT field normally named params, to store additional information about records; this is often used to store data that determines how a record will be displayed. The data held in these fields is encoded as INI strings (which we handle using the JParameter class). Before using a parameter field, we should carefully consider the data we intend to store in the field. Data should only be stored in a parameter field if all of the following criteria are true: Not used for sorting records Not used in searches Only exists for some records Not part of a database relationship Schema Example Imagine we have an extension called 'My Extension' and an entity called foobar. The name of the table is #__myextension_foobars. This schema describes the table: Field Datatype NOT NULL AUTO INC UNSIGNED DEFAULT id INTEGER X X X NULL content TEXT X       checked_out INTEGER X   X 0 checked_out_time DATETIME X     0000-00-00 00:00:00 params TEXT X       ordering INTEGER X   X 0 hits INTEGER X   X 0 published TINYINT(1) X   X 0 This table uses all of the common fields and uses an auto-incrementing primary keyID field. When we come to define our own tables we must ensure that we use thecorrect data types and NOT NULL, AUTO INC, UNSIGNED and DEFAULT values. The SQL displayed below will create the table described in the above schema: CREATE TABLE `#__myextension_foobars` ( `id` INTEGER UNSIGNED NOT NULL DEFAULT NULL AUTO_INCREMENT, `content` TEXT NOT NULL DEFAULT '', `checked_out` INTEGER UNSIGNED NOT NULL DEFAULT 0, `checked_out_time` DATETIME NOT NULL DEFAULT '0000-00-00 00:00:00', `params` TEXT NOT NULL DEFAULT '', `ordering` INTEGER UNSIGNED NOT NULL DEFAULT 0, `hits` INTEGER UNSIGNED NOT NULL DEFAULT 0, `published` INTEGER UNSIGNED NOT NULL DEFAULT 0, PRIMARY KEY(`id`)) CHARACTER SET `utf8` COLLATE `utf8_general_ci`; Date Fields We regularly use datetime fields to record the date and time at which an action has taken place. When we use these fields, it is important that we are aware of the effect of time zones. All dates and times should be recorded in UTC+0 (GMT / Z). When we come to display dates and times we can use the JDate class. The JDate class allows us to easily parse dates, output them in different formats, and apply UTC time-zone offsets. For more information about time zones, please refer to http://www.timeanddate.com. We often use parsers before we display data to make the data safe or to apply formatting to the data. We need to be careful how we store data that is going to be parsed. If the data is ever going to be edited, we must store the data in its RAW state. If the data is going to be edited extremely rarely and if the parsing is reversible, we may want to consider building a 'reverse-parser'. This way we can store the data in its parsed format, eradicating the need for parsing when we view the data and reducing the load on the server. Another option available tous is to store the data in both formats. This way we only have to parse data when we save it. Dealing with Multilingual Requirements Unlike ASCII and ANSII, Unicode is a multi-byte character set; it uses more than eight bits (one byte) per character. When we use UTF-8 encoding, character byte lengths vary. Unfortunately, MySQL versions prior to 4.1.2 assume that characters are always eight bits (one byte), which poses some problems. To combat the issue when installing extensions we have the ability to define different SQL files for servers, that do and do not support UTF-8. In MySQL servers that do not support UTF-8, when we create fields, which define a character length, we are actually defining the length in bytes. Therefore, if we try to store UTF-8 characters that are longer than one byte, we may exceed the size of the field. To combat this, we increase the length of fields to try to accommodate UTF-8strings. For example, a varchar(20) field becomes a varchar(60) field. We triple the size of fields because, although UTF-8 characters can be more than three bytes, the majority of common characters are a maximum of three bytes. This poses another issue, if we use a varchar(100) field, scaling it up for a MySQL server, which does not support UTF-8, we would have to define it as a varchar(300) field. We cannot do this because varchar fields have a maximum size of 255. The next step is slightly more drastic. We must redefine the field type so as it will accommodate at least three hundred bytes. Therefore, a varchar(100) field becomes a text field. As an example, the core #__content table includes a field named title. For MySQL severs that support UTF-8, the field is defined as: `title` varchar(255) NOT NULL default '' For MySQL severs that do not support UTF-8, the field is defined as: `title` text NOT NULL default '' We should also be aware that using a version of MySQL that does not support UTF-8 would affect the MySQL string handling functions. For example ordering by a string field may yield unexpected results. While we can overcome this using postprocessing in our scripts using the JString class, the recommended resolution is to upgrade to the latest version of MySQL.

This article by Javier Collado expands the set of web scraping techniques shown in his previous article by looking closely into a more complex problem that cannot be solved with the tools that were explained there. For those who missed out on that article, here's the link. Web Scraping with Python This article will show how to extract the desired information using the same three steps when the web page is not written directly using HTML, but is auto-generated using JavaScript to update the DOM tree. As you may remember from that article, web scraping is the ability to extract information automatically from a set of web pages that were designed only to display information nicely to humans; but that might not be suitable when a machine needs to retrieve that information. The three basic steps that were recommended to be followed when performing a scraping task were the following: Explore the website to find out where the desired information is located in the HTML DOM tree Download as many web pages as needed Parse downloaded web pages and extract the information from the places found in the exploration step What should be taken into account when the content is not directly coded in the HTML DOM tree? The main difference, as you probably have already noted, is that using the downloading methods that were suggested in the previous article (urllib2 or mechanize) just don't work. This is because they generate an HTTP request to get the web page and deliver the received HTML directly to the scraping script. However, the pieces of information that are auto-generated by the JavaScript code are not yet in the HTML file because the code is not executed in any virtual machine as it happens when the page is displayed in a web browser. Hence, instead of relying on a library that generates HTTP requests, we need a library that behaves as a real web browser, or even better, a library that interacts with a real web browser. So that we are sure that we obtain the same data as we see when manually opening a page in a web browser. Please remember that the aim of web scraping is actually parsing the data that a human user sees, so interacting with a real web browser would be a really nice feature. Is there any tool out there to perform that? Fortunately, the answer is yes. In particular, there are a couple of tools used for web testing automation that can be used to solve the JavaScript execution problem: Selenium and Windmill . For the code samples in the sections below, Windmill is used. Any choice would be fine as both of them are well documented and stable tools ready to be used for production. Let's now follow the same three steps that were suggested in the previous article to solve the scraping of the contents of a web page that is partly generated using JavaScript code. Explore Imagine that you are a fan of NASA Image of the day gallery. You want to get a list of the names of all the images in the gallery together with the link to the whole resolution picture just in case you decide to download it later to use as a desktop wallpaper. The first thing to do is to locate the data that has to be extracted on the desired web page. In the case of the Image of the day gallery (see screenshot below), there are three elements that are important to note: Title of the image that is being currently displayed Link to the image full resolution file Next link to make it possible navigate through all the images To find out the location of each piece of interesting information, as it was already suggested in the previous article, it's better to use a tool such as Firebug whose inspect functionality can be really useful. The following picture, for example, shows the location of the image title inside an h3 tag: The other two fields can be located as easily as the title, so no further explanation will be given here. Please refer to the previous article for further information. Download As explained in the introduction, to download the content of the web page, we will use Windmill as it allows the JavaScript code to execute in the web browser before getting the page content. Because Windmill is mostly a testing library, instead of writing a script that calls the Windmill API, I will write a test case for Windmill to navigate through all the image web pages. The code for the test should be as follows: 1 def test_scrape_iotd_gallery(): 2 """ 3 Scrape NASA Image of the Day Gallery 4 """ 5 # Extra data massage for BeautifulSoup 6 my_massage = get_massage() 7 8 # Open main gallery page 9 client = WindmillTestClient(__name__) 10 client.open(url='http://www.nasa.gov/multimedia/imagegallery/iotd.html') 11 12 # Page isn't completely loaded until image gallery data 13 # has been updated by javascript code 14 client.waits.forElement(xpath=u"//div[@id='gallery_image_area']/img", 15 timeout=30000) 16 17 # Scrape all images information 18 images_info = {} 19 while True: 20 image_info = get_image_info(client, my_massage) 21 22 # Break if image has been already scrapped 23 # (that means that all images have been parsed 24 # since they are ordered in a circular ring) 25 if image_info['link'] in images_info: 26 break 27 28 images_info[image_info['link']] = image_info 29 30 # Click to get the information for the next image 31 client.click(xpath=u"//div[@class='btn_image_next']") 32 33 # Print results to stdout ordered by image name 34 for image_info in sorted(images_info.values(), 35 key=lambda image_info: image_info['name']): 36 print ("Name: %(name)sn" 37 "Link: %(link)sn" % image_info) As it can be seen, the usage of Windmill is similar to other libraries such as mechanize. For example, first of all a client object has to be created to interact with the browser, (line 9) and later, the main web page, that is going to be used to navigate through all the information, has to be opened (line 10). Nevertheless, it also includes some facilities that take into account JavaScript code as shown at line 14. In this line, the waits.forElement method has been used to look for DOM element that is filled by the JavaScript code so when that element, in this case the big image in the image gallery, is displayed, the rest of the script can proceed. It is important to note here that the web page processing doesn't start when the page is downloaded (this happens after line 10), but when there's some evidence that JavaScript code has finished the DOM tree manipulation. For navigating through all the pages that contain the information needed, this is just a matter of pressing over the next arrow (line 30). As the images are ordered in a circular buffer, the point when it is decided to stop is when the same image link has been parsed twice (line 25). To execute the script, instead of launching it as we would normally do for a python script, we should call it through the Windmill script to properly initialize the environment: $ windmill firefox test=nasa_iotd.py As it can be seen in the following screenshot, Windmill takes care of opening a browser (Firefox in this case) window and a controller window in which it's possible to see the commands that the script is executing (several clicks on next in the example): The controller window is really interesting because not only does it display the progress of the test cases, but also allows to enter/record actions interactively, which is a nice feature when trying things out. In particular, the recording may be used under some situations to replace Firebug in the exploration step. This is because the captured actions may be stored in a script without spending much time in xpath expressions. For more information about how to use Windmill and the complete API, please refer to the Windmill documentation.

This is the third article in the article mini-series on Python LDAP applications by Matt Butcher. The first part deals with the installation and configuration of Python-LDAP library, and the binding-unbinding operations, and changing of the LDAP password. The second article takes a look at some of LDAP operations. In this article we will see some more LDAP operations such as add operation, delete operation etc. Then we will take a look at LDAP URL Library. The ModRDN Operation Another simple write operation that can be done through the Python-LDAP API is the ModRDN operation. This operation is used to change the relative DN (RDN) of a record. We can change an RDN using the modrdn() or modrdn_s() method. These two methods take three parameters: The full DN The new RDN An optional flag indicating whether the attribute corresponding to the RDN should be deleted from the record For example, if we want to change the UID attribute for uid=manny,ou=users,dc=example,dc=com, we will need to use a ModRDN operation, since this attribute is used in the DN. Here's an example for changing the UID from manny to immanuel. >>> l.modrdn_s('uid=manny,ou=users,dc=example,dc=com',... 'uid=immanuel', False)(109, [])>>> l.compare_s('uid=immanuel,ou=users,dc=example,dc=com','uid',... 'immanuel')1>>> In this example, we first use modrdn_s() to change the DN of a record from uid=manny,ou=users,dc=example,dc=com to uid=immanuel,ou=users,dc=example,dc=com. The False flag at the end of the modrdn_s() method indicates that the old UID (uid=manny) should be left in the record. The LDIF for uid=immanuel's record now, after the ModRDN operation, looks something like this: dn: uid=immanuel,ou=Users,dc=example,dc=comcn: Manny KantgivenName: MannyobjectClass: personobjectClass: organizationalPersonobjectClass: inetOrgPersonou: Userssn: Kantuid: immanueluid: manny If we had set the last flag to True instead of False, the manny attribute value of uid would have been deleted. More sophisticated DN modifications can be made with the rename() and rename_s() methods. But your OpenLDAP server will need to be running the HDB backend for all of the renaming features to work. The Add Operation The LDAP add operation is used to add new (complete) records to the directory information tree. Here, we will look at adding records through the add() and add_s() methods of the LDAPObject class. Both of these methods take only two parameters: The string DN of the new record A list of attribute tuples While the first parameter is straightforward, we've looked at dozens of DNs already; the second attribute is a little trickier. The addition list looks something like this: add_record = [ ('objectclass', ['person','organizationalperson','inetorgperson']), ('uid', ['francis']), ('cn', ['Francis Bacon'] ), ('sn', ['Bacon'] ), ('userpassword', ['secret']), ('ou', ['users'])] If there is only one value in the attribute value list, the value can be just a string – it need not be a list. Example: ('ou', 'user') is an acceptable alternative to ('ou', ['user']). The list of attributes is made up of two-value tuples, where the first item of each tuple is the attribute name, and the second value is a list of attribute values. All of the values are expected to be strings. If you have values in a dictionary, where the attribute name is the key and the attribute values are stored in a list in the dictionary value, you can use the ldap.modlist module's addModList() function to create an attributes list in the form specified above. Once you have a list in the correct format, writing it to the directory is just a matter of executing the add() or add_s() method. >>> l.add_s('uid=francis,ou=users,dc=example,dc=com', add_record)(105, [])>>> This line performs an LDAP add operation, sending this new data to the server. The server ensures that the new record adheres to the appropriate schemas (e.g. the schemas for the person, organizationalPerson, and inetOrgPerson object classes), and then writes the entry to the directory. As might be expected, the add() method functions the same way that the add_s() method does, except that it returns an ID number. The result must be retrieved using the result() method. We can dump the new entry from the server (using the dump_record.py program developed earlier in the series) to verify that the record is as we expect it to be: $ ./dump_record.py 'uid=matt,ou=users,dc=example,dc=com' 'uid=francis, ou=users,dc=example,dc=com'Password for uid=matt,ou=users,dc=example,dc=com:dn: uid=francis,ou=users,dc=example,dc=comcn: Francis BaconobjectClass: personobjectClass: organizationalPersonobjectClass: inetOrgPersonou: userssn: Baconuid: francisuserPassword: secret We can tell by comparing this record with the add_record list above that the record is correct. The main error encountered when adding is violating the schema, either by adding attributes that are not supported, or by failing to add required attributes. When one of these conditions is met, an exception will be raised. For example, if no structural object class is specified in the attributes, an OTHER exception will be raised. If a record does not contain the attributes used in the UID, a NAMING_VIOLATION will be raised. If a record is missing an attribute required by a structural object class, an OBJECT_CLASS_VIOLATION will be raised, and so on. Of course, since all of these are subclasses of LDAPError, these numerous exceptions can all be caught in a try/except clause like this: >>> try:... l.add_s('uid=william,ou=users,dc=example,dc=com', attrs )... except ldap.LDAPError, e:... print e.message['info']... This will catch any of the LDAP exceptions, and display some of the error text, rather than showing the stack trace. Now we are ready to move on to the most complicated of writing operations: the LDAP modify operation. The Modify Operation Here we will look at the LDAP modify operation, which is used for modifying attributes – adding, replacing, or removing them from already-existing records. The OpenLDAP command line tool ldapmodify provides one way of performing this operation. In the Python-LDAP library, the modify() and modify_s() methods provide asynchronous and synchronous methods for performing modifications to the directory information tree. The signature of these methods is same as that of the add methods. There are two parameters: the DN and a list of modification tuples. The main difference is that the form of the tuples in this modification list is different than those in the add methods. A tuple in a modification list has three items: The modification type The attribute name A list of attribute values Modification type is one of three different constants defined in the ldap module: MOD_ADD: This is used to add an attribute value. If the attribute already exists (and the schema permits multiple values), the new value will be added, and the old value will remain. MOD_DELETE: The attribute value will be removed, if it exists. MOD_REPLACE: The given attribute values will replace all other values for that attribute name. In other words, all old values for the attribute will be deleted, and then this value will be added. For example, a simple list for adding a new givenName to an existing entry might look like this: mod_attrs = [( ldap.MOD_ADD, 'givenName', 'Francis' )] This list contains only one attribute to be modified. It will (if successful) add a new givenName attribute to the specified record. The modification can then be done with code like this: >>> mod_attrs = [( ldap.MOD_ADD, 'givenName', 'Francis' )]>>> l.modify_s('uid=francis,ou=users,dc=example,dc=com', mod_attrs)(103, [])>>> This will add the specified attribute value to the uid=francis record that we created above. As a result, dumping the LDIF record will show the newly added attribute: dn: uid=francis,ou=users,dc=example,dc=comcn: Francis BacongivenName: FrancisobjectClass: personobjectClass: organizationalPersonobjectClass: inetOrgPersonou: userssn: Baconuid: francisuserPassword: secret The highlighted line above shows the newly added attribute value. The modifyModList() function in the ldap.modlist module can help convert modification lists stored in dictionaries to the appropriate tuple-based format. What if Francis decided that he preferred to go by Frank? We could perform a slightly more sophisticated modification, changing his givenName to Frank, and adding a second CN value: >>> mod_attrs = [... ( ldap.MOD_REPLACE, 'givenName', 'Frank' ),... ( ldap.MOD_ADD, 'cn', 'Frank Bacon' )... ]>>> l.modify_s('uid=francis,ou=users,dc=example,dc=com', mod_attrs)(103, [])>>> Notice that our modification list now has two different modifications. First, it will replace givenName. Second, it will add a new cn attribute value. The result will be something like this: dn: uid=francis,ou=users,dc=example,dc=comcn: Francis Baconcn: Frank BacongivenName: FrankobjectClass: personobjectClass: organizationalPersonobjectClass: inetOrgPersonou: userssn: Baconuid: francisuserPassword: secret If we wanted to change the UID attribute, we would have to use the modrdn() or modrdn_s() method, since uid is used in the DN. If we try to change it with modify_s() or modify(), we will get a NAMING_VIOLATION exception. Finally, we can use the modify methods to remove attribute values: >>> mod_attrs = [ (ldap.MOD_DELETE, 'cn','Francis Bacon') ]>>> l.modify_s('uid=francis,ou=users,dc=example,dc=com', mod_attrs)(103, [])>>> This will remove only the attribute value Francis Bacon from the cn attribute. If no such value exists, a NO_SUCH_ATTRIBUTE exception will be raised. Otherwise, the value will be discarded. Note that some attributes are required by the record's object classes to be present in an entry. Attempting to delete the last value for such an attribute will result in an OBJECT_CLASS_EXCEPTION being raised. Removing All Attribute Values Sometimes it is necessary to remove all of the values for an attribute in a record, instead of just one specific value, as we did above. Let's look at an example. First, we add a few attribute values – two descriptions: >>> mod_attrs = [ ... (ldap.MOD_ADD, 'description', 'Author of New Organon'),... (ldap.MOD_ADD, 'description', 'British empiricist') ... ]>>> l.modify_s('uid=francis,ou=users,dc=example,dc=com', mod_attrs)(103, []) Now we have a record with two new descriptions. We can perform a very specific search to verify this. >>> l.search_s('uid=francis,ou=users,dc=example,dc=com', ... ldap.SCOPE_BASE, '(uid=francis)',['description'])[('uid=francis,ou=users,dc=example,dc=com', {'description': ['Author of New Organon', 'British empiricist']})] This search looks at just the uid=francis record, and shows just the description attributes. Now, how can we delete both of these attribute values without having to supply the exact attribute values for each? We can do this removal by creating a modification entry that uses None instead of a string for the final item in the attribute tuple: >>> mod_attrs = [( ldap.MOD_DELETE, 'description', None )]>>> l.modify_s('uid=francis,ou=users,dc=example,dc=com', mod_attrs)(103, [])>>> A simple search will verify that both description attribute values have been deleted: >>> l.search_s('uid=francis,ou=users,dc=example,dc=com', ... ldap.SCOPE_BASE, '(uid=francis)',['description'])[('uid=francis,ou=users,dc=example,dc=com', {})]>>> The server returned one entry – one with the DN for uid=francis – but since there were no description attribute values, the dictionary is empty.

Introduction The major topics that we'll cover in this article are: Testing your application (in other words, uncovering problems before your users do it for you). Asking the right questions when users do discover problems. Using logging to help troubleshoot your problems. We'll also examine two important new Notes/Domino 7 features that can be critical for troubleshooting applications: Domino Domain Monitoring (DDM) Agent Profiler   For more troubleshooting issues visit: TroubleshootingWiki.org Testing your Application Testing an application before you roll it out to your users may sound like an obvious thing to do. However, during the life cycle of a project, testing is often not allocated adequate time or money. Proper testing should include the following: A meaningful amount of developer testing and bug fixing: This allows you to catch most errors, which saves time and frustration for your user community. User representative testing: A user representative, who is knowledgeable about the application and how users use it, can often provide more robust testing than the developer. This also provides early feedback on features. Pilot testing: In this phase, the product is assumed to be complete, and a pilot group uses it in production mode. This allows for limited stress testing as well as more thorough testing of the feature set. In addition to feature testing, you should test the performance of the application. This is the most frequently skipped type of testing, because some consider it too complex and difficult. In fact, it can be difficult to test user load, but in general, it's not difficult to test data load. So, as part of any significant project, it is a good practice to programmatically create the projected number of documents that will exist within the application, one or two years after it has been fully deployed, and have a scheduled agent trigger the appropriate number of edits-per-hour during the early phases of feature testing. Although this will not give a perfect picture of performance, it will certainly help ascertain whether and why the time to create a new document is unacceptable (for example, because the @Db formulas are taking too long, or because the scheduled agent that runs every 15 minutes takes too long due to slow document searches). Asking the Right Questions Suppose that you've rolled out your application and people are using it. Then the support desk starts getting calls about a certain problem. Maybe your boss is getting an earful at meetings about sluggish performance or is hearing gripes about error messages whenever users try to click a button to perform some action. In this section, we will discuss a methodology to help you troubleshoot a problem when you don't necessarily have all the information at your disposal. We will include some specific questions that can be asked verbatim for virtually any application. The first key to success in troubleshooting an application problem is to narrow down where and when it happens. Let's take these two very different problems suggested above (slow performance and error messages), and pose questions that might help unravel them: Does the problem occur when you take a specific action? If so, what is that action? Your users might say, "It's slow whenever I open the application", or "I get an error when I click this particular button in this particular form". Does the problem occur for everyone who does this, or just for certain people? If just certain people, what do they have in common? This is a great way to get your users to help you help them. Let them be a part of the solution, not just "messengers of doom". For example, you might ask questions such as, "Is it slow only for people in your building or your floor? Is it slow only for people accessing the application remotely? Is it slow only for people who have your particular access (for example, SalesRep)?" Does this problem occur all the time, at random times, or only at certain times? It's helpful to check whether or not the time of day or the day of week/month is relevant. So typical questions might be similar to the following: "Do you get this error every time you click the button or just sometimes? If just sometimes, does it give you the error during the middle of the day, but not if you click it at 7 AM when you first arrive? Do you only get the error on Mondays or some other day of the week? Do you only see the error if the document is in a certain status or has certain data in it? If it just happens for a particular document, please send me a link to that document so that I can inspect it carefully to see if there is invalid or unexpected data." Logging Ideally, your questions have narrowed down the type of problem it could be. So at this point, the more technical troubleshooting can start. You will likely need to gather concrete information to confirm or refine what you're hearing from the users. For example, you could put a bit of debugging code into the button that they're clicking so that it gives more informative errors, or sends you an email (or creates a log document) whenever it's clicked or whenever an error occurs. Collecting the following pieces of information might be enough to diagnose the problem very quickly: Time/date User name Document UNID (if the button is pushed in a document) Error Status or any other likely field that might affect your code By looking for common denominators (such as the status of the documents in question, or access or roles of the users), you will likely be able to further narrow down the possibilities of why the problem is happening. This doesn't solve your problem of course, but it helps in advancing you a long way towards that goal. A trickier problem to troubleshoot might be one we mentioned earlier: slow performance. Typically, after you've determined that there is some kind of performance delay, it's a good idea to first collect some server logging data. Set the following Notes.ini variables in the Server Configuration document in your Domino Directory, on the Notes.ini tab: Log_Update=1Log_AgentManager=1 These variables instruct the server to write output to the log.nsf database in the Miscellaneous Events view. Note that they may already be set in your environment. If not, they're fairly unobtrusive, and shouldn't trouble your administration group. Set them for a 24-hour period during a normal business week, and then examine the results to see if anything pops out as being suspicious. For view indexing, you should look for lines like these in the Miscellaneous Events (Log_Update=1): 07/01/2006 09:29:57 AM Updating views in appsSalesPipeline.nsf07/01/2006 09:30:17 AM Finished updating views in appsSalesPipeline.nsf07/01/2006 09:30:17 AM Updating views in appsTracking.nsf07/01/2006 09:30:17 AM Finished updating views in appsTracking.nsf07/01/2006 09:30:17 AM Updating views in appsZooSchedule.nsf07/01/2006 09:30:18 AM Finished updating views in appsZooSchedule.nsf And lines like these for Agent execution (Log_AgentManager=1): 06/30/2006 09:43:49 PM AMgr: Start executing agent 'UpdateTickets' in 'appsSalesPipeline.nsf ' by Executive '1'06/30/2006 09:43:52 PM AMgr: Start executing agent 'ZooUpdate' in 'appsZooSchedule.nsf ' by Executive '2'06/30/2006 09:44:44 PM AMgr: Start executing agent 'DirSynch' in 'appsTracking.nsf ' by Executive '1' Let's examine these lines to see whether or not there is anything we can glean from them. Starting with the Log_Update=1 setting, we see that it gives us the start and stop times for every database that gets indexed. We also see that the database file paths appear alphabetically. This means that, if we search for the text string updating views and pull out all these lines covering (for instance) an hour during a busy part of the day, and copy/paste these lines into a text editor so that they're all together, then we should see complete database indexing from A to Z on your server repeating every so often. In the log.nsf database, there may be many thousands of lines that have nothing to do with your investigation, so culling the important lines is imperative for you to be able to make any sense of what's going on in your environment. You will likely see dozens or even hundreds of databases referenced. If you have hundreds of active databases on your server, then culling all these lines might be impractical, even programmatically. Instead, you might focus on the largest group of databases. You will notice that the same databases are referenced every so often. This is the Update Cycle, or view indexing cycle. It's important to get a sense of how long this cycle takes, so make sure you don't miss any references to your group of databases. Imagine that SalesPipeline.nsf and Tracking.nsf were the two databases that you wanted to focus on. You might cull the lines out of the log that have updating views and which reference these two databases, and come up with something like the following: 07/01/2006 09:29:57 AM Updating views in appsSalesPipeline.nsf07/01/2006 09:30:17 AM Finished updating views in appsSalesPipeline.nsf07/01/2006 09:30:17 AM Updating views in appsTracking.nsf07/01/2006 09:30:20 AM Finished updating views in appsTracking.nsf07/01/2006 10:15:55 AM Updating views in appsSalesPipeline.nsf07/01/2006 10:16:33 AM Finished updating views in appsSalesPipeline.nsf07/01/2006 10:16:33 AM Updating views in appsTracking.nsf07/01/2006 10:16:43 AM Finished updating views in appsTracking.nsf07/01/2006 11:22:31 AM Updating views in appsSalesPipeline.nsf07/01/2006 11:23:33 AM Finished updating views in appsSalesPipeline.nsf07/01/2006 11:23:33 AM Updating views in appsTracking.nsf07/01/2006 11:23:44 AM Finished updating views in appsTracking.nsf This gives us some very important information: the Update task (view indexing) is taking approximately an hour to cycle through the databases on the server; that's too long. The Update task is supposed to run every 15 minutes, and ideally should only run for a few minutes each time it executes. If the cycle is an hour, then that means update is running full tilt for that hour, and as soon as it stops, it realizes that it's overdue and kicks off again. It's possible that if you examine each line in the log, you'll find that certain databases are taking the bulk of the time, in which case it might be worth examining the design of those databases. But it might be that every database seems to take a long time, which might be more indicative of a general server slowdown. In any case, we haven't solved the problem; but at least we know that the problem is probably server-wide. More complex applications, and newer applications, tend to reflect server‑performance problems more readily, but that doesn't necessarily mean they carry more responsibility for the problem. In a sense, they are the "canary in the coal mine". If you suspect the problem is confined to one database (or a few), then you can increase the logging detail by setting Log_Update=2. This will give you the start time for every view in every database that the Update task indexes. If you see particular views taking a long time, then you can examine the design of those views. If no database(s) stand out, then you might want to see if the constant indexing occurs around the clock or just during business hours. If it's around the clock, then this might point to some large quantities of data that are changing in your databases. For example, you may be programmatically synchronizing many gigabytes of data throughout the day, not realizing the cost this brings in terms of indexing. If slow indexing only occurs during business hours, then perhaps the user/data load has not been planned out well for this server. As the community of users ramps up in the morning, the server starts falling behind and never catches up until evening. There are server statistics that can help you determine whether or not this is the case. (These server statistics go beyond the scope of this book, but you can begin your investigation by searching on the various Notes/Domino forums for "server AND performance AND statistics".) As may be obvious at this point, troubleshooting can be quite time-consuming. The key is to make sure that you think through each step so that it either eliminates something important, or gives you a forward path. Otherwise, you can find yourself still gathering information weeks and months later, with users and management feeling very frustrated. Before moving on from this section, let's take a quick look at agent logging. Agent Manager can run multiple agents in different databases, as determined by settings in your server document. Typically, production servers only allow two or three concurrent agents to run during business hours, and these are marked in the log as Executive '1', Executive '2', and so on. If your server is often busy with agent execution, then you can track Executive '1' and see how many different agents it runs, and for how long. If there are big gaps between when one agent starts and when the next one does (for Executive '1'), this might raise suspicion that the first agent took that whole time to execute. To verify this, turn up the logging by setting the Notes.ini variable debug_amgr=*. (This will output a fair amount of information into your log, so it's best not to leave it on for too long, but normally one day is not a problem.) Doing this will give you a very important piece of information: the number of "ticks" it took for the agent to run. One second equals 100 ticks, so if the agent takes 246,379 ticks, this equals 2,463 seconds (about 41 minutes). As a general rule, you want scheduled agents to run in seconds, not minutes; so any agent that is taking this long will require some examination. In the next section, we will talk about some other ways you can identify problematic agents. Domino Domain Monitoring (DDM) Every once in a while, a killer feature is introduced—a feature so good, so important, so helpful, that after using it, we just shake our heads and wonder how we ever managed without it for so long. Domino Domain Monitor (DDM) is just such a feature. DDM is too large to be completely covered in this one section, so we will confine our overview to what it can do in terms of troubleshooting applications. For a more thorough explanation of DDM and all its features, see the book, Upgrading to Lotus Notes and Domino (www.packtpub.com/upgrading_lotus/book). In the events4.nsf database, you will find a new group of documents you can create for tracking agent or application performance. On Domino 7 servers, a new database is created automatically with the filename ddm.nsf. This stores the DDM output you will examine. For application troubleshooting, some of the most helpful areas to track using DDM are the following: Full-text index needs to be built. If you have agents that are creating a full‑text index on the fly because the database has no full‑text index built, DDM can track that potential problem for you. Especially useful is the fact that DDM compiles the frequency per database, so (for instance) you can see if it happens once per month or once per hour. Creating full‑text indexes on the fly can result in a significant demand on server resources, so having this notification is very useful. We discuss an example of this later in this section. Agent security warnings. You can manually examine the log to try to find errors about agents not being able to execute due to insufficient access. However, DDM will do this for you, making it much easier to find (and therefore fix) such problems. Resource utilization. You can track memory, CPU, and time utilization of your agents as run by Agent Manager or by the HTTP task. This means that at any time you can open the ddm.nsf database and spot the worst offenders in these categories, over your entire server/domain. We will discuss an example of CPU usage later in this section. The following illustration shows the new set of DDM views in the events4.nsf (Monitoring configuration) database: The following screenshot displays the By Probe Server view after we've made a few document edits: Notice that there are many probes included out-of-the-box (identified by the property "author = Lotus Notes Template Development") but set to disabled. In this view, there are three that have been enabled (ones with checkmarks) and were created by one of the authors of this book. If you edit the probe document highlighted above, Default Application Code/Agents Evaluated By CPU Usage (Agent Manager), the document consists of three sections. The first section is where you choose the type of probe (in this case Application Code) and the subtype (in this case Agents Evaluated By CPU Usage). The second section allows you to choose the servers to run against, and whether you want this probe to run against agents/code executed by Agent Manager or by the HTTP task (as shown in the following screenshot). This is an important distinction. For one thing, they are different tasks, and therefore one can hit a limit while the other still has room to "breathe". But perhaps more significantly, if you choose a subtype of Agents Evaluated By Memory Usage, then the algorithms used to evaluate whether or not an agent is using too much memory are very different. Agents run by the HTTP task will be judged much more harshly than those run by the Agent Manager task. This is because with the HTTP task, it is possible to run the same agent with up to hundreds of thousands of concurrent executions. But with Agent Manager, you are effectively limited to ten concurrent instances, and none within the same database. The third section allows you to set your threshold for when DDM should report the activity: You can select up to four levels of warning: Fatal, Failure, Warning (High), and Warning (Low). Note that you do not have the ability to change the severity labels (which appear as icons in the view). Unless you change the database design of ddm.nsf, the icons displayed in the view and documents are non-configurable. Experiment with these settings until you find the approach that is most useful for your corporation. Typically, customers start by overwhelming themselves with information, and then fine-tuning the probes so that much less information is reported. In this example, only two statuses are enabled: one for six seconds, with a label of Warning (High), and one for 60 seconds, with a label of Failure. Here is a screenshot of the DDM database: Notice that there are two Application Code results, one with a status of Failure (because that agent ran for more than 60 seconds), and one with a status of Warning (High) (because that agent ran for more than six seconds but less than 60 seconds). These are the parameters set in the Probe document shown previously, which can easily be changed by editing that Probe document. If you want these labels to be different, you must enable different rows in the Probe document. If you open one of these documents, there are three sections. The top section gives header information about this event, such as the server name, the database and agent name, and so on. The second section includes the following table, with a tab for the most recent infraction and a tab for previous infractions. This allows you to see how often the problem is occurring, and with what severity. The third section provides some possible solutions, and (if applicable) automation. For example, in our example, you might want to "profile" your agent. (We will profile one of our agents in the final section of this article.) DDM can capture full-text operations against a database that is not full‑text indexed. It tracks the number of times this happens, so you can decide whether to full‑text index the database, change the agent, or neither. For a more complete list of the errors and problems that DDM can help resolve, check the Domino 7 online help or the product documentation (www.lotus.com). Agent Profiler If any of the troubleshooting tips or techniques we've discussed in this article causes you to look at an agent and think, "I wonder what makes this agent so slow", then the Agent Profiler should be the next tool to consider. Agent Profiler is another new feature introduced in Notes/Domino 7. It gives you a breakdown of many methods/properties in your LotusScript agent, telling you how often each one was executed and how long they took to execute. In Notes/Domino 7, the second (security) tab of Agent properties now includes a checkbox labeled Profile this agent. You can select this option if you want an agent to be profiled. The next time the agent runs, a profile document in the database is created and filled with the information from that execution. This document is then updated every time the agent runs. You can view these results from the Agent View by highlighting your agent and selecting Agent | View Profile Results. The following is a profile for an agent that performed slow mail searches: Although this doesn't completely measure (and certainly does not completely troubleshoot) your agents, it is an important step forward in troubleshooting code. Imagine the alternative: dozens of print statements, and then hours of collating results! Summary In closing, we hope that this article has opened your eyes to new possibilities in troubleshooting, both in terms of techniques and new Notes/Domino 7 features. Every environment has applications that users wish ran faster, but with a bit of care, you can troubleshoot your performance problems and find resolutions. After you have your servers running Notes/Domino 7, you can use DDM and Agent Profiler (both exceptionally easy to use) to help nail down poorly performing code in your applications. These tools really open a window on what had previously been a room full of mysterious behavior. Full-text indexing on the fly, code that uses too much memory, and long running agents are all quickly identified by Domino Domain Monitoring (DDM). Try it!

Introduction WPF, an acronym for Windows Presentation Foundation is a subsystem of class libraries for WinFX and it enables the user to get a richer experience bringing together UI, Documents, media etc. A XAML (Extensible Application Markup Language) file which is at the heart of a WPF project can be created in several ways that includes the Notepad text editor, the Expression Blend which requires another download from Microsoft, but may not provide a easy to use XAML file to use in VS, and the Visual Studio editions except the express edition. XAML is presently specific to windows platform and is a XML formatting language and not an application programming interface. I will be mostly showing how to get some hands-on experience with a WPF project using the Visual Studio 2005 interface and the template files that you may access with the Windows SDK installed. Creating a WPF Project From File | New | Project click open the New Project window as shown in the next figure. Click on Visual Basic and expand its contents. Under .NET 3.0 FrameWork (It is assumed that you have installed NET 3.0 Framework) choose the Windows Application (WPF). Now highlight the Windows Application (WPF) and change the name of the application to some name of your choice. For this article it is changed to AppWPF. Click on the OK button after typing a name of your choice. This creates the necessary file/folders for the application as shown in the next figure. There are two XAML files created in the project. The App.xaml and the Windows1.xaml file. Delete the Windows1.xaml and add a new item as shown with the name BasicControls.xaml. With this new item added you may need to change the App.xaml file as shown below. <Application x_Class="App" StartupUri="BasicControls.xaml"> <Application.Resources> </Application.Resources> </Application> The StartupUri has been changed from the original Windows1.xaml to BasicControls.xaml. With this change made you can now display the BasicControls.xaml file together with its design as shown in the next figure. This represents a 300 X 300 window which can be used as a container for other controls. You also notice the reference to the namespaces that are required and the XML syntax with the attribute of the project for the window. Placing Controls on the Window Placing Controls automatically creates XAML code. Placing controls on this window is as easy as dragging from the Tools and dropping on to this window. The next picture shows a button and a textbox dragged and dropped onto this window. The necessary code for these controls gets automatically added as the controls are placed. After the two controls are added, the xaml file gets changed as shown. The Button and Textbox properties are the defaults which may be modified as will be seen later in the article. <Window x_Class="BasicControls" Title="AppWPF" Height="300" Width="300" > <Grid> <Button Height="23" Margin="94,0,123,39" Name="Button1" VerticalAlignment="Bottom">Button</Button> <TextBox HorizontalAlignment="Left" Margin="43, 126, 0,115" Name="TextBox1" Width="100"></TextBox> </Grid> </Window> Adding code automatically updates the window design. Inserting a declarative code into the BasicControls.xaml file will automatically add the control defined by that code to the design window. Add this code to the xaml file after as shown in the next paragraph. As soon as you type "<", the intellisense gets fired up and you will see a drop-down list of items that you can insert as shown in the next figure. Now you click on the Textbox (or whatever else you wish to place). This adds to the xaml file. Now to the opening tag of the textbox, you add a name attribute and call it TextBox2. Intellisense is also used in adding attributes as you will get a context sensitive listing of attributes for the chosen control. Also add other attributes such as width, height, alignment etc. With the code added as shown in the next paragraph you will see that the design pane has a new textbox as shown in the next figure.   <Textbox Name="TextBox2" Height="20" Margin="89.5,96.5,0,0" VerticalAlignment="Top" HorizontalAlignment="Left" Width="50"></TextBox*gt; The property window for the TextBox2 shown can also be used to make changes. You can also move, or adjust the dimensions of the controls using the mouse. The various controls provide a very rich interface for the designer in manipulating the controls. Event Handling All 'Hello World' programs used a button click to demonstrate the workings of the code or how the events were handled. In this tutorial also you will demonstrate the click event along the same lines. In the Solution Explorer only a few items are seen but there are lot more files in the project. Click on the middle toolbar just above the project as shown in the next figure. This will allow you to see all the files / folders in the project displayed (every folder expanded out) as shown. This is vastly different from a legacy windows project. The references to the Presentation Foundation are all in the three references, PresentationCore, PresentationDesignDeveloper and PresentationDesignFramework. In order to appreciate the rich designer support you have to go to the ,Object Browser and look at the references. For example just the PresentationCore has the following namespaces shown in the next figure. The BasicControls.xaml file also has the code behind file, BasicControls.xaml.vb, as shown in the next figure. In the code page, the drop-down control displaying BasicControls presently has all the objects on this window listed in its menu. You can find the Button as well. With the button chosen you can use the second drop-down to access all the events of the Button in the second drop-down (presently showing Declarations). In this manner the button click event was chosen from the second drop-down. Here the Button1_Click has been set to display "Click is registered" in Textbox1 when the button is clicked. You can find the reference to this in the Object Browser as shown in the next figure. Object Browser is an extremely valuable resource that you should seek out to understand the underlying logic, the arguments of a function call, etc. When you build and execute the program and click on the button this is what you will see displayed. The top part is the design window and the bottom is the window when clicked. At this point you might be wondering how to improve the look and feel. Indeed the form looks drab since none of the properties have been used except for the most basic. The next figure shows how you may change the appearance by inserting the property attributes directly into the XMAL file. You will be better off using the intellisense rather than trying to guess the property based on your previous 'Windows' experience as shown in the next figure. You may also add attributes from the property window of the object which you can view when the object is highlighted (or clicked) in the design pane. The variety of attributes is just too many and when in doubt you will be able to drill down to the one you want to use in the Object Browser. The next code listing shows a few more attributes added to the Textbox1. As you might have seen in the intellisense pop-up windows, there is a large number of properties that you can tweak and events that you can trigger. Notice the [.] notation for the TextElement in the code listing, FontFamily being the child of the parent TextElement. Listing 1 <TextBox HorizontalAlignment="Left" Margin="43,126,0,115" Name="TextBox1" Width="150" TextElement.FontFamily="Times Roman" ToolTip="Xaml TextBox" FontWeight="Bold" AutoWordSelection="True" Foreground="BlueViolet" Background="Aquamarine" TextDecorations="Underline" > </TextBox> When the program is executed you will see the following displayed. Summary The article describes the steps to create a WPF project. The Design <-->Declarative Code interactivity is also described. The placing of controls and adding event handling code to the code behind page is explained with an example. While testing the "AutoWordSelection" did not function as it should by its definition. You may look up this in the 'Help'.

The Repeater control in ASP.NET is a data-bound container control that can be used to automate the display of a collection of repeated list items. These items can be bound to either of the following data sources: Database Table XML File In a Repeater control, the data is rendered as DataItems that are defined using one or more templates. You can even use HTML tags such as <li>, <ul>, or <div> if required. Similar to the DataGrid, DataList, or GridView controls, the Repeater control has a DataSource property that is used to set the DataSource of this control to any ICollection, IEnumerable, or IListSource instance. Once this is set, the data from one of these types of data sources can be easily bound to the Repeater control using itsDataBind() method. However, the Repeater control by itself does not support paging or editing of data. The Repeater control is light weight and does not contain so many features as the DataGrid contains. However, it enables you to place HTML code in its templates. It is great in situations where you need to display the data quickly and format the data to be displayed easily. Using the Repeater Control The Repeater control is a data-bound control that uses templates to display data. It does not have any built-in support for paging, editing, or sorting of the data that is rendered through one or more of its templates. The Repeater control works by looping through the records in your data source and then repeating the rendering of one of its templates called the ItemTemplate, one that contains the records that the control needs to render. To use this control, drag and drop the control in the design view of the web form onto a web form from the toolbox. Refer to the following screenshot: You can also drag and drop the Repeater control from the toolbox onto the source view directly. This is shown in the following screenshot: For customizing the behavior of this control, you have to use the built-in templates that this control comes with. These templates are actually blocks of HTML code. The Repeater control contains the following five templates: HeaderTemplate ItemTemplate AlternatingItemTemplate SeparatorTemplate FooterTemplate The following screenshot shows how a Repeater control looks when populated with data. Note that the templates (Header, Item, Footer, Alternate and Separator) have all been used. The following code snippet is an example of the order in which the templates of the Repeater control are used. <asp:Repeater id="repEmployee" runat="server"><HeaderTemplate>...</HeaderTemplate><ItemTemplate></ItemTemplate><FooterTemplate>...</FooterTemplate></asp:Repeater> When the Repeater control is bound to a data source, the data from the data source is displayed using the ItemTemplate element and any other optional elements, if used. Note that the contents of the HeaderTemplate and the FooterTemplate are rendered once for each Repeater control. The contents of the ItemTemplate are rendered for each record in the control. You can also use the additional AlternatingItemTemplate element after the ItemTemplate element for specifying the appearance of each alternate record. You can also use the SeparatorTemplate element between each record for specifying the separators for the records. Displaying Data Using the Repeater Control This section discusses how we can display data using the Repeater control. As discussed earlier, the Repeater control uses templates for formatting the data that it displays. The following code snippet displays the code in an .aspx file that contains a Repeater control. Note that we would be making use of templates and that the data would be bound to the control from the code-behind file using the DataManager class. The Repeater control is populated with data in the Page_Load event by reusing the DataManager(). Note how the SeparatorTemplate and the AlternatingItemTemplate have been used in the previous code example. Further, the DataBinder.Eval() method has been used to display the values of the corresponding fields from the data container, (in our case, the DataSet instance) in the Repeater control. The FooterTemplate uses the Total Records variable and substitutes its value to display the total number of records displayed by the control. The following is the output on execution. The Header and the Footer templates of the Repeater control are still rendered even if the data source does not contain any data. If you want to suppress their display, you can use the Visible property of the Repeater control and use it to suppress the display of these templates with a simple logic. Here is how you specify the Visible property of this control in your .aspx file to achieve this_Visible="<%# Repeater1.Items.Count > 0 %>"When you specify the Visible property as shown here, the Repeater is made visible only if there are records in your data source.   Displaying Checkboxes in a Repeater Control Let us now understand how we can display checkboxes in a Repeater Control and retrieve the number of checked items. We will use a Button control and a Label control in our page. When you click on the Button control, the number of checked items in the Repeater Control will be displayed in the Label control. The output on execution is similar to what is shown in the following screenshot: Here is the code that we will use in the .aspx file to display checkboxes in a Repeater control. The data is bound to the Repeater control in the Page_Load event as follows: Note that we have used the Page.IsPostBack to check whether the page has posted back in the Page_Load method. If you don't bind data by checking whether the page has posted back, the Repeater control will be rebound to data once again after a postback and all the checkboxes in your web page will be reset to the unchecked state. The source code for the click event of the Button control that we have used is as follows: When you execute the application, the Repeater control is displayed with records from the employee table. Now you check one or more of the checkboxes and then click on the Button control just beneath the Repeater control as follows: Note that the number of checked records is displayed in the Label Control. Summary This article discussed the Repeater control and how we can use it in ourASP.NET applications. Though this control does not support all the functionalities of other data controls, like DataGrid and GridView, it is still a good choice if you want faster rendering of data as it is light weight, and is very flexible.

Fundamentals of XHTML MP Since XHTML MP is based on XHTML, certain syntactical rules must be followed. Making syntactical errors is a good way to learn a programming language, but so that you don't get frustrated with them, here are some rules you must follow with XHTML MP! Remember, HTML is very forgiving in terms of syntax, but make a small syntax error in XHTML MP and the browser may refuse to show your page! Overall, XHTML elements consist of a start tag—element name and its attributes, element content, and closing tag. The format is like: <element attribute="value">element content</element> XHTML Documents Must be Well Formed Since XHTML is based on XML, all XHTML documents must adhere to thebasic XML syntax and be well formed. The document must also have a DOCTYPE declaration. Tags Must be Closed! All open tags must be closed. Even if it is an empty tag like "<br>", it must be used in the self-closed form like "<br />". Note the extra space before the slash. It's not mandatory, but makes things work with some older browsers. If you can validate within your editor, make it a practice to do that. Also cultivate the habit of closing a tag that you start immediately—even before you put in the content. That will ensure you don't miss closing it later on! Elements Must be Properly Nested You cannot start a new paragraph until you complete the previous one. You must close tags to ensure correct nesting. Overlapping is not allowed. So the following is not valid in XHTML MP: <p><b>Pizzas are <i>good</b>.</i></p> It should be written as: <p><b>Pizzas are <i>good</i>.</b></p> Elements and Attributes Must be in Lowercase XHTML MP is case sensitive. And you must keep all the element tags and all their attributes in lowercase, although values and content can be in any case. Attribute Values Must be Enclosed within Quotes HTML allowed skipping the quotation marks around attribute values. This will not work with XHTML MP as all attribute values must be enclosed within quotes—either single or double. So this will not work: <div align=center>Let things be centered!</div> It must be written as: <div align="center">Let things be centered!</div> Attributes Cannot be Minimized Consider how you would do a drop down in HTML: <select> <option value="none">No toppings</option> <option value="cheese" selected>Extra Cheese</option> <option value="olive">Olive</option> <option value="capsicum">Capsicum</option> </select> The same drop down in XHTML is done as: <select> <option value="none">No toppings</option> <option value="cheese" selected="selected">Extra Cheese</option> <option value="olive">Olive</option> <option value="capsicum">Capsicum</option> </select> The "selected" attribute of the "option" element has only one possible value and, with HTML, you can minimize the attribute and specify only the attribute without its value. This is not allowed in XHTML, so you must specify the attribute as well as its value, enclosed in quotes. Another similar case is the "checked" attribute in check boxes. XHTML Entities Must be Handled Properly If you want to use an ampersand in your XHTML code, you must use it as &amp; and not just &. & is used as a starting character for HTML entities—e.g. &nbsp;, &quot;, &lt;, &gt; etc. Just using & to denote an ampersand confuses the XML parser and breaks it. Similarly, use proper HTML Entities instead of quotation marks, less than/greater than signs, and other such characters. You can refer to http://www.webstandards.org/learn/reference/charts/entities/ for more information on XHTML entities. Most Common HTML Elements are Supported The following table lists different modules in HTML and the elements within them that are supported in XHTML MP version 1.2. You can use this as a quick reference to check what's supported. Module Element Structure body, head, html, title Text abbr, acronym, address, blockquote, br, cite, code, dfn, div, em, h1, h2, h3, h4, h5, h6, kbd, p, pre, q, samp, span, strong, var Presentation b, big, hr, i, small Style Sheet style element and style attribute Hypertext a List dl, dt, dd, ol, ul, li Basic Forms form, input, label, select, option, textarea, fieldset, optgroup Basic Tables caption, table, td, th, tr Image img Object object, param Meta Information meta Link link Base base Legacy start attribute on ol, value attribute on li Most of these elements and their attributes work as in HTML. Table support in mobile browsers is flaky, so you should avoid tables or use them minimally. We will discuss specific issues of individual elements as we go further. XHTML MP Does Not Support Many WML Features If you have developed WAP applications, you would be interested in finding the differences between WML (Wireless Markup Language—the predecessor of XHTML MP) and XHTML MP; apart from the obvious syntactical differences. You need to understand this also while porting an existing WML-based application to XHTML MP. Most of WML is easily portable to XHTML MP, but some features require workarounds. Some features are not supported at all, so if you need them, you should use WML instead of XHTML MP. WML 1.x will be supported in any mobile device that conforms to XHTML MP standards. Here is a list of important WML features that are not available in XHTML MP: There is no metaphor of decks and cards. Everything is a page. This means you cannot pre-fetch content in different cards and show a card based on some action. With XHTML MP, you either have to make a new server request for getting new content, or use named anchors and link within the page. You could use the <do> tag in WML to program the left and right softkeys on the mobile device. Programming softkeys is not supported in XHTML MP; the alternative is to use accesskey attribute in the anchor tag (<a>) to specify a key shortcut for a link. WML also supports client-side scripting using WMLScript—a language similar to JavaScript. This is not supported in XHTML MP yet, but will come in near future in the form of ECMA Script Mobile Profile (ECMP). WML also supported client-side variables. This made it easier to process form data, validate them on the client side, and to reuse user-filled data across cards. This is not supported in XHTML MP. With XHTML MP, you have to submit a form with a submit button. WML allowed this on a link. WML also had a format attribute on the input tag—specifying the format in which input should be accepted. You need to use CSS to achieve this with XHTML MP. There are no timers in XHTML MP. This was a useful WML feature making it easier to activate certain things based on a timer. You can achieve a similar effect in XHTML MP using a meta refresh tag. The WML events ontimer, onenterbackward, onenterforward, and onpick are not available in XHTML MP. You can do a workaround for the ontimer event, but if you need others, you have to stick to using WML for development. XHTML MP also does not support the <u> tag, or align attribute on the <p> tag, and some other formatting options. All these effects can be achieved using CSS though. Summary In this article, we had a look at the fundamentals of XHTML MP and also at the grammar that must be followed for development with it. Next, we listed different modules in HTML and the elements within them that are supported in XHTML MP version 1.2. We finished the article by listing the important WML features that are not available in XHTML MP.