How-To Tutorials

  Blender 2.5 Lighting and Rendering Bring your 3D world to life with lighting, compositing, and rendering Render spectacular scenes with realistic lighting in any 3D application using interior and exterior lighting techniques Give an amazing look to 3D scenes by applying light rigs and shadow effects Apply color effects to your scene by changing the World and Lamp color values A step-by-step guide with practical examples that help add dimensionality to your scene        Getting the right files Before we get started, we need a scene to work with. There are three scenes provided for our use—an outdoor scene, an indoor scene, and a hybrid scene that incorporates elements that are found both inside as well as outside. All these files can be downloaded from http://www.cgshark.com/lightingand-rendering/ The file we are going to use for this scene is called exterior.blend. This scene contains a tricycle, which we will light as if it were a product being promoted for a company. To download the files for this tutorial, visit http://www.cgshark.com/lighting-and-rendering/ and select exterior.blend. Blender render settings In computer graphics, a two-dimensional image is created from three-dimensional data through a computational process known as rendering. It's important to understand how to customize Blender's internal renderer settings to produce a final result that's optimized for our project, be it a single image or a full-length film. With the settings Blender provides us, we can set frame rates for animation, image quality, image resolution, and many other essential parts needed to produce that optimized final result. The Scene menu We can access these render settings through the Scene menu. Here, we can adjust a myriad of settings. For the sake of these projects, we are only going to be concerned with: Which window Blender will render our image in How render layers are set up Image dimensions Output location and file type Render settings The first settings we see when we look at the Scene menu are the Render settings. Here, we can tell Blender to render the current frame or an animation using the render buttons. We can also choose what type of window we want Blender to render our image in using the Display options. The first option (and the one chosen by default) is Full Screen. This renders our image in a window that overlaps the three-dimensional window in our scene. To restore the three-dimensional view, select the Back to Previous button at the top of the window. The next option is the Image Editor that Blender uses both for rendering as well as UV editing. This is especially useful when using the Compositor, allowing us to see our result alongside our composite node setup. By default, Blender replaces the three-dimensional window with the Image Editor. The last option is the option that Blender has used, by default, since day one—New Window. This means that Blender will render the image in a newly created window, separate from the rest of the program's interface. For the sake of these projects, we're going to keep this setting at the default setting—Full Screen. Dimensions settings These are some of the most important settings that we can set when dealing with optimizing our project output. We can set the image size, frame rate, frame range, and aspect ratio of our render. Luckily for us, Blender provides us with preset render settings, common in the film industry: HDTV 1080P HDTV 720P TV NTSC TV PAL TV PAL 16:9 Because we want to keep our render times relatively low for our projects, we're going to set our preset dimensions to TV NTSC, which results in an image 720 pixels wide by 480 pixels high. If you're interested in learning more about how the other formats behave, feel free to visit http://en.wikipedia.org/wiki/Display_resolution. Output settings These settings are an important factor when determining how we want our final product to be viewed. Blender provides us with numerous image and video types to choose from. When rendering an animation or image sequence, it's always easier to manually set the folder we want Blender to save to. We can tell Blender where we want it to save by establishing the path in the output settings. By default on Macintosh, Blender saves to the /tmp/ folder. Now that we understand how Blender's renderer works, we can start working with our scene! Establishing a workflow The key to constantly producing high-quality work is to establish a well-tested and efficient workflow. Everybody's workflow is different, but we are going to follow this series of steps: Evaluate what the scene we are lighting will require. Plan how we want to lay out the lamps in our scene. Set lamp positions, intensities, colors, and shadows, if applicable. Add materials and textures. Tweak until we're satisfied. Evaluating our scene Before we even begin to approach a computer, we need to think about our scene from a conceptual perspective. This is important, because knowing everything about our scene and the story that's taking place will help us produce a more realistic result. To help kick start this process, we can ask ourselves a series of questions that will get us thinking about what's happening in our scene. These questions can pertain to an entire array of possibilities and conditions, including: Weather What is the weather like on this particular day? What was it like the day before or the day after? Is it cloudy, sunny, or overcast? Did it rain or snow? Source of light Where is the light coming from? Is it in front of, to the side, or even behind the object? Remember, light is reflected and refracted until all energy is absorbed; this not only affects the color of the light, but the quality as well. Do we need to add additional light sources to simulate this effect? Scale of light sources What is the scale of our light sources in relation to our three-dimensional scene? Believe it or not, this factor carries a lot of weight when it comes to the quality of the final render. If any lights feel out of place, it could potentially affect the believability of the final product. The goal of these questions is to prove to ourselves that the scene we're lighting has the potential to exist in real life. It's much harder, if not impossible, to light a scene if we don't know how it could possibly act in the real world. Let's take a look at these questions. What is the weather like? In our case, we're not concerned with anything too challenging, weather wise. The goal of this tutorial is to depict our tricycle in an environment that reflects the effects of a sunny, cloudless day. To achieve this, we are going to use lights with blue and yellow hues for simulating the effect the sun and sky will have on our tricycle. What are the sources of our light and where are they coming from in relation to our scene? In a real situation, the sun would provide most of the light, so we'll need a key light that simulates how the sun works. In our case, we can use a Sun lamp. The key to positioning light sources within a three-dimensional scene is to find a compromise between achieving the desired mood of the image and effectively illuminating the object being presented. What is the scale of our light sources? The sun is rather large, but because of the nature of the Sun lamp in Blender, we don't have to worry about the scale of the lamp in our three-dimensional scene. Sometimes—more commonly when working with indoor scenes, such as the scene we'll approach later—certain light sources need to be of certain sizes in relation to our scene, otherwise the final result will feel unnatural. Although we will be using a realistic approach to materials, textures, and lighting, we are going to present this scene as a product visualization. This means that we won't explicitly show a ground plane, allowing the viewer to focus on the product being presented, in this case, our tricycle.

  ASP.NET Site Performance Secrets Simple and proven techniques to quickly speed up your ASP.NET website Speed up your ASP.NET website by identifying performance bottlenecks that hold back your site's performance and fixing them Tips and tricks for writing faster code and pinpointing those areas in the code that matter most, thus saving time and energy Drastically reduce page load times Configure and improve compression – the single most important way to improve your site's performance Written in a simple problem-solving manner – with a practical hands-on approach and just the right amount of theory you need to make sense of it all. The reader can benefit from the previous articles on Pinpointing bottlenecks for better Database Access in ASP.Net and Fixing bottlenecks for better Database Access in ASP.Net. Locking In this section, you'll see how to determine which queries are involved in excessive locking delays, and how to prevent those delays from happening. Gathering detailed locking information You can find out which queries are involved in excessive locking delays by tracing the event "Blocked process report" in SQL Server Profiler. This event fires when the lock wait time for a query exceeds the "blocked process threshold". To set this threshold to, for example, 30 seconds, run the following lines in a query window in SSMS (locking.sql in the downloaded code bundle): DBCC TRACEON(1222,-1) Then, start the trace in Profiler: Start SQL Profiler. Click on Start | Programs | Microsoft SQL Server 2008 | Performance Tools | SQL Server Profiler. In SQL Profiler, click on File | New Trace. Click on the Events Selection tab. Select Show all events checkbox to see all events. Also select Show all columns to see all the data columns. In the main window, expand Errors and Warnings and select the Blocked process report event. Make sure the checkbox in the TextData column is checked—scroll horizontally if needed to find it. If you need to investigate deadlocks, also expand Locks and select the Deadlock graph event. To get additional information about deadlocks, have SQL Server write information about each deadlock event to its error log, by executing the following from an SSMS query window: ALTER DATABASE mydatabase SET SINGLE_USER WITH ROLLBACK IMMEDIATE; ALTER DATABASE mydatabase SET READ_COMMITTED_SNAPSHOT ON; ALTER DATABASE mydatabase SET MULTI_USER; Uncheck all the other events, unless you are interested in them. Click on Run to start the trace. Save the template, so that you don't have to recreate it the next time. Click on File | Save As | Trace Template. Fill in a descriptive name and click on OK. Next time you create a new trace by clicking on File | New Trace, you can retrieve the template from the Use the template drop-down. Once you have captured a representative sample, click File | Save to save the trace to a trace file for later analysis. You can load a trace file by clicking on File | Open. (Move the mouse over the image to enlarge.) When you click a Blocked process report event in Profiler, you'll find information about the event in the lower pane, including the blocking query and the blocked query. You can get details about Deadlock graph events the same way. To check the SQL Server error log for deadlock events: In SSMS expand the database server, expand Management and expand SQL Server Logs. Then double-click on a log. In the Log File Viewer, click on Search near the top of the window and search for "deadlock-list". In the lines that chronologically come after the deadlock-list event, you'll find much more information about the queries involved in the deadlock. Reducing blocking Now that you identified the queries involved in locking delays, it's time to reduce those delays. The most effective way to do this is to reduce the length of time locks are held as follows: Optimize queries. The lesser time your queries take, the lesser time they hold locks. Use stored procedures rather than ad hoc queries. This reduces time spent compiling execution plans and time spent sending individual queries over the network. If you really have to use cursors, commit updates frequently. Cursor processing is much slower than set-based processing. Do not process lengthy operations while locks are held, such as sending e-mails. Do not wait for user input while keeping a transaction open. Instead, use optimistic locking, as described in: Optimistic Locking in SQL Server using the ROWVERSION Data Type http://www.mssqltips.com/tip.asp?tip=1501 A second way to reduce lock wait times is to reduce the number of resources being locked: Do not put a clustered index on frequently updated columns. This requires a lock on both the clustered index and all non-clustered indexes, because their row locator contains the value you are updating. Consider including a column in a non-clustered index. This would prevent a query from having to read the table record, so it won't block another query that needs to update an unrelated column in the same record. Consider row versioning. This SQL Server feature prevents queries that read a table row from blocking queries that update the same row and vice versa. Queries that need to update the same row still block each other.Read versioning works by storing rows in a temporary area (in tempdb) before they are updated, so that reading queries can access the stored version while the update is taking place. This does create an overhead in maintaining the row versions—test this solution before taking it live. Also, in case you set the isolation level of transactions, row versioning only works with the Read Committed isolation mode, which is the default isolation mode.To implement row versioning, set the READ_COMMITTED_SNAPSHOT option as shown in the following code (locking.sql in the downloaded code bundle). When doing this, you can have only one connection open—the one used to set the option. You can make that happen by switching the database to single user mode; warn your users first. Be careful when applying this to a production database, because your website won't be able to connect to the database while you are carrying out this operation. select is_read_committed_snapshot_on from sys.databases where name='mydatabase' To check whether row versioning is in use for a database, run: SET LOCK_TIMEOUT 5000 Finally, you can set a lock timeout. For example, to abort statements that have been waiting for over five seconds (or 5000 milliseconds), issue the following command: SET TRANSACTION ISOLATION LEVEL REPEATABLE READ BEGIN TRAN SELECT Title FROM dbo.Book UPDATE dbo.Book SET Author='Charles Dickens' WHERE Title='Oliver Twist' COMMIT Use 1 to wait indefinitely. Use 0 to not wait at all. Reducing deadlocks Deadlock is a situation where two transactions are waiting for each other to release a lock. In a typical case, transaction 1 has a lock on resource A and is trying to get a lock on resource B, while transaction 2 has a lock on resource B and is trying to get a lock on resource A. Neither transaction can now move forward, as shown below: One way to reduce deadlocks is to reduce lock delays in general, as shown in the last section. That reduces the time window in which deadlocks can occur. A second way is suggested by the diagram—always lock resources in the same order. If, as shown in the diagram, you get transaction 2 to lock the resources in the same order as transaction 1 (first A, then B), then transaction 2 won't lock resource B before it starts waiting for resource A. Hence, it doesn't block transaction 1. Finally, watch out for deadlocks caused by the use of HOLDLOCK or Repeatable Read or Serializable Read isolation levels. Take for example the following code: SET TRANSACTION ISOLATION LEVEL REPEATABLE READ BEGIN TRAN SELECT Title FROM dbo.Book WITH(UPDLOCK) UPDATE dbo.Book SET Author='Charles Dickens' WHERE Title='Oliver Twist' COMMIT Imagine two transactions running this code at the same time. Both acquire a Select lock on the rows in the Book table when they execute the SELECT. They hold onto the lock because of the Repeatable Read isolation level. Now, both try to acquire an Update lock on a row in the Book table to execute the UPDATE. Each transaction is now blocked by the Select lock the other transaction is still holding. To prevent this from happening, use the UPDLOCK hint on the SELECT statement. This causes the SELECT to acquire an Update lock, so that only one transaction can execute the SELECT. The transaction that did get the lock can then execute its UPDATE and free the locks, after which the other transaction comes through. The code is as follows: SELECT b.Title, a.AuthorName FROM dbo.Book b JOIN dbo.Author a ON b.LeadAuthorId=a.Authorid WHERE BookId=5

  ASP.NET Site Performance Secrets Simple and proven techniques to quickly speed up your ASP.NET website Speed up your ASP.NET website by identifying performance bottlenecks that hold back your site's performance and fixing them Tips and tricks for writing faster code and pinpointing those areas in the code that matter most, thus saving time and energy Drastically reduce page load times Configure and improve compression – the single most important way to improve your site's performance Written in a simple problem-solving manner – with a practical hands-on approach and just the right amount of theory you need to make sense of it all           Read more about this book       (For more resources on ASP.Net, see here.) The reader can benefit from the previous article on Pinpointing bottlenecks for better Database Access in ASP.Net. Now that you have pinpointed the bottlenecks to prioritize, skip to the appropriate subsection to find out how to fix those bottlenecks. Missing indexes Just as using an index in a book to find a particular bit of information is often much faster than reading all pages, SQL Server indexes can make finding a particular row in a table dramatically faster by cutting down the number of read operations. This section first discusses the two types of indexes supported by SQL Server: clustered and non-clustered. It also goes into included columns, a feature of nonclustered indexes. After that, we'll look at when to use each type of index. Clustered index Take the following table (missingindexes.sql in the downloaded code bundle): CREATE TABLE [dbo].[Book]( [BookId] [int] IDENTITY(1,1) NOT NULL, [Title] [nvarchar](50) NULL, [Author] [nvarchar](50) NULL, [Price] [decimal](4, 2) NULL) Because this table has no clustered index, it is called a heap table. Its records are unordered, and to get all books with a given title, you have to read all the records. It has a very simple structure: Let's see how long it takes to locate a record in this table. That way, we can compare against the performance of a table with an index. To do that in a meaningful way, first insert a million records into the table (code to do this is in missingindexes.sql in the downloaded code bundle). Tell SQL Server to show I/O and timing details of each query we run: SET STATISTICS IO ONSET STATISTICS TIME ON Also, before each query, flush the SQL Server memory cache: CHECKPOINTDBCC DROPCLEANBUFFERS Now, run the query below with a million records in the Book table: SELECT Title, Author, Price FROM dbo.Book WHERE BookId = 5000 The results on my machine are: reads: 9564, CPU time: 109 ms, elapsed time: 808 ms. SQL Server stores all data in 8-KB pages. This shows that it read 9564 pages, that is, the entire table. Now, add a clustered index: ALTER TABLE BookADD CONSTRAINT [PK_Book] PRIMARY KEY CLUSTERED ([BookId] ASC) This puts the index on column BookId, making WHERE and JOIN statements on BookId faster. It sorts the table by BookId and adds a structure called a B-tree to speed up access: BookId is now used the same way as a page number in a book. Because the pages in a book are sorted by page number, finding a page by page number is very fast. Now, run the same query again to see the difference: SELECT Title, Author, Price FROM dbo.Book WHERE BookId = 5000 The results are: reads: 2, CPU time: 0 ms, elapsed time: 32 ms. The number of reads of 8-KB pages has gone from 9564 to 2, CPU time from 109ms to less than 1 ms, and elapsed time from 808 ms to 32 ms. That's a dramatic improvement. Non-clustered index Now let's select by Title instead of BookId: SELECT Title, Author FROM dbo.Book WHERE Title = 'Don Quixote' These results are pretty similar to what we got with the heap table, which is no wonder, seeing that there is no index on Title. The solution obviously is to put an index on Title. However, because a clustered index involves sorting the table records on the index field, there can be only one clustered index. We've already sorted on BookId, and the table can't be sorted on Title at the same time. The solution is to create a non-clustered index. This is essentially a duplicate of the table records, this time sorted by Title. To save space, SQL Server leaves out the other columns, such as Author and Price. You can have up to 249 non-clustered indexes on a table. Because we still want to access those other columns in queries though, we need a way to get from the non-clustered index records to the actual table records. The solution is to add the BookId to the non-clustered records. Because BookId has the clustered index, once we have found a BookId via the non-clustered index, we can use the clustered index to get to the actual table record. This second step is called a key lookup. Why go through the clustered index? Why not put the physical address of the table record in the non-clustered index record? The answer is that when you update a table record, it may get bigger, causing SQL Server to move subsequent records to make space. If non-clustered indexes contained physical addresses, they would all have to be updated when this happens. It's a tradeoff between slightly slower reads and much slower updates. If there is no clustered index or if it is not unique, then non-clustered index records do have the physical address. To see what a non-clustered index will do for us, first create it as follows: CREATE NONCLUSTERED INDEX [IX_Title] ON [dbo].[Book]([Title] ASC) Now, run the same query again: SELECT Title, Author FROM dbo.Book WHERE Title = 'Don Quixote' The results are: reads: 4, CPU time: 0 ms, elapsed time: 46 ms. The number of reads has gone from 9146 to 4, CPU time from 156 ms to less than 1 ms, and elapsed time from 1653 ms to 46 ms. This means that having a non-clustered index is not quite as good as having a clustered index, but still dramatically better than having no index at all. Included columns You can squeeze a bit more performance out of a non-clustered index by cutting out the key lookup—the second step where SQL Server uses the clustered index to find the actual record. Have another look at the test query—it simply returns Title and Author. Title is already present in the non-clustered index record. If you were to add Author to the non-clustered index record as well, there would be no longer any need for SQL Server to access the table record, enabling it to skip the key lookup. It would look similar to the following: This can be done by including Author in the non-clustered index: CREATE NONCLUSTERED INDEX [IX_Title] ON [dbo].[Book]([Title] ASC)INCLUDE(Author)WITH drop_existing Now, run the query again: SELECT Title, Author FROM dbo.Book WHERE Title = 'Don Quixote' The results are: reads: 2, CPU time: 0 ms, elapsed time: 26 ms. The number of reads has gone from 4 to 2, and elapsed time from 46 ms to 26 ms; that's almost 50 percent improvement. In absolute terms, the gain isn't all that great, but for a query that is executed very frequently, this may be worthwhile. Don't overdo this—the bigger you make the non-clustered index records, the fewer fit on an 8KB page, forcing SQL Server to read more pages. Selecting columns to give an index Because indexes do create overhead, you want to carefully select the columns to give indexes. Before starting the selection process, keep the following in mind: Putting a Primary Key on a column by default gives it a clustered index (unless you override the default). So, you may already have many columns in your database with an index. As you'll see later in the When to use a clustered index section, putting the clustered index on the ID column of a record is almost always a good idea. Putting an index on a table column affects all queries that use that table. Don't focus on just one query. Before introducing an index on your live database, test the index in development to make sure it really does improve performance. Let's look at when and when not to use an index, and when to use a clustered index. When to use an index You can follow this decision process when selecting columns to give an index: Start by looking at the most expensive queries. Look at putting an index on at least one column involved in every JOIN. Consider columns used in ORDER BY and GROUP BY clauses. If there is an index on such a column, than SQL Server doesn't have to sort the column again because the index already keeps the column values in sorted order. Consider columns used in WHERE clauses, especially if the WHERE will select a small number of records. However, keep in mind the following: A WHERE clause that applies a function to the column value can't use an index on that column, because the output of the function is not in the index. Take for example the following: SELECT Title, Author FROM dbo.Book WHERE LEFT(Title, 3) = 'Don' Putting an index on the Title column won't make this query any faster. Likewise, SQL Server can't use an index if you use LIKE in a WHERE clause with a wild card at the start of the search string, as in the following: SELECT Title, Author FROM dbo.Book WHERE Title LIKE '%Quixote' However, if the search string starts with constant text instead of a wild card, an index can be used: SELECT Title, Author FROM dbo.Book WHERE Title LIKE 'Don%' Consider columns that have a UNIQUE constraint. Having an index on the column makes it easier for SQL Server to check whether a new value would not be unique. The MIN and MAX functions benefit from working on a column with an index. Because the values are sorted, there is no need to go through the entire table to find the minimum or maximum. Think twice before putting an index on a column that takes a lot of space. If you use a non-clustered index, the column values will be duplicated in the index. If you use a clustered index, the column values will be used in all nonclustered indexes. The increased sizes of the index records means fewer fit in each 8-KB page, forcing SQL Server to read more pages. The same applies to including columns in non-clustered indexes. When not to use an index Having too many indexes can actually hurt performance. Here are the main reasons not to use an index on a column: The column gets updated often The column has low specificity, meaning it has lots of duplicate values Let's look at each reason in turn. Column updated often When you update a column without an index, SQL Server needs to write one 8KB page to disk, provided there are no page splits. However, if the column has a non-clustered index, or if it is included in a nonclustered index, SQL Server needs to update the index as well, so it has to write at least one additional page to disk. It also has to update the B-tree structure used in the index, potentially leading to more page writes. If you update a column with a clustered index, the non-clustered index records that use the old value need to be updated too, because the clustered index key is used in the non-clustered indexes to navigate to the actual table records. Secondly, remember that the table records themselves are sorted based on the clustered index. If the update causes the sort order of a record to change, that may mean more writes. Finally, the clustered index needs to keep its B-tree up-to-date. This doesn't mean you cannot have indexes on columns that get updated; just be aware that indexes slow down updates. Test the effect of any indexes you add. If an index is critical but rarely used, for example only for overnight report generation, consider dropping the index and recreating it when it is needed. Low specificity Even if there is an index on a column, the query optimizer won't always use it. Remember, each time SQL Server accesses a record via an index, it has to go through the index structure. In the case of a non-clustered index, it may have to do a key lookup as well. If you're selecting all books with price $20, and lots of books happen to have that price, than it might be quicker to simply read all book records rather than going through an index over and over again. In that case, it is said that the $20 price has low specificity. You can use a simple query to determine the average selectivity of the values in a column. For example, to find the average selectivity of the Price column in the Book table, use (missingindexes.sql in downloaded code bundle): SELECT COUNT(DISTINCT Price) AS 'Unique prices', COUNT(*) AS 'Number of rows', CAST((100 * COUNT(DISTINCT Price) / CAST(COUNT(*) AS REAL)) AS nvarchar(10)) + '%' AS 'Selectivity'FROM Book If every book has a unique price, selectivity will be 100 percent. However, if half the books cost $20 and the other half $30, then average selectivity will be only 50 percent. If the selectivity is 85 percent or less, an index is likely to incur more overhead than it would save. Some prices may occur a lot more often than other prices. To see the specificity of each individual price, you would run (missingindexes.sql in downloaded code bundle): DECLARE @c realSELECT @c = CAST(COUNT(*) AS real) FROM BookSELECT Price, COUNT(BookId) AS 'Number of rows', CAST((1 - (100 * COUNT(BookId) / @c)) AS nvarchar(20)) + '%' AS 'Selectivity'FROM BookGROUP BY PriceORDER BY COUNT(BookId) The query optimizer is unlikely to use a non-clustered index for a price whose specificity is below 85 percent. It figures out the specificity of each price by keeping statistics on the values in the table. When to use a clustered index You saw that there are two types of indexes, clustered and non-clustered, and that you can have only one clustered index. How do you determine the lucky column that will have the clustered index? To work this out, let's first look at the characteristics of a clustered index against a non-clustered index: Characteristic Clustered index compared to a non-clustered index Reading Faster: Because there is no need for key lookups. No difference if all the required columns are included in the non-clustered index. Updating Slower: Not only the table record, but also all non-clustered index records potentially need to be updated. Inserting/Deleting Faster: With a non-clustered index, inserting a new record in the table means inserting a new record in the non-clustered index as well. With a clustered index, the table is effectively part of the index, so there is no need for the second insert. The same goes for deleting a record. On the other hand, when the record is inserted at any place in the table but the very end, the insert may cause a page split where half the content of the 8-KB page is moved to another page. Having a page split in a non-clustered index is less likely, because its records are smaller (they normally don't have all columns that a table record has), so more records fit on a page. When the record is inserted at the end of the table, there won't be a page split. Column Size Needs to be kept short and fast - Every non-clustered index contains a clustered index value, to do the key lookup. Every access via a non-clustered index has to use that value, so you want it to be fast for the server to process. That makes a column of type int a lot better to put a clustered index on than a column of type nvarchar(50).   If only one column requires an index, this comparison shows that you'll probably want to give it the clustered index rather than a non-clustered index. If multiple columns need indexes, you'll probably want to put the clustered index on the primary key column: Reading: The primary key tends to be involved in a lot of JOIN clauses, making read performance important. Updating: The primary key should never or rarely get updated, because that would mean changing referring foreign keys as well. Inserting/Deleting: Most often you'll make the primary key an IDENTITY column, so each new record is assigned a unique, ever increasing number. This means that if you put the clustered index on the primary key, new records are always added at the end of the table. When a record is added at the end of a table with a clustered index and there is no space in the current page, the new record goes into a new page but the rest of the data in the current page stays in the page. In other words, there is no expensive page split. Size: Most often, the primary key is of type int, which is short and fast. Indeed, when you set the primary key on a column in the SSMS table designer, SSMS gives that column the clustered index by default, unless another column already has the clustered index.

  PostgreSQL 9.0 High Performance A clear, step-by-step guide to optimizing and scaling up PostgreSQL database servers Learn the right techniques to obtain optimal PostgreSQL database performance, from initial design to routine maintenance Discover the techniques used to scale successful database installations Avoid the common pitfalls that can slow your system down Filled with advice about what you should be doing; how to build experimental databases to explore performance topics, and then move what you've learned into a production database environment Covers versions 8.1 through 9.0           Read more about this book       The main tunable settings for PostgreSQL are in a plain text file named postgresql.conf that's located at the base of the database directory structure. This will often be where $PGDATA is set to on UNIX-like systems, making the file $PGDATA/postgresql.conf on those platforms. This article by Gregory Smith, author of PostgreSQL 9.0 High Performance, mirrors the general format of the official documentation's look at these parameters at http://www.postgresql.org/docs/current/static/runtime-config.html. However, it is more focused on guidelines for setting the most important values, from the perspective of someone interested in performance tuning, rather than describing the meaning of every parameter. This should be considered a supplement to rather than a complete replacement for the extensive material in the manual. Logging General logging setup is important but it is somewhat outside the scope of this article. You may need to set parameters such as log_destination, log_directory, and log_filename to save your log files in a way compatible with the system administrations requirements of your environment. These will all be set to reasonable defaults to get started with on most systems. On UNIX-like systems, it's common for some of the database logging to be set in the script that starts and stops the server, rather than directly in the postgresql.conf file. If you instead use the pg_ctl command to manually start the server, you may discover that logging ends up on your screen instead. You'll need to look at the script that starts the server normally (commonly /etc/init.d/postgresql) to determine what it does, if you want to duplicate that behavior. In most cases, you just need to add –l logfilename to the pg_ctl command line to redirect its output to the standard location. log_line_prefix The default log_line_prefix is empty, which is not what you want. A good starting value here is the following: log_line_prefix='%t:%r:%u@%d:[%p]: ' This will put the following into every log line: %t: Timestamp %u: Database user name %r: Remote host connection is from %d: Database connection is to %p: Process ID of connection It may not be obvious what you'd want all of these values for initially, particularly, the process ID. Once you've tried to chase down a few performance issues, the need for saving these values will be more obvious, and you'll be glad to already have this data logged. Another approach worth considering is setting log_line_prefix such that the resulting logs will be compatible with the pgFouine program. That is a reasonable, general purpose logging prefix, and many sites end up needing to do some sort of query analysis eventually. log_statement The options for this setting are as follows: none: Do not log any statement-level information. ddl: Log only Data Definition Language (DDL) statements such as CREATE and DROP. This can normally be left on even in production, and is handy to catch major changes introduced accidentally or intentionally by administrators. mod: Log any statement that modifies a value, which is essentially everything except for simple SELECT statements. If your workload is mostly SELECT based with relatively few data changes, this may be practical to leave enabled all the time. all: Log every statement. This is generally impractical to leave on in production due to the overhead of the logging. However, if your server is powerful enough relative to its workload, it may be practical to keep it on all the time. Statement logging is a powerful technique for finding performance issues. Analyzing the information saved by log_statement and related sources for statement-level detail can reveal the true source for many types of performance issues. You will need to combine this with appropriate analysis tools. log_min_duration_statement Once you have some idea of how long a typical query statement should take to execute, this setting allows you to log only the ones that exceed some threshold you set. The value is in milliseconds, so you might set: log_min_duration_statement=1000 And then you'll only see statements that take longer than one second to run. This can be extremely handy for finding out the source of "outlier" statements that take much longer than most to execute. If you are running 8.4 or later, you might instead prefer to use the auto_explain module: http://www.postgresql.org/docs/8.4/static/auto-explain.html instead of this feature. This will allow you to actually see why the queries that are running slowly are doing so by viewing their associated EXPLAIN plans. Vacuuming and statistics PostgreSQL databases require two primary forms of regular maintenance as data is added, updated, and deleted. VACUUM cleans up after old transactions, including removing information that is no longer visible and returning freed space to where it can be re-used. The more often you UPDATE and DELETE information from the database, the more likely you'll need a regular vacuum cleaning regime. However, even static tables with data that never changes once inserted still need occasional care here. ANALYZE looks at tables in the database and collects statistics about them— information like estimates of how many rows they have and how many distinct values are in there. Many aspects of query planning depend on this statistics data being accurate. autovacuum As both these tasks are critical to database performance over the long-term, starting in PostgreSQL 8.1 there is an autovacuum daemon available that will run in the background to handle these tasks for you. Its action is triggered by the number of changes to the database exceeding a threshold it calculates based on the existing table size. The parameter for autovacuum is turned on by default in PostgreSQL 8.3, and the default settings are generally aggressive enough to work out of the box for smaller database with little manual tuning. Generally you just need to be careful that the amount of data in the free space map doesn't exceed max_fsm_pages, and even that requirement is automated away from being a concern as of 8.4. Enabling autovacuum on older versions If you have autovacuum available but it's not turned on by default, which will be the case with PostgreSQL 8.1 and 8.2, there are a few related parameters that must also be enabled for it to work, as covered in http://www.postgresql.org/docs/8.1/interactive/maintenance.html or http://www.postgresql.org/docs/8.2/interactive/routine-vacuuming.html. The normal trio to enable in the postgresql.conf file in these versions are: stats_start_collector=truestats_row_level=trueautovacuum=on Note that as warned in the documentation, it's also wise to consider adjusting superuser_reserved_connections to allow for the autovacuum processes in these earlier versions. The autovacuum you'll get in 8.1 and 8.2 is not going to be as efficient as what comes in 8.3 and later. You can expect it to take some fine tuning to get the right balance of enough maintenance without too much overhead, and because there's only a single worker it's easier for it to fall behind on a busy server. This topic isn't covered at length here. It's generally a better idea to put time into planning an upgrade to a PostgreSQL version with a newer autovacuum than to try and tweak an old one extensively, particularly if there are so many other performance issues that cannot be resolved easily in the older versions, too. maintainance_work_mem A few operations in the database server need working memory for larger operations than just regular sorting. VACUUM, CREATE INDEX, and ALTER TABLE ADD FOREIGN KEY all can allocate up to maintainance_work_mem worth of memory instead. As it's unlikely that many sessions will be doing one of these operations at once, it's possible to set this value much higher than the standard per-client work_mem setting. Note that at least autovacuum_max_workers (defaulting to 3 starting in version 8.3) will allocate this much memory, so consider those sessions (perhaps along with a session or two doing a CREATE INDEX) when setting this value. Assuming you haven't increased the number of autovacuum workers, a typical high setting for this value on a modern server would be at five percent of the total RAM, so that even five such processes wouldn't exceed a quarter of available memory. This works out to approximately 50 MB of maintainance_work_mem per GB of server RAM. default_statistics_target PostgreSQL makes its decisions about how queries execute based on statistics collected about each table in your database. This information is collected by analyzing the tables, either with the ANALYZE statement or via autovacuum doing that step. In either case, the amount of information collected during the analyze step is set by default_statistics_target. Increasing this value makes analysis take longer, and as analysis of autovacuum happens regularly this turns into increased background overhead for database maintenance. But if there aren't enough statistics about a table, you can get bad plans for queries against it. The default value for this setting used to be the very low (that is,10), but was increased to 100 in PostgreSQL 8.4. Using that larger value was popular in earlier versions, too, for general improved query behavior. Indexes using the LIKE operator tended to work much better with values greater than 100 rather than below it, due to a hard-coded change at that threshold. Note that increasing this value does result in a net slowdown on your system if you're not ever running queries where the additional statistics result in a change to a better query plan. This is one reason why some simple benchmarks show PostgreSQL 8.4 as slightly slower than 8.3 at default parameters for each, and in some cases you might return an 8.4 install to a smaller setting. Extremely large settings for default_statistics_target are discouraged due to the large overhead they incur. If there is just a particular column in a table you know that needs better statistics, you can use ALTER TABLE SET STATISTICS on that column to adjust this setting just for it. This works better than increasing the system-wide default and making every table pay for that requirement. Typically, the columns that really require a lot more statistics to work properly will require a setting near the maximum of 1000 (increased to 10,000 in later versions) to get a serious behavior change, which is far higher than you'd want to collect data for on every table in the database.

Scopes provide a way to divide a complex business process into hierarchically organized parts—scopes. Scopes provide behavioral contexts for activities. In other words scopes address the problem that we identified in the previous article and allow us to define different fault handlers for different activities (or sets of activities gathered under a common structured activity, such as <sequence> or <flow>). In addition to fault handlers, scopes also provide a way to declare variables and partner links that are visible only within the scope. Scopes also allow us to define local correlation sets, compensation handlers, event handlers, termination handler, and message exchanges. The following code excerpt shows how scopes are defined in BPEL. We can specify <partnerLinks>, <messageExchanges>, <variables>, <correlationSets>, <faultHandlers>, <compensationHandler>, <terminationHandler>, and <eventHandlers> locally for the scope. All are optional: <scope> <partnerLinks> <!-- Partner link definitions local to scope. --> </partnerLinks> <messageExchanges> <!-- Message exchanges local to scope.--> </messageExchanges> <variables> <!-- Variable definitions local to scope. --> </variables> <correlationSets> <!-- Correlation sets local to scope.--> </correlationSets> <faultHandlers> <!-- Fault handlers local to scope. --> </faultHandlers> <compensationHandler> <!-- Compensation handlers local to scope.--> </compensationHandler> <terminationHandler> <!-- Termination handler local to scope. --> </terminationHandler> <eventHandlers> <!-- Event handlers local to scope. --> </eventHandlers> activity </scope> Each scope has a primary activity. This is similar to the overall process structure, where we have said that a BPEL process also has a primary activity. The primary activity, which is often a <sequence> or <flow>, defines the behavior of a scope for normal execution. Fault handlers and other handlers define the behavior for abnormal execution scenarios. The primary activity of a scope can be a basic activity such as <invoke>, or it can be a structured activity such as <sequence> or <flow>. Enclosing the <invoke> activity with a scope and defining the fault handlers is equivalent to using inline fault handlers. The inline fault handler shown in the previous article is equal to the following scope: <scope> <faultHandlers> <catch faultName="emp:WrongEmployeeName" > <!-- Perform an activity --> </catch> <catch faultName="emp:TravelNotAllowed" faultVariable="Description" > <!-- Perform an activity --> </catch> <catchAll> <!-- Perform an activity --> </catchAll> </faultHandlers> <invoke partnerLink="employeeTravelStatus" portType="emp:EmployeeTravelStatusPT" operation="EmployeeTravelStatus" inputVariable="EmployeeTravelStatusRequest" outputVariable="EmployeeTravelStatusResponse" > </invoke> </scope> If the primary activity of a scope is a structured activity, it can have many nested activities where the nesting depth is arbitrary. The scope is shared by all nested activities. A scope can also have nested scopes with arbitrary depth. The variables defined within a scope are only visible within that scope. Fault handlers attached to a scope handle faults of all nested activities of a scope. By default behavior, faults not caught in a scope are rethrown to the enclosing scope. Scopes in which faults have occurred are considered to have ended abnormally, even if a fault handler has caught the fault and not rethrown it. Similarly as for the <process>, we can define the exitOnStandardFault for a scope as well. If set to no, which is the default, the scope can handle the faults using the corresponding fault handlers. If set to yes, then the scope must exit immediately if a fault occurs (similarly to if it reached an <exit> activity). If we do not set this attribute, it inherits the value from its enclosing <scope> or <process>. How scopes can be used in BPEL processes To demonstrate how scopes can be used in BPEL processes, we will rewrite our asynchronous travel process example and introduce three scopes: In the first scope, we will retrieve the employee travel status(RetrieveEmployeeTravelStatus) In the second scope, we will check the flight availability with both airlines(CheckFlightAvailability) In the third scope, we will call back to the client (CallbackClient) We will also declare those variables that are limited to a scope locally within the scope. This will reduce the number of global variables and make the business process easier to understand. The major benefit of scopes is the capability to define custom fault handlers, which we will also implement. The high-level structure of our travel process will be as follows: <process ...> <partnerLinks/>.</partnerLinks> <variables>...</variables> <faultHandlers> <catchAll>...</catchAll> </faultHandlers> <sequence> <!-- Receive the initial request for business travel from client --> <receive .../> <scope name="RetrieveEmployeeTravelStatus"> <variables>...</variables> <faultHandlers> <catchAll>...</catchAll> </faultHandlers> <sequence> <!-- Prepare the input for Employee Travel Status Web Service --> <!-- Synchronously invoke the Employee Travel Status Web Service --> <!-- Prepare the input for AA and DA --> </sequence> </scope> <scope name="CheckFlightAvailability"> <variables>...</variables> <faultHandlers> <catchAll>...</catchAll> </faultHandlers> <sequence> <!-- Make a concurrent invocation to AA and DA --> <flow> <!-- Async invoke the AA web service and wait for the callback --> <!-- Async invoke the DA web service and wait for the callback --> </flow> <!-- Select the best offer and construct the TravelResponse --> </sequence> </scope> <scope name="CallbackClient"> <faultHandlers>...</faultHandlers> <!-- Check if the ticket is approved --> </scope> </sequence> </process> To signal faults to the BPEL process client, we will use the ClientCallbackFault operation on the client partner link, which we defined in the previous article. This operation has a string message, which we will use to describe the fault. In real-world scenarios, the fault message is more complex and includes a fault code and other relevant information. Let us start with the example. The process declaration and the partner links have not changed: <process name="Travel" targetNamespace="http://packtpub.com/bpel/travel/" > <partnerLinks> <partnerLink name="client" partnerLinkType="trv:travelLT" myRole="travelService" partnerRole="travelServiceCustomer"/> <partnerLink name="employeeTravelStatus" partnerLinkType="emp:employeeLT" partnerRole="employeeTravelStatusService"/> <partnerLink name="AmericanAirlines" partnerLinkType="aln:flightLT" myRole="airlineCustomer" partnerRole="airlineService"/> <partnerLink name="DeltaAirlines" partnerLinkType="aln:flightLT" myRole="airlineCustomer" partnerRole="airlineService"/> </partnerLinks> The variables section will now define only global variables. These are TravelRequest, FlightDetails, TravelResponse, and TravelFault. We have reduced the number of global variables, but we will have to declare other variables within scopes: <variables> <!-- input for this process --> <variable name="TravelRequest" messageType="trv:TravelRequestMessage"/> <!-- input for the Employee Travel Status web service --> <variable name="FlightDetails" messageType="aln:FlightTicketRequestMessage"/> <!-- output from BPEL process --> <variable name="TravelResponse" messageType="aln:TravelResponseMessage"/> <!-- fault to the BPEL client --> <variable name="TravelFault" messageType="trv:TravelFaultMessage"/> </variables> Next we define the global fault handlers section. Here we use the <catchAll> activity, through which we handle all faults not handled within scopes. We will signal the fault to the BPEL client: <faultHandlers> <catchAll> <sequence> <!-- Create the TravelFault variable --> <assign> <copy> <from>string('Other fault')</from> <to variable="TravelFault" part="error" /> </copy> </assign> <invoke partnerLink="client" portType="trv:ClientCallbackPT" operation="ClientCallbackFault" inputVariable="TravelFault" /> </sequence> </catchAll> </faultHandlers> The main activity of the BPEL process will still be <sequence>, and we will also specify the <receive> activity to wait for the incoming message from the client: <sequence> <!-- Receive the initial request for business travel from client --> <receive partnerLink="client" portType="trv:TravelApprovalPT" operation="TravelApproval" variable="TravelRequest" createInstance="yes" />

ASP.NET Site Performance Secrets Simple and proven techniques to quickly speed up your ASP.NET website Speed up your ASP.NET website by identifying performance bottlenecks that hold back your site's performance and fixing them Tips and tricks for writing faster code and pinpointing those areas in the code that matter most, thus saving time and energy Drastically reduce page load times Configure and improve compression – the single most important way to improve your site's performance Written in a simple problem-solving manner – with a practical hands-on approach and just the right amount of theory you need to make sense of it all In this section, we'll identify the biggest bottlenecks. Missing indexes and expensive queries You can greatly improve the performance of your queries by reducing the number of reads executed by those queries. The more reads you execute, the more potentially you stress the disk, CPU, and memory. Secondly, a query reading a resource normally blocks another query from updating that resource. If the updating query has to wait while holding locks itself, it may then delay a chain of other queries. Finally, unless the entire database fits in memory, each time data is read from disk, other data is evicted from memory. If that data is needed later, it then needs to be read from the disk again. The most effective way to reduce the number of reads is to create sufficient indexes on your tables. Just as an index in a book, an SQL Server index allows a query to go straight to the table row(s) it needs, rather than having to scan the entire table. Indexes are not a cure-all though—they do incur overhead and slow down updates, so they need to be used wisely. In this section, we'll see: How to identify missing indexes that would reduce the number of reads in the database How to identify those queries that create the greatest strain, either because they are used very often, or because they are just plain expensive How to identify superfluous indexes that take resources but provide little benefit Missing indexes SQL Server allows you to put indexes on table columns, to speed up WHERE and JOIN statements on those columns. When the query optimizer optimizes a query, it stores information about those indexes it would have liked to have used, but weren't available. You can access this information with the Dynamic Management View (DMV) dm_db_missing_index_details (indexesqueries.sql in the code bundle): select d.name AS DatabaseName, mid.* from sys.dm_db_missing_index_details mid join sys.databases d ON mid.database_id=d.database_id The most important columns returned by this query are: ColumnDescriptionDatabaseNameName of the database this row relates to.equality_columnsComma-separated list of columns used with the equals operator, such as: column=valueinequality_columnsComma-separated list of columns used with a comparison operator other than the equals operator, such as: column>valueincluded_columnsComma-separated list of columns that could profitably be included in an index.statementName of the table where the index is missing. This information is not persistent—you will lose it after a server restart. An alternative is to use Database Engine Tuning Advisor, which is included with SQL Server 2008 (except for the Express version). This tool analyzes a trace of database operations and identifies an optimal set of indexes that takes the requirements of all queries into account. It even gives you the SQL statements needed to create the missing indexes it identified. The first step is to get a trace of database operations during a representative period. If your database is the busiest during business hours, then that is probably when you want to run the trace: Start SQL Profiler. Click on Start | Programs | Microsoft SQL Server 2008 | Performance Tools | SQL Server Profiler. In SQL Profiler, click on File | New Trace. Click on the Events Selection tab. You want to minimize the number of events captured to reduce the load on the server. Deselect every event, except SQL:BatchCompleted and RPC:Completed. It is those events that contain resource information for each batch, and so are used by Database Engine Tuning Advisor to analyze the workload. Make sure that the TextData column is selected for both the events. To capture events related only to your database, click on the Column Filters button. Click on DatabaseName in the left column, expand Like in the righthand pane, and enter your database name. Click on OK. (Move the mouse over the image to enlarge.) To further cut down the trace and only trace calls from your website, put a filter on ApplicationName, so only events where this equals ".Net SqlClient Data Provider" will be recorded. Click on the Run button to start the trace. You will see batch completions scrolling through the window. At any stage, you can click on File | Save or press Ctrl + S. to save the trace to a file. Save the template so that you don't have to recreate it next time. Click on File | Save As | Trace Template. Fill in a descriptive name and click on OK. Next time you create a new trace by clicking on File | New Trace, you can retrieve the template from the Use the template drop-down.Sending all these events to your screen takes a lot of server resources. You probably won't be looking at it all day anyway. The solution is to save your trace as a script and then use that to run a background trace. You'll also be able to reuse the script later on. Click on File | Export | Script Trace Definition | For SQL Server 2005 – 2008. Save the file with a .sql extension. You can now close SQL Server Profiler, which will also stop the trace. In SQL Server Management Studio, open the .sql file you just created. Find the string InsertFileNameHere and replace it with the full path of the file where you want the log stored. Leave off the extension; the script will set it to .trc. Press Ctrl + S to save the .sql file. To start the trace, press F5 to run the .sql file. It will tell you the trace ID of this trace. To see the status of this trace and any other traces in the system, execute the following command in a query window: select * from ::fn_trace_getinfo(default) Find the row with property 5 for your trace ID. If the value column in that row is 1, your trace is running. The trace with trace ID 1 is a system trace. To stop the trace after it has captured a representative period, assuming your trace ID is two, run the following command: exec sp_trace_setstatus 2,0 To restart it, run: exec sp_trace_setstatus 2,1 To stop and close it so that you can access the trace file, run: exec sp_trace_setstatus 2,0 exec sp_trace_setstatus 2,2 Now, run Database Engine Tuning Advisor: Start SQL Profiler. Click on Start | Programs | Microsoft SQL Server 2008 | Performance Tools | Database Engine Tuning Advisor. In the Workload area, select your trace file. In the Database for workload analysis drop-down, select the first database you want to be analyzed. Under Select databases and tables to tune, select the databases for which you want index recommendations. Especially with a big trace, Database Engine Tuning Advisor may take a long time to do its analysis. On the Tuning Options tab, you can tell it when to stop analyzing. This is just a limit; if it is done sooner, it will produce results as soon as it is done. To start the analysis, click on the Start Analysis button in the toolbar. Keep in mind that Database Engine Tuning Advisor is just a computer program. Consider its recommendations, but make up your own mind. Be sure to give it a trace with a representative workload, otherwise its recommendations may make things worse rather than better. For example, if you provide a trace that was captured at night when you process few transactions but execute lots of reporting jobs, its advice is going to be skewed towards optimizing reporting, not transactions.

Transaction Auto-wrapping for the data access layer This article by Jason Dentler, author of NHibernate 3.0 Cookbook, shows how we can set up the data access layer to wrap all data access in NHibernate transactions automatically. Getting ready Complete the Eg.Core model and mappings. Download code (ch:1) How to do it... Create a new class library named Eg.Core.Data. Add a reference to NHibernate.dll and the Eg.Core project. Add the following two DAO classes: public class DataAccessObject<T, TId> where T : Entity<TId> { private readonly ISessionFactory _sessionFactory; private ISession session { get { return _sessionFactory.GetCurrentSession(); } } public DataAccessObject(ISessionFactory sessionFactory) { _sessionFactory = sessionFactory; } public T Get(TId id) { return Transact(() => session.Get<T>(id)); } public T Load(TId id) { return Transact(() => session.Load<T>(id)); } public void Save(T entity) { Transact(() => session.SaveOrUpdate(entity)); } public void Delete(T entity) { Transact(() => session.Delete(entity)); } private TResult Transact<TResult>(Func<TResult> func) { if (!session.Transaction.IsActive) { // Wrap in transaction TResult result; using (var tx = session.BeginTransaction()) { result = func.Invoke(); tx.Commit(); } return result; } // Don't wrap; return func.Invoke(); } private void Transact(Action action) { Transact<bool>(() => { action.Invoke(); return false; }); } } public class DataAccessObject<T> : DataAccessObject<T, Guid> where T : Entity { } How it works... NHibernate requires that all data access occurs inside an NHibernate transaction and this can be easily accomplished with AOP. Remember, the ambient transaction created by TransactionScope is not a substitute for an NHibernate transaction. This recipe shows a more explicit approach. To ensure that at least all our data access layer calls are wrapped in transactions, we create a private Transact function that accepts a delegate, consisting of some data access methods, such as session.Save or session.Get. This Transact function first checks if the session has an active transaction. If it does, Transact simply invokes the delegate. If it doesn't, it creates an explicit NHibernate transaction, then invokes the delegate, and finally commits the transaction. If the data access method throws an exception, the transaction will be rolled back automatically as the exception bubbles up through the using block. There's more... This transactional auto-wrapping can also be set up using SessionWrapper from the unofficial NHibernate AddIns project at http://code.google.com/p/unhaddins. This class wraps a standard NHibernate session. By default, it will throw an exception when the session is used without an NHibernate transaction. However, it can be configured to check for and create a transaction automatically, much in the same way I've shown you here. See also Setting up an NHibernate repository

Getting ready Download the latest release of the Common Service Locator from http://commonservicelocator.codeplex.com, and extract Microsoft.Practices.ServiceLocation.dll to your solution's libs folder. Complete the previous recipe, Setting up an NHibernate repository. Following the Fast testing with SQLite in-memory database recipe in the previous article, create a new NHibernate test project named Eg.Core.Data.Impl.Test. Include the Eg.Core.Data.Impl assembly as an additional mapping assembly in your test project's App.Config with the following xml: <mapping assembly="Eg.Core.Data.Impl"/> How to do it... In the Eg.Core.Data project, add a folder for the Queries namespace. Add the following IQuery interfaces: public interface IQuery { } public interface IQuery<TResult> : IQuery { TResult Execute(); } Add the following IQueryFactory interface: { TQuery CreateQuery<TQuery>() where TQuery :IQuery; } Change the IRepository interface to implement the IQueryFactory interface, as shown in the following code: public interface IRepository<T> : IEnumerable<T>, IQueryFactory where T : Entity { void Add(T item); bool Contains(T item); int Count { get; } bool Remove(T item); } In the Eg.Core.Data.Impl project, change the NHibernateRepository constructor and add the _queryFactory field, as shown in the following code: private readonly IQueryFactory _queryFactory; public NHibernateRepository(ISessionFactory sessionFactory, IQueryFactory queryFactory) : base(sessionFactory) { _queryFactory = queryFactory; } Add the following method to NHibernateRepository: public TQuery CreateQuery<TQuery>() where TQuery : IQuery { return _queryFactory.CreateQuery<TQuery>(); } In the Eg.Core.Data.Impl project, add a folder for the Queries namespace. To the Eg.Core.Data.Impl project, add a reference to Microsoft.Practices.ServiceLocation.dll. To the Queries namespace, add this QueryFactory class: public class QueryFactory : IQueryFactory { private readonly IServiceLocator _serviceLocator; public QueryFactory(IServiceLocator serviceLocator) { _serviceLocator = serviceLocator; } public TQuery CreateQuery<TQuery>() where TQuery : IQuery { return _serviceLocator.GetInstance<TQuery>(); } } Add the following NHibernateQueryBase class: public abstract class NHibernateQueryBase<TResult> : NHibernateBase, IQuery<TResult> { protected NHibernateQueryBase( ISessionFactory sessionFactory) : base(sessionFactory) { } public abstract TResult Execute(); } Add an empty INamedQuery interface, as shown in the following code: public interface INamedQuery { string QueryName { get; } } Add a NamedQueryBase class, as shown in the following code: public abstract class NamedQueryBase<TResult> : NHibernateQueryBase<TResult>, INamedQuery { protected NamedQueryBase(ISessionFactory sessionFactory) : base(sessionFactory) { } public override TResult Execute() { var nhQuery = GetNamedQuery(); return Transact(() => Execute(nhQuery)); } protected abstract TResult Execute(IQuery query); protected virtual IQuery GetNamedQuery() { var nhQuery = session.GetNamedQuery( ((INamedQuery) this).QueryName); SetParameters(nhQuery); return nhQuery; } protected abstract void SetParameters(IQuery nhQuery); public virtual string QueryName { get { return GetType().Name; } } } In Eg.Core.Data.Impl.Test, add a test fixture named QueryTests inherited from NHibernateFixture. Add the following test and three helper methods: [Test] public void NamedQueryCheck() { var errors = new StringBuilder(); var queryObjectTypes = GetNamedQueryObjectTypes(); var mappedQueries = GetNamedQueryNames(); foreach (var queryType in queryObjectTypes) { var query = GetQuery(queryType); if (!mappedQueries.Contains(query.QueryName)) { errors.AppendFormat( "Query object {0} references non-existent " + "named query {1}.", queryType, query.QueryName); errors.AppendLine(); } } if (errors.Length != 0) Assert.Fail(errors.ToString()); } private IEnumerable<Type> GetNamedQueryObjectTypes() { var namedQueryType = typeof(INamedQuery); var queryImplAssembly = typeof(BookWithISBN).Assembly; var types = from t in queryImplAssembly.GetTypes() where namedQueryType.IsAssignableFrom(t) && t.IsClass && !t.IsAbstract select t; return types; } private IEnumerable<string> GetNamedQueryNames() { var nhCfg = NHConfigurator.Configuration; var mappedQueries = nhCfg.NamedQueries.Keys .Union(nhCfg.NamedSQLQueries.Keys); return mappedQueries; } private INamedQuery GetQuery(Type queryType) { return (INamedQuery) Activator .CreateInstance(queryType, new object[] { SessionFactory }); } For our example query, in the Queries namespace of Eg.Core.Data, add the following interface: public interface IBookWithISBN : IQuery<Book> { string ISBN { get; set; } } Add the implementation to the Queries namespace of Eg.Core.Data.Impl using the following code: public class BookWithISBN : NamedQueryBase<Book>, IBookWithISBN { public BookWithISBN(ISessionFactory sessionFactory) : base(sessionFactory) { } public string ISBN { get; set; } protected override void SetParameters( NHibernate.IQuery nhQuery) { nhQuery.SetParameter("isbn", ISBN); } protected override Book Execute(NHibernate.IQuery query) { return query.UniqueResult<Book>(); } } Finally, add the embedded resource mapping, BookWithISBN.hbm.xml, to Eg.Core.Data.Impl with the following xml code: <?xml version="1.0" encoding="utf-8" ?> <hibernate-mapping > <query name="BookWithISBN"> <![CDATA[ from Book b where b.ISBN = :isbn ]]> </query> </hibernate-mapping>

What is a grid? Ext JS grids are similar to a spreadsheet; there are two main parts to each spreadsheet: Columns Rows Here our columns are Title, Released, Genre, and Price. Each of the rows contains movies such as The Big Lebowski, Super Troopers, and so on. The rows are really our data; each row in the grid represents a record of data held in a data store. GridPanel is databound Like many Ext JS Components, the GridPanel class is bound to a data store which provides it with the data shown in the user interface. So the first step in creating our GridPanel is creating and loading a store. The data store in Ext JS gives us a consistent way of reading different data formats such as XML and JSON, and using this data in a consistent way throughout all of the Ext JS widgets. Regardless of whether this data is originally provided in JSON, XML, an array, or even a custom data type of our own, it's all accessed in the same way thanks to the consistency of the data store and how it uses a separate reader class which interprets the raw data. Instead of using a pre-configured class, we will explicitly define the classes used to define and load a store. The record definition We first need to define the fields which a Record contains. A Record is a single row of data and we need to define the field names, which we want to be read into our store. We define the data type for each field, and, if necessary, we define how to convert from the raw data into the field's desired data type. What we will be creating is a new class. We actually create a constructor which will be used by Ext JS to create records for the store. As the 'create' method creates a constructor, we reference the resulting function with a capitalized variable name, as per standard CamelCase syntax: var Movie = Ext.data.Record.create([ 'id', 'coverthumb', 'title', 'director', 'runtime', {name: 'released', type: 'date', dateFormat: 'Y-m-d'}, 'genre', 'tagline', {name: 'price', type: 'float'}, {name: 'available', type: 'bool'} ]); Each element in the passed array defines a field within the record. If an item is just a string, the string is used as the name of the field, and no data conversion is performed; the field's value will be whatever the Reader object (which we will learn more about soon) finds in the raw data. If data conversion is required, then a field definition in the form of an object literal instead of a string may contain the following config options: Name: The name by which the field will be referenced. type: The data type in which the raw data item will be converted to when stored in the record. Values may be 'int', 'float', 'string', 'date', or 'bool'. dateFormat: If the type of data to be held in the field is a date type, then we need to specify a format string as used by the Date.parseDate function. Defining the data type can help to alleviate future problems, instead of having to deal with all string type data defining the data type, and lets us work with actual dates, Boolean values, and numbers. The following is a list of the built in data types:   Field type Description Information string String data   int Number Uses JavaScript's parseInt function float Floating point number Uses JavaScript's parseFloat function boolean True/False data   date Date data dateFormat config required to interpret incoming data. Now that the first step has been completed, and we have a simple Record definition in place, we can move on to the next step of configuring a Reader that is able to understand the raw data. The Reader A store may accept raw data in several different formats. Raw data rows may be in the form of a simple array of values, or an object with named properties referencing the values, or even an XML element in which the values are child nodes. We need to configure a store with a Reader object which knows how to extract data from the raw data that we are dealing with. There are three built in Reader classes in Ext JS. ArrayReader The ArrayReader class can create a record from an array of values. By default, values are read out of the raw array into a record's fields in the order that the fields were declared in the record definition we created. If fields are not declared in the same order that the values occur in the rows, the field's definition may contain a mapping config which specifies the array index from which we retrieve the field's value. JsonReader This JSONReader class is the most commonly used, and can create a record from raw JSON by decoding and reading the object's properties. By default, the field's name is used as the property name from which it takes the field's value. If a different property name is required, the field's definition may contain a mapping config which specifies the name of the property from which to retrieve the field's value. XmlReader An XMLReader class can create a record from an XML element by accessing child nodes of the element. By default, the field's name is used as the XPath mapping (not unlike HTML) from which to take the field's value. If a different mapping is required, the field's definition may contain a mapping config which specifies the mapping from which to retrieve the field's value. Loading our data store In our first attempt, we are going to create a grid that uses simple local array data stored in a JavaScript variable. The data we're using below in the movieData variable is taken from a very small movie database of some of my favorite movies. The data store needs two things: the data itself, and a description of the data—or what could be thought of as the fields. A reader will be used to read the data from the array, and this is where we define the fields of data contained in our array. The following code should be placed before the Ext JS OnReady function: var movieData = [ [ 1, "Office Space", "Mike Judge", 89, "1999-02-19", 1, "Work Sucks", "19.95", 1 ],[ 3, "Super Troopers", "Jay Chandrasekhar", 100, "2002-02-15", 1, "Altered State Police", "14.95", 1 ] //...more rows of data removed for readability...// ]; var store = new Ext.data.Store({ data: movieData, , reader: new Ext.data.ArrayReader({idIndex: 0}, Movie) }); If we view this code in a browser we would not see anything—that's because a data store is just a way of loading and keeping track of our data. The web browser's memory has our data in it. Now we need to configure the grid to display our data to the user. Displaying structured data with a GridPanel Displaying data in a grid requires several Ext JS classes to cooperate: A Store: A data store is a client-side analogue of a database table. It encapsulates a set of records, each of which contains a defined set of fields. A Record definition: This defines the fields (or "columns" in database terminology) which make up each record in the Store. Field name and datatype are defined here. A Reader which uses a Record definition to extract field values from a raw data object to create the records for a Store. A ColumnModel which specifies the details of each column, including the column header to display, and the name of the record field to be displayed for each column. A GridPanel: A panel subclass which provides the controller logic to bind the above classes together to generate the grid's user interface. If we were to display the data just as the store sees it now, we would end up with something like this: Now that is ugly—here's a breakdown of what's happening: The Released date has been type set properly as a date, and interpreted from the string value in our data. It's provided in a native JavaScript date format—luckily Ext JS has ways to make this look pretty. The Price column has been type set as a floating point number. Note that there is no need to specify the decimal precision. The Avail column has been interpreted as an actual Boolean value, even if the raw data was not an actual Boolean value. As you can see, it's quite useful to specify the type of data that is being read, and apply any special options that are needed so that we don't have to deal with converting data elsewhere in our code. Before we move on to displaying the data in our grid, we should take a look at how the convert config works, as it can come in quite useful. Converting data read into the store If we need to, we can convert data as it comes into the store, massage it, remove any quirky parts, or create new fields all together. This should not be used as a way to change the display of data; that part will be handled elsewhere. A common task might be to remove possible errors in the data when we load it, making sure it's in a consistent format for later actions. This can be done using a convert function, which is defined in the 'convert' config by providing a function, or reference to a function. In this case we are going to create a new field by using the data from another field and combining it with a few standard strings. var store = new Ext.data.Store({ data: movieData, reader: new Ext.data.ArrayReader({id:'id'}, [ 'id', 'title', 'director', {name: 'released', type: 'date', dateFormat: 'Y-m-d'}, 'genre', 'tagline', 'price', 'available', {name:'coverthumb',convert:function(v, rawData){ return 'images/'+rawData[0]+'m.jpg'; }} ]) }); This convert function when used in this manner will create a new field of data that looks like this: 'images/5m.jpg' We will use this new field of data shortly, so let's get a grid up and running.

What is Direct? Part of the power of any client-side library is its ability to tap nearly any server-side technology. That said, with so many server-side options available there were many different implementations being written for accessing the data. Direct is a means of marshalling those server-side connections, creating a 'one-stop-shop' for handling your basic Create, Read, Update, and Delete actions against that remote data. Through some basic configuration, we can now easily create entire server-side API's that we may programmatically expose to our Ext JS applications. In the process, we end up with one set of consistent, predefined methods for managing that data access. Building server-side stacks There are several examples of server-side stacks already available for Ext JS, directly from their site's Direct information. These are examples, showing you how you might use Direct with a particular server-side technology, but Ext provides us with a specification so that we might write our own. Current stack examples are available for: PHP .NET Java ColdFusion Ruby Perl These are examples written directly by the Ext team, as guides, as to what we can do. Each of us writes applications differently, so it may be that our application requires a different way of handling things at the server level. The Direct specification, along with the examples, gives us the guideposts we need for writing our own stacks when necessary. We will deconstruct one such example here to help illustrate this point. Each server-side stack is made up of three basic components: Configuration— denoting which components/classes are available to Ext JS API— client-side descriptors of our configuration Router— a means to 'route' our requests to their proper API counterparts To illustrate each of these pieces of the server-side stack we will deconstruct one of the example stacks provided by the Ext JS team. I have chosen the ColdFusion stack because: It is a good example of using a metadata configuration DirectCFM (the ColdFusion stack example) was written by Aaron Conran, who is the Senior Software Architect and Ext Services Team Leader for Ext, LLC Each of the following sections will contain a "Stack Deconstruction" section to illustrate each of the concepts. These are to show you how these concepts might be written in a server-side language, but you are welcome to move on if you feel you have a good grasp of the material. Configuration Ultimately the configuration must define the classes/objects being accessed, the functions of those objects that can be called, and the length (number) of arguments that the method is expecting. Different servers will allow us to define our configuration in different ways. The method we choose will sometimes depend upon the capabilities or deficiencies of the platform we're coding to. Some platforms provide the ability to introspect components/classes at runtime to build configurations, while others require a far more manual approach. You can also include an optional formHandler attribute to your method definitions, if the method can take form submissions directly. There are four basic ways to write a configuration. Programmatic A programmatic configuration may be achieved by creating a simple API object of key/value pairs in the native language. A key/value pair object is known by many different names, depending upon the platform to which we're writing for: HashMap, Structure, Object, Dictionary, or an Associative Array. For example, in PHP you might write something like this: $API = array( 'Authors'=>array( 'methods'=>array( 'GetAll'=>array( 'len'=>0 ), 'add'=>array( 'len'=>1 ), 'update'=>array( 'len'=>1 ) ) ) ); Look familiar? It should, in some way, as it's very similar to a JavaScript object. The same basic structure is true for our next two methods of configuration as well. JSON and XML For this configuration, we can pass in a basic JSON configuration of our API: { Authors:{ methods:{ GetAll:{ len:0 }, add:{ len:1 }, update:{ len:1 } } } } Or we could return an XML configuration object: <Authors> <methods> <method name="GetAll" len="0" /> <method name="add" len="1" /> <method name="update" len="1" /> </methods> </Authors> All of these forms have given us the same basic outcome, by providing a basic definition of server-side classes/objects to be exposed for use with our Ext applications. But, each of these methods require us to build these configurations basically by hand. Some server-side options make it a little easier. Metadata There are a few server-side technologies that allow us to add additional metadata to classes and function definitions, using which we can then introspect objects at runtime to create our configurations. The following example demonstrates this by adding additional metadata to a ColdFusion component (CFC): <cfcomponent name="Authors" ExtDirect="true"> <cffunction name="GetAll" ExtDirect="true"> <cfreturn true /> </cffunction> <cffunction name="add" ExtDirect="true"> <cfargument name="author" /> <cfreturn true /> </cffunction> <cffunction name="update" ExtDirect="true"> <cfargument name="author" /> <cfreturn true /> </cffunction> </cfcomponent> This is a very powerful method for creating our configuration, as it means adding a single name/value attribute (ExtDirect="true") to any object and function we want to make available to our Ext application. The ColdFusion server is able to introspect this metadata at runtime, passing the configuration object back to our Ext application for use. Stack deconstruction—configuration The example ColdFusion Component provided with the DirectCFM stack is pretty basic, so we'll write one slightly more detailed to illustrate the configuration. ColdFusion has a facility for attaching additional metadata to classes and methods, so we'll use the fourth configuration method for this example, Metadata. We'll start off with creating the Authors.cfc class: <cfcomponent name="Authors" ExtDirect="true"> </cfcomponent> Next we'll create our GetAll method for returning all the authors in the database: <cffunction name="GetAll" ExtDirect="true"> <cfset var q = "" /> <cfquery name="q" datasource="cfbookclub"> SELECT AuthorID, FirstName, LastName FROM Authors ORDER BY LastName </cfquery> <cfreturn q /> </cffunction> We're leaving out basic error handling and stuff, but these are the basics behind it. The classes and methods we want to make available will all contain the additional metadata. Building your API So now that we've explored how to create a configuration at the server, we need to take the next step by passing that configuration to our Ext application. We do this by writing a server-side template that will output our JavaScript configuration. Yes, we'll actually dynamically produce a JavaScript include, calling the server-side template directly from within our <script> tag: <script src="Api.cfm"></script> How we write our server-side file really depends on the platform, but ultimately we just want it to return a block of JavaScript (just like calling a .js file) containing our API configuration description. The configuration will appear as part of the actions attribute, but we must also pass the url of our Router, the type of connection, and our namespace. That API return might look something like this: Ext.ns("com.cc"); com.cc.APIDesc = { "url": "/remote/Router.cfm", "type": "remoting" "namespace": "com.cc", "actions": { "Authors": [{ "name": "GetAll", "len": 0 },{ "name": "add", "len": 1 },{ "name": "update", "len": 1 }] } }; This now exposes our server-side configuration to our Ext application. Stack deconstruction—API The purpose here is to create a JavaScript document, dynamically, of your configuration. Earlier we defined configuration via metadata. The DirectCFM API now has to convert that metadata into JavaScript. The first step is including the Api.cfm in a <script> tag on the page, but we need to know what's going on "under the hood." Api.cfm: <!--- Configure API Namespace and Description variable names ---> <cfset args = StructNew() /> <cfset args['ns'] = "com.cc" /> <cfset args['desc'] = "APIDesc" /> <cfinvoke component="Direct" method="getAPIScript" argumentcollection="#args#" returnVariable="apiScript" /> <cfcontent reset="true" /> <cfoutput>#apiScript#</cfoutput> Here we set a few variables, that will then be used in a method call. The getAPIScript method, of the Direct.cfc class, will construct our API from metadata. Direct.cfc getAPIScript() method: <cffunction name="getAPIScript"> <cfargument name="ns" /> <cfargument name="desc" /> <cfset var totalCFCs = '' /> <cfset var cfcName = '' /> <cfset var CFCApi = '' /> <cfset var fnLen = '' /> <cfset var Fn = '' /> <cfset var currFn = '' /> <cfset var newCfComponentMeta = '' /> <cfset var script = '' /> <cfset var jsonPacket = StructNew() /> <cfset jsonPacket['url'] = variables.routerUrl /> <cfset jsonPacket['type'] = variables.remotingType /> <cfset jsonPacket['namespace'] = ARGUMENTS.ns /> <cfset jsonPacket['actions'] = StructNew() /> <cfdirectory action="list" directory="#expandPath('.')#" name="totalCFCs" filter="*.cfc" recurse="false" /> <cfloop query="totalCFCs"> <cfset cfcName = ListFirst(totalCFCs.name, '.') /> <cfset newCfComponentMeta = GetComponentMetaData(cfcName) /> <cfif StructKeyExists(newCfComponentMeta, "ExtDirect")> <cfset CFCApi = ArrayNew(1) /> <cfset fnLen = ArrayLen(newCFComponentMeta.Functions) /> <cfloop from="1" to="#fnLen#" index="i"> <cfset currFn = newCfComponentMeta.Functions[i] /> <cfif StructKeyExists(currFn, "ExtDirect")> <cfset Fn = StructNew() /> <cfset Fn['name'] = currFn.Name/> <cfset Fn['len'] = ArrayLen(currFn.Parameters) /> <cfif StructKeyExists(currFn, "ExtFormHandler")> <cfset Fn['formHandler'] = true /> </cfif> <cfset ArrayAppend(CFCApi, Fn) /> </cfif> </cfloop> <cfset jsonPacket['actions'][cfcName] = CFCApi /> </cfif> </cfloop> <cfoutput><cfsavecontent variable="script">Ext.ns('#arguments. ns#');#arguments.ns#.#desc# = #SerializeJson(jsonPacket)#;</ cfsavecontent></cfoutput> <cfreturn script /> </cffunction> The getAPIScript method sets a few variables (including the 'actions' array), pulls a listing of all ColdFusion Components from the directory, loops over that listing, and finds any components containing "ExtDirect" in their root meta. With every component that does contain that meta, it then loops over each method, finds methods with "ExtDirect" in the function meta, and creates a structure with the function name and number of arguments, which is then added to an array of methods. When all methods have been introspected, the array of methods is added to the 'actions' array. Once all ColdFusion Components have been introspected, the entire packet is serialized into JSON, and returned to API.cfm for output. One item to note is that the script, when introspecting method metadata, also looks for a "ExtFormHandler" attribute. If it finds the attribute, it will include that in the method struct prior to placing the struct in the 'actions' array.

Business processes specified using BPEL will interact with their partners through operation invocations of web services. Web services are based on loosely coupled Service Oriented Architecture (SOA) . The communication between web services is done over Internet connections that may or may not be highly reliable. Web services could also raise faults due to logical errors and execution errors arising from defects in the infrastructure. Therefore, BPEL business processes will need to handle faults appropriately. BPEL processes may also need to signal faults themselves. Fault handling and signaling is an important aspect of business processes designed using BPEL. Faults in BPEL can arise in various situations: When a BPEL process invokes a synchronous web service operation, the operation might return a WSDL fault message, which results in a BPEL fault. A BPEL process can explicitly signal (throw) a fault. A fault can be thrown automatically, for example, when a join failure has occurred. The BPEL server might encounter error conditions in the runtime environment, network communications, or any other such area. BPEL defines several standard faults. WSDL faults WSDL faults occur due to synchronous operation invocations on partner web services. In WSDL, such faults are denoted with the &ltfault> element within the &ltoperation> declaration. In BPEL, WSDL faults are identified by the qualified name of the fault and the target namespace of the corresponding port type used in the operation declaration. The TravelApproval operation used on the TravelApprovalPT port type with input and output messages is shown in the following WSDL excerpt: <portType name="TravelApprovalPT"> <operation name="TravelApproval"> <input message="tns:TravelRequestMessage" /> <output message="aln:TravelResponseMessage" /> </operation> </portType> To add fault information to the operation, we first need to define a corresponding message. For simplicity, this message will be of the xs:string type: <message name="TravelFaultMessage"> <part name="error" type="xs:string" /> </message> Now we will add the fault declaration to the operation signature shown previously: <portType name="TravelApprovalPT"> <operation name="TravelApproval"> <input message="tns:TravelRequestMessage" /> <output message="aln:TravelResponseMessage" /> <fault name="fault" message="tns:TravelFaultMessage" /> </operation> </portType> WSDL does not require that we use unique fault names within the namespace used to define the operation. This implies that faults that have the same name and are defined within the same namespace will be considered as the same fault in BPEL. Keep this in mind when designing services that can potentially become partners of BPEL business processes, because this can lead to conflicts in fault handling during execution. Signaling faults A business process may sometimes need to explicitly signal a fault. For such a situation, BPEL provides the &ltthrow> activity. It has the following syntax: <throw faultName="name" /> BPEL does not require that we define fault names in advance, prior to their use in the &ltthrow> activity. This flexible approach can also be error-prone, because there is no compile-time checking of fault names. Therefore, a typo could result in a situation where a misspelled fault might not be handled by the designated fault handler. Faults can also have an associated variable that usually contains data related to the fault. If such a variable is associated with the fault, we need to specify it when throwing the fault. This is done by using the optional faultVariable attribute as shown here: <throw faultName="name" faultVariable="variable-name" /> The following example shows the most straightforward use of the &ltthrow> activity; where a WrongEmployeeName fault is thrown, no variable is needed. Remember that fault names are not declared in advance: <throw faultName="WrongEmployeeName" /> The faults raised with the &ltthrow> activity have to be handled in the BPEL process. Faults that are not handled will not be automatically propagated to the client as is the case in modern programming languages (Java for example). Rather, the BPEL process will terminate abnormally. Sometimes, however, we may want to signal faults to clients. Signaling faults to clients in synchronous replies A BPEL process offers operations to its clients through the &ltreceive> activity. If the process wants to provide a synchronous request/response operation, it sends a &ltreply> activity in response to the initial &ltreceive>. Remember that the type of the operation is defined in the WSDL document of the BPEL process. A synchronous request/response operation is defined as an operation that has an input and an output message, and an optional fault message. If such an operation has the fault part specified, we can use the &ltreply> activity to return a fault instead of the output message. The syntax of the &ltreply> activity in this case is: <reply partnerLink="partner-link-name" portType="port-type-name" operation="operation-name" variable="variable-name" <!-- optional --> faultName="fault-name" > </reply> When we specify a fault name to be returned through the &ltreply> activity, the variable name is optional. If we specify a variable name, then the variable has to be of the fault message type as defined in WSDL. Example Let's modify the BPEL process definition in the synchronous travel example and signal the fault (TravelFaultMessage) to the client by using the &ltreply> activity. First, we need to declare an additional variable that will hold the fault description to return to the client. The variable is of the TravelFaultMessage type: <variables> <!-- fault to the BPEL client --> <variable name="TravelFault" messageType="trv:TravelFaultMessage"/> </variables> ... Then we return the fault to the BPEL process client. We will need to check if something went wrong in the travel process. For the purpose of this example, we will check whether the selected flight ticket has been approved. This information is stored in the confirmationData part of the TravelResponse variable in the Approved element. Note that this is an oversimplification, but it demonstrates how to return faults. We can use an &ltif> activity to determine whether the ticket is approved; then we construct the fault variable and use the &ltreply> activity to return it to the client. This is shown in the following code: <!-- Check if the ticket is approved --> <if> <condition> $TravelResponse.confirmationData/aln:Approved='true' </condition> <!-- Send a response to the client --> <reply partnerLink="client" portType="trv:TravelApprovalPT" operation="TravelApproval" variable="TravelResponse"/> <else> <sequence> <!-- Create the TravelFault variable with fault description --> <assign> <copy> <from>string('Ticket not approved')</from> <to variable="TravelFault" part="error" /> </copy> </assign> <!-- Send a fault to the client --> <reply partnerLink="client" portType="trv:TravelApprovalPT" operation="TravelApproval" variable="TravelFault" faultName="fault" /> </sequence> </else> </if> If the ticket is not approved, the following fault is signaled to the client: <TravelFault> <part name="error"> <error > Ticket not approved </error> </part> </TravelFault> We have seen that signaling faults in synchronous replies is easy. Let us now discuss signaling faults in asynchronous scenarios. Signaling faults to clients in asynchronous scenarios If an asynchronous BPEL process needs to notify the client about a fault, it cannot use the &ltreply> activity. Remember that in asynchronous scenarios the client does not wait for the reply; rather, the process uses a callback. To return a fault in callback scenarios, we usually define additional callback operations on the same port type. Through these callback operations, we can signal that an exceptional situation has prevented normal completion of the process. To demonstrate how faults can be propagated to the client using a callback operation, we will use the asynchronous travel process example. First, we need to modify the travel BPEL process WSDL and introduce another operation called ClientCallbackFault. This operation consists of an input message called tns:TravelFaultMessage. The message is of the string type. The declaration of the operation and the message is shown in the following code excerpt: <message name="TravelFaultMessage"> <part name="error" type="xs:string" /> </message> <portType name="ClientCallbackPT"> <operation name="ClientCallback"> <input message="aln:TravelResponseMessage" /> </operation> <operation name="ClientCallbackFault"> <input message="tns:TravelFaultMessage" /> </operation> </portType> We can use the &ltif> activity to determine whether the ticket has been approved. If the ticket is not approved, however, we &ltinvoke> the ClientCallbackFault operation instead of using the &ltreply> activity to signal the fault to the client. This is shown in the following code excerpt: <!-- Check if the ticket is approved --> <if> <condition> $TravelResponse.confirmationData/aln:Approved='true' </condition> <!-- Make a callback to the client --> <invoke partnerLink="client" portType="trv:ClientCallbackPT" operation="ClientCallback" inputVariable="TravelResponse" /> <else> <sequence> <!-- Create the TravelFault variable with fault description --> <assign> <copy> <from>string('Ticket not approved')</from> <to variable="TravelFault" part="error" /> </copy> </assign> <!-- Send a fault to the client --> <invoke partnerLink="client" portType="trv:ClientCallbackPT" operation="ClientCallbackFault" inputVariable="TravelFault" /> </sequence> </else> </if> In the next section, we will look at how to handle faults thrown in BPEL processes. Handling faults Now that we are familiar with how faults are signaled, let us consider how the business process handles faults. When a fault occurs within a business process (this can be a WSDL fault, a fault thrown by the BPEL process, or any other type of fault), it means that the process may not complete successfully. The process can complete successfully only if the fault is handled within a scope. Business processes handle faults through fault handlers. A business process can handle a fault through one or more fault handlers. Within a fault handler, the business process defines custom activities that are used to recover from the fault and recover the partial (unsuccessful) work of the activity in which the fault has occurred. The fault handlers are specified before the first activity of the BPEL process, after the partner links and variables. The overall structure is shown in the following code excerpt: <process ...> <partnerLinks> ... </partnerLinks> <variables> ... </variables> <faultHandlers> <catch ... > <!-- Perform an activity --> </catch> <catch ... > <!-- Perform an activity --> </catch> ... <catchAll> <!-- catchAll is optional --> <!-- Perform an activity --> </catchAll> </faultHandlers> <sequence> </sequence> </process> We can see that within the fault handlers, we specify several &ltcatch> activities where we indicate the fault that we would like to catch and handle. Within a fault handler, we have to specify at least one &ltcatch> or a &ltcatchAll> activity. Of course, the &ltcatchAll> activity can be specified only once within a fault handler. Usually we will specify several &ltcatch> activities to handle specific faults and use the &ltcatchAll> to handle all other faults. The &ltcatch> activity has two attributes, of which we have to specify at least one: faultName: Specifies the name of the fault to be handled. faultVariable: Specifies the variable type used for fault data. Additionally, we can specify one of the following attributes (both are optional, but we may specify one, not both): faultMessageType: Specifies the WSDL message type of the fault to be handled. faultElement: Specifies the XML element type of the fault to be handled. The flexibility of &ltcatch> activities is high and several variations are permissible. Listed ahead are the most common: <faultHandlers> <catch faultName="trv:TicketNotApproved" > <!-- First fault handler --> <!-- Perform an activity --> </catch> <catch faultName="trv:TicketNotApproved" faultVariable="TravelFault" > <!-- Second fault handler --> <!-- Perform an activity --> </catch> <catch faultVariable="TravelFault" > <!-- Third fault handler --> <!-- Perform an activity --> </catch> <catchAll> <!-- Perform an activity --> </catchAll> </faultHandlers> We can see that fault handlers in BPEL are very similar to try/catch clauses found in modern programming languages.

Moodle 2.0 First Look Discover what's new in Moodle 2.0, how the new features work, and how it will impact you Get an insight into the new features of Moodle 2.0 Discover the benefits of brand new additions such as Comments and Conditional Activities Master the changes in administration with Moodle 2.0 The first and only book that covers all of the fantastic new features of Moodle 2.0         Read more about this book       (For more resources on Moodle, see here.)   Question: What are the basic requirements for Moodle 2.0 to function? Answer: It's important that either you (if you're doing this yourself) or your Moodle admin or webhost are aware of the requirements for Moodle 2.0. It needs: PHP must be 5.2.8 or later One of the following databases: MySQL 5.0.25 or later (InnoDB storage engine highly recommended) PostgreSQL 8.3 or later Oracle 10.2 or later MS SQL 2005 or later One of the following browsers: Firefox 3 or later Safari 3 or later Google Chrome 4 or later Opera 9 or later MS Internet Explorer 7 or later   Question: How can I upgrade to Moodle 2.0? Answer: If you already have an installation of Moodle, you will find instructions for upgrading in the docs on the main Moodle site here http://docs.moodle.org/en/Upgrading_to_Moodle_2.0. If you are upgrading from an earlier version of Moodle (such as 1.8) then you should upgrade to Moodle 1.9 first before going to 2.0. You must update incrementally; shortcuts – for example. updating from 1.7 directly to 2.0 -- are simply not possible. Read the docs carefully if you are planning on upgrading from very early versions such as 1.5 or 1.6.   Question: What are the potential problems with upgrading? Answer: There are a few challenges that one may come across while upgrading from Moodle 1.9 to 2.0 which are listed below: Themes: The way themes work has changed completely. While this allows for more flexible coding and templating, it does mean that if you had a customized theme it will not transfer over to Moodle 2 without some redesigning beforehand. Third party add-ons and custom code: The same applies to third party add-ons and custom code: it is highly unlikely they will work without significant alterations. Backup and Restore: Making courses from 1.9 or earlier restore into Moodle 2. 0 has proved very problematic and is still not entirely achievable. Although this is a priority for the Moodle developers, there is at the time of writing only a workaround involving restoring your course to a 1.9 site and then upgrading it to 2.0.   Question: How can teachers and students manage their learning? Answer: The two new features of Moodle 2.0 help teacher and students manage their learning: Conditional activities: A way to organize a course so that tasks are only available dependent on certain grades being obtained or criteria being met beforehand. Completion tracking: A way for students to have checkboxes next to their tasks that are either automatically marked as complete or which students themselves can manually mark if they feel they've finished the exercise – or alternatively a way for whole courses to be checked off as finished.   Question: What are the changes in the Themes structure for Moodle 2.0? Answer: The themes structure has been completely rewritten for Moodle 2.0. Themes that worked in 1.9 needed to be updated to work in 2.0. There is a wide variety of attractive new themes available. If you need to update your own theme or would like information on Moodle 2.0 theming, you will find the documentation at http://docs.moodle.org/en/Development:Themes_2.0 helpful. New to Moodle 2.0 are the following: Designer Mode: Turn this on so you're not served cached versions of themes, if you are designing themes or developing code. Allow theme changes in the URL: Enabling this will let users alter their theme via their Moodle URL using the syntax Allow blocks to use the dock: Enabling this will allow users to dock blocks if the theme supports it.   Question: Can we customize the MyMoodle page in Moodle 2.0? Answer: Yes, we can customize the default MyMoodle page. It's worth noting that on the MyMoodle page we can add blocks to the middle as well as the sides. With editing turned on, we're given the option to move a block to a central location.   Question: Can we Comment on the Moodle blog? Answer: Commenting on the Moodle blog is a bit of a workaround really; the Moodle blog doesn't really have a built-in commenting facility like, say WordPress. Rather, Moodle is making use of the new Comments feature which ordinarily appears as a block anywhere you want to add it.   Question: What are the improvements in the Blog option in Moodle 2.0 as compared to the previous version? Answer: There has always been a blogging option in a standard Moodle install. However, some users have found it unsatisfactory because of the following reasons: The blog is attached to the user profile so you can only have one blog There is no way to attach a blog or blog entry to a particular course There is no way for other people to comment on your blog For this reason, alternative blog systems (such as the contributed OU blog module) have become popular as they give users a wider range of options. The standard blog in Moodle 2.0 has changed, and now: A blog entry can optionally be associated with a course It is possible to comment on a blog entry Blog entries from outside of Moodle can be copied in It is now possible to search blog entries   Question: How to enable/disable the docking facility in Moodle 2.0? Answer: The docking facility can be managed in Moodle 2.0 as follows: The "docking" facility may be enabled or disabled for themes in Site Administration | Appearance | Themes | Theme settings. If we click the icon shown in the following screenshot, we also have the option of "docking" this over to the far left as a narrow tab.   Question: Has the HTML editor been replaced by some other editing tool? What is its advantage? Answer: In Moodle 2.0, the HTML editor has been replaced with a version known as Tiny MCE, a very popular Open Source editor you might have encountered in content management systems or blogging software such as WordPress. Along with Internet Explorer and Firefox, it will work with web browsers such as Safari, Chrome, and Opera, unlike Moodle's previous HTML editor. The following screenshot shows the new editor (on the bottom) with the original editor (on the top): There are many more options available to us when adding descriptions of our materials or summaries of our courses. However, one of the most powerful new features is the ability to add and embed media directly from within this new HTML editor.   Question: What have been the improvements related to Moodle Quiz? Answer: The following are the improvements to Moodle Quiz: The set up page has been simplified Creating questions has been simplified It's possible to flag questions for later referral Questions can be accessed with one click in the post-quiz review and correct/ incorrect questions are color-coded in an easy-to access navigation block   Question: What are Cohorts? Answer: Cohort is Moodle 2.0's take on the long wished for site-wide groups. When we click on the link we're taken to the following screen where we click on Add to enter details of the cohort we want to create:   Question: Has there been any modification in the Filters menu as compared to the previous versions On/Off options? Answer: The Manage Filters in Moodle 2.0 equates to the Filters menu in Moodle 1.9. The Manage Filters screen looks like the following screenshot (note—the screenshot only displays the first three filters): Previously, filters were either On or Off. Now we have three choices: Disabled: Nobody, in any course, can enable a filter. On: A filter is enabled by default and teachers can disable if they wish to. Off but available: A filter is off but teachers can enable it in their own courses.   Question: What are the changes in Site Administration? Answer: Perhaps the simplest way to explore this is to look at how this menu has altered since Moodle 1.9: Notifications/Registrations: A small but important change in Moodle 1.9, the Notifications screen contained a button you could click to register your site with http://moodle.org/. The page this took you to now has its own billing in Moodle 2.0, as the Registration link. Community hubs: The main Moodle community hub is known as MOOCH and you register with it here. You can also register your site with other community hubs. If you register with hubs, then teachers can add a Community block in their courses where users can search for a suitable course to enroll in or download. Summary In this article we took a look at the queries regarding what Moodle 2.0 has to offer with the exciting new modules and enhanced features, and the major overhauls in the file uploading and navigation system. Further resources on this subject: Moodle 1.9 Math [Book] Moodle Administration [Book] Moodle 1.9 for Teaching Special Education Children (5-10): Beginner's Guide [Book] Moodle 2.0: What's New in Add a Resource [Article] What's New in Moodle 2.0 [Article]

  jQuery Plugin Development Beginner's Guide A practical straightforward guide for creating your first jQuery plugin Utilize jQuery's plugin framework to create a wide range of useful jQuery plugins from scratch Understand development patterns and best practices and move up the ladder to master plugin development Discover the ins and outs of some of the most popular jQuery plugins in action A Beginner's Guide packed with examples and step-by-step instructions to quickly get your hands dirty in developing high quality jQuery plugins         Read more about this book       (For more resources on jQuery, see here.) Defining our own default plugin structure To make things easier to remember and apply, we are going to start off with the definition of what we will be referring to when speaking of the basic template that all the plugins we are going to develop should conform to. Actually, we have already had a quick look at it earlier in the previous chapter, but there's something more definitely worth saying. From now on, we will call the following code the default structure for our plugins. This is what we will promptly copy and paste into each file we're going to write a plugin into. jQuery.fn.PLUGINNAME = function() { return this.each(function() { // code });} Needless to say, the this.each loop iterates all of the matching elements. We return the jQuery object (this) to allow chaining. We extend the jQuery.fn object; all of the code will be put inside the function. Also: The file name of every plugin we're going to develop will bejquery.PLUGINNAME.js. For the moment, remember to always avoid referring to the jQuery object with the dollar sign ($), to prevent possible conflicts with other libraries. We'll get to using aliases very soon. All of the functions that we write to make our plugin work should be private and not accessible from outside, in an attempt to avoid cluttering and possible backwards incompatibility. If not from the very start, at least at the end, a user will be able to specify options to control the plugin behavior. Default options for the plugin will be publicly accessible to allow for easier customization with minimal code. The directory that the plugin resides in will also contain two other files, by default: index.html: This is our test page. jquery.js: This is the jQuery library that we need to make things work. Setting the basics for our first plugin As our first plugin, we might want to create something uncomplicated but somewhat impressive: what about something that, when the cursor is hovering over an element, substitutes the text contained with some words of our choice? Time for action – our first plugin, Part I Getting started in creating jQuery plugins in not difficult at all. For example, creating this simple plugin should help us in understanding how things actually work. Given that our plugin name is txtHover, create all the directories and files we need by default, and copy over the jquery.js file. Copy the default structure for our plugins to the plugin file and make sure the function is named accordingly. It should look like this: jQuery.fn.txtHover = function() { return this.each(function() { // code });}; Nothing's easier to do than change the text contained in some element. Inside the plugin function, write the following to let the trick happen: jQuery(this).text("Text changed"); To test this in action, we can modify the HTML document to look like this: <!DOCTYPE html><html><head> <script src="jquery.js"></script> <script src="jquery.txthover.js"></script> <script> $(document).ready(function() { $("p#one").txtHover(); }); </script></head><body> <p id="one">Some text.</p></body></html> Unfortunately, the result is neither fancy nor satisfactory—something we've experienced earlier too. But we're just getting started; we won't stop here this time! What just happened? The plugin is working correctly so far. When the page loads, the text is changed to what we had defined into the plugin code. However, there are a couple of things to pay attention to: The function text() from the jQuery API expects either one or no arguments to be passed: if there is no argument the function returns the current content of the selected element. The text string passed as an argument substitutes the element text otherwise. There are, however, some similarities with the html() function, which treats the text it operates on as if it were HTML code. That is, passing any tag to the html() function results in having the element possibly containing other elements after the operation (also, the same applies for getting HTML code from within the element), whereas the this function will just treat the HTML code as plain text and not affect any element hierarchy. The fact that the text cannot be changed, unless we directly modify the code of the plugin. Getting a step farther Despite the good realization, our plugin still misses the point. Our goal was to activate the text substitution whenever the mouse pointer hovered over the text to be replaced, whereas our current implementation does it right after the page is loaded. We put it inside the "document ready" statement, after all! Time for action – our first plugin, Part II: Hovering By adding little pieces of code one at a time, we can easily understand what we are going to do and which is the best layout for our plugin. Activating the plugin whenever the mouse pointer hovers over the selected elements is surely another little step that adds up to reach our final goal. Back to our plugin file, we have to change the code so that the whole mechanism activates when the hover event is triggered. jQuery provides a function called hover(), which we can use to achieve our objective: jQuery.fn.txtHover = function() { return this.each(function() { jQuery(this).hover(function() { jQuery(this).text("Mouse hovered"); }); });}; Now, on to testing. Once the mouse pointer hovers over the text, it is effectively replaced by our string. But even if the mouse is moved, the text doesn't revert to the original. In fact, looking at the hover documentation, we see the function can also take a second argument, that is, the pointer to a function to be executed when the mouse goes off the element. Our modified code will now look like the following: jQuery.fn.txtHover = function() { return this.each(function() { var oldTxt = jQuery(this).text(); jQuery(this).hover(function() { jQuery(this).text("Mouse hover"); }, function() { jQuery(this).text(oldTxt); }); });}; The result is somewhat better now: the text is changed when we leave the pointer on the paragraph, and is changed again to its original form once the pointer is moved away. What just happened? We might be a little more satisfied with this evolution of our plugin, even though it's far from complete. Its functioning is fairly straightforward: we have made use of a variable (oldTxt) to store the old content, and we then have proceeded to using two anonymous functions (function(){ }), passed as arguments to the hover() function, to handle the mouse hover event (write our string) and the mouse out event (text back to original). There's still something to do though: We have used far too many instances of $(this) in our code and, on a larger scale application, a lot of memory would be wasted this way. It will be incredibly better for performance to make up a variable and use it every time we refer to the element with $(this). The text cannot be changed, unless we directly modify the code of the plugin. Have a go hero – html () versus text () Read the documentation for the html() function. Create a plugin that, once the mouse pointer hovers over an element, displays the HTML code of its content. The content should then change back to normal once the mouse pointer is moved away from that space. What happens if the html() and text() functions are used one inside the other, that is, $(sel).text($(sel).html()) or $(sel).html($(sel).text())? Just play around a little bit.

Most businesses today use Business Intelligence (BI), the process of obtaining business information from available data, to control their affairs. If you're new to Business Intelligence, then this definition may leave you with the following questions: What is data? What is the information obtained from it? What is the difference between data and the information obtained from it? You may be confused even more if you learn that data represents groups of information related to an object or a set of objects. Depending on your needs, though, such groups of information may or may not be immediately useful, and often require additional processing such as filtering, formatting, and/or calculating to take on a meaning. For example, information about your customers may be organized in a way that is stored in several database tables related to each other. For security purposes, some pieces of information stored in this way may be encoded, or just represented in binary, and therefore not immediately readable. It's fairly obvious that some processing must be applied before you can make use of such information. So, data can be thought of as the lowest level of abstraction from which meaningful information is derived. But what is information anyway? Well, a piece of information normally represents an answer to a certain question. For example, you want to know how many new customers have registered on your site this year. An answer to this question can be obtained with a certain query issued against the table containing customer registration dates, giving you the information you asked for. Data, information, and Business Intelligence Although the terms data and information refer to similar things, they aren't really interchangeable as there is some difference in their meaning and spirit. Talking about data, as a rule, involves its structure, format, storage, as well as ways in which you can access and manipulate it. In contrast, when talking about information, you mean food for your decision-making process. So, data can be viewed as low-level information structures, where the internal representation matters. Therefore, the ways in which you can extract useful information from data entirely depend on the structure and storage of that data. The following diagram gives a conceptual view of delivering information from different data sets: As you can see from the figure, information can be derived from different data sources, and by different means. Once it's derived, though, it doesn't matter where it has come from, letting its consumers concentrate on the business aspects rather than on the specifics of the internal structure. For example, you might derive some pieces of data from the Web, using the Oracle Database's XQuery feature, and then process it as native database data. To produce meaningful information from your data, you will most likely need to perform several processing steps, load new data, and summarize the data. This is why the Business Intelligence layer usually sits on top of many data sources, consolidating information from various business systems and heterogeneous platforms. The following figure gives a graphical depiction of a Business Intelligence system. In particular, it shows you that the Business Intelligence layer consumes information derived from various sources and heterogeneous platforms. It is intuitively clear that the ability to solve problems is greatly enhanced if you can effectively handle all the information you're getting. On the other hand, extracting information from data coming in from different sources may become a nightmare if you try to do it on your own, with only the help of miscellaneous tools. Business Intelligence comes to the rescue here, ensuring that the extraction, transformation, and consolidation of data from disparate sources becomes totally transparent to you. For example, when using a Business Intelligence application for reporting, you may never figure out exactly what happens behind the scenes when you instruct the system to prepare another report. The information you need for such a report may be collected from many different sources, hiding the complexities associated with handling heterogeneous data. But, without Business Intelligence, that would be a whole different story, of course. Imagine for a moment that you have to issue several queries against different systems, using different tools, and you then have to consolidate the results somehow—all just to answer a single business question such as: what are the top three customers for the preceding quarter? As you have no doubt realized, the software at the Business Intelligence layer is used to provide a business-centric view of data, eliminating as much of the technology-specific logic as possible. What this means in practice is that information consumers working at the Business Intelligence layer may not even know that, say, customer records are stored in a Lightweight Directory Access Protocol (LDAP) database, but purchase orders are kept in a relational database. The kind of business questions you may need to answer As you just learned, Business Intelligence is here to consolidate information from disparate sources so that you need not concern yourself with it. Okay, but why might you need to gather and process heterogeneous data? The answer is clear. You might need it in order to answer analytical questions that allow you to understand and run your business better. In the following two sections, you'll look at some common questions that Business Intelligence can help you answer. Then, you'll see how you can ask those questions with the help of Business Intelligence tools. Answering basic business questions The set of questions you may need your Business Intelligence system to answer will vary depending on your business and, of course, your corresponding functions. However, to give you a taste of what Business Intelligence can do for you, let's firrst look at some questions that are commonly brought up by business users: What is the average salary throughout the entire organization? Which customers produce the most revenue? What is the amount of revenue each salesman brought in over the preceding quarter? What is the profitability of each product? If you run your business online, you may be also interested in hit counting and traffic analysis questions, such as the following: How much traffic does a certain account generate over a month? What pages in your site are most visited? What are the profits made online? Looking at the business analysis requests presented here, a set of questions related to your own business may flash into your mind. Answering probing analytical questions In the preceding section, you looked at some common questions a business analyst is usually interested in asking. But bowing to the reality, you may have to answer more probing questions in your decision-making process, in order to determine changes in the business and find ways to improve it. Here are some probing analytical questions you might need to find answers to: How do sales for this quarter compare to sales for the preceding quarter? What factors impact our sales? Which products are sold better together? What are ten top-selling products in this region? What are the factors influencing the likelihood of purchase? As you can see, each of these questions reflects a certain business problem. Looking through the previous list, though, you might notice that some of the questions shown here can be hard to formulate with the tools available in a computer application environment. There's nothing to be done here; computers like specific questions. Unlike humans, machines can give you exactly what you ask for, not what you actually mean. So, even an advanced Business Intelligence application will require you to be as specific as possible when it comes to putting a question to it. It's fairly clear that the question about finding the factors impacting sales needs to be rephrased to become understandable for a Business Intelligence application. How you would rephrase it depends on the specifics of your business, of course. Often, it's good practice to break apart a problem into simpler questions. For example, the first question on the above list—the one about comparing quarter sales—might be logically divided into the following two questions: What are the sales figures for this quarter? What are the sales figures for the last quarter? Once you get these questions answered, you can compare the results, thus answering the original, more generically phrased question. It can also provide one definition or variation for drill down. In the above example, it's fairly obvious what specific questions can be derived from the generic question. There may be probing questions, though, whose derived questions are not so obvious. For example, consider the following question: What motivates a customer to buy? This could perhaps be broken down into the following questions: Where did visitors come from? Which pages did they visit before reaching the product page? Of course, the above list does not seem to be complete—some other questions might be added. Asking business questions using data-access tools As you might guess, although all these questions sound simple when formulated in plain English, they are more difficult to describe when using data-access tools. If you're somewhat familiar with SQL, you might notice that most of the analytical questions discussed here cannot be easily expressed with the help of SQL statements, even if the underlying data is relational. For example, the problem of finding the top three salespersons for a year may require you to write a multi-line SQL request including several sub-queries. Here is what such a query might look like: SELECT emp.ename salesperson, top_emp_orders.sales sales FROM (SELECT all_orders.sales_empno empno, all_orders.total_sales FROM (SELECT sales_empno, SUM(ord_total) total_sales, RANK() OVER (ORDER BY SUM(ord_total) DESC) sal_rank FROM orders WHERE EXTRACT(YEAR FROM ord_dt) = 2009 GROUP BY sales_empno )all_orders WHERE all_orders.sal_rank<=3 )top_emp_orders, employees emp WHERE top_emp_orders.empno = emp.empno ORDER BY sales DESC; This might produce something like this: If you're not an SQL guru of course, writing the above query and then debugging it could easily take a couple of hours. Determining profitability by customer, for example, might take you another couple of hours to write a proper SQL query. In other words, business questions are often somewhat tricky (if possible at all) to implement with SQL. All this does not mean that SQL is not used in the area of Business Intelligence. Quite the contrary, SQL is still indispensable here. In fact, SQL has a lot to offer when it comes to data analysis. As you just saw, though, composing complex queries assumes solid SQL skills. Thankfully, most Business Intelligence tools use SQL behind the scenes totally transparently to users. Now let's look at a simple example illustrating how you can get an analytical question answered with a Business Intelligence tool—Oracle BI Discoverer Plus in this particular example. Suppose you simply want to calculate the average salary sum over the organization. This example could use the records from the hr.employees demonstration table. Creating a worksheet representing the records of a database table in the Discoverer Plus focuses on issues related to analyzing data, and creating reports with the tools available through the Oracle Business Intelligence suite. For now, look at the following screenshot to see what such a worksheet might look like: As you can see in the previous screenshot, a Discoverer Plus worksheet is similar to one in MS Excel. As in Excel, there are toolbars and menus offering a lot of options for manipulating and analyzing data presented on the worksheet. In addition, Discoverer Plus offers Item Navigator, which enables you to add data to (or remove it from) the worksheet. The data structure you can see in Item Navigator is retrieved from the database. When we return to our example, answering the question: "what is the average salary across the organization?"Similarly, in Excel, it is as simple as selecting the Salary SUM column on the worksheet, choosing an appropriate menu, and setting some parameters in the dialog shown next. After you click the OK button in this dialog box, the calculated average will be added to the worksheet in the position specified. So, the Total dialog shown in the following screenshot provides an efficient means for automating the process of creating a total on a specified data column: As you can see, this approach doesn't require you to write an SQL query on your own. Instead, Discoverer Plus will do it for you implicitly, thus allowing you to concentrate on business issues rather than data access issues. This previous example should have given you a taste of what Business Intelligence can do for you.

  WS-BPEL 2.0 for SOA Composite Applications with IBM WebSphere 7 Define, model, implement, and monitor real-world BPEL 2.0 business processes with SOA-powered BPM Develop BPEL and SOA composite solutions with IBM's WebSphere SOA platform Automate business processes with WS-BPEL 2.0 and develop SOA composite applications efficiently Detailed explanation of advanced topics, such as security, transactions, human workflow, dynamic processes, fault handling, and more—enabling you to work smarter      Introduction IBM provides comprehensive support for SOA through various products within the WebSphere product line. In this article, we will look at the most important products that provide support for BPEL. These will be: WebSphere Integration Developer, which is used for the development of BPEL processes (and several other SOA artifacts, such as services) Websphere Process Server, which is used for the execution of BPEL processes (and other SOA artifacts) In this article, we use WebSphere Integration Developer 7.0 and WebSphere Process Server 7.0. Let us now have a look at the BPEL support in WebSphere Integration Developer (WID). WID provides comprehensive support for BPEL and includes activities as specified by the BPEL specification. WID also provides some IBM-specific extensions for BPEL, which we will discuss later in this section. First, we will look at long-running processes and microflows. Long-running processes and microflows We can use BPEL to orchestrate services. Such orchestrations can range from relatively simple integration scenarios, where a BPEL process invokes several services within the enterprise, to complex processes, where several services from different enterprises are invoked together with human tasks. Human tasks in BPEL processes allow us to include human interactions. From the performance and transactional perspective, a process server can treat these two kinds of BPEL processes in a different way. Relatively simple BPEL processes that contain synchronous invocation of services only and execute in a relatively short time can be executed within a single transaction. A process server does not have to persist such processes. Persisting a process means that the process server saves the states of activities, including the values of variables, to the persistent storage (database). We can look at the process flow as it has been executed and monitor the state of variables. This is sometimes called dehydration. BPEL processes that include more complex, asynchronous invocations and also human tasks cannot be executed in a relatively short time. In special cases, such processes can take hours, days, weeks, or even months to execute. Such processes should be persisted and cannot be executed within a single transaction. IBM WebSphere Process Server treats short-running and long-running processes differently. The developer has to decide at the time of development what type of process it will use. There are two options: Long-running processes Microflows Microflows are best suited for short BPEL processes that contain a short series of activities that should be executed very quickly in the runtime environment. Microflows are not interruptible. They are contained within a single ACID (Atomicity, Consistency, Isolation, and Durability) transaction. Therefore, the microflow should not wait for external input (such as callback from a service using &ltreceive> or &gtpick<). Microflows also cannot contain human tasks. The process server does not persist the state of microflows. Long-running processes are best suited for BPEL processes that contain many activities and include parallel flows, external inputs, or human tasks. Long-running processes execute within several ACID transactions. Therefore, we have to set transaction boundaries. We will talk more about transaction boundaries later in this section. Also, please do not confuse ACID transactions and compensating transactions. A long-running process will use several ACID transactions; however, it will be able to compensate activities using the BPEL compensation handlers. Usually we select the type of the process when we create a new BPEL process in WID. We select a New Business Process. Then we define the name of the process, as shown in the following screenshot: Next, we select Long-running process or Microflow: Here we also select whether we want to use the WebSphere BPEL extensions. We will discuss more about the WebSphere extensions later in this article. Overview of BPEL activities We develop BPEL processes in WID using a graphical development interface, where we drag-and-drop the various BPEL activities from the palette on the left-hand side to the main process flow: The BPEL activities in WID are structured into the following groups: Basic Actions: Invoke, assign, receive, receive choice, reply, wait, empty action, snippet, data map Structures: scope, parallel activities, sequence, choice, while loop, repeat until loop, for each, generalized flow Faults: compensate, throw, rethrow, terminate Human Workflow: human tasks, collaboration scope On the left, we can see familiar BPEL activities, such as Invoke, Assign, Receive, and so on. After dragging-and-dropping them to the process, we have to define several properties. We use the Properties tab: We can set several different properties; some of them are a part of the BPEL language, and some of them part of the WebSphere execution environment. Although WID in most cases uses the standard BPEL naming for activities, there are some cases where the naming differs. We list the differences in the following table: Instead of using the graphical business process editor, we can always switch to the raw XML view of the BPEL process. In this case, we will see that behind the scenes the BPEL code is generated.