How-To Tutorials

In this article by Andrew J Wagner, author of the book Learning Swift, we will cover: What is an optional? How to unwrap an optional Optional chaining Implicitly unwrapped optionals How to debug optionals The underlying implementation of an optional (For more resources related to this topic, see here.) Introducing optionals So, we know that the purpose of optionals in Swift is to allow the representation of the absent value, but what does that look like and how does it work? An optional is a special type that can wrap any other type. This means that you can make an optional String, optional Array, and so on. You can do this by adding a question mark (?) to the type name: var possibleString: String? var possibleArray: [Int]? Note that this code does not specify any initial values. This is because all optionals, by default, are set to no value at all. If we want to provide an initial value, we can do so like any other variable: var possibleInt: Int? = 10 Also note that, if we leave out the type specification (: Int?), possibleInt would be inferred to be of the Int type instead of an Int optional. It is pretty verbose to say that a variable lacks a value. Instead, if an optional lacks a variable, we say that it is nil. So, both possibleString and possibleArray are nil, while possibleInt is 10. However, possibleInt is not truly 10. It is still wrapped in an optional. You can see all the forms a variable can take by putting the following code in to a playground: var actualInt = 10 var possibleInt: Int? = 10 var nilInt: Int? println(actualInt) // "10" println(possibleInt) // "Optional(10)" println(nilInt) // "nil" As you can see, actualInt prints out as we expect it to, but possibleInt prints out as an optional that contains the value 10 instead of just 10. This is a very important distinction because an optional cannot be used as if it were the value it wraps. The nilInt optional just reports that it is nil. At any point, you can update the value within an optional, including the fact that you can give it a value for the first time using the assignment operator (=): nilInt = 2 println(nilInt) // "Optional(2)" You can even remove the value within an optional by assigning it to nil: nilInt = nil println(nilInt) // "nil" So, we have this wrapped form of a variable that may or may not contain a value. What do we do if we need to access the value within an optional? The answer is that we must unwrap it. Unwrapping an optional There are multiple ways to unwrap an optional. All of them essentially assert that there is truly a value within the optional. This is a wonderful safety feature of Swift. The compiler forces you to consider the possibility that an optional lacks any value at all. In other languages, this is a very commonly overlooked scenario that can cause obscure bugs. Optional binding The safest way to unwrap an optional is using something called optional binding. With this technique, you can assign a temporary constant or variable to the value contained within the optional. This process is contained within an if statement, so that you can use an else statement for when there is no value. An optional binding looks like this: if let string = possibleString {    println("possibleString has a value: \(string)") } else {    println("possibleString has no value") } An optional binding is distinguished from an if statement primarily by the if let syntax. Semantically, this code says "if you can let the constant string be equal to the value within possibleString, print out its value; otherwise, print that it has no value." The primary purpose of an optional binding is to create a temporary constant that is the normal (nonoptional) version of the optional. It is also possible to use a temporary variable in an optional binding: possibleInt = 10 if var int = possibleInt {    int *= 2 } println(possibleInt) // Optional(10) Note that an astrix (*) is used for multiplication in Swift. You should also note something important about this code, that is, if you put it into a playground, even though we multiplied int by 2, the value does not change. When we print out possibleInt later, the value still remains Optional(10). This is because even though we made the int variable (otherwise known as mutable), it is simply a temporary copy of the value within possibleInt. No matter what we do with int, nothing will be changed about the value within possibleInt. If we need to update the actual value stored within possibleInt, we need to simply assign possibleInt to int after we are done modifying it: possibleInt = 10 if var int = possibleInt {    int *= 2    possibleInt = int } println(possibleInt) // Optional(20) Now the value wrapped inside possibleInt has actually been updated. A common scenario that you will probably come across is the need to unwrap multiple optional values. One way of doing this is by simply nesting the optional bindings: if let actualString = possibleString {    if let actualArray = possibleArray {        if let actualInt = possibleInt {            println(actualString)            println(actualArray)            println(actualInt)        }    } } However, this can be a pain as it increases the indentation level each time to keep the code organized. Instead, you can actually list multiple optional bindings in a single statement separated by commas: if let actualString = possibleString,    let actualArray = possibleArray,    let actualInt = possibleInt {    println(actualString)    println(actualArray)    println(actualInt) } This generally produces more readable code. This way of unwrapping is great, but saying that optional binding is the safe way to access the value within an optional implies that there is an unsafe way to unwrap an optional. This way is called forced unwrapping. Forced unwrapping The shortest way to unwrap an optional is by forced unwrapping. This is done using an exclamation mark (!) after the variable name when it is used: possibleInt = 10 possibleInt! *= 2   println(possibleInt) // "Optional(20)" However, the reason it is considered unsafe is that your entire program crashes if you try to unwrap an optional that is currently nil: nilInt! *= 2 // fatal error The full error you get is "unexpectedly found as nil while unwrapping an optional value". This is because forced unwrapping is essentially your personal guarantee that the optional truly holds a value. This is why it is called forced. Therefore, forced unwrapping should be used in limited circumstances. It should never be used just to shorten up the code. Instead, it should only be used when you can guarantee, from the structure of the code, that it cannot be nil, even though it is defined as an optional. Even in this case, you should check whether it is possible to use a nonoptional variable instead. The only other place you may use it is when your program truly cannot recover if an optional is nil. In these circumstances, you should at least consider presenting an error to the user, which is always better than simply having your program crash. An example of a scenario where forced unwrapping may be used effectively is with lazily calculated values. A lazily calculated value is a value that is not created until the first time it is accessed. To illustrate this, let's consider a hypothetical class that represents a filesystem directory. It would have a property that lists its contents that are lazily calculated. The code would look something like this: class FileSystemItem {} class File: FileSystemItem {} class Directory: FileSystemItem {    private var realContents: [FileSystemItem]?    var contents: [FileSystemItem] {        if self.realContents == nil {           self.realContents = self.loadContents()        }        return self.realContents!    }      private func loadContents() -> [FileSystemItem] {        // Do some loading        return []    } } Here, we defined a superclass called FileSystemItem that both File and Directory inherit from. The contents of a directory is a list of any kind of FileSystemItem. We define content as a calculated variable and store the real value within the realContents property. The calculated property checks whether there is a value yet loaded for realContents; if there isn't, it loads the contents and puts it into the realContents property. Based on this logic, we know with 100 percent certainty that there will be a value within realContents by the time we get to the return statement, so it is perfectly safe to use forced unwrapping. Nil coalescing In addition to optional binding and forced unwrapping, Swift also provides an operator called the nil coalescing operator to unwrap an optional. This is represented by a double question mark (??). Basically, this operator lets us provide a default value for a variable or operation result in case it is nil. This is a safe way to turn an optional value into a nonoptional value and it would look something like this: var possibleString: String? = "An actual string" println(possibleString ?? "Default String")   // "An Actual String" Here, we ask the program to print out possibleString unless it is nil, in which case, it will just print Default String. Since we did give it a value, it printed out that value and it is important to note that it printed out as a regular variable, not as an optional. This is because one way or another, an actual value will be printed. This is a great tool for concisely and safely unwrapping an optional when a default value makes sense. Optional chaining A common scenario in Swift is to have an optional that you must calculate something from. If the optional has a value you want to store the result of the calculation on, but if it is nil, the result should just be set to nil: var invitee: String? = "Sarah" var uppercaseInvitee: String? if let actualInvitee = invitee {    uppercaseInvitee = actualInvitee.uppercaseString } This is pretty verbose. To shorten this up in an unsafe way, we could use forced unwrapping: uppercaseInvitee = invitee!.uppercaseString However, optional chaining will allow us to do this safely. Essentially, it allows optional operations on an optional. When the operation is called, if the optional is nil, it immediately returns nil; otherwise, it returns the result of performing the operation on the value within the optional: uppercaseInvitee = invitee?.uppercaseString So in this call, invitee is an optional. Instead of unwrapping it, we will use optional chaining by placing a question mark (?) after it, followed by the optional operation. In this case, we asked for the uppercaseInvitee property on it. If invitee is nil, uppercaseInvitee is immediately set to nil without it even trying to access uppercaseString. If it actually does contain a value, uppercaseInvitee gets set to the uppercaseString property of the contained value. Note that all optional chains return an optional result. You can chain as many calls, both optional and nonoptional, as you want in this way: var myNumber: String? = "27" myNumber?.toInt()?.advancedBy(10).description This code attempts to add 10 to myNumber, which is represented by String. First, the code uses an optional chain in case myNumber is nil. Then, the call to toInt uses an additional optional chain because that method returns an optional Int type. We then call advancedBy, which does not return an optional, allowing us to access the description of the result without using another optional chain. If at any point any of the optionals are nil, the result will be nil. This can happen for two different reasons: This can happen because myNumber is nil This can also happen because toInt returns nil as it cannot convert String to the Int type If the chain makes it all the way to advanceBy, there is no longer a failure path and it will definitely return an actual value. You will notice that there are exactly two question marks used in this chain and there are two possible failure reasons. At first, it can be hard to understand when you should and should not use a question mark to create a chain of calls. The rule is that you should always use a question mark if the previous element in the chain returns an optional. However, since you are prepared, let's look at what happens if you use an optional chain improperly: myNumber.toInt() // Value of optional type 'String?' not unwrapped In this case, we try to call a method directly on an optional without a chain so that we get an error. We also have the case where we try to inappropriately use an optional chain: var otherNumber = "10" otherNumber?.toInt() // Operand of postfix '?'   should have optional type Here, we get an error that says a question mark can only be used on an optional type. It is great to have a good sense of catching errors, which you will see when you make mistakes, so that you can quickly correct them because we all make silly mistakes from time to time. Another great feature of optional chaining is that it can be used for method calls on an optional that does not actually return a value: var invitees: [String]? = [] invitee?.removeAll(keepCapacity: false) In this case, we only want to call removeAll if there is truly a value within the optional array. So, with this code, if there is a value, all the elements are removed from it: otherwise, it remains nil. In the end, option chaining is a great choice for writing concise code that still remains expressive and understandable. Implicitly unwrapped optionals There is a second type of optional called an implicitly unwrapped optional. There are two ways to look at what an implicitly unwrapped optional is. One way is to say that it is a normal variable that can also be nil. The other way is to say that it is an optional that you don't have to unwrap to use. The important thing to understand about them is that like optionals, they can be nil, but like a normal variable, you do not have to unwrap them. You can define an implicitly unwrapped optional with an exclamation mark (!) instead of a question mark (?) after the type name: var name: String! Just like with regular optionals, implicitly unwrapped optionals do not need to be given an initial value because they are nil by default. At first, this may sound like it is the best of both worlds, but in reality, it is more like the worst of both worlds. Even though an implicitly unwrapped optional does not have to be unwrapped, it will crash your entire program if it is nil when used: name.uppercaseString // Crash A great way to think about them is that every time an implicitly unwrapped optional is used, it is implicitly performing a forced unwrapping. The exclamation mark is placed in its type declaration instead of using it every time. This is particularly bad because it appears the same as any other variable except for how it is declared. This means that it is very unsafe to use, unlike a normal optional. So, if implicitly unwrapped optionals are the worst of both worlds and are so unsafe, why do they even exist? The reality is that in rare circumstances, they are necessary. They are used in circumstances where a variable is not truly optional, but you also cannot give an initial value to it. This is almost always true in the case of custom types that have a member variable that is nonoptional, but cannot be set during initialization. A rare example of this is a view in iOS. UIKit, as we discussed earlier, is the framework that Apple provides for iOS development. In it, Apple has a class called UIView that is used for displaying content on the screen. Apple also provides a tool in Xcode called Interface Builder that lets you design these views in a visual editor instead of in code. Many views designed in this way need references to other views that can be accessed programmatically later. When one of these views is loaded, it is initialized without anything connected and then all the connections are made. Once all the connections are made, a function called awakeFromNib is called on the view. This means that these connections are not available for use during initialization, but are available once awakeFromNib is called. This order of operations also ensures that awakeFromNib is always called before anything actually uses the view. This is a circumstance where it is necessary to use an implicitly unwrapped optional. A member variable may not be defined until the view is initialized and when it is completely loaded: import UIKit class MyView: UIView {    @IBOutlet var button : UIButton!    var buttonOriginalWidth : CGFloat!      override func awakeFromNib() {        self.buttonOriginalWidth = self.button.frame.size.width    } } Note that we have actually declared two implicitly unwrapped optionals. The first is a connection to button. We know this is a connection because it is preceded by @IBOutlet. This is declared as an implicitly unwrapped optional because the connections are not set up until after initialization, but they are still guaranteed to be set up before any other methods are called on the view. This also then leads us to make our second variable, buttonOriginalWidth, implicitly unwrapped because we need to wait until the connection is made before we can determine the width of button. After awakeFromNib is called, it is safe to treat both button and buttonOriginalWidth as nonoptional. You may have noticed that we had to dive pretty deep in to app development in order to find a valid use case for implicitly unwrapped optionals, and this is arguably only because UIKit is implemented in Objective-C. Debugging optionals We already saw a couple of compiler errors that we commonly see because of optionals. If we try to call a method on an optional that we intended to call on the wrapped value, we will get an error. If we try to unwrap a value that is not actually optional, we will get an error that the variable or constant is not optional. We also need to be prepared for runtime errors that optionals can cause. As discussed, optionals cause runtime errors if you try to forcefully unwrap an optional that is nil. This can happen with both explicit and implicit forced unwrapping. If you followed my advice so far in this article, this should be a rare occurrence. However, we all end up working with third-party code, and maybe they were lazy or maybe they used forced unwrapping to enforce their expectations about how their code should be used. Also, we all suffer from laziness from time to time. It can be exhausting or discouraging to worry about all the edge cases when you are excited about programming the main functionality of your app. We may use forced unwrapping temporarily while we worry about that main functionality and plan to come back to handle it later. After all, during development, it is better to have a forced unwrapping crash the development version of your app than it is for it to fail silently if you have not yet handled that edge case. We may even decide that an edge case is not worth the development effort of handling because everything about developing an app is a trade-off. Either way, we need to recognize a crash from forced unwrapping quickly, so that we don't waste extra time trying to figure out what went wrong. When an app tries to unwrap a nil value, if you are currently debugging the app, Xcode shows you the line that tries to do the unwrapping. The line reports that there was EXC_BAD_INSTRUCTION and you will also get a message in the console saying fatal error: unexpectedly found nil while unwrapping an Optional value:   You will also sometimes have to look at which code currently calls the code that failed. To do that, you can use the call stack in Xcode. When your program crashes, Xcode automatically displays the call stack, but you can also manually show it by going to View | Navigators | Show Debug Navigator. This will look something as follows:   Here, you can click on different levels of code to see the state of things. This becomes even more important if the program crashes within one of Apple's framework, where you do not have access to the code. In that case, you should move up the call stack to the point where your code is called in the framework. You may also be able to look at the names of the functions to help you figure out what may have gone wrong. Anywhere on the call stack, you can look at the state of the variables in the debugger, as shown in the following screenshot:   If you do not see this variable's view, you can display it by clicking on the button at the bottom-left corner, which is second from the right that will be grayed out. Here, you can see that invitee is indeed nil, which is what caused the crash. As powerful as the debugger is, if you find that it isn't helping you find the problem, you can always put println statements in important parts of the code. It is always safe to print out an optional as long as you don't forcefully unwrap it like in the preceding example. As we saw earlier, when an optional is printed, it will print nil if it doesn't have a value or it will print Optional(<value>) if it does have a value. Debugging is an extremely important part of becoming a productive developer because we all make mistakes and create bugs. Being a great developer means that you can identify problems quickly and understand how to fix them soon after that. This will largely come from practice, but it will also come when you have a firm grasp of what really happens with your code instead of simply adapting some code you find online to fit your needs through trial and error. The underlying implementation At this point, you should have a pretty strong grasp of what an optional is and how to use and debug it, but it is valuable to look deeper at optionals and see how they actually work. In reality, the question mark syntax for optionals is just a special shorthand. Writing String? is equivalent to writing Optional<String>. Writing String! is equivalent to writing ImplicitlyUnwrappedOptional<String>. The Swift compiler has shorthand versions because they are so commonly used This allows the code to be more concise and readable. If you declare an optional using the long form, you can see Swift's implementation by holding command and clicking on the word Optional. Here, you can see that Optional is implemented as an enumeration. If we simplify the code a little, we have: enum Optional<T> {    case None    case Some(T) } So, we can see that Optional really has two cases: None and Some. None stands for the nil case, while the Some case has an associated value, which is the value wrapped inside Optional. Unwrapping is then the process of retrieving the associated value out of the Some case. One part of this that you have not seen yet is the angled bracket syntax (<T>). This is a generic and essentially allows the enumeration to have an associated value of any type. Realizing that optionals are simply enumerations will help you to understand how to use them. It also gives you some insight into how concepts are built on top of other concepts. Optionals seem really complex until you realize that they are just two-case enumerations. Once you understand enumerations, you can pretty easily understand optionals as well. Summary We only covered a single concept, optionals, in this article, but we saw that this is a pretty dense topic. We saw that at the surface level, optionals are pretty straightforward. They offer a way to represent a variable that has no value. However, there are multiple ways to get access to the value wrapped within an optional, which have very specific use cases. Optional binding is always preferred as it is the safest method, but we can also use forced unwrapping if we are confident that an optional is not nil. We also have a type called implicitly unwrapped optional to delay the assigning of a variable that is not intended to be optional, but we should use it sparingly because there is almost always a better alternative. Resources for Article: Further resources on this subject: Network Development with Swift [article] Flappy Swift [article] Playing with Swift [article]

JDeveloper built-in schema validation Oracle JDeveloper 11g has built-in support for XML schema validation. If an XML document includes a reference to an XML schema, the XML document may be validated with the XML schema using the built-in feature. An XML schema may be specified in an XML document using the xsi:noNamespaceSchemaLocation attribute or the xsi:namespaceSchemaLocation attribute. Before we discuss when to use which attribute, we need to define the target namespace. A schema is a collection of type definitions and element declarations whose names belong to a particular namespace called a target namespace. Thus, a target namespace distinguishes between type definitions and element declarations from different collections. An XML schema doesn't need to have a target namespace. If the XML schema has a target namespace, specify the schema's location in an XML document using the xsi:namespaceSchemaLocation attribute. If the XML schema does not have a target namespace, specify the schema location using the xsi:noNamespaceSchemaLocation attribute. The xsi:noNamespaceSchemaLocation and xsi:namespaceSchemaLocation attributes are a hint to the processor about the location of an XML schema document. The example XML schema document that we shall create is catalog.xsd and is listed here: <?xml version="1.0" encoding="utf-8"?><xsd:schema > <xsd:element name="catalog"type="catalogType"/> <xsd:complexType name="catalogType"> <xsd:sequence> <xsd:element ref="journal" minOccurs="0"maxOccurs="unbounded"/> </xsd:sequence> </xsd:complexType> <xsd:element name="journal" type="journalType"/> <xsd:complexType name="journalType"> <xsd:sequence> <xsd:element ref="article" minOccurs="0" maxOccurs="unbounded"/> </xsd:sequence> <xsd:attribute name="title" type="xsd:string"/> <xsd:attribute name="publisher" type="xsd:string"/> <xsd:attribute name="edition" type="xsd:string"/> </xsd:complexType> <xsd:element name="article" type="articleType"/> <xsd:complexType name="articleType"> <xsd:sequence> <xsd:element name="title" type="xsd:string"/> <xsd:element name="author" type="xsd:string"/> </xsd:sequence> <xsd:attribute name="section" type="xsd:string"/> </xsd:complexType></xsd:schema> The XML document instance that we shall generate from the schema is catalog.xml and is listed as follows: <?xml version="1.0" encoding="utf-8"?><catalog><journal title="Oracle Magazine" publisher="OraclePublishing" edition="September-October 2008"> <article section="Features"> <title>Share 2.0</title> <author>Alan Joch</author> </article></journal><journal title="Oracle Magazine" publisher="OraclePublishing" edition="March-April 2008"> <article section="Oracle Developer"> <title>Declarative Data Filtering</title> <author>Steve Muench</author> </article></journal></catalog> Specify the XML schema location in the XML document using the following attribute declaration: xsi:noNamespaceSchemaLocation="catalog.xsd" The XML schema may be in any directory. The example XML document does not include any namespace elements. Therefore, the schema is specified with the xsi:noNamespaceSchemaLocation attribute in the root element catalog. The XML schema may be specified with a relative URL, or a file, or an HTTP URL. The xsi:noNamespaceSchemaLocation attribute we added specifies the relative path to the XML schema document catalog.xsd. To validate the XML document with the XML schema, right-click on the XML document and select Validate XML . The XML document gets validated with the XML schema and the output indicates that the XML document does not have any validation errors. To demonstrate validation errors, add a non-valid element to the XML document. As an example, add the following element to the catalog element after the first journal element: <article></article> To validate the modified XML document, right-click on the XML document and select Validate XML. The output indicates validation errors. All the elements after the non-valid element become non-valid. For example, the journal element is valid as a subelement of the catalog element, but because the second journal element is after the non-valid article element, the journal element also becomes non-valid as indicated in the validation output. XDK 11g also provides a schema validation-specific API known as XSDValidator to validate an XML document with an XML schema. The choice of validation method depends on the additional functionality required in the validation application. XSDValidator is suitable for validation if all that is required is schema validation. Setting the environment Create an application (SchemaValidation, for example) and a project (SchemaValidation, for example) in JDeveloper. To create an application and a project select File | New. In the New Gallery window, select Categories | General and Items | Generic Application. Click on OK. In the Create Generic Application window, specify an Application Name and click on Next. In the Name your Generic project window, specify a Project Name and click on Finish. An application and a project get created. Next, add some XDK 11g JAR files to the project classpath. Select the project node in Application Navigator, and select Tools | Project Properties. In the Project Properties window, select Libraries and Classpath. Click on the Add Library button to add a library. In the Add Library window, select the Oracle XML Parser v2 library and click on the OK button. The Oracle XML Parser v2 library gets added to the project Libraries. Select the Add JAR/Directory button to add JAR file xml.jar from the C:OracleMiddlewarejdevelopermodulesoracle.xdk_11.1.1 directory. First, create an XML document and an XML schema in JDeveloper. To create an XML document, select File | New. In the New Gallery window select Categories | General | XML. In the Items listed select XML Document, and click on the OK button. In the Create XML File wizard, specify the XML file name, catalog.xml, and click on the OK button. An XML document gets added to the SchemaValidation project in Application Navigator. To add an XML schema, select File | New, and General | XML in the New Gallery window. Select XML schema in the Items listed. Click on the OK button. An XML schema document gets added to SchemaValidation project. The example XML document, catalog.xml, consists of a journal catalog. Copy the XML document to the catalog.xml file in the JDeveloper project. The example XML document does not specify the location of the XML schema document to which the XML document must conform to, because we will be setting the XML schema document in the schema validation application. If the XML schema document is specified in the XML document and the schema validation application, the schema document set in the schema validation application is used. Next, copy the example XML schema document, catalog.xsd to catalog.xsd in the JDeveloper project Schema Validation. Each XML schema is required to be in the XML schema namespace http://www.w3.org/2001/XMLSchema. The XML schema namespace is specified with a namespace declaration in the root element, schema, of the XML schema. A namespace declaration is of the format > Next, we will create Java classes for schema validation. Select File | New and subsequently Categories | General and Items | Java Class in the New Gallery window to create a Java class for schema validation. Click on the OK button. In the Create Java Class window specify a Class Name, XMLSchemaValidator, and a package name, schemavalidation, and click on the OK button. A Java class gets added to the SchemaValidation project. Similarly, add Java classes, DOMValidator and SAXValidator. The schema validation applications are shown in the Application Navigator.

  (For more resources on IBM, see here.) Dumping namespaces To diagnose a problem, you might need to collect WAS JNDI information. WebSphere Application Server provides a utility that dumps the JNDI namespace. The dumpNamespace.sh script dumps information about the WAS namespace and is very useful when debugging applications when JNDI errors are seen in WAS logs. You can use this utility to dump the namespace to see the JNDI tree that the WAS name server (WAS JNDI lookup service provider) is providing for applications. This tool is very useful in JNDI problem determination, for example, when debugging incorrect JNDI resource mappings in the case where an application resource is not mapped correctly to a WAS-configured resource or the application is using direct JNDI lookups when really it should be using indirect lookups. For this tool to work, WAS must be running when this utility is run. To run the utility, use the following syntax: ./dumpNameSpace.sh -<command_option> There are many options for this tool and the following table lists the command-line options available by typing the command <was_root>/dumpsnameSpace.sh -help: Command option Description -host <host> Bootstrap host, that is, the WebSphere host whose namespace you want to dump. Defaults to localhost. -port <port> Bootstrap port. Defaults to 2809. -user <name> Username for authentication if security is enabled on the server. Acts the same way as the -username keyword. -username <name> Username for authentication if security is enabled on the server. Acts the same way as the -user keyword. -password <password> Password for authentication, if security is enabled in the server. -factory <factory> The initial context factory to be used to get the JNDI initial context. Defaults to com.ibm.websphere.naming. WsnInitialContextFactory and normally does not need to be changed. -root [ cell | server | node | host | legacy | tree | default ] Scope of the namespace to dump. For WS 5.0 or later: cell: DumpNameSpace default. Dump the tree starting at the cell root context. server: Dump the tree starting at the server root context. node: Dump the tree starting at the node root context. (Synonymous with host) For WS 4.0 or later: legacy: DumpNameSpace default. Dump the tree starting at the legacy root context. host: Dump the tree starting at the bootstrap host root context. (Synonymous with node) tree: Dump the tree starting at the tree root context. For all WebSphere and other name servers: default: Dump the tree starting at the initial context, which JNDI returns by default for that server type. This is the only -root choice that is compatible with WebSphere servers prior to 4.0 and with non-WebSphere name servers. The WebSphere initial JNDI context factory (default) obtains the desired root by specifying a key specific to the server type when requesting an initial CosNaming NamingContext reference. The default roots and the corresponding keys used for the various server types are listed as follows: WebSphere 5.0: Server root context. This is the initial reference registered under the key of NameServiceServerRoot on the server. WebSphere 4.0: Legacy root context. This context is bound under the name domain/legacyRoot, in the initial context registered on the server, under the key NameService. WebSphere 3.5: Initial reference registered under the key of NameService, on the server. Non-WebSphere: Initial reference registered under the key of NameService, on the server. -url <url> The value for the java.naming.provider.url property used to get the initial JNDI context. This option can be used in place of the -host, -port, and -root options. If the -url option is specified, the -host,-port, and -root options are ignored. -startAt <some/subcontext/ in/the/tree> The path from the requested root context to the top-level context, where the dump should begin. Recursively dumps (displays a tree-like structure) the sub-contexts of each namespace context. Defaults to empty string, that is, root context requested with the -root option. -format [ jndi | ins ] jndi: Display name components as atomic strings. ins: Display name components parsed per INS rules (id.kind) The default format is jndi. -report [ short | long ] short: Dumps the binding name and bound object type, which is essentially what JNDI Context. list() provides. long: Dumps the binding name, bound object type, local object type, and string representation of the local object, that is, Interoperable Object References (IORs) string values, and so on, are printed). The default report option is short. -traceString <some.package. to.trace.*=all> Trace string of the same format used with servers, with output going to the DumpNameSpaceTrace.out file. Example name space dump To see the result of using the namespace tool, navigate to the <was_root>/bin directory on your Linux server and type the following command: For Linux: ./dumpNameSpace.sh -root cell -report short -username wasadmin -password wasadmin >> /tmp/jnditree.txt For Windows: ./dumpNameSpace.bat -root cell -report short -username wasadmin -password wasadmin > c:tempjnditree.txt The following screenshot shows a few segments of the contents of an example jnditree.txt file which would contain the output of the previous command. EAR expander Sometimes during application debugging or automated application deployment, you may need to enquire about the contents of an Enterprise Archive (EAR) file. An EAR file is made up of one or more WAR files (web applications), one or more Enterprise JavaBeans (EJBs), and there can be shared JAR files as well. Also, within each WAR file, there may be JAR files as well. The EARExpander.sh utility allows all artifacts to be fully decompressed much as expanding a TAR file. Usage syntax: EARExpander -ear (name of the input EAR file for the expand operation or name of the output EAR file for the collapse operation) -operationDir (directory to which the EAR file is expanded or directory from which the EAR file is collapsed) -operation (expand | collapse) [-expansionFlags (all | war)] [-verbose] To demonstrate the utility, we will expand the HRListerEAR.ear file. Ensure that you have uploaded the HRListerEAR.ear file to a new folder called /tmp/EARExpander on your Linux server or an appropriate alternative location and run the following command: For Linux: <was_root>/bin/EARExpander.sh -ear /tmp/HRListerEAR.ear -operationDir /tmp/expanded -operation expand -expansionFlags all -verbose For Windows: <was_root>binEARExpander.bat -ear c:tempHRListerEAR.ear -operationDir c:tempexpanded -operation expand -expansionFlags all -verbose The result will be an expanded on-disk structure of the contents of the entire EAR file, as shown in the following screenshot: An example of everyday use could be that EARExpander.sh is used as part of a deployment script where an EAR file is expanded and hardcoded properties files are searched and replaced. The EAR is then re-packaged using the EARExpander -operation collapse option to recreate the EAR file once the find-and-replace routine has completed. An example of how to collapse an expanded EAR file is as follows: For Linux: <was_root>/bin/EARExpander.sh -ear /tmp/collapsed/HRListerEAR.ear -operationDir /tmp/expanded -operation collapse -expansionFlags all -verbose For Windows: <was_root>binEARExpander.bat -ear c:tempcollapsedHRListerEAR. ear -operationDir c:tempexpanded -operation collapse -expansionFlags all -verbose In the previous command line examples, the folder called EARExpander contains an expanded HRListerEAR.ear file, which was created when we used the -expand command example previously. To collapse the files back into an EAR file, use the -collapse option, as shown previously in the command line example. Collapsing the EAR folders results in a file called HRListerEAR.ear, which is created by collapsing the expanded folder contents back into a single EAR file. IBM Support Assistant IBM Support Assistant can help you locate technical documents and fixes, and discover the latest and most useful support resources available. IBM Support Assistant can be customized for over 350 products and over 20 tools, not just WebSphere Application Server. The following is a list of the current features in IBM Support Assistant: Search Information Search and filter results from a number of different websites and IBM Information Centers with just one click. Product Information Provides you with a page full of related resources specific to the IBM software you are looking to support. It also lists the latest support news and information, such as the latest fixes, APARs, Technotes, and other support data for your IBM product. Find product education and training materials Using this feature, you can search for online educational materials on how to use your IBM product. Media Viewer The media viewer allows you search and find free education and training materials available on the IBM Education Assistant sites. You can also watch Flash-based videos, read documentation, view slide presentations, or download for offline access. Automate data collection and analysis Support Assistant can help you gather the relevant diagnostic information automatically so you do not have to manually locate the resources that can explain the cause of the issue. With its automated data collection capabilities, ISA allows you to specify the troublesome symptom and have the relevant information automatically gathered in an archive. You can then look through this data, analyze it with the IBM Support Assistant tool, and even forward data to IBM support. Generate IBM Support Assistant Lite packages for any product addon that has data collection scripts. You can then export a lightweight Java application that can easily be transferred to remote systems for remote data connection. Analysis and troubleshooting tools for IBM products ISA contains tools that enable you to troubleshoot system problems. These include: analyzing JVM core dumps and garbage collector data, analyzing system ports, and also getting remote assistance from IBM support. Guided Troubleshooter This feature provides a step-by-step troubleshooting wizard that can be used to help you look for logs, suggest tools, or recommend steps on fixing the problems you are experiencing. Remote Agent technology Remote agent capabilities through the feature pack provide the ability to perform data collection and file transfer through the workbench from remote systems. Note that the Remote agents must be installed and configured with appropriate 'root-level' access. ISA is a very detailed tool and we cannot cover every feature in this article. However, for a demonstration, we will install ISA and then update ISA with an add-on called the Log Analyzer. We will use the Log Analyzer to analyze a WAS SystemOut.log file. Downloading the ISA workbench To download ISA you will require your IBM user ID. The download can be found at the following URL: http://www-01.ibm.com/software/support/isa/download.html It is possible to download both Windows and Linux versions.

(For more resources on Axis2, see here.) Handler In any messaging system, the interceptor has its factual meaning in the context of messaging, where it intercepts the flow of messaging and does whatever task it is assigned to do. In fact, an interceptor is the smallest execution unit in a messaging system, and an Axis2 handler is also an interceptor. Handlers in Axis are stateless, that is, they do not keep their pass execution states in the memory. A handler can be considered as a logic invoker with the input for the logic evaluation taken from the MessageContext. A Handler has both read and write access permissions to MessageContext (MC) or to an incoming SOAP message. We can consider MessageContext as a property bag that keeps incoming or outgoing messages (maybe both) and other required parameters. It may also include properties to carry the message through the execution chain. On the other hand, we can access the whole system including the system runtime, global parameters, and property service operations via the MC. In most cases, a handler only touches the header block part of the SOAP message, which will either read a header (or headers), add a header(s), or remove a header(s). (This does not mean that the handler cannot touch the SOAP body, nor does it mean that it is not going to touch the SOAP body.) During reading, if a header is targeted to a handler and is not executing properly (the message might be faulty), then it should throw an exception, and the next driver in the chain (in Axis2, it is the Axis engine) would take the necessary action. A typical SOAP message with few headers is shown in the figure given below: Any handler in Axis2 has the capability to pause the message execution, which means that the handler can terminate the message flow if it cannot continue. Reliable messaging (RM) is a good example or use case for that scenario, when it needs to pause the flow depending on some of the preconditions and the postconditions as well and it works on a message sequence. If a service invocation consists of more than one message, and if the second message comes before the first one, then the RM handler will stop (or rather pause) the execution of the message invocation corresponding to the second message until it gets the first one. And when it gets, the first message is invoked, and thereafter it invokes or resumes the second message. Writing a Simple Handler Just learning the concepts will not help us in remembering what we have discussed. For that, we need to write a handler and see how it works. Writing a handler in Axis2 is very simple. If you want to write a handler, you either have to extend the AbstractHandler class or implement the Handler interface. A simple handler that extends the AbstractHandler class will appear as follows: public class SimpleHandler extends AbstractHandler{ public SimpleHandler() { }public InvocationResponse invoke(MessageContext msgContext) throws AxisFault { //Write the processing logic here // DO something return InvocationResponse.CONTINUE; }} Note the return value of the invoke method. We can have the following three values as the return value of the invoke method: Continue: The handler thinks that the message is ready to go forward. Suspend: The handler thinks that the message cannot be sent forward since some conditions are not satisfied; so the execution is suspended. Abort: The handler thinks that there is something wrong with the message, and cannot therefore allow the message to go forward. In most cases, handlers will return InvocationResponse.CONTINUE as the return value. When a message is received by the Axis engine, it calls the invoke method of each of the handlers by passing the argument to the corresponding MessageContext. As a result of this, we can implement all the processing logic inside that method. A handler author has full access to the SOAP message, and also has the required properties to process the message via the MessageContext. In addition, if the handler is not satisfied with the invocation of some precondition, the invocation can be paused as we have discussed earlier (Suspend). If some handler suspends the execution, then it is its responsibility to store the message context, and to forward the message when the conditions are satisfied. For example, the RM handler performs in a similar manner. Phase The concept of phase is introduced by Axis2, mainly to support the dynamic ordering of handlers. A phase can be defined in a number of ways: It can be considered a logical collection of handlers. It can be considered a specific time interval in the message execution. It can be considered a bucket into which to put a handler. One can consider a phase as a handler too. A flow or an execution chain can be considered as a collection of phases. Even though it was mentioned earlier that an Axis engine calls the invoke method of a handler, that is not totally correct. In fact, what the engine really does is call the invoke method of each phase in a given flow, and then the phase will sequentially invoke all the handlers in it (refer to the following figure). As we know, we can extend AbstractHandler and create a new handler; in the same way one can extend the Phase class and then create a new phase. But remember that we need not always extend the Phase class to create a new phase. We can do it by just adding an entry into axis2.xml (All the configuration that requires starting axis2 is obtained from axis2.xml). A phase has two important methods—precondition checking and postcondition checking. Therefore, if we are writing a custom phase, we need to consider the methods that have been mentioned. However, writing a phase is not a common case; you need to know how to write a handler.

In this article by Symeon Huang, author of the book Qt 5 Blueprints, explains typical and basic GUI components in Qt 5 (For more resources related to this topic, see here.) Design UI in Qt Creator Qt Creator is the official IDE for Qt application development and we're going to use it to design application's UI. At first, let's create a new project: Open Qt Creator. Navigate to File | New File or Project. Choose Qt Widgets Application. Enter the project's name and location. In this case, the project's name is layout_demo. You may wish to follow the wizard and keep the default values. After this creating process, Qt Creator will generate the skeleton of the project based on your choices. UI files are under Forms directory. And when you double-click on a UI file, Qt Creator will redirect you to integrated Designer, the mode selector should have Design highlighted and the main window should contains several sub-windows to let you design the user interface. Here we can design the UI by dragging and dropping. Qt Widgets Drag three push buttons from the widget box (widget palette) into the frame of MainWindow in the center. The default text displayed on these buttons is PushButtonbut you can change text if you want, by double-clicking on the button. In this case, I changed them to Hello, Hola, and Bonjouraccordingly. Note that this operation won't affect the objectName property and in order to keep it neat and easy-to-find, we need to change the objectName! The right-hand side of the UI contains two windows. The upper right section includes Object Inspector and the lower-right includes the Property Editor. Just select a push button, we can easily change objectName in the Property Editor. For the sake of convenience, I changed these buttons' objectName properties to helloButton, holaButton, and bonjourButton respectively. Save changes and click on Run on the left-hand side panel, it will build the project automatically then run it as shown in the following screenshot: In addition to the push button, Qt provides lots of commonly used widgets for us. Buttons such as tool button, radio button, and checkbox. Advanced views such as list, tree, and table. Of course there are input widgets, line edit, spin box, font combo box, date and time edit, and so on. Other useful widgets such as progress bar, scroll bar, and slider are also in the list. Besides, you can always subclass QWidget and write your own one. Layouts A quick way to delete a widget is to select it and press the Delete button. Meanwhile, some widgets, such as the menu bar, status bar, and toolbar can't be selected, so we have to right-click on them in Object Inspector and delete them. Since they are useless in this example, it's safe to remove them and we can do this for good. Okay, let's understand what needs to be done after the removal. You may want to keep all these push buttons on the same horizontal axis. To do this, perform the following steps: Select all the push buttons either by clicking on them one by one while keeping the Ctrl key pressed or just drawing an enclosing rectangle containing all the buttons. Right-click and select Layout | LayOut Horizontally. The keyboard shortcut for this is Ctrl + H. Resize the horizontal layout and adjust its layoutSpacing by selecting it and dragging any of the points around the selection box until it fits best. Hmm…! You may have noticed that the text of the Bonjour button is longer than the other two buttons, and it should be wider than the others. How do you do this? You can change the property of the horizontal layout object's layoutStretch property in Property Editor. This value indicates the stretch factors of the widgets inside the horizontal layout. They would be laid out in proportion. Change it to 3,3,4, and there you are. The stretched size definitely won't be smaller than the minimum size hint. This is how the zero factor works when there is a nonzero natural number, which means that you need to keep the minimum size instead of getting an error with a zero divisor. Now, drag Plain Text Edit just below, and not inside, the horizontal layout. Obviously, it would be neater if we could extend the plain text edit's width. However, we don't have to do this manually. In fact, we could change the layout of the parent, MainWindow. That's it! Right-click on MainWindow, and then navigate to Lay out | Lay Out Vertically. Wow! All the children widgets are automatically extended to the inner boundary of MainWindow; they are kept in a vertical order. You'll also find Layout settings in the centralWidget property, which is exactly the same thing as the previous horizontal layout. The last thing to make this application halfway decent is to change the title of the window. MainWindow is not the title you want, right? Click on MainWindow in the object tree. Then, scroll down its properties to find windowTitle. Name it whatever you want. In this example, I changed it to Greeting. Now, run the application again and you will see it looks like what is shown in the following screenshot: Qt Quick Components Since Qt 5, Qt Quick has evolved to version 2.0 which delivers a dynamic and rich experience. The language it used is so-called QML, which is basically an extended version of JavaScript using a JSON-like format. To create a simple Qt Quick application based on Qt Quick Controls 1.2, please follow following procedures: Create a new project named HelloQML. Select Qt Quick Application instead of Qt Widgets Application that we chose previously. Select Qt Quick Controls 1.2 when the wizard navigates you to Select Qt Quick Components Set. Edit the file main.qml under the root of Resources file, qml.qrc, that Qt Creator has generated for our new Qt Quick project. Let's see how the code should be. import QtQuick 2.3 import QtQuick.Controls 1.2   ApplicationWindow {    visible: true    width: 640    height: 480    title: qsTr("Hello QML")      menuBar: MenuBar {        Menu {            title: qsTr("File")            MenuItem {                text: qsTr("Exit")                shortcut: "Ctrl+Q"                onTriggered: Qt.quit()            }        }    }      Text {        id: hw        text: qsTr("Hello World")        font.capitalization: Font.AllUppercase        anchors.centerIn: parent    }      Label {        anchors { bottom: hw.top; bottomMargin: 5; horizontalCenter: hw.horizontalCenter }        text: qsTr("Hello Qt Quick")    } } If you ever touched Java or Python, then the first two lines won't be too unfamiliar for you. It simply imports the Qt Quick and Qt Quick Controls. And the number behind is the version of the library. The body of this QML source file is really in JSON style, which enables you understand the hierarchy of the user interface through the code. Here, the root item is ApplicationWindow, which is basically the same thing as QMainWindow in Qt/C++. When you run this application in Windows, you can barely find the difference between the Text item and Label item. But on some platforms, or when you change system font and/or its colour, you'll find that Label follows the font and colour scheme of the system while Text doesn't. Run this application, you'll see there is a menu bar, a text, and a label in the application window. Exactly what we wrote in the QML file: You may miss the Design mode for traditional Qt/C++ development. Well, you can still design Qt Quick application in Design mode! Click on Design in mode selector when you edit main.qml file. Qt Creator will redirect you into Design mode where you can use mouse drag-and-drop UI components: Almost all widgets you use in Qt Widget application can be found here in a Qt Quick application. Moreover, you can use other modern widgets such as busy indicator in Qt Quick while there's no counterpart in Qt Widget application. However, QML is a declarative language whose performance is obviously poor than C++. Therefore, more and more developers choose to write UI with Qt Quick in order to deliver a better visual style, while keep core functions in Qt/C++. Summary In this article, we had a brief contact with various GUI components of Qt 5 and focus on the Design mode in Qt Creator. Two small examples used as a Qt-like "Hello World" demonstrations. Resources for Article: Further resources on this subject: Code interlude – signals and slots [article] Program structure, execution flow, and runtime objects [article] Configuring Your Operating System [article]

Activating the administration interface The administration interface comes as a Django application. To activate it, we will follow a simple procedure that is similar to enabling the user authentication system. The administration application is located in the django.contrib.admin package. So the first step is adding the path of this package to the INSTALLED_APPS variable. Open the settings.py file, locate INSTALLED_APPS, and edit it as follows: INSTALLED_APPS = ( 'django.contrib.auth', 'django.contrib.admin', 'django.contrib.contenttypes', 'django.contrib.sessions', 'django.contrib.sites', 'django.contrib.comments', 'django_bookmarks.bookmarks',) Next, run the following command to create the necessary tables for the administration application: $ python manage.py syncdb Now we need to make the administration interface accessible from within our site by adding URL entries for it. The admin application defines many views (as we will see later), so manually adding a separate entry for each view can become a tedious task. Therefore, the admin interface provides a shortcut for this. There is a single object that encapsulates all the admin views. To use it, open the urls.py file and edit it as follows: from django.contrib import adminadmin.autodiscover()urlpatterns = ('', [...] # Admin interface (r'^admin/(.*)', admin.site.root),) Here, we are importing the admin module, calling a method in it, and mapping all the URLs under the path ^admin/ to a view called admin.site.root. This will make the views of the administration interface accessible from within our project. One last thing remains before we see the administration page in action. We need to tell Django what models can be managed in the administration interface. This is done by creating a new file called the admin.py file in the bookmarks directory. Create the bookmarks/admin.py file and add the following code to it: from django.contrib import adminfrom bookmarks.models import *class LinkAdmin(admin.ModelAdmin): passadmin.site.register(Link, LinkAdmin) We created a class derived from the admin.ModelAdmin class and mapped it to the Link model using the admin.site.register method. This effectively tells Django to enable the Link model in the administration interface. The keyword pass means that the class is empty. Later, we will use this class to customize the administration page; so it won't remain empty. Do the same to the Bookmark, Tag, and SharedBookmark models and add it to the bookmarks/admin.py file. Now, create an empty admin class for each of them and register it. The User model is provided by Django and, therefore, we don't have control over it. But fortunately, it already has an Admin class so it's available in the administration interface by default. Next, launch the development server and direct your browser to http://127.0.0.1:8000/admin/. You will be greeted by a login page. The superuser account after writing the database model is the account that you have to use in order to log in: Next, you will see a list of the models that are available to the administration interface. As discussed earlier, only models that have admin classes in the bookmarks/admin.py file will appear on this page. If you click on a model name, you will get a list of the objects that are stored in the database under this model. You can use this page to view or edit a particular object, or to add a new one. The following figure shows the listing page for the Link model: The edit form is generated according to the fields that exist in the model. The Link form, for example, contains a single text field called Url. You can use this form to view and change the URL of a Link object. In addition, the form performs proper validation of fields before saving the object. So if you try to save a Link object with an invalid URL, you will receive an error message asking you to correct the field. The following figure shows a validation error when trying to save an invalid link: Fields are mapped to form widgets according to their type. For example, date fields are edited using a calendar widget, whereas foreign key fields are edited using a list widget, and so on. The following figure shows a calendar widget from the user edit page. Django uses it for date and time fields. As you may have noticed, the administration interface represents models by using the string returned by the __unicode__ method. It was indeed a good idea to replace the generic strings returned by the default __unicode__ method with more helpful ones. This greatly helps when working with the administration page, as well as with debugging. Experiment with the administration pages. Try to create, edit, and delete objects. Notice how changes made in the administration interface are immediately reflected on the live site. Also, the administration interface keeps a track of the actions that you make and lets you review the history of changes for each object. This section has covered most of what you need to know in order to use the administration interface provided by Django. This feature is actually one of the main advantages of using Django. You get a fully featured administration interface from writing only a few lines of code! Next, we will see how to tweak and customize the administration pages. As a bonus, we will learn more about the permissions system offered by Django.

(For more resources on Plone, see here.) Introduction CMS Plone provides: A Means of adding, editing, and managing content A database to store content A mechanism to serve content in HTML or other formats Fortunately, it also supplies the tools to do all these things in an incredibly easy and powerful way. For example, content producers can create a new article without worrying how it will look or what other information will be surrounding the main information. To do this Plone must compose a single HTML output file (if we are talking from a web browser viewpoint) by joining and rendering several sources of data according to the place, importance, and target they are meant for. As it is built upon the Zope application server, all these jobs are easy for Plone. However, they have a tremendous impact as far as work and performance goes. If enough care is not taken, then a whole website could be stuck due to a couple of user requests. In this article, we'll look at the various performance improvements and how to measure these enhancements. We are not going to make a comprehensive review of all the options to tweak or set up a Zope-based web application, like configuring a like configuring a proxy cache or a load balancer. There are lots of places, maybe too many, where you can find information about these topics. We invite you to read these articles and tutorials and subscribe or visit Zope and Plone mailing lists: http://projects.zestsoftware.nl/guidelines/guidelines/caching/ caching1_background.html http://plone.org/documentation/tutorial/buildout/a-deployment-configuration/ http://plone.org/documentation/tutorial/optimizing-plone Installing CacheFu with a policy product When a user requests HTML pages from a website, many things can be expressed about the downloading files by setting special headers in the HTTP response. If managed cautiously, the server can save lots of time and, consequently, work by telling the browser how to store and reuse many of the resources it has got. CacheFu is the Plone add-on product that streamlines HTTP header handling in order to obtain the required performance. We could add a couple of lines to the buildout.cfg file to download and install CacheFu. Then we could add some code in our end user content type products (pox.video and Products.poxContentTypes) to configure CacheFu properly to deliver them in an efficient way. However, if we do so, we would be forcing these products to automatically install CacheFu, even if we were testing them in a development environment. To prevent this, we are going to create a policy product and add some code to install and configure CacheFu. A policy product is a regular package that will take care of general customizations to meet customer requirements. For information on how to create a policy product see Creating a policy product. Getting ready To achieve this we'll use pox.policy, the policy product created in Creating a policy product. How to do it... Automatically fetch dependencies of the policy product: Open setup.py in the root pox.policy folder and modify the install_requires variable of the setup call: setup(name='pox.policy', ... install_requires=['setuptools', # -*- Extra requirements: -*- 'Products.CacheSetup', ], Install dependencies during policy product installation. In the profiles/default folder, modify the metadata.xml file: <?xml version="1.0"?><metadata> <version>1</version> <dependencies> <dependency>profile-Products.CacheSetup:default</dependency> </dependencies></metadata You could also add here all the other products you plan to install as dependencies, instead of adding them individually in the buildout.cfg file. Configure products during the policy product installation. Our policy product already has a <genericsetup:importStep /> directive in its main component configuration file (configure.zcml). This import step tells GenericSetup to process a method in the setuphandlers module (we could have several steps, each of them with a matching method). Then modify the setupVarious method to do what we want, that is, to apply some settings to CacheFu. from zope.app.component.hooks import getSitefrom Products.CMFCore.utils import getToolByNamefrom config import * def setupVarious(context): if context.readDataFile('pox.policy_various.txt') is None: return portal = getSite() # perform custom operations # Get portal_cache_settings (from CacheFu) and # update plone-content-types rule pcs = getToolByName(portal, 'portal_cache_settings') rules = pcs.getRules() rule = getattr(rules, 'plone-content-types') rule.setContentTypes(list(rule.getContentTypes()) + CACHED_CONTENT) The above code has been shortened for clarity's sake. Check the accompanying code bundle for the full version. Add or update a config.py file in your package with all configuration options: # Content types that should be cached in plone-content-types# rule of CacheFuCACHED_CONTENT = ['XNewsItem', 'Video',] Build your instance up again and launch it: ./bin/buildout./bin/instance fg After installing the pox.policy product (it's automatically installed during buildout as explained in Creating a policy product) we should see our content types—Video and XNewsItem—listed within the cached content types. The next screenshot corresponds to the following URL: http://localhost:8080/plone/portal_cache_settings/with-caching-proxy/rules/plone-content-types. The with-caching-proxy part of the URL matches the Cache Policy field; and the plone-content-types part matches the Short Name field. As we added Python code, we must test it. Create this doctest in the README.txt file in the pox.policy package folder: Check that our content types are properly configured >>> pcs = getToolByName(self.portal, 'portal_cache_settings')>>> rules = pcs.getRules()>>> rule = getattr(rules, 'plone-content-types')>>> 'Video' in rule.getContentTypes()True>>> 'XNewsItem' in rule.getContentTypes()True Modify the tests module by replacing the ptc.setupPloneSite() line with these ones: # We first tell Zope there's a CacheSetup product availableztc.installProduct('CacheSetup') # And then we install pox.policy product in Plone.# This should take care of installing CacheSetup in Plone alsoptc.setupPloneSite(products=['pox.policy']) And then uncomment the ZopeDocFileSuite: # Integration tests that use PloneTestCaseztc.ZopeDocFileSuite( 'README.txt', package='pox.policy', test_class=TestCase), Run this test suite with the following command: ./bin/instance test -s pox.policy How it works... In the preceding steps, we have created a specific procedure to install and configure other products (CacheFu in our case). This will help us in the final production environment startup as well as on installation of other development environments we could need (when a new member joins the development team, for instance). In Step 1 of the How to do it... section, we modified setup.py to download and install a dependency package during the installation process, which is done on instance buildout. Getting dependencies in this way is possible when products are delivered in egg format thanks to Python eggs repositories and distribution services. If you need to get an old-style product, you'll have to add it to the [productdistros] part in buildout.cfg. Products.CacheSetup is the package name for CacheFu and contains these dependencies: CMFSquidTool, PageCacheManager, and PolicyHTTPCacheManager. There's more... For more information about CacheFu visit the project home page at http://plone.org/products/cachefu. You can also check for its latest version and release notes at Python Package Index (PyPI, a.k.a. The Cheese Shop): http://pypi.python.org/pypi/Products.CacheSetup. The first link that we recommended in the Introduction is a great help in understanding how CacheFu works: http://projects.zestsoftware.nl/guidelines/guidelines/caching/caching1_background.html. See also Creating a policy product Installing and configuring an egg repository

[box type="note" align="" class="" width=""]The following two-part tutorial is an excerpt from the book Mastering Machine Learning with Spark 2.x by Alex Tellez, Max Pumperla and Michal Malohlava. [/box] When collecting real-world data between individual measures or events, there are usually very intricate and highly complex relationships to observe. The guiding example for this tutorial is the observation of click events that users generate on a website and its subdomains. Such data is both interesting and challenging to investigate. It is interesting, as there are usually many patterns that groups of users show in their browsing behavior and certain rules they might follow. Gaining insights about user groups, in general, is of interest, at least for the company running the website and might be the focus of their data science team. Methodology aside, putting a production system in place that can detect patterns in real time, for instance, to find malicious behavior, can be very challenging technically. It is immensely valuable to be able to understand and implement both the algorithmic and technical sides. In this tutorial, we will look into doing pattern mining in Spark. The tutorial is split up into two main sections. In the first, we will first introduce the three available pattern mining algorithms that Spark currently comes with and then apply them to an interesting dataset. In particular, you will learn the following from this two-part tutorial: The basic principles of frequent pattern mining. Useful and relevant data formats for applications. Understanding and comparing three pattern mining algorithms available in Spark, namely FP-growth, association rules, and prefix span. Frequent pattern mining When presented with a new data set, a natural sequence of questions is: What kind of data do we look at; that is, what structure does it have? Which observations in the data can be found frequently; that is, which patterns or rules can we identify within the data? How do we assess what is frequent; that is, what are the good measures of relevance and how do we test for it? On a very high level, frequent pattern mining addresses precisely these questions. While it's very easy to dive head first into more advanced machine learning techniques, these pattern mining algorithms can be quite informative and help build an intuition about the data. To introduce some of the key notions of frequent pattern mining, let's first consider a somewhat prototypical example for such cases, namely shopping carts. The study of customers being interested in and buying certain products has been of prime interest to marketers around the globe for a very long time. While online shops certainly do help in further analyzing customer behavior, for instance, by tracking the browsing data within a shopping session, the question of what items have been bought and what patterns in buying behavior can be found applies to purely offline scenarios as well. We will see a more involved example of clickstream data accumulated on a website soon; for now, we will work under the assumption that only the events we can track are the actual payment transactions of an item. Just this given data, for instance, for groceries shopping carts in supermarkets or online, leads to quite a few interesting questions, and we will focus mainly on the following three: Which items are frequently bought together? For instance, there is anecdotal evidence suggesting that beer and diapers are often brought together in one shopping session. Finding patterns of products that often go together may, for instance, allow a shop to physically place these products closer to each other for an increased shopping experience or promotional value even if they don't belong together at first sight. In the case of an online shop, this sort of analysis might be the base for a simple recommender system. Based on the previous question, are there any interesting implications or rules to observe in shopping behavior?, continuing with the shopping cart example, can we establish associations such as if bread and butter have been bought, we also often find cheese in the shopping cart? Finding such association rules can be of great interest, but also need more clarification of what we consider to be often, that is, what does frequent mean. Note that, so far, our shopping carts were simply considered a bag of items without additional structure. At least in the online shopping scenario, we can endow data with more information. One aspect we will focus on is that of the sequentiality of items; that is, we will take note of the order in which the products have been placed into the cart. With this in mind, similar to the first question, one might ask, which sequence of items can often be found in our transaction data? For instance, larger electronic devices bought might be followed up by additional utility items. The reason we focus on these three questions, in particular, is that Spark MLlib comes with precisely three pattern mining algorithms that roughly correspond to the aforementioned questions by their ability to answer them. Specifically, we will carefully introduce FP- growth, association rules, and prefix span, in that order, to address these problems and show how to solve them using Spark. Before doing so, let's take a step back and formally introduce the concepts we have been motivated for so far, alongside a running example. We will refer to the preceding three questions throughout the following subsection. Pattern mining terminology We will start with a set of items I = {a1, ..., an}, which serves as the base for all the following concepts. A transaction T is just a set of items in I, and we say that T is a transaction of length l if it contains l item. A transaction database D is a database of transaction IDs and their corresponding transactions. To give a concrete example of this, consider the following situation. Assume that the full item set to shop from is given by I = {bread, cheese, ananas, eggs, donuts, fish, pork, milk, garlic, ice cream, lemon, oil, honey, jam, kale, salt}. Since we will look at a lot of item subsets, to make things more readable later on, we will simply abbreviate these items by their first letter, that is, we'll write I = {b, c, a, e, d, f, p, m, g, i, l, o, h, j, k, s}. Given these items, a small transaction database D could look as follows:   Transaction ID Transaction 1 a, c, d, f, g, i, m, p 2 a, b, c, f, l, m, o 3 b, f, h, j, o 4 b, c, k, s, p 5 a, c, e, f, l, m, n, p Table 1: A small shopping cart database with five transactions Frequent pattern mining problem Given the definition of a transaction database, a pattern P is a transaction contained in the transactions in D and the support, supp(P), of the pattern is the number of transactions for which this is true, divided or normalized by the number of transactions in D: supp(s) = suppD(s) = |{ s' ∈ S | s < s'}| / |D| We use the < symbol to denote s as a subpattern of s' or, conversely, call s' a superpattern of s. Note that in the literature, you will sometimes also find a slightly different version of support that does not normalize the value. For example, the pattern {a, c, f} can be found in transactions 1, 2, and 5. This means that {a, c, f} is a pattern of support 0.6 in our database D of five items. Support is an important notion, as it gives us a first example of measuring the frequency of a pattern, which, in the end, is what we are after. In this context, for a given minimum support threshold t, we say P is a frequent pattern if and only if supp(P) is at least t. In our running example, the frequent patterns of length 1 and minimum support 0.6 are {a}, {b}, {c}, {p}, and {m} with support 0.6 and {f} with support 0.8. In what follows, we will often drop the brackets for items or patterns and write f instead of {f}, for instance. Given a minimum support threshold, the problem of finding all the frequent patterns is called the frequent pattern mining problem and it is, in fact, the formalized version of the aforementioned first question. Continuing with our example, we have found all frequent patterns of length 1 for t = 0.6 already. How do we find longer patterns? On a theoretical level, given unlimited resources, this is not much of a problem, since all we need to do is count the occurrences of items. On a practical level, however, we need to be smart about how we do so to keep the computation efficient. Especially for databases large enough for Spark to come in handy, it can be very computationally intense to address the frequent pattern mining problem. One intuitive way to go about this is as follows: Find all the frequent patterns of length 1, which requires one full database scan. This is how we started with in our preceding example. For patterns of length 2, generate all the combinations of frequent 1-patterns, the so-called candidates, and test if they exceed the minimum support by doing another scan of D. Importantly, we do not have to consider the combinations of infrequent patterns, since patterns containing infrequent patterns can not become frequent. This rationale is called the apriori principle. For longer patterns, continue this procedure iteratively until there are no more patterns left to combine. This algorithm, using a generate-and-test approach to pattern mining and utilizing the apriori principle to bound combinations, is called the apriori algorithm. There are many variations of this baseline algorithm, all of which share similar drawbacks in terms of scalability. For instance, multiple full database scans are necessary to carry out the iterations, which might already be prohibitively expensive for huge datasets. On top of that, generating candidates themselves is already expensive, but computing their combinations might simply be infeasible. In the next section, we will see how a parallel version of an algorithm called FP-growth, available in Spark, can overcome most of the problems just discussed. The association rule mining problem To advance our general introduction of concepts, let's next turn to association rules, as first introduced in Mining Association Rules between Sets of Items in Large Databases, available at http:/ /arbor. ee. ntu. edu. tw/~chyun/ dmpaper/agrama93. pdf. In contrast to solely counting the occurrences of items in our database, we now want to understand the rules or implications of patterns. What I mean is, given a pattern P1 and another pattern P2, we want to know whether P2 is frequently present whenever P1 can be found in D, and we denote this by writing P1 ⇒ P2. To make this more precise, we need a concept for rule frequency similar to that of support for patterns, namely confidence. For a rule P1 ⇒ P2, confidence is defined as follows: conf(P1 ⇒ P2) = supp(P1 ∪ P2) / supp(P1) This can be interpreted as the conditional support of P2 given to P1; that is, if it were to restrict D to all the transactions supporting P1, the support of P2 in this restricted database would be equal to conf(P1 ⇒ P2). We call P1 ⇒ P2 a rule in D if it exceeds a minimum confidence threshold t, just as in the case of frequent patterns. Finding all the rules for a confidence threshold represents the formal answer to the second question, association rule mining. Moreover, in this situation, we call P1 the antecedent and P2 the consequent of the rule. In general, there is no restriction imposed on the structure of either the antecedent or the consequent. However, in what follows, we will assume that the consequent's length is 1, for simplicity. In our running example, the pattern {f, m} occurs three times, while {f, m, p} is just present in two cases, which means that the rule {f, m} ⇒ {p} has confidence 2/3. If we set the minimum confidence threshold to t = 0.6, we can easily check that the following association rules with an antecedent and consequent of length 1 are valid for our case: {a} ⇒ {c}, {a} ⇒ {f}, {a} ⇒ {m}, {a} ⇒ {p} {c} ⇒ {a}, {c} ⇒ {f}, {c} ⇒ {m}, {c} ⇒ {p} {f} ⇒ {a}, {f} ⇒ {c}, {f} ⇒ {m} {m} ⇒ {a}, {m} ⇒ {c}, {m} ⇒ {f}, {m} ⇒ {p} {p} ⇒ {a}, {p} ⇒ {c}, {p} ⇒ {f}, {p} ⇒ {m} From the preceding definition of confidence, it should now be clear that it is relatively straightforward to compute the association rules once we have the support value of all the frequent patterns. In fact, as we will soon see, Spark's implementation of association rules is based on calculating frequent patterns upfront. [box type="info" align="" class="" width=""]At this point, it should be noted that while we will restrict ourselves to the measures of support and confidence, there are many other interesting criteria available that we can't discuss in this book; for instance, the concepts of conviction, leverage, or lift. For an in-depth comparison of the other measures, refer to http:/ / www. cse. msu. edu/ ~ptan/ papers/ IS. pdf.[/box] The sequential pattern mining problem Let's move on to formalizing, the third and last pattern matching question we tackle in this chapter. Let's look at sequences in more detail. A sequence is different from the transactions we looked at before in that the order now matters. For a given item set I, a sequence S in I of length l is defined as follows: s = <s1, s2, ..., sl> Here, each individual si is a concatenation of items, that is, si = (ai1 ... aim), where aij is an item in I. Note that we do care about the order of sequence items si but not about the internal ordering of the individual aij in si. A sequence database S consists of pairs of sequence IDs and sequences, analogous to what we had before. An example of such a database can be found in the following table, in which the letters represent the same items as in our previous shopping cart example:   Sequence ID Sequence 1 <a(abc)(ac)d(cf)> 2 <(ad)c(bc)(ae)> 3 <(ef)(ab)(df)cb> 4 <eg(af)cbc> Table 2: A small sequence database with four short sequences. In the example sequences, note the round brackets to group individual items into a sequence item. Also note that we drop these redundant braces if the sequence item consists of a single item. Importantly, the notion of a subsequence requires a little more carefulness than for unordered structures. We call u = (u1, ..., un) a subsequence of s = (s1, ..., sl) and write u < s if there are indices 1 ≤ i1 < i2 < ... < in ≤ m so that we have the following: u1 < si1, ..., un < sin Here, the < signs in the last line mean that uj is a subpattern of sij. Roughly speaking, u is a subsequence of s if all the elements of u are subpatterns of s in their given order. Equivalently, we call s a supersequence of u. In the preceding example, we see that <a(ab)ac> and a(cb)(ac)dc> are examples of subsequences of <a(abc)(ac)d(cf)> and that <(fa)c> is an example of a subsequence of <eg(af)cbc>. With the help of the notion of supersequences, we can now define the support of a sequence s in a given sequence database S as follows: suppS(s) = supp(s) = |{ s' ∈ S | s < s'}| / |S| Note that, structurally, this is the same definition as for plain unordered patterns, but the < symbol means something else, that is, a subsequence. As before, we drop the database subscript in the notation of support if the information is clear from the context. Equipped with a notion of support, the definition of sequential patterns follows the previous definition completely analogously. Given a minimum support threshold t, a sequence s in S is said to be a sequential pattern if supp(s) is greater than or equal to t. The formalization of the third question is called the sequential pattern mining problem, that is, find the full set of sequences that are sequential patterns in S for a given threshold t. Even in our little example with just four sequences, it can already be challenging to manually inspect all the sequential patterns. To give just one example of a sequential pattern of support 1.0, a subsequence of length 2 of all the four sequences is <ac>. Finding all the sequential patterns is an interesting problem, and we will learn about the so-called prefix span algorithm that Spark employs to address the problem in the following section. Next time, in part 2 of the tutorial, we will see how to use Spark to solve the above three pattern mining problems using the algorithms introduced. If you enjoyed this tutorial, an excerpt from the book Mastering Machine Learning with Spark 2.x by Alex Tellez, Max Pumperla and Michal Malohlava, check out the book for more.

Background Performance Tuning is one of the most critical area of Oracle databases and having a good knowledge on SQL tuning helps DBAs in tuning production databases on a daily basis. Over the years Oracle optimizer has gone through several enhancements and each release presents a best among all optimizer versions. Oracle 12c is no different. Oracle has improved the optimizer and added new features in this release to make it better than previous release. In this article we are going to see some of the explicit new features of Oracle optimizer which helps us in tuning our queries. Objective In this article, Advait Deo and Indira Karnati, authors of the book OCP Upgrade 1Z0-060 Exam guide discusses new features of Oracle 12c optimizer and how it helps in improving the SQL plan. It also discusses some of the limitations of optimizer in previous release and how Oracle has overcome those limitations in this release. Specifically, we are going to discuss about dynamic plan and how it works (For more resources related to this topic, see here.) SQL Tuning Before we go into the details of each of these new features, let us rewind and check what we used to have in Oracle 11g. Behavior in Oracle 11g R1 Whenever an SQL is executed for the first time, an optimizer will generate an execution plan for the SQL based on the statistics available for the different objects used in the plan. If statistics are not available, or if the optimizer thinks that the existing statistics are of low quality, or if we have complex predicates used in the SQL for which the optimizer cannot estimate the cardinality, the optimizer may choose to use dynamic sampling for those tables. So, based on the statistics values, the optimizer generates the plan and executes the SQL. But, there are two problems with this approach: Statistics generated by dynamic sampling may not be of good quality as they are generated in limited time and are based on a limited sample size. But a trade-off is made to minimize the impact and try to approach a higher level of accuracy. The plan generated using this approach may not be accurate, as the estimated cardinality may differ a lot from the actual cardinality. The next time the query executes, it goes for soft parsing and picks the same plan. Behavior in Oracle 11g R2 To overcome these drawbacks, Oracle enhanced the dynamic sampling feature further in Oracle11g Release 2. In the 11.2 release, Oracle will automatically enable dynamic sample when the query is run if statistics are missing, or if the optimizer thinks that current statistics are not up to the mark. The optimizer also decides the level of the dynamic sample, provided the user does not set the non-default value of the OPTIMIZER_DYNAMIC_SAMPLING parameter (default value is 2). So, if this parameter has a default value in Oracle11g R2, the optimizer will decide when to spawn dynamic sampling in a query and at what level to spawn the dynamic sample. Oracle also introduced a new feature in Oracle11g R2 called cardinality feedback. This was in order to further improve the performance of SQLs, which are executed repeatedly and for which the optimizer does not have the correct cardinality, perhaps because of missing statistics, or complex predicate conditions, or because of some other reason. In such cases, cardinality feedback was very useful. The way cardinality feedback works is, during the first execution, the plan for the SQL is generated using the traditional method without using cardinality feedback. However, during the optimization stage of the first execution, the optimizer notes down all the estimates that are of low quality (due to missing statistics, complex predicates, or some other reason) and monitoring is enabled for the cursor that is created. If this monitoring is enabled during the optimization stage, then, at the end of the first execution, some cardinality estimates in the plan are compared with the actual estimates to understand how significant the variation is. If the estimates vary significantly, then the actual estimates for such predicates are stored along with the cursor, and these estimates are used directly for the next execution instead of being discarded and calculated again. So when the query executes the next time, it will be optimized again (hard parse will happen), but this time it will use the actual statistics or predicates that were saved in the first execution, and the optimizer will come up with better plan. But even with these improvements, there are drawbacks: With cardinality feedback, any missing cardinality or correct estimates are available for the next execution only and not for the first execution. So the first execution always go for regression. The dynamic sample improvements (that is, the optimizer deciding whether dynamic sampling should be used and the level of the dynamic sampling) are only applicable to parallel queries. It is not applicable to queries that aren't running in parallel. Dynamic sampling does not include joins and groups by columns. Oracle 12c has provided new improvements, which eliminates the drawbacks of Oracle11g R2. Adaptive execution plans – dynamic plans The Oracle optimizer chooses the best execution plan for a query based on all the information available to it. Sometimes, the optimizer may not have sufficient statistics or good quality statistics available to it, making it difficult to generate optimal plans. In Oracle 12c, the optimizer has been enhanced to adapt a poorly performing execution plan at run time and prevent a poor plan from being chosen on subsequent executions. An adaptive plan can change the execution plan in the current run when the optimizer estimates prove to be wrong. This is made possible by collecting the statistics at critical places in a plan when the query starts executing. A query is internally split into multiple steps, and the optimizer generates multiple sub-plans for every step. Based on the statistics collected at critical points, the optimizer compares the collected statistics with estimated cardinality. If the optimizer finds a deviation in statistics beyond the set threshold, it picks a different sub-plan for those steps. This improves the ability of the query-processing engine to generate better execution plans. What happens in adaptive plan execution? In Oracle12c, the optimizer generates dynamic plans. A dynamic plan is an execution plan that has many built-in sub-plans. A sub-plan is a portion of plan that the optimizer can switch to as an alternative at run time. When the first execution starts, the optimizer observes statistics at various critical stages in the plan. An optimizer makes a final decision about the sub-plan based on observations made during the execution up to this point. Going deeper into the logic for the dynamic plan, the optimizer actually places the statistics collected at various critical stages in the plan. These critical stages are the places in the plan where the optimizer has to join two tables or where the optimizer has to decide upon the optimal degree of parallelism. During the execution of the plan, the statistics collector buffers a portion of the rows. The portion of the plan preceding the statistics collector can have alternative sub-plans, each of which is valid for the subset of possible values returned by the collector. This means that each of the sub-plans has a different threshold value. Based on the data returned by the statistics collector, a sub-plan is chosen which falls in the required threshold. For example, an optimizer can insert a code to collect statistics before joining two tables, during the query plan building phase. It can have multiple sub-plans based on the type of join it can perform between two tables. If the number of rows returned by the statistics collector on the first table is less than the threshold value, then the optimizer might go with the sub-plan containing the nested loop join. But if the number of rows returned by the statistics collector is above the threshold values, then the optimizer might choose the second sub-plan to go with the hash join. After the optimizer chooses a sub-plan, buffering is disabled and the statistics collector stops collecting rows and passes them through instead. On subsequent executions of the same SQL, the optimizer stops buffering and chooses the same plan instead. With dynamic plans, the optimizer adapts to poor plan choices and correct decisions are made at various steps during runtime. Instead of using predetermined execution plans, adaptive plans enable the optimizer to postpone the final plan decision until statement execution time. Consider the following simple query: SELECT a.sales_rep, b.product, sum(a.amt) FROM sales a, product b WHERE a.product_id = b.product_id GROUP BY a.sales_rep, b.product When the query plan was built initially, the optimizer will put the statistics collector before making the join. So it will scan the first table (SALES) and, based on the number of rows returned, it might make a decision to select the correct type of join. The following figure shows the statistics collector being put in at various stages: Enabling adaptive execution plans To enable adaptive execution plans, you need to fulfill the following conditions: optimizer_features_enable should be set to the minimum of 12.1.0.1 optimizer_adapive_reporting_only should be set to FALSE (default) If you set the OPTIMIZER_ADAPTIVE_REPORTING_ONLY parameter to TRUE, the adaptive execution plan feature runs in the reporting-only mode—it collects the information for adaptive optimization, but doesn't actually use this information to change the execution plans. You can find out if the final plan chosen was the default plan by looking at the column IS_RESOLVED_ADAPTIVE_PLAN in the view V$SQL. Join methods and parallel distribution methods are two areas where adaptive plans have been implemented by Oracle12c. Adaptive execution plans and join methods Here is an example that shows how the adaptive execution plan will look. Instead of simulating a new query in the database and checking if the adaptive plan has worked, I used one of the queries in the database that is already using the adaptive plan. You can get many such queries if you check V$SQL with is_resolved_adaptive_plan = 'Y'. The following queries will list all SQLs that are going for adaptive plans. Select sql_id from v$sql where is_resolved_adaptive_plan = 'Y'; While evaluating the plan, the optimizer uses the cardinality of the join to select the superior join method. The statistics collector starts buffering the rows from the first table, and if the number of rows exceeds the threshold value, the optimizer chooses to go for a hash join. But if the rows are less than the threshold value, the optimizer goes for a nested loop join. The following is the resulting plan: SQL> SELECT * FROM TABLE(DBMS_XPLAN.display_cursor(sql_id=>'dhpn35zupm8ck',cursor_child_no=>0; Plan hash value: 3790265618 ------------------------------------------------------------------------------------------------- | Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time | ------------------------------------------------------------------------------------------------- | 0 | SELECT STATEMENT | | | | 445 (100)| | | 1 | SORT ORDER BY | | 1 | 73 | 445 (1)| 00:00:01| | 2 | NESTED LOOPS | | 1 | 73 | 444 (0)| 00:00:01| | 3 | NESTED LOOPS | | 151 | 73 | 444 (0)| 00:00:01| |* 4 | TABLE ACCESS BY INDEX ROWID BATCHED| OBJ$ | 151 | 7701 | 293 (0)| 00:00:01| |* 5 | INDEX FULL SCAN | I_OBJ3 | 1 | | 20 (0)| 00:00:01| |* 6 | INDEX UNIQUE SCAN | I_TYPE2 | 1 | | 0 (0)| | |* 7 | TABLE ACCESS BY INDEX ROWID | TYPE$ | 1 | 22 | 1 (0)| 00:00:01| ------------------------------------------------------------------------------------------------- Predicate Information (identified by operation id): --------------------------------------------------- 4 - filter(SYSDATE@!-"O"."CTIME">.0007) 5 - filter("O"."OID$" IS NOT NULL) 6 - access("O"."OID$"="T"."TVOID") 7 - filter(BITAND("T"."PROPERTIES",8388608)=8388608) Note ----- - this is an adaptive plan If we check this plan, we can see the notes section, and it tells us that this is an adaptive plan. It tells us that the optimizer must have started with some default plan based on the statistics in the tables and indexes, and during run time execution it changed the join method for a sub-plan. You can actually check which step optimizer has changed and at what point it has collected the statistics. You can display this using the new format of DBMS_XPLAN.DISPLAY_CURSOR – format => 'adaptive', resulting in the following: DEO>SELECT * FROM TABLE(DBMS_XPLAN.display_cursor(sql_id=>'dhpn35zupm8ck',cursor_child_no=>0,format=>'adaptive')); Plan hash value: 3790265618 ------------------------------------------------------------------------------------------------------ | Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time | ------------------------------------------------------------------------------------------------------ | 0 | SELECT STATEMENT | | | | 445 (100)| | | 1 | SORT ORDER BY | | 1 | 73 | 445 (1)| 00:00:01 | |- * 2 | HASH JOIN | | 1 | 73 | 444 (0)| 00:00:01 | | 3 | NESTED LOOPS | | 1 | 73 | 444 (0)| 00:00:01 | | 4 | NESTED LOOPS | | 151 | 73 | 444 (0)| 00:00:01 | |- 5 | STATISTICS COLLECTOR | | | | | | | * 6 | TABLE ACCESS BY INDEX ROWID BATCHED| OBJ$ | 151 | 7701 | 293 (0)| 00:00:01 | | * 7 | INDEX FULL SCAN | I_OBJ3 | 1 | | 20 (0)| 00:00:01 | | * 8 | INDEX UNIQUE SCAN | I_TYPE2 | 1 | | 0 (0)| | | * 9 | TABLE ACCESS BY INDEX ROWID | TYPE$ | 1 | 22 | 1 (0)| 00:00:01 | |- * 10 | TABLE ACCESS FULL | TYPE$ | 1 | 22 | 1 (0)| 00:00:01 | ------------------------------------------------------------------------------------------------------ Predicate Information (identified by operation id): --------------------------------------------------- 2 - access("O"."OID$"="T"."TVOID") 6 - filter(SYSDATE@!-"O"."CTIME">.0007) 7 - filter("O"."OID$" IS NOT NULL) 8 - access("O"."OID$"="T"."TVOID") 9 - filter(BITAND("T"."PROPERTIES",8388608)=8388608) 10 - filter(BITAND("T"."PROPERTIES",8388608)=8388608) Note ----- - this is an adaptive plan (rows marked '-' are inactive) In this output, you can see that it has given three extra steps. Steps 2, 5, and 10 are extra. But these steps were present in the original plan when the query started. Initially, the optimizer generated a plan with a hash join on the outer tables. During runtime, the optimizer started collecting rows returned from OBJ$ table (Step 6), as we can see the STATISTICS COLLECTOR at step 5. Once the rows are buffered, the optimizer came to know that the number of rows returned by the OBJ$ table are less than the threshold and so it can go for a nested loop join instead of a hash join. The rows indicated by - in the beginning belong to the original plan, and they are removed from the final plan. Instead of those records, we have three new steps added—Steps 3, 8, and 9. Step 10 of the full table scan on the TYPE$ table is changed to an index unique scan of I_TYPE2, followed by the table accessed by index rowed at Step 9. Adaptive plans and parallel distribution methods Adaptive plans are also useful in adapting from bad distributing methods when running the SQL in parallel. Parallel execution often requires data redistribution to perform parallel sorts, joins, and aggregates. The database can choose from among multiple data distribution methods to perform these options. The number of rows to be distributed determines the data distribution method, along with the number of parallel server processes. If many parallel server processes distribute only a few rows, the database chooses a broadcast distribution method and sends the entire result set to all the parallel server processes. On the other hand, if a few processes distribute many rows, the database distributes the rows equally among the parallel server processes by choosing a "hash" distribution method. In adaptive plans, the optimizer does not commit to a specific broadcast method. Instead, the optimizer starts with an adaptive parallel data distribution technique called hybrid data distribution. It places a statistics collector to buffer rows returned by the table. Based on the number of rows returned, the optimizer decides the distribution method. If the rows returned by the result are less than the threshold, the data distribution method switches to broadcast distribution. If the rows returned by the table are more than the threshold, the data distribution method switches to hash distribution. Summary In this article we learned the explicit new features of Oracle optimizer which helps us in tuning our queries. Resources for Article: Further resources on this subject: Oracle Essbase System 9 Components [article] Oracle E-Business Suite: Adjusting Items in Inventory and Classifying Items [article] Oracle Business Intelligence : Getting Business Information from Data [article]

In this article, by Jonathan Hobson, the author of Troubleshooting CentOS, we will learn about memory management, swap, and swappiness. (For more resources related to this topic, see here.) A deeper understanding of the underlying active processes in CentOS 7 is an essential skill for any troubleshooter. From high load averages to slow response times, system overloads to dead and dying processes, there comes a time when every server may start to feel sluggish, act impoverished, or fail to respond, and as a consequence, it will require your immediate attention. Regardless of how you look at it, the question of memory usage remains critical to the life cycle of a system, and whether you are maintaining system health or troubleshooting a particular service or application, you will always need to remember that the use of memory is a critical resource to your system. For this reason, we will begin by calling the free command in the following way: # free -m The main elements of the preceding command will look similar to this:          Total   used   free   shared   buffers   cached Mem:     1837     274   1563         8         0       108 -/+ buffers/cache: 164   1673 Swap:     2063       0   2063 In the example shown, I have used the -m option to ensure that the output is formatted in megabytes. This makes it easier to read, but for the sake of troubleshooting, rather than trying to understand every numeric value shown, let's reduce the scope of the original output to highlight the relevant area of concern: -/+ buffers/cache: 164   1673 The importance of this line is based on the fact that it accounts for the associated buffers and caches to illustrate what memory is currently being used and what is held in reserve. Where the first value indicates how much memory is being used, the second value tells us how much memory is available to our applications. In the example shown, this instance translates into 164 MB of used memory and 1673 MB of available memory. Bearing this in mind, let me draw your attention to the final line in order that we can examine the importance of swap: Swap:     2063       0   2063 Swapping typically occurs when memory usage is impacting performance. As we can see from the preceding example, the first value tells us that there is a total amount of system swap set at 2063 MB, with the second value indicating how much swap is being used (0 MB), while the third value shows the amount of swap that is still available to the system as a whole (2063 MB). So yes, based on the example data shown here, we can conclude that this is a healthy system, and no swap is being used, but while we are here, let's use this time to discover more about the swap space on your system. To begin, we will revisit the contents of the proc directory and reveal the total and used swap size by typing the following command: # cat /proc/swaps Assuming that you understand the output shown, you should then investigate the level of swappiness used by your system with the following command: # cat /proc/sys/vm/swappiness Having done this, you will now see a numeric value between the ranges of 0-100. The numeric value is a percentage and it implies that, if your system has a value of 30, for example, it will begin to use swap memory at 70 percent occupation of RAM. The default for all Linux systems is usually set with a notional value between 30 to 60, but you can use either of the following commands to temporarily change and modify the swappiness of your system. This can be achieved by replacing the value of X with a numeric value from 1-100 by typing: # echo X > /proc/sys/vm/swappiness Or more specifically, this can also be achieved with: # sysctl -w vm.swappiness=X If you change your mind at any point, then you have two options in order to ensure that no permanent changes have been made. You can either repeat one of the preceding two commands and return the original values, or issue a full system reboot. On the other hand, if you want to make the change persist, then you should edit the /etc/sysctl.conf file and add your swappiness preferences in the following way: vm.swappiness=X When complete, simply save and close the file to ensure that the changes take effect. The level of swappiness controls the tendency of the kernel to move a process out of the physical RAM on to a swap disk. This is memory management at work, but it is important to realize that swapping will not occur immediately, as the level of swappiness is actually expressed as a percentage value. For this reason, the process of swapping should be viewed more as a measurement of preference when using the cache, and as every administrator will know, there is an option for you to clear the swap by using the commands swapoff -a and swapon -a to achieve the desired result. The golden rule is to realize that a system displaying a level of swappiness close to the maximum value (100) will prefer to begin swapping inactive pages. This is because a value of 100 is a representative of 0 percent occupation of RAM. By comparison, the closer your system is to the lowest value (0), the less likely swapping is to occur as 0 is representative of 100 percent occupation of RAM. Generally speaking, we would all probably agree that systems with a very large pool of RAM would not benefit from aggressive swapping. However, and just to confuse things further, let's look at it in a different way. We all know that a desktop computer will benefit from a low swappiness value, but in certain situations, you may also find that a system with a large pool of RAM (running batch jobs) may also benefit from a moderate to aggressive swap in a fashion similar to a system that attempts to do a lot but only uses small amounts of RAM. So, in reality, there are no hard and fast rules; the use of swap should be based on the needs of the system in question rather than looking for a single solution that can be applied across the board. Taking this further, special care and consideration should be taken while making changes to the swapping values as RAM that is not used by an application is used as disk cache. In this situation, by decreasing swappiness, you are actually increasing the chance of that application not being swapped-out, and you are thereby decreasing the overall size of the disk cache. This can make disk access slower. However, if you do increase the preference to swap, then because hard disks are slower than memory modules, it can lead to a slower response time across the overall system. Swapping can be confusing, but by knowing this, we can also appreciate the hidden irony of swappiness. As Newton's third law of motion states, for every action, there is an equal and opposite reaction, and finding the optimum swappiness value may require some additional experimentation. Summary In this article, we learned some basic yet vital commands that help us gauge and maintain server performance with the help of swapiness. Resources for Article: Further resources on this subject: Installing CentOS [article] Managing public and private groups [article] Installing PostgreSQL [article]

Customizing Google Maps Google Maps has a comprehensive API for interacting with maps on your website. MooTools can be used to load the Google Maps engine at the correct time. It can also act as a bridge between the map and other HTML elements on your site. To get started, you will first need to get an API key to use Google Maps on your domain. You can sign up for a free key at http://code.google.com/apis/maps/signup.html. Even if you are working on your local computer, you still need the key. For instance, if the base URL of your Joomla installation is http://localhost/joomla, you will enter localhost as the domain for your API key. Once you have an API key ready, create the file basicmap.js in the /components/com_js folder, and fill it with the following code: window.addEvent('domready', function() { if (GBrowserIsCompatible()) { var map = new GMap2($('map_canvas')); map.setCenter(new GLatLng(38.89, -77.04), 12); window.onunload=function() { GUnload(); }; }}); The entire script is wrapped within a call to the MooTools-specific addEvent() member function of window. Because we want this code to execute once the DOM is ready, the first parameter is the event name 'domready'. The second parameter is an anonymous function containing our code. What does the call to function() do?Using function() in JavaScript is a way of creating an anonymous function. This way, you can create functions that are used in only one place (such as event handlers) without cluttering the namespace with a needless function name. Also, the code within the anonymous function operates within its own scope; this is referred to as a closure. Closures are very frequently used in modern JavaScript frameworks, for event handling and other distinct tasks. Once inside of the function, GBrowserIsCompatible() is used to determine if the browser is capable of running Google Maps. If it is, a new instance of GMap2() is declared and bound to the HTML element that has an id of 'map_canvas' and is stored into map. The call to $('map_canvas') is a MooTools shortcut for document.GetElementById(). Next, the setCenter() member function of map is called to tell Google Maps where to center the map and how far to zoom in. The first parameter is a GLatLng() object, which is used to set the specific latitude and longitude of the map's center. The other parameter determines the zoom level, which is set to 12 in this case. Finally, the window.onunload event is set to a function that calls GUnload(). When the user navigates away from the page, this function removes Google Maps from memory, to prevent memory leaks. With our JavaScript in place, it is now time to add a function to the controller in /components/com_js/js.php that will load it along with some HTML. Add the following basicMap() function to this file: function basicMap(){ $key = 'DoNotUseThisKeyGetOneFromCodeDotGoogleDotCom'; JHTML::_('behavior.mootools'); $document =& JFactory::getDocument(); $document->addScript('http://maps.google.com/maps?file=api&v= 2&key=' . $key); $document->addScript( JURI::base() . 'components/com_js/basicmap.js'); ?> <div id="map_canvas" style="width: 500px; height: 300px"></div> <?php} The basicMap() function starts off by setting $key to the API key received from Google. You should replace this value with the one you receive at http://code.google.com/apis/maps/signup.html. Next, JHTML::_('behavior.mootools'); is called to load MooTools into the <head> tag of the HTML document. This is followed by getting a reference to the current document object through the getDocument() member function of JFactory. The addScript() member function is called twice—once to load in the Google Maps API (using our key), then again to load our basicmap.js script. (The Google Maps API calls in all of the functions and class definitions beginning with a capital 'G'.) Finally, a <div> with an id of 'map_canvas' is sent to the browser. Once this function is in place and js.php has been saved, load index.php?option=com_js&task=basicMap in the browser. Your map should look like this: We can make this map slightly more interesting by adding a marker to a specific address. To do so, add the highlighted code below to the basicmap.js file: window.addEvent('domready', function() { if (GBrowserIsCompatible()) { var map = new GMap2($('map_canvas')); map.setCenter(new GLatLng(38.89, -77.04), 12); var whitehouse = new GClientGeocoder(); whitehouse.getLatLng('1600 Pennsylvania Ave NW', function(latlng) { marker = new GMarker( latlng ); marker.bindInfoWindowHtml('<strong>The White House</strong>'); map.addOverlay(marker); }); window.onunload=function(){ GUnload(); }; }}); This code sets whitehouse as an instance of the GClientGeocoder class. Next, the getLatLng() member function of GClientGeocoder is called. The first parameter is the street address to be looked up. The second parameter is an anonymous function where the GLatLng object is passed once the address lookup is complete. Within this function, marker is set as a new GMarker object, which takes the passed-in latlng object as a parameter. The bindInfoWindowHTML() member function of GMarker is called to add an HTML message to appear in a balloon above the marker. Finally, the maker is passed into the addOverlay() member function of GMap2, to place it on the map. Save basicmap.js and then reload index.php?option=com_js&task=basicMap. You should now see the same map, only with a red pin. When you click on the red pin, your map should look like this: Interactive Maps These two different maps show the basic functionality of getting Google Maps on your own website. These maps are very basic; you could easily create them at maps.google.com then embed them in a standard Joomla! article with the HTML code they provide you. However, you would not have the opportunity to add functions that interact with the other elements on your page. To do that, we will create some more HTML code and then write some MooTools-powered JavaScript to bridge our content with Google Maps. Open the /components/com_js/js.php file and add the following selectMap() function to the controller: function selectMap(){ $key = 'DoNotUseThisKeyGetOneFromCodeDotGoogleDotCom'; JHTML::_('behavior.mootools'); $document =& JFactory::getDocument(); $document->addScript('http://maps.google.com/maps?file=api&v =2&key=' . $key); $document->addScript( JURI::base() . 'components/com_js/selectmap.js'); ?> <div id="map_canvas" style="width: 500px; height: 300px"></div> <select id="map_selections"> <option value="">(select...)</option> <option value="1200 K Street NW">Salad Surprises</option> <option value="1221 Connecticut Avenue NW">The Daily Dish</option> <option value="701 H Street NW">Sushi and Sashimi</option> </select><?php} This function is almost identical to basicMap() except for two things—selectmap.js is being added instead of basicmap.js, and a <select> element has been added beneath the <div>. The <select> element has an id that will be used in the JavaScript. The options of the <select> element are restaurants, with different addresses as values. The JavaScript code will bind a function to the onChange event so that the marker will move as different restaurants are selected.

Managing composite lifecycles Every composite has a state, mode, and associated metadata. The state can be up (started) or down (shut down). The mode can either be active or retired. Metadata is stored in the Metadata Store (MDS), which is a database-based repository used by Oracle SOA Suite 11g, and consists of information that includes default revision number, last modification date, deployment and redeployment times, and instance statistics. Before walking through how to manage the state and mode of composites, we will begin by describing composite revisions. Understanding revisions When a HelloWorld composite is deployed to the server, a revision is required during the deployment. Thus, the service's Web Services Description Language (WSDL) can be accessed via a URL similar to the following, clearly indicating a revision of "1.0" after the composite name: http://soahost1:8001/soa-infra/services/default/HelloWorld!1.0/HelloWorld.wsdl However, there may be a case where a new version of the service needs to be deployed and that this version has a different implementation from the existing one. Overwriting the existing version may not be the right option as it would break all client applications that are already utilizing the service. Thus, it makes sense to deploy the new service using a different revision, such as revision "2.0", and thus make both the versions available simultaneously. It would, therefore, be accessible at a different URL: http://soahost1:8001/soa-infra/services/default/HelloWorld!2.0/HelloWorld.wsdl Now, the old and new services are both available and accessible. Clients accessing revision 1.0 of the composite may transition to revision 2.0 at their own pace. If multiple revisions of the same service are deployed, one of them must be specified as the default revision. This can be specified during deployment time or changed at runtime. The default revision would thus be accessed at a revision-independent URL: http://soahost1:8001/soa-infra/services/default/HelloWorld/HelloWorld.wsdl Typically, client applications will access the default revision. Revisions are advantageous in environments where maintaining old and new versions of the same composite is required, particularly if it involves breaking changes. As shown in the following screenshot, default revisions are indicated by a green dot in the list of composites for a given partition. Changing the composite default revision at runtime If a composite is not the default revision, the Set As Default... button will appear in the composite page, as shown in the following screenshot. By clicking on this button, it is possible at this point to set the revision of this composite as the default revision if you choose to. If a default composite application is undeployed, the default revision is automatically changed to the last deployed revision. Deploying, redeploying, and undeploying composites Composites are deployed and redeployed as SOA Archives or SARs, which are similar to traditional JAR files. Oracle Enterprise Manager Fusion Middleware Control provides the ability to deploy, redeploy, and undeploy a SAR from the convenience and simplicity of a web browser. Though deploying via the console is extremely easy, the following two important points should be considered: A SAR file is a special JAR file that requires a prefix of sca_ and may include environment-specific information bundled within the JAR file. For example, the composite may reference a web service on some external development server. The URL of this web service is hardcoded in the JAR file. Deploying the same JAR to a production server would not be valid. Deployment of multiple composites via the console is cumbersome and time consuming. Using ant is the preferred method for deploying multiple SARs and this will be covered in a subsequent section in this article. Deploying a composite To deploy a single composite from the console: On the navigator, expand Farm_[Domain] | SOA and right-click on soa-infra. Navigate to SOA Deployment | Deploy. In the field labeled Archive is on the machine where the web browser is running , click on the Browse... button and locate your SAR file (for example, C:svnSOA11gHelloWorlddeploysca_HelloWorld_rev1.0.jar). Click on Next. From the drop-down list, select the partition to which you wish to deploy this composite. Click on Next. Choose the radio button Deploy as default revision or Do not change the default revision. Click on the Deploy button. When the HelloWorld composite is deployed as revision 1.0 and as the default version, an entry is logged in the soa_server1.out file (located under $MW_HOME/user_projects/domains/soa_domain/servers/soa_server1/logs/soa_server1.out) as follows: INFO: DeploymentEventPublisher.invoke Publishing deploy event for default/HelloWorld!1.0*soa_fe9ee226-4f29-4db7-b4be- d7410bbc13ffdefault/HelloWorld!1.0*soa_fe9ee226-4f29-4db7-b4be- d7410bbc13ff Once it is deployed, the service becomes available immediately. If the composite uses inbound resources (such as the JMS Adapter, which consumes from a JMS queue), the consumption begins immediately once the composite is deployed. The rest of the instructions in this chapter assume that the HelloWorld composite is deployed to the default partition. Redeploying a composite To redeploy a single composite from the console: On the navigator, expand Farm_[Domain] | SOA | soa-infra. Expand the partition (for example, default) and right-click on the composite name that you wish to redeploy (for example, HelloWorld). Navigate to SOA Deployment | Redeploy. In the field labeled Archive is on the machine where the web browser is running, click on the Browse button and locate your SAR file (for example, C:svnSOA11gHelloWorlddeploysca_HelloWorld_rev1.0.jar). Click on Next. Choose the radio button Deploy as default revision or Do not change the default revision. Click on the Redeploy button. Redeploying a composite overwrites the existing revision. The state of the instances of the older revision are all changed to stale. Undeploying a composite To undeploy a single composite from the console: On the navigator, expand Farm_[Domain] | SOA | soa-infra. Expand the partition (for example, default) and right-click on the composite name that you wish to undeploy (for example, HelloWorld). Navigate to SOA Deployment | Undeploy. Click on the Undeploy button. In addition to the service no longer being available, undeploying a composite (or a composite revision) changes the state of all historical instances to stale, denoted by the icon . If the default revision of the composite is undeployed, the last deployed revision of the composite becomes the default. Starting up and shutting down composites Composites are automatically started up when they are deployed. If a composite is shut down, all requests to the composite are rejected, including callbacks. New requests are not served and new instances are not created. However, all running instances are allowed to complete. Though starting up and shutting down composites via the console is extremely easy, if you require to start up or shut down multiple composites, two approaches are available (discussed in detail later in this article): Composites deployed to the same partition can all be started up or all be shut down with a single operation. Ant can be used to automate the startup and shutdown of composites. Starting a composite To start up a single composite from the console: On the navigator, expand Farm_[Domain] | SOA | soa-infra. Expand the default partition, and click on the HelloWorld composite and the revision. Click on the Start Up button, which will only appear if the composite is already shut down. Shutting down a composite To shut down a single composite from the console: On the navigator, expand Farm_[Domain] | SOA | soa-infra. Expand the partition (for example, default), choose from among the deployed composites, and click on the composite name and the revision (for example, HelloWorld[1.0]). Click on the Shut Down button, which will only appear if the composite is already started up. Retiring and activating composites Composites have two modes—active and retired. These are often confused with composite states, which can be up (started) and down (shut down). Composites are automatically activated when they are deployed (in fact, they are also started up as well, so active and started composites are really identical in nature). However, when a composite is retired, new instances cannot be created. Existing instances, however, continue to completion. This includes instances that receive callbacks. The ability to receive callbacks and time based waits is the primary difference between a retired composite and a composite that has been shut down. The only difference between activating a composite and starting up a composite is that activating the composite affects the retired mode, while starting up a composite affects the shutdown state. Retiring a composite To retire a single composite from the console: On the navigator, expand Farm_[Domain] | SOA | soa-infra. Expand the partition (for example, default) and click on the composite name (for example, HelloWorld). Click on the Retire button, which will only appear if the composite is already active. Activating a composite To activate a single composite from the console: On the navigator, expand Farm_[Domain] | SOA | soa-infra. Expand the partition (for exampple, default) and click on the composite name (for exampple, HelloWorld). Click on the Activate button, which will only appear if the composite is already retired. Deleting instances When an SOA composite application is invoked, a new composite instance is created. Every instance has a unique ID and its details can be retrieved from Oracle Enterprise Manager Fusion Middleware Control. Administrators are expected to delete completed instances and free up their data periodically to control growth. Too much instance-related data requires additional storage and it also impacts the performance of the console. Deleting instances is quite easy as demonstrated in the following steps: On the navigator, expand Farm_[Domain] | SOA | soa-infra. Expand the partition (for example, default) and click on the composite name (for example, HelloWorld). Click on the Instances tab. At this point, you can delete instances in one of the following two ways: Highlight the list of instances (press the Ctrl key and click on each composite one by one) and click on the Delete Selected button. Click on the Delete With Options button. From here you can delete instances older than a specific time or delete all instances within a time frame that have a certain state. We can also bulk delete/purge composite instances from the underlying database dehydration store through the use of SQL scripts. Structuring composite deployments with partitions Prior to Oracle SOA Suite11gPS2 (11.1.1.3), as hundreds of composites were deployed to the SOA server, they were all listed in alphabetical order on the console, which made it a burden to manage and was not very structured. Oracle recognized the lack of structure and, therefore, introduced the concept of partitions to help better organize where to deploy your composites. However, partitions are just logical separations to group your composites together. Domain libraries, extension modules, server Java Naming and Directory Interface (JNDI), and infrastructure properties are shared across all partitions. Partitions do not have their own configuration or logging. They serve no purpose other than grouping composites into separate categories. Thus, for example, code for your Human Resources integrations can reside in a partition separate from your EBS integrations, offering better structuring and organization. There are a few bulk lifecycle management tasks that can be performed on all SOA composite applications in a partition, as we will describe in this section. For example, all composites within a partition can be shut down with a single operation. The preceding screenshot shows a list of partitions in the navigator under soa-infra. Each partition may have one or more composite applications deployed to it. Partitions cannot be cascaded (that is, a partition cannot have a child partition). The default partition Oracle SOA Suite11gshould have, as a minimum, one partition. The default partition is created automatically when the product is installed, but it can be deleted afterwards if you choose to. You must always have at least one partition to allow you to deploy composites. Managing partitions You can perform several management tasks pertaining to partitions. These tasks include: Creating a partition Deleting a partition, including all composites within the partition Starting up and shutting down all composites in a partition Retiring and activating all composites in a partition Undeploying all composites in a partition The simplest method to manage partitions is via the Manage Partitions page. Simply navigate to this page to create, delete, or perform bulk lifecycle management operations on the partitions: Right-click on soa-infra, then click on Manage Partitions to access the Manage Partitions page. At this point, you can do one of the following four things: Click on the Create button to create a partition. Highlight an existing partition and click on the Delete button to delete the partition. Highlight an existing partition and click on the Composites Control button to start up, shut down, activate, or retire all composites within that partition. Highlight an existing partition and click on the Deployment button to undeploy all composites within this partition, or to deploy a single composite to this partition. The Manage Partitions page with each of its action buttons is shown in the following screenshot. The Composites Control and Deployment buttons are only activated when a partition is highlighted. Partitions do not have a state or a mode. Thus, for example, you are not shutting down the partition, you are actually shutting down all composites within the partition. Creating a partition When creating a partition, be mindful of the following naming conventions: Letters, numbers, underscores, and dashes are allowed (dashes are not allowed as the first character) Spaces are not allowed Also, be aware that partitions cannot be renamed once they are created. Deleting a partition When considering deleting partitions, remember that there always needs to be one partition in existence. If you delete all partitions, it will not be possible to deploy any code to the server. If you delete a partition, all composites within that partition are automatically undeployed. Grouping SOA composite applications into partitions Typically developers choose a partition to which a particular composite should be deployed, but as an administrator, you must understand its implications. When composites are deployed—whether through JDeveloper, the console, or ant—a partition name must be specified. Code deployed to the default partition will result in a different WSDL URL than that deployed to, for example, the HumanResources partition as shown here: http://soahost1:8001/soa-infra/services/default/HelloWorld/HelloWorld.wsdl http://soahost1:8001/soa-infra/services/HumanResources/HelloWorld/HelloWorld.wsdl Considerations for partition management There are some considerations regarding partitions that you should be aware of: Avoid creating partitions called Dev, Test, and Prod. Though possible, partitions are not designed to separate by environment. Domain libraries and SOA extensions (such as MQs and AQs) are shared by all partitions, so it is not possible to have different versions of these libraries or extensions for each partition. It is not possible to have the same JNDI address for outbound connection pools in Resource Adapters pointing to different queue manager or data sources for composites deployed in different partitions. Oracle SOA Suite 11 g parameters such as timeouts, threads, and recovery configurations are defined by WebLogic Server domain, not by partition. If composites that use inbound adapters (such as the inbound AQ Adapter, in which messages are automatically dequeued from an Oracle AQ) are deployed to multiple partitions, it is not guaranteed which composite will dequeue the inbound message (that is, they will compete with each other). Setting up ant for automated composite management All component management tasks that can be performed manually through the web-based Oracle Enterprise Manager Fusion Middleware Control console can also be executed with a script through the command-line utility ant. In this section, we describe how to use ant to start up, shut down, activate, and retire composites, as well as package and deploy them. Oracle SOA Suite11gships all necessary ant scripts to perform these tasks, and they are quite easy to use. Setting the environment Here, we will describe how to set both Linux and Microsoft Windows based environments to allow you to run your ant commands through the command line. Your ant scripts do not have to be installed on the same machine running Oracle SOA Suite 11g. In fact, it is not unusual to dedicate a single machine or server, which would host your ant scripts, allowing you to centralize the startup, shutdown, and deployment of your SOA composites to multiple target environments. You will also see how ant enables automated build management for your environment in later sections of this artilce. Setting the environment path for ant In your environment, we assume that Oracle SOA Suite 11 g is installed, which is recommended, as it will include all the required binaries to run ant. The Middleware Home, the Oracle SOA Suite 11 g Home, the Java Home, WebLogic Server username and password, and SOA server host and port will need to be updated appropriately to reflect your environment. Directory locations and JDK versions may differ depending on the patchset of Oracle SOA Suite11ginstalled. These commands must be executed to set your environment paths before running any ant command. On Linux/Unix In this article, we will assume that your code will reside under $CODE under the same Unix account where the Middleware Home and other binaries are installed. This is because the ant scripts require access to specific product libraries. The scripts assume a bash-based shell, so some changes may be required if other shells are used. To set your environment, we recommend first creating a shell script setAntEnv.sh with the following content while keeping in mind to replace the highlighted values to suit your environment and installation: export USERNAME=weblogic export PASSWORD=welcome1 export SOAHOST=soahost1 export SOAPORT=8001 export SOAURL=http://${SOAHOST}:${SOAPORT}export CODE=/u01/svn/SOA11g export MW_HOME=/u01/app/oracle/Middleware export ORACLE_HOME=$MW_HOME/Oracle_SOA1 export JAVA_HOME=$MW_HOME/jdk160_24 export ANT_HOME=$MW_HOME/modules/org.apache.ant_1.7.1 export PATH=$JAVA_HOME/bin:$ANT_HOME/bin:$ANT_HOME/lib:$PATH:. Don't forget to change the permissions of the script to executable: chmod 750 setAntEnv.sh Prior to running any ant command in the remainder of the chapter, simply source this shell script once to set your environment for your session as follows: source setAntEnv.sh Finally, make sure to change to the $ORACLE_HOME/bin directory before running any of the ant commands: cd $ORACLE_HOME/bin On Windows We will assume that your code will reside under %CODE%. To set your environment, we recommend first creating a shell script setAntEnv.bat with the following content: set USERNAME=weblogic set PASSWORD=welcome1 set SOAHOST=soahost1 set SOAPORT=8001 set SOAURL=http://soaHost:soaPort set CODE=c:svnSOA11g set MW_HOME=C:OracleMiddleware set ORACLE_HOME=%MW_HOME%Oracle_SOA1 set JAVA_HOME=%ORACLE_HOME%jdk160_24 set ANT_HOME=%ORACLE_HOME%modulesorg.apache.ant_1.7.1 set PATH=%JAVA_HOME%bin;%ANT_HOME%bin;%ANT_HOME%lib;%PATH% Make sure to update the highlighted text in the preceding script, to reflect your actual environment and installation. In your Windows environment, if you do not have Oracle SOA Suite11gand instead only have Oracle JDeveloper11ginstalled, only a few modifications are required, and you should use the following commands instead: set USERNAME=weblogic set PASSWORD=welcome1 set SOAHOST=soahost1 set SOAPORT=8001 set SOAURL=http://%SOAHOST%:%SOAPORT% set CODE=c:svnSOA11g set ORACLE_HOME=C:Oraclejdevjdeveloper set JAVA_HOME=%ORACLE_HOME%..jdk160_24 set ANT_HOME=%ORACLE_HOME%..modulesorg.apache.ant_1.7.1 set PATH=%JAVA_HOME%bin;%ANT_HOME%bin;%ANT_HOME%lib;%PATH% But only for installations of Oracle JDeveloper 11g, you must also perform a one-time copy of ant-contrib-1.0b3.jar. The file can be downloaded from http://sourceforge.net/projects/ant-contrib/files/ant-contrib/. Simply copy the file to %ANT_HOME%lib as follows: copy c:tempant-contrib-1.0b3.jar %ANT_HOME%lib Now that your batch script is created, simply run it once in the command prompt to set your environment for your session: setAntEnv.bat Finally, make sure to change to the %ORACLE_HOME%bin directory before running any of the ant commands: cd %ORACLE_HOME%bin All ant commands in the remainder of this chapter will be Linux based. For Windows, simply replace the Linux specific environment variables such as $USERNAME and $SOAHOST with their Windows equivalent of %USERNAME% and %SOAHOST%. Also ensure that the slashes are reversed in the code paths. For example, $CODE/HelloWorld/deploy/sca_HelloWorld_rev1.0.jar in Linux would be %CODE%HelloWorlddeploysca_HelloWorld_rev1.0.jar in Windows.

Revision graphs provide an easy way for you to tell at-a-glance what is going on with your project. They provide a map, in easy-to-understand tree form, of the revision history of your project, including copies, branches, and tags. One useful feature of revision graphs is that you can export them into a vector graphics format (WMF is a good option because they scale well and produce fairly small file size images. If you need a more widely supported format, then PNG is a good option) for inclusion with your source code, or on your project's website, giving everyone an easy overview of the status of your project. You can view revision graphs for files, directories, or the whole project. Time for action – viewing a revision graph To view a revision graph, go to your working copy, and right-click inside the project, and then select TortoiseSVN | Revision Graph. The graph that appears in the following screenshot shows the history of any branches and tags created, in an easy-to-understand, tree-like structure: If you prefer to read from top to bottom, rather than having the newest node at the top of the screen, then click the Show oldest node at top button (this can be found two buttons to the right of the drop-down which allows you to change the zoom level). This will invert the view, as shown in the following screenshot: You can view information about a particular branch by right-clicking on it and selecting Show Log: You can also use the same context menu to merge revisions, switch your working copy to a particular branch or tag, browse the repository, and collapse trees. What just happened? You have just used the revision graph to get an overview of the revisions, branches, and tags in your project. The last revision graph that we saw is a simple one, taken from our example project. In a real-world scenario, it is likely that the revision graph would be much more complex, and it is in these complex projects that having a revision graph becomes so useful. When there are dozens of branches which are being created and merged, it can become difficult to keep track of what happened when, and why. A revision graph gives you a clear high-level view of everything that is happening in your software project. To fully understand the revision graph, it helps to understand what each node means. The following table will help with this: ItemShapeDefault ColorItems which have been added or copied Rounded rectangle GreenItems which have been deleted   RedItems which have been renamed   BlueBranch HEAD revisions (if you have elected to show these)   PlainWorking copy revisions Oval Plain with bold outline (a red outline indicates modifications)Modified working copies Oval Plain with bold red outlineMoved items Edged Rectangle BlueAll other items Rectangle Plain   You can use the graph to get more detailed information about the differences between revisions. Just Ctrl-click on the two revisions you are interested in, right-click to bring up the context-sensitive menu, and then select Compare Revisions. You will be able to see a list of the revisions made to each file. Compare them using TortoiseDiff, as shown in the following screenshot: You can also compare HEAD revisions, and view the unified differences using this method. Changing your view If your software project is quite large and complex, then you may find it useful to change the view used in the revision graph. There are several options that you can use to change the view you are using, and they can all be found under the view menu. Rather than replicating the TortoiseSVN documentation by describing every single option, only the more interesting options will be described here. The other options are mostly clearly labelled, and otherwise are explained in the online help for TortoiseSVN: Group by Branch: This option is off by default, so all rows are sorted by revision. This can be a problem if you have branches with a long life and a few commits, because those branches will occupy a whole column, making the graph expand unnecessarily. This is demonstrated in the following screenshot: Turning on Group by Branch will change this so that revisions on a branch will be shown on consecutive lines, and branches will be grouped into columns, keeping the graph slim. The previous screenshot shows the default appearance of the revision graph (a shorter revision graph has been used here, for ease of viewing), the next screenshot shows the same revision graph with Group by Branch: With this fairly simple tree layout, the difference isn't immediately clear, but if you have a lot of branches, you'll find that the group by branch feature keeps the layout much neater, and avoids needless scrolling. Oldest on top: This option switches the graph so that the oldest revisions are shown at the top of the screen. By default, the oldest revisions are at the bottom, and the 'tree' grows upwards. Align trees on top: This option forces trees to grow down, rather than appearing in their natural revision order, or aligned at the bottom of the window. Reduce cross lines: This option cleans up the revision graph if there are lots of crossing lines. In some cases, this option can make the layout appear less logical, and can also make the graph take up a larger area of the screen. Differential path names: This option makes the path names in the node boxes as short as possible—so if you create a branch called /branches/katakana/images/characters out of /trunk/images/characters, the branch would be shown as merely /branches/katakana/.. the remainder of the path has not changed. Exact copy sources: The default behavior of the revision graph is to show branches as being taken from the last node where the change was made. In practice, many people make branches from the HEAD rather than from a specific revision. If you have a reason for needing to know which revision was used to create a copy, then you can use this option to show those details. Fold tags: If your project has a lot of tags, then you may find that they take up unnecessary screen space, hiding the information that you are interested in. You can use this option to hide the nodes for tags. If you still need to find a tag, you will find them displayed as tooltips on the node that they were copied from. Each source node that had a tag made from it will have an icon on the right-hand side indicating that a tag was made. Tree stripes: No, this option isn't related to landscape gardening. The tree stripes option tells TortoiseSVN to use alternating background colors so that it is easy to distinguish between different trees in the graph. Keeping your view up-to-date If you are viewing a revision graph of an active project, you may want to check for updates. Just as you would in your web browser, you can refresh the revision graph by pressing F5. This will connect you to the server (if you have been working offline) and check to see if there have been any new commits. Pressing F5 to refresh works for most screens in TortoiseSVN. You can update your log dialog, for example, by pressing F5 too. Pruning trees Large software projects can end up with lots of trees. This can make the revision graph look excessively complex, and can make it harder for you to find the information that you need. The good news is that you can tame the trees in your graph, shrinking and expanding them as you need them. To shrink a tree or a branch, simply hover your mouse over the point where the branch begins (where the node link enters the node), and you will be given the option to collapse the related tree (-), or expand it (+). If applicable, you will also be presented with the option to split a sub-tree into a separate graph (x), or re-attach a tree that had been split (o): Summary In this article you learned how to view revision graphs, and how to manipulate your view of the graph to give a clearer view of the things that you are interested in. Revision graphs are useful for even small development teams. The good thing about the way TortoiseSVN displays them is that they are highly customizable, and scale well for larger projects. So, if you need to keep up with the lifecycle of branches and tags in a big project, then you can do so easily—thanks to the highly customizable views offered by TortoiseSVN! Further resources on this subject: Working with Revision Logs in TortoiseSVN [Article] Change Control for Personal Projects - Subversion Style [Article] Managing Software Development with Trac and Subversion [Book]

When writing Chapter 12, "It's All about the Data," of Learning Ext JS, I switched things up a bit and switched the server-side processes to utilizing Adobe's ColdFusion application server, instead of the PHP we had been using in the rest of the book. There were a few reasons we decided to do this. To show that Ext JS can work with any server-side technology. ColdFusion 8 includes Ext JS 1.1 for it's new Ajax form components. Adobe uses a custom format for the serialized JSON return of query data, making it perfect for our example needs. I'm a ColdFusion programmer. Some time ago, before writing Chapter 12, I had begun to use a Custom Data Reader that I had found on the Ext JS forums. Another Ext user and ColdFusion programmer, John Wilson, had written the custom reader to consume Adobe's custom JSON return for queries. First, let me show you why Adobe's format differs from the generally expected serialized JSON return of a query. Here's an example of a typical query response. { 'results': 2, 'rows': [ { 'id': 1, 'firstname': 'Bill', occupation: 'Gardener' }, // a row object { 'id': 2, 'firstname': 'Ben' , occupation: 'Horticulturalist' } // another row object ] } And here's an example of how ColdFusion returns a query response.     {        "COLUMNS":["INTPROPERTIESID","STRDEVELOPMENT","INTADDRESSID", "STRSTREET","STRSTREET2", "STRCITY","CHSTATEID","INTZIP"],        "DATA":[            [2,"Abbey Road",6,"456 Abbey Road","Villa 5","New York","NY",12345],            [6,"Splash",39,"566 aroundthe bend dr",null,"Nashville","TN",37221]        ]    } You can see, when examining the two formats that they are very divergent. The typical format returns an array of row objects of the query's results, whereas ColdFusion's format is an array (DATA) of arrays (each row of the query result), with each row array only containing the data. The ColdFusion format has extracted the column names into it's own array (COLUMNS), as opposed to the name/value pairing found in the object notation of the typical return. It's actually very smart, on Adobe's part, to return the data in this fashion, as it would ultimately mean smaller data sets returned from a remote call, especially with large recordsets. John's CFJsonReader, a custom data reader and an extended component of Ext's base DataReader, was able to translate ColdFusion's data returns by properly parsing the JSON return into Records of an Ext Store. It worked fairly well, with a few minor exceptions. it didn't handle the column aliasing you could do with any other Ext JS data reader (name:'development',mapping:'STRDEVELOPMENT') it didn't allow data type association with a value, as other Ext JS data readers (INTZIP is of type 'int', STRDEVELOPMENT is of type 'string', etc) So, it worked, but ultimately was limited. When I was writing Chapter 13, "Code for Reuse: Extending Ext JS", I really dove into extending existing Ext JS components. This helped me gain a better understanding of what John had done, when writing CFJsonReader. But, after really reviewing the code, I saw there was a better way of handling ColdFusion's JSON return. What it basically came down to was that John was extending Ext's base DataReader object, and then hand parsing almost the entire return. Looking at the above examples, you'll notice that Adobe's implementation is an array of arrays, rather than an array of objects. Ext JS already comes with an ArrayReader object, so I knew that by writing a custom data reader that extended it I would be able to get the desired results. Half an hour later, I had "built a better mousetrap" and we now have a Custom Data Reader for properly parsing ColdFusion's JSON return, without the previous limitations. /* * Ext JS Library 2.0 * Copyright(c) 2006-2007, Ext JS, LLC. * licensing@extjs.com * * http://extjs.com/license * ******************************************* * Steve 'Cutter' Blades (CutterBl) no.junkATcutterscrossingDOTcom * http://blog.cutterscrossing.com * * Inspired by the CFJsonReader, originally writtin by John Wilson (Daemach) * http://extjs.com/forum/showthread.php?t=21408&highlight=cfjsonreader * * This Custom Data Reader will take the JSON return of a ColdFusion * Query object, rather returned straight up, or via the ColdFusion * QueryForGrid() method. * * The CFQueryReader constructor takes two arguments * @meta : object containing single key/value pair for the 'id' of each record * @recordType : field mapping object * * The recordType object allows you to alias the returned ColdFusion column * name (which is always passed in upper case) to any 'name' you wish, as * well as assign a data type, which your ExtJS app will attempt to cast * whenever the value is referenced. * * ColdFusion's JSON return, for a ColdFusion Query object, will appear in the * following format: * * {"COLUMNS":["INTVENDORTYPEID","STRVENDORTYPE","INTEXPENSECATEGORIESID", * "STREXPENSECATEGORIES"],"DATA" :[[2,"Carpet Cleaning",1,"Cleaining"], * [1,"Cleaning Service",1,"Cleaining"]]} * * The ColdFusion JSON return on any query that is first passed through * ColdFusion's QueryForGrid() method will return the object in the * following format: * * {"TOTALROWCOUNT":3, "QUERY":{"COLUMNS":["MYIDFIELD","DATA1","DATA2"], * "DATA":[[1,"Bob","Smith"],[6,"Jim","Brown"]]}} * * The Ext.data.CFQueryReader is designed to accomodate either format * automatically. You would create your reader instance in much the same * way as the CFJsonReader was created: * * var myDataModel = [ * {name: 'myIdField', mapping: 'MYIDFIELD'}, * {name: 'data1', mapping: 'DATA1'}, * {name: 'data2', mapping: 'DATA2'} * ]; * * var myCFReader = new Ext.data.CFJsonReader({id:'myIdField'},myDataModel); * * Notice that the 'id' value mirrors the alias 'name' of the record's field. */ Ext.data.CFQueryReader = function(meta, recordType){ this.meta = meta || {}; Ext.data.CFQueryReader.superclass.constructor.call(this, meta, recordType || meta.fields); }; Ext.extend(Ext.data.CFQueryReader, Ext.data.ArrayReader, { read : function(response){ var json = response.responseText; var o = eval("("+json+")"); if(!o) { throw {message: "JsonReader.read: Json object not found"}; } if(o.TOTALROWCOUNT){ this.totalRowCount = o.TOTALROWCOUNT; } return this.readRecords(((o.QUERY)? o.QUERY : o)); }, readRecords : function(o){ var sid = this.meta ? this.meta.id : null; var recordType = this.recordType, fields = recordType.prototype.fields; var records = []; var root = o.DATA; // give sid an integer value that equates to it's mapping sid = fields.indexOfKey(sid); // re-assign the mappings to line up with the column position // in the returned json response for(var a = 0; a < o.COLUMNS.length; a++){ for(var b = 0; b < fields.length; b++){ if(fields.items[b].mapping == o.COLUMNS[a]){ fields.items[b].mapping = a; } } } for(var i = 0; i < root.length; i++){ var n = root[i]; var values = {}; var id = ((sid || sid === 0) && n[sid] !== undefined && n[sid] !== "" ? n[sid] : null); for(var j = 0, jlen = fields.length; j < jlen; j++){ var f = fields.items[j]; var k = f.mapping !== undefined && f.mapping !== null ? f.mapping : j; var v = n[k] !== undefined ? n[k] : f.defaultValue; v = f.convert(v, n); values[f.name] = v; } var record = new recordType(values, id); record.json = n; records[records.length] = record; } if(!this.totalRowCount){ this.totalRowCount = records.length; } return { records : records, totalRecords : this.totalRowCount }; } }); So, this changes our examples for Chapter 12 just a little bit. First of all, we'll need to have the CFQueryReader included, in place of the CFJsonReader. You can change the script tags in the samples for Examples 3 and 4. ... <script language="javascript" type="text/javascript" src="/scripts/custom-ext/CFQueryReader.js"></script> ... Next, we'll change the scripts for these two examples. We'll remove our configuration references for CFJsonReader, and replace them with the updated configuration for the CFQueryReader. /* * Chapter 12 Example 3 * Data Store from custom reader * * Revised: SGB (Cutter): 12.17.08 * Replaced CFJsonReader with CFQueryReader */ // Save all processing until the // DOM is completely loaded Ext.onReady(function(){ var ourStore = new Ext.data.Store({ url:'Chapter12Example.cfc', baseParams:{ method: 'getFileInfoByPath', returnFormat: 'JSON', queryFormat: 'column', startPath: '/images/' }, reader: new Ext.data.CFQueryReader({ id: 'NAME', // This is supposed to match the 'mapping' fields:[ {name:'file_name',mapping:'NAME'}, {name:'file_size',mapping:'SIZE'}, {name:'type',mapping:'TYPE'}, {name:'lastmod',mapping:'DATELASTMODIFIED'}, {name:'file_attributes',mapping:'ATTRIBUTES'}, {name:'mode',mapping:'MODE'}, {name:'directory',mapping:'DIRECTORY'} ] }), fields: recordModel, listeners:{ beforeload:{ fn: function(store, options){ if (options.startPath && (options.startPath.length > 0)){ store.baseParams.startPath = options.startPath; } }, scope:this }, load: { fn: function(store,records,options){ console.log(records); } }, scope:this } }); ourStore.load(); }); /* * Chapter 12 Example 4 * Data Store from custom reader - Filtering * * Revised: SGB (Cutter): 12.17.08 * Replaced CFJsonReader with CFQueryReader */ // Simple function/object to 'clone' objects cloneConfig = function (config) { for (i in config) { if (typeof config[i] == 'object') { this[i] = new cloneConfig(config[i]); } else this[i] = config[i]; } } // Save all processing until the // DOM is completely loaded Ext.onReady(function(){ var initialBaseParams = { method: 'getDirectoryContents', returnFormat: 'JSON', queryFormat: 'column', startPath: '/testdocs/' }; var ourStore = new Ext.data.Store({ url:'Chapter12Example.cfc', baseParams: new cloneConfig(initialBaseParams), reader: new Ext.data.CFQueryReader({ id: 'NAME', // This is supposed to match the 'mapping' fields:[ {name:'file_name',mapping:'NAME'}, {name:'file_size',mapping:'SIZE'}, {name:'type',mapping:'TYPE'}, {name:'lastmod',mapping:'DATELASTMODIFIED'}, {name:'file_attributes',mapping:'ATTRIBUTES'}, {name:'mode',mapping:'MODE'}, {name:'directory',mapping:'DIRECTORY'} ] }), listeners:{ beforeload:{ fn: function(store, options){ for(var i in options){ if(options[i].length > 0){ store.baseParams[i] = options[i]; } } }, scope:this }, load: { fn: function(store, records, options){ console.log(records); }, scope: this }, update: { fn: function(store, record, operation){ switch (operation){ case Ext.record.EDIT: // Do something with the edited record break; case Ext.record.REJECT: // Do something with the rejected record break; case Ext.record.COMMIT: // Do something with the committed record break; } }, scope:this } } }); ourStore.load({recurse:true}); filterStoreByType = function (type){ ourStore.load({dirFilter:type}); } filterStoreByFileType = function (fileType){ ourStore.load({fileFilter:fileType}); } clearFilters = function (){ ourStore.baseParams = new cloneConfig(initialBaseParams); ourStore.load(); } }); Summary These very basic changes have no overall effect on our examples. They function exactly as they did before. The new Custom Data Reader loads the data, returned from ColdFusion, exactly as it should. Now, we can also work with these data stores in the same manor as we would with any other data store set up through Ext JS, having the ability to alias columns, define field data types, and more.

In this article by Ben Frain, the author of Responsive Web Design with HTML5 and CSS3 Second Edition, we'll cover the following topics: What are pseudo classes The :last-child selector The nth-child selectors The nth rules nth-based selection in responsive web design (For more resources related to this topic, see here.) CSS3 gives us more power to select elements based upon where they sit in the structure of the DOM. Let's consider a common design treatment; we're working on the navigation bar for a larger viewport and we want to have all but the last link over on the left. Historically, we would have needed to solve this problem by adding a class name to the last link so that we could select it, like this: <nav class="nav-Wrapper"> <a href="/home" class="nav-Link">Home</a> <a href="/About" class="nav-Link">About</a> <a href="/Films" class="nav-Link">Films</a> <a href="/Forum" class="nav-Link">Forum</a> <a href="/Contact-Us" class="nav-Link nav-LinkLast">Contact Us</a> </nav> This in itself can be problematic. For example, sometimes, just getting a content management system to add a class to a final list item can be frustratingly difficult. Thankfully, in those eventualities, it's no longer a concern. We can solve this problem and many more with CSS3 structural pseudo-classes. The :last-child selector CSS 2.1 already had a selector applicable for the first item in a list: div:first-child { /* Styles */ } However, CSS3 adds a selector that can also match the last: div:last-child { /* Styles */ } Let's look how that selector could fix our prior problem: @media (min-width: 60rem) { .nav-Wrapper { display: flex; } .nav-Link:last-child { margin-left: auto; } } There are also useful selectors for when something is the only item: :only-child and the only item of a type: :only-of-type. The nth-child selectors The nth-child selectors let us solve even more difficult problems. With the same markup as before, let's consider how nth-child selectors allow us to select any link(s) within the list. Firstly, what about selecting every other list item? We could select the odd ones like this: .nav-Link:nth-child(odd) { /* Styles */ } Or, if you wanted to select the even ones: .nav-Link:nth-child(even) { /* Styles */ } Understanding what nth rules do For the uninitiated, nth-based selectors can look pretty intimidating. However, once you've mastered the logic and syntax you'll be amazed what you can do with them. Let's take a look. CSS3 gives us incredible flexibility with a few nth-based rules: nth-child(n) nth-last-child(n) nth-of-type(n) nth-last-of-type(n) We've seen that we can use (odd) or (even) values already in an nth-based expression but the (n) parameter can be used in another couple of ways: As an integer; for example, :nth-child(2) would select the 
second item As a numeric expression; for example, :nth-child(3n+1) would start at 1 and then select every third element The integer based property is easy enough to understand, just enter the element number you want to select. The numeric expression version of the selector is the part that can be a little baffling for mere mortals. If math is easy for you, I apologize for this next section. For everyone else, let's break it down. Breaking down the math Let's consider 10 spans on a page (you can play about with these by looking at example_05-05): <span></span> <span></span> <span></span> <span></span> <span></span> <span></span> <span></span> <span></span> <span></span> <span></span> By default they will be styled like this: span { height: 2rem; width: 2rem; background-color: blue; display: inline-block; } As you might imagine, this gives us 10 squares in a line: OK, let's look at how we can select different ones with nth-based selections. For practicality, when considering the expression within the parenthesis, I start from the right. So, for example, if I want to figure out what (2n+3) will select, I start with the right-most number (the three here indicates the third item from the left) and know it will select every second element from that point on. So adding this rule: span:nth-child(2n+3) { color: #f90; border-radius: 50%; } Results in this in the browser: As you can see, our nth selector targets the third list item and then every subsequent second one after that too (if there were 100 list items, it would continue selecting every second one). How about selecting everything from the second item onwards? Well, although you could write :nth-child(1n+2), you don't actually need the first number 1 as unless otherwise stated, n is equal to 1. We can therefore just write :nth-child(n+2). Likewise, if we wanted to select every third element, rather than write :nth-child(3n+3), we can just write :nth-child(3n) as every third item would begin at the third item anyway, without needing to explicitly state it. The expression can also use negative numbers, for example, :nth-child(3n-2) starts at -2 and then selects every third item. You can also change the direction. By default, once the first part of the selection is found, the subsequent ones go down the elements in the DOM (and therefore from left to right in our example). However, you can reverse that with a minus. For example: span:nth-child(-2n+3) { background-color: #f90; border-radius: 50%; } This example finds the third item again, but then goes in the opposite direction to select every two elements (up the DOM tree and therefore from right to left in our example): Hopefully, the nth-based expressions are making perfect sense now? The nth-child and nth-last-child differ in that the nth-last-child variant works from the opposite end of the document tree. For example, :nth-last-child(-n+3) starts at 3 from the end and then selects all the items after it. Here's what that rule gives us in the browser: Finally, let's consider :nth-of-type and :nth-last-of-type. While the previous examples count any children regardless of type (always remember the nth-child selector targets all children at the same DOM level, regardless of classes), :nth-of-type and :nth-last-of-type let you be specific about the type of item you want to select. Consider the following markup (example_05-06): <span class="span-class"></span> <span class="span-class"></span> <span class="span-class"></span> <span class="span-class"></span> <span class="span-class"></span> <div class="span-class"></div> <div class="span-class"></div> <div class="span-class"></div> <div class="span-class"></div> <div class="span-class"></div> If we used the selector: .span-class:nth-of-type(-2n+3) { background-color: #f90; border-radius: 50%; } Even though all the elements have the same span-class, we will only actually be targeting the span elements (as they are the first type selected). Here is what gets selected: We will see how CSS4 selectors can solve this issue shortly. CSS3 doesn't count like JavaScript and jQuery! If you're used to using JavaScript and jQuery you'll know that it counts from 0 upwards (zero index based). For example, if selecting an element in JavaScript or jQuery, an integer value of 1 would actually be the second element. CSS3 however, starts at 1 so that a value of 1 is the first item it matches. nth-based selection in responsive web designs Just to close out this little section I want to illustrate a real life responsive web design problem and how we can use nth-based selection to solve it. Remember the horizontal scrolling panel from example_05-02? Let's consider how that might look in a situation where horizontal scrolling isn't possible. So, using the same markup, let's turn the top 10 grossing films of 2014 into a grid. For some viewports the grid will only be two items wide, as the viewport increases we show three items and at larger sizes still we show four. Here is the problem though. Regardless of the viewport size, we want to prevent any items on the bottom row having a border on the bottom. You can view this code at example_05-09. Here is how it looks with four items wide: See that pesky border below the bottom two items? That's what we need to remove. However, I want a robust solution so that if there were another item on the bottom row, the border would also be removed on that too. Now, because there are a different number of items on each row at different viewports, we will also need to change the nth-based selection at different viewports. For the sake of brevity, I'll show you the selection that matches four items per row (the larger of the viewports). You can view the code sample to see the amended selection at the different viewports. @media (min-width: 55rem) { .Item { width: 25%; } /* Get me every fourth item and of those, only ones that are in the last four items */ .Item:nth-child(4n+1):nth-last-child(-n+4), /* Now get me every one after that same collection too. */ .Item:nth-child(4n+1):nth-last-child(-n+4) ~ .Item { border-bottom: 0; } } You'll notice here that we are chaining the nth-based pseudo-class selectors. It's important to understand that the first doesn't filter the selection for the next, rather the element has to match each of the selections. For our preceding example, the first element has to be the first item of four and also be one of the last four. Nice! Thanks to nth-based selections we have a defensive set of rules to remove the bottom border regardless of the viewport size or number of items we are showing. Summary In this article, we've learned what are structural pseudo-classes. We've also learned what nth rules do. We have also showed the nth-based selection in responsive web design. Resources for Article:   Further resources on this subject: CSS3 Animation[article] A look into responsive design frameworks[article] Linking Dynamic Content from External Websites[article]