How-To Tutorials - Programming

Many of you as (Java) programmers generate business purpose code, like "confirming an order" or "find available products". At times, you may also want to connect to external systems and services, since your application in isolation alone will not provide you the required functionality. When the number of such connections increases, you would be generating more and more of "integration code", mixed along with your business code. For single or simple systems and services this is fine, but what if your "Enterprise" has got many (say 100? or even more...) such systems and services to be integrated together? Here, integration becomes a prime concern, which is separate from fulfilling your business concerns. In the SOA context, we will have services fulfilling your business use cases. Existing Java tools help us to define services. But are they enough to support Service Oriented Integration (SOI)? Perhaps not, and this is where JSR-208 (Java Specification Request) introduces the Java Business Integration (JBI) specification. And in the world of integration, we have multiple Architectures to follow including the Point-to-Point, Hub-and-Spoke, Message-Bus and the Service-Bus. Each of them have their own advantages and disadvantages, and the Enterprise Service Bus (ESB) is an Architectural pattern best suited for doing SOI. This book provides a consistent style and visual representations to describe the message flows in sample scenarios, which helps the reader to understand the code samples fully. This book also presents practical advice on designing code that connects services together, based again on practical experiences gathered over the last one decade in java business integration. I believes in "Practice What You Preach" and hence equips you with enough tools to "Practice What You Read". What does the book have to offer? or What does it teach? This book introduces ESB - The book guarantees you understand ESB and can also code for ESB. This book introduces JBI - The book don't reproduce specification, but give you just enough highlights alone. Teaches you ServiceMix - ServiceMix is an Apache Open source Java ESB. The book teach you from step 1 of installation Teaches you to implement practical scenarios - Proxies, Web Services gateway, web services over JMS, service versioning, etc. Implementation for EIP - Gives you code on how to implement Enterprise Integration Patterns by Gregor Hohpe and Bobby Woolf For more, have a glance through the Table of Contents [PDF] Who would benefit from it? Any Java/J2EE enthusiast, who wants to know something more than daily POJO, Spring & Hibernate Developers & Architects who deals with integration Developers & Architects who don't consciously deal with integration - its high time for you to seperate out spaghetti integration aspects from your business purpose code - for that, you need to first understand and sense integration! Even people with Non-Java background - My .NET peers, don't envy on the lightweight approaches described in this book using java tools. The integration is done mostly in XML configurations with minimum java code, and you too can benefit from the literature. Anything special about the book First book published on JBI First book published on Apache ServiceMix First book which shows you how to integrate following ESB, using lightweight tools. A book with code, which makes you feel running and seeing the code in action, even without actually running the code (nothing prevents you from trying the samples). You can go though a Sample Chapter here: JBI-Bind-Web-Services-in-ESB-Gateway.pdf [1 MB] No heavy Workshops, IDEs, Studios, Plugins or 4GB RAM required - Use a text editor and Apache Ant, and you can run the samples. Based on existing knowledge on web services Authored and reviewed by practicing Architects, who are developers too in their everyday role. What this book is not about? Not a collection of white papers alone - The book provide you implementation samples. Not a repetition of ServiceMix online documentation - The book provide practical scenarios as samples Not about JBI Service Provider Interface (SPI) - Hence this book is not for tool vendors, but for developers Fine, if you think you need some starters before the real chill, you can go through the article titled Aggregate Services in ServiceMix JBI ESB

Authentication and authorization in Java EE Container Security There are two aspects covered by Java EE container security: authentication and authorization. Authentication is the process of verifying that users are who they claim to be. Typically this is performed by the user providing credentials such as a password. Authorization, or access control, is the process of restricting operations to specific users or categories of users. The EJB specification provides two kinds of authorization: declarative and programmatic, as we shall see later in the article. The Java EE security model introduces a few concepts common to both authentication and authorization. A principal is an entity that we wish to authenticate. The format of a principal is application-specific but an example is a username. A role is a logical grouping of principals. For example, we can have administrator, manager, and employee roles. The scope over which a common security policy applies is known as a security domain, or realm. Authentication For authentication, every Java EE compliant application server provides the Java Authentication and Authorization Service (JAAS) API. JAAS supports any underlying security system. So we have a common API regardless of whether authentication is username/password verification against a database, iris or fingerprint recognition for example. The JAAS API is fairly low level and most application servers provide authentication mechanisms at a higher level of abstraction. These authentication mechanisms are application-server specific however. We will not cover JAAS any further here, but look at authentication as provided by the GlassFish application server. GlassFish Authentication There are three actors we need to define on the GlassFish application server for authentication purposes: users, groups, and realms. A user is an entity that we wish to authenticate. A user is synonymous with a principal. A group is a logical grouping of users and is not the same as a role. A group's scope is global to the application server. A role is a logical grouping of users whose scope is limited to a specific application. Of course for some applications we may decide that roles are identical to groups. For other applications we need some mechanism for mapping the roles onto groups. We shall see how this is done later. A realm, as we have seen, is the scope over which a common security policy applies. GlassFish provides three kinds of realms: file, certificate, and admin-realm. The file realm stores user, group, and realm credentials in a file named keyfile. This file is stored within the application server file system. A file realm is used by web clients using http or EJB application clients. The certificate realm stores a digital certificate and is used for authenticating web clients using https. The admin-realm is similar to the file realm and is used for storing administrator credentials. GlassFish comes pre-configured with a default file realm named file. We can add, edit, and delete users, groups, and realms using the GlassFish administrator console. We can also use the create-file-user option of the asadmin command line utility. To add a user named scott to a group named bankemployee, in the file realm, we would use the command: <target name="create-file-user"> <exec executable="${glassfish.home}/bin/asadmin" failonerror="true" vmlauncher="false"> <arg line="create-file-user --user admin --passwordfile userpassword --groups bankemployee scott"/> </exec> </target> --user specifies the GlassFish administrator username, admin in our example. --passwordfile specifies the name of the file containing password entries. In our example this file is userpassword. Users, other than GlassFish administrators, are identified by AS_ADMIN_USERPASSWORD. In our example the content of the userpassword file is: AS_ADMIN_USERPASSWORD=xyz This indicates that the user's password is xyz. --groups specifies the groups associated with this user (there may be more than one group). In our example there is just one group, named bankemployee. Multiple groups are colon delineated. For example if the user belongs to both the bankemployee and bankcustomer groups, we would specify: --groups bankemployee:bankcustomer The final entry is the operand which specifies the name of the user to be created. In our example this is scott. There is a corresponding asadmin delete-file-user option to remove a user from the file realm. Mapping Roles to Groups The Java EE specification specifies that there must be a mechanism for mapping local application specific roles to global roles on the application server. Local roles are used by an EJB for authorization purposes. The actual mapping mechanism is application server specific. As we have seen in the case of GlassFish, the global application server roles are called groups. In GlassFish, local roles are referred to simply as roles. Suppose we want to map an employee role to the bankemployee group. We would need to create a GlassFish specific deployment descriptor, sun-ejb-jar.xml, with the following element: <security-role-mapping> <role-name>employee</role-name> <group-name>bankemployee</group-name> </security-role-mapping> We also need to access the configuration-security screen in the administrator console. We then disable the Default Principal To Role Mapping flag. If the flag is enabled then the default is to map a group onto a role with the same name. So the bankemployee group will be mapped to the bankemployee role. We can leave the default values for the other properties on the configuration-security screen. Many of these features are for advanced use where third party security products can be plugged in or security properties customized. Consequently we will give only a brief description of these properties here. Security Manager: This refers to the JVM security manager which performs code-based security checks. If the security manager is disabled GlassFish will have better performance. However, even if the security manager is disabled, GlassFish still enforces standard Java EE authentication/authorization. Audit Logging: If this is enabled, GlassFish will provide an audit trail of all authentication and authorization decisions through audit modules. Audit modules provide information on incoming requests, outgoing responses and whether authorization was granted or denied. Audit logging applies for web-tier and ejb-tier authentication and authorization. A default audit module is provided but custom audit modules can also be created. Default Realm: This is the default realm used for authentication. Applications use this realm unless they specify a different realm in their deployment descriptor. The default value is file. Other possible values are admin-realm and certificate. We discussed GlassFish realms in the previous section. Default Principal: This is the user name used by GlassFish at run time if no principal is provided. Normally this is not required so the property can be left blank. Default Principal Password: This is the password of the default principal. JACC: This is the class name of a JACC (Java Authorization Contract for Containers) provider. This enables the GlassFish administrator to set up third-party plug in modules conforming to the JACC standard to perform authorization. Audit Modules: If we have created custom modules to perform audit logging, we would select from this list. Mapped Principal Class: This is only applicable when Default Principal to Role Mapping is enabled. The mapped principal class is used to customize the java.security.Principal implementation class used in the default principal to role mapping. If no value is entered, the com.sun.enterprise.deployment.Group implementation of java.security.Principal is used. Authenticating an EJB Application Client Suppose we want to invoke an EJB, BankServiceBean, from an application client. We also want the application client container to authenticate the client. There are a number of steps we first need to take which are application server specific. We will assume that all roles will have the same name as the corresponding application server groups. In the case of GlassFish we need to use the administrator console and enable Default Principal To Role Mapping. Next we need to define a group named bankemployee with one or more associated users. An EJB application client needs to use IOR (Interoperable Object Reference) authentication. The IOR protocol was originally created for CORBA (Common Object Request Broker Architecture) but all Java EE compliant containers support IOR. An EJB deployed on one Java EE compliant vendor may be invoked by a client deployed on another Java EE compliant vendor. Security interoperability between these vendors is achieved using the IOR protocol. In our case the client and target EJB both happen to be deployed on the same vendor, but we still use IOR for propagating security details from the application client container to the EJB container. IORs are configured in vendor specific XML files rather than the standard ejb-jar.xml file. In the case of GlassFish, this is done within the <ior-security-config> element within the sun-ejb-jar.xml deployment descriptor file. We also need to specify the invoked EJB, BankServiceBean, in the deployment descriptor. An example of the sun-ejb-jar.xml deployment descriptor is shown below: <?xml version="1.0" encoding="UTF-8"?> <!DOCTYPE sun-ejb-jar PUBLIC "-//Sun Microsystems, Inc.//DTD       Application Server 9.0 EJB 3.0//EN"       "http://www.sun.com/software/appserver/dtds/sun-ejb-jar_3_0-0.dtd"> <sun-ejb-jar>   <enterprise-beans>     <ejb>       <ejb-name>BankServiceBean</ejb-name>         <ior-security-config>           <as-context>              <auth-method>USERNAME_PASSWORD</auth-method>              <realm>default</realm>              <required>true</required>           </as-context>         </ior-security-config>     </ejb>   </enterprise-beans> </sun-ejb-jar> The as in <as-context> stands for the IOR authentication service. This specifies authentication mechanism details. The <auth-method> element specifies the authentication method. This is set to USERNAME_PASSWORD which is the only value for an application client. The <realm> element specifies the realm in which the client is authenticated. The <required> element specifies whether the above authentication method is required to be used for client authentication. When creating the corresponding EJB JAR file, the sun-ejb-jar.xml file should be included in the META-INF directory, as follows: <target name="package-ejb" depends="compile">     <jar jarfile="${build.dir}/BankService.jar">         <fileset dir="${build.dir}">              <include name="ejb30/session/**" />                           <include name="ejb30/entity/**" />               </fileset>               <metainf dir="${config.dir}">             <include name="persistence.xml" />                          <include name="sun-ejb-jar.xml" />         </metainf>     </jar> </target> As soon as we run the application client, GlassFish will prompt with a username and password form, as follows: If we reply with the username scott and password xyz the program will run. If we run the application with an invalid username or password we will get the following error message: javax.ejb.EJBException: nested exception is: java.rmi.AccessException: CORBA NO_PERMISSION 9998 ..... EJB Authorization Authorization, or access control, is the process of restricting operations to specific roles. In contrast with authentication, EJB authorization is completely application server independent. The EJB specification provides two kinds of authorization: declarative and programmatic. With declarative authorization all security checks are performed by the container. An EJB's security requirements are declared using annotations or deployment descriptors. With programmatic authorization security checks are hard-coded in the EJBs code using API calls. However, even with programmatic authorization the container is still responsible for authentication and for assigning roles to principals. Declarative Authorization As an example, consider the BankServiceBean stateless session bean with methods findCustomer(), addCustomer() and updateCustomer(): package ejb30.session; import javax.ejb.Stateless; import javax.persistence.EntityManager; import ejb30.entity.Customer; import javax.persistence.PersistenceContext; import javax.annotation.security.RolesAllowed; import javax.annotation.security.PermitAll; import java.util.*; @Stateless @RolesAllowed("bankemployee") public class BankServiceBean implements BankService { @PersistenceContext(unitName="BankService") private EntityManager em; private Customer cust; @PermitAll public Customer findCustomer(int custId) { return ((Customer) em.find(Customer.class, custId)); } public void addCustomer(int custId, String firstName, String lastName) { cust = new Customer(); cust.setId(custId); cust.setFirstName(firstName); cust.setLastName(lastName); em.persist(cust); } public void updateCustomer(Customer cust) { Customer mergedCust = em.merge(cust); } } We have prefixed the bean class with the annotation: @RolesAllowed("bankemployee") This specifies the roles allowed to access any of the bean's method. So only users belonging to the bankemployee role may access the addCustomer() and updateCustomer() methods. More than one role can be specified by means of a brace delineated list, as follows: @RolesAllowed({"bankemployee", "bankcustomer"}) We can also prefix a method with @RolesAllowed, in which case the method annotation will override the class annotation. The @PermitAll annotation allows unrestricted access to a method, overriding any class level @RolesAllowed annotation. As with EJB 3 in general, we can use deployment descriptors as alternatives to the @RolesAllowed and @PermitAll annotations. Denying Authorization Suppose we want to deny all users access to the BankServiceBean.updateCustomer() method. We can do this using the @DenyAll annotation: @DenyAll public void updateCustomer(Customer cust) { Customer mergedCust = em.merge(cust); } Of course if you have access to source code you could simply delete the method in question rather than using @DenyAll. However suppose you do not have access to the source code and have received the EJB from a third party. If you in turn do not want your clients accessing a given method then you would need to use the <exclude-list> element in the ejb-jar.xml deployment descriptor: <?xml version="1.0" encoding="UTF-8"?> <ejb-jar version="3.0"                         xsi_schemaLocation="http://java.sun.com/xml/ns/javaee             http://java.sun.com/xml/ns/javaee/ejb-jar_3_0.xsd"> <enterprise-beans> <session> <ejb-name>BankServiceBean</ejb-name> </session> </enterprise-beans> <assembly-descriptor> <exclude-list><method> <ejb-name>BankServiceBean</ejb-name> <method-name>updateCustomer</method-name></method></exclude-list> </assembly-descriptor> </ejb-jar> EJB Security Propagation Suppose a client with an associated role invokes, for example, EJB A. If EJB A then invokes, for example, EJB B then by default the client's role is propagated to EJB B. However, you can specify with the @RunAs annotation that all methods of an EJB execute under a specific role. For example, suppose the addCustomer() method in the BankServiceBean EJB invokes the addAuditMessage() method of the AuditServiceBean EJB: @Stateless @RolesAllowed("bankemployee") public class BankServiceBean implements BankService { private @EJB AuditService audit; ....      public void addCustomer(int custId, String firstName,                                                          String lastName) {              cust = new Customer();              cust.setId(custId);              cust.setFirstName(firstName);              cust.setLastName(lastName);              em.persist(cust);              audit.addAuditMessage(1, "customer add attempt");      }      ... } Note that only a client with an associated role of bankemployee can invoke addCustomer(). If we prefix the AuditServiceBean class declaration with @RunAs("bankauditor") then the container will run any method in AuditServiceBean as the bankauditor role, regardless of the role which invokes the method. Note that the @RunAs annotation is applied only at the class level, @RunAs cannot be applied at the method level. @Stateless @RunAs("bankauditor") public class AuditServiceBean implements AuditService { @PersistenceContext(unitName="BankService") private EntityManager em; @TransactionAttribute( TransactionAttributeType.REQUIRES_NEW) public void addAuditMessage (int auditId, String message) { Audit audit = new Audit(); audit.setId(auditId); audit.setMessage(message); em.persist(audit); } } Programmatic Authorization With programmatic authorization the bean rather than the container controls authorization. The javax.ejb.SessionContext object provides two methods which support programmatic authorization: getCallerPrincipal() and isCallerInRole(). The getCallerPrincipal() method returns a java.security.Principal object. This object represents the caller, or principal, invoking the EJB. We can then use the Principal.getName() method to obtain the name of the principal. We have done this in the addAccount() method of the BankServiceBean as follows: Principal cp = ctx.getCallerPrincipal(); System.out.println("getname:" + cp.getName()); The isCallerInRole() method checks whether the principal belongs to a given role. For example, the code fragment below checks if the principal belongs to the bankcustomer role. If the principal does not belong to the bankcustomer role, we only persist the account if the balance is less than 99. if (ctx.isCallerInRole("bankcustomer")) {     em.persist(ac); } else if (balance < 99) {            em.persist(ac);   } When using the isCallerInRole() method, we need to declare all the security role names used in the EJB code using the class level @DeclareRoles annotation: @DeclareRoles({"bankemployee", "bankcustomer"}) The code below shows the BankServiceBean EJB with all the programmatic authorization code described in this section: package ejb30.session; import javax.ejb.Stateless; import javax.persistence.EntityManager; import ejb30.entity.Account; import javax.persistence.PersistenceContext; import javax.annotation.security.RolesAllowed; import java.security.Principal; import javax.annotation.Resource; import javax.ejb.SessionContext; import javax.annotation.security.DeclareRoles; import java.util.*; @Stateless @DeclareRoles({"bankemployee", "bankcustomer"}) public class BankServiceBean implements BankService { @PersistenceContext(unitName="BankService") private EntityManager em; private Account ac; @Resource SessionContext ctx; @RolesAllowed({"bankemployee", "bankcustomer"}) public void addAccount(int accountId, double balance, String accountType) { ac = new Account(); ac.setId(accountId); ac.setBalance(balance); ac.setAccountType(accountType); Principal cp = ctx.getCallerPrincipal(); System.out.println("getname:" + cp.getName()); if (ctx.isCallerInRole("bankcustomer")) { em.persist(ac); } else if (balance < 99) { em.persist(ac); } } ..... } Where we have a choice declarative authorization is preferable to programmatic authorization. Declarative authorization avoids having to mix business code with security management code. We can change a bean's security policy by simply changing an annotation or deployment descriptor instead of modifying the logic of a business method. However, some security rules, such as the example above of only persisting an account within a balance limit, can only be handled by programmatic authorization. Declarative security is based only on the principal and the method being invoked, whereas programmatic security can take state into consideration. Because an EJB is typically invoked from the web-tier by a servlet, JSP page or JSF component, we will briefly mention Java EE web container security. The web-tier and EJB tier share the same security model. So the web-tier security model is based on the same concepts of principals, roles and realms.

Writing a Package Its intents are: To shorten the time needed to set up everything before starting the real work, in other words the boiler-plate code To provide a standardized way to write packages To ease the use of a test-driven development approach To facilitate the releasing process It is organized in the following four parts: A common pattern for all packages that describes the similarities between all Python packages, and how distutils and setuptools play a central role How generative programming (http://en.wikipedia.org/wiki/Generative_programming) can help this through the template-based approach The package template creation, where everything needed to work is set Setting up a development cycle A Common Pattern for All Packages The easiest way to organize the code of an application is to split it into several packages using eggs. This makes the code simpler, and easier to understand, maintain, and change. It also maximizes the reusability of each package. They act like components. Applications for a given company can have a set of eggs glued together with a master egg. Therefore, all packages can be built using egg structures. This section presents how a namespaced package is organized, released, and distributed to the world through distutils and setuptools. Writing an egg is done by layering the code in a nested folder that provides a common prefix namespace. For instance, for the Acme company, the common namespace can be acme. The result is a namespaced package. For example, a package whose code relates to SQL can be called acme.sql. The best way to work with such a package is to create an acme.sql folder that contains the acme and then the sql folder: setup.py, the Script That Controls Everything The root folder contains a setup.py script, which defines all metadata as described in the distutils module, combined as arguments in a call to the standard setup function. This function was extended by the third-party library setuptools that provides most of the egg infrastructure. The boundary between distutils and setuptools is getting fuzzy, and they might merge one day. Therefore, the minimum content for this file is: from setuptools import setupsetup(name='acme.sql') name gives the full name of the egg. From there, the script provides several commands that can be listed with the -help-commands option. $ python setup.py --help-commands Standard commands:  build             build everything needed to install  ...  install           install everything from build directory  sdist         create a source distribution  register      register the distribution  bdist         create a built (binary) distributionExtra commands:  develop       install package in 'development mode'  ...  test          run unit tests after in-place build alias         define a shortcut  bdist_egg     create an "egg" distribution The most important commands are the ones left in the preceding listing. Standard commands are the built-in commands provided by distutils, whereas Extra commands are the ones created by third-party packages such as setuptools or any other package that defines and registers a new command. sdist The sdist command is the simplest command available. It creates a release tree where everything needed to run the package is copied. This tree is then archived in one or many archived files (often, it just creates one tar ball). The archive is basically a copy of the source tree. This command is the easiest way to distribute a package from the target system independently. It creates a dist folder with the archives in it that can be distributed. To be able to use it, an extra argument has to be passed to setup to provide a version number. If you don't give it a version value, it will use version = 0.0.0: from setuptools import setupsetup(name='acme.sql', version='0.1.1' This number is useful to upgrade an installation. Every time a package is released, the number is raised so that the target system knows it has changed. Let's run the sdist command with this extra argument: $ python setup.py sdistrunning sdist...creating disttar -cf dist/acme.sql-0.1.1.tar acme.sql-0.1.1gzip -f9 dist/acme.sql-0.1.1.tarremoving 'acme.sql-0.1.1' (and everything under it)$ ls dist/acme.sql-0.1.1.tar.gz Under Windows, the archive will be a ZIP file. The version is used to mark the name of the archive, which can be distributed and installed on any system having Python. In the sdist distribution, if the package contains C libraries or extensions, the target system is responsible for compiling them. This is very common for Linux-based systems or Mac OS because they commonly provide a compiler. But it is less usual to have it under Windows. That's why a package should always be distributed with a pre-built distribution as well, when it is intended to run under several platforms. The MANIFEST.in File When building a distribution with sdist, distutils browse the package directory looking for files to include in the archive. distutils will include: All Python source files implied by the py_modules, packages, and scripts option All C source files listed in the ext_modules option Files that match the glob pattern test/test*.py README, README.txt, setup.py, and setup.cfg files Besides, if your package is under Subversion or CVS, sdist will browse folders such as .svn to look for files to include .sdist builds a MANIFEST file that lists all files and includes them into the archive. Let's say you are not using these version control systems, and need to include more files. Now, you can define a template called MANIFEST.in in the same directory as that of setup.py for the MANIFEST file, where you indicate to sdist which files to include. This template defines one inclusion or exclusion rule per line, for example: include HISTORY.txtinclude README.txtinclude CHANGES.txtinclude CONTRIBUTORS.txtinclude LICENSErecursive-include *.txt *.py The full list of commands is available at http://docs.python.org/dist/sdist-cmd.html#sdist-cmd. build and bdist To be able to distribute a pre-built distribution, distutils provide the build command, which compiles the package in four steps: build_py: Builds pure Python modules by byte-compiling them and copying them into the build folder. build_clib: Builds C libraries, when the package contains any, using Python compiler and creating a static library in the build folder. build_ext: Builds C extensions and puts the result in the build folder like build_clib. build_scripts: Builds the modules that are marked as scripts. It also changes the interpreter path when the first line was set (!#) and fixes the file mode so that it is executable. Each of these steps is a command that can be called independently. The result of the compilation process is a build folder that contains everything needed for the package to be installed. There's no cross-compiler option yet in the distutils package. This means that the result of the command is always specific to the system it was build on. Some people have recently proposed patches in the Python tracker to make distutils able to cross-compile the C parts. So this feature might be available in the future. When some C extensions have to be created, the build process uses the system compiler and the Python header file (Python.h). This include file is available from the time Python was built from the sources. For a packaged distribution, an extra package called python-dev often contains it, and has to be installed as well. The C compiler used is the system compiler. For Linux-based system or Mac OS X, this would be gcc. For Windows, Microsoft Visual C++ can be used (there's a free command-line version available) and the open-source project MinGW as well. This can be configured in distutils. The build command is used by the bdist command to build a binary distribution. It calls build and all dependent commands, and then creates an archive in the same was as sdist does. Let's create a binary distribution for acme.sql under Mac OS X: $ python setup.py bdistrunning bdistrunning bdist_dumbrunning build...running install_scriptstar -cf dist/acme.sql-0.1.1.macosx-10.3-fat.tar .gzip -f9 acme.sql-0.1.1.macosx-10.3-fat.tarremoving 'build/bdist.macosx-10.3-fat/dumb' (and everything under it)$ ls dist/acme.sql-0.1.1.macosx-10.3-fat.tar.gz    acme.sql-0.1.1.tar.gz Notice that the newly created archive's name contains the name of the system and the distribution it was built under (Mac OS X 10.3). The same command called under Windows will create a specific distribution archive: C:acme.sql> python.exe setup.py bdist...C:acme.sql> dir dist25/02/2008  08:18     <DIR>          .25/02/2008  08:18     <DIR>          ..25/02/2008  08:24             16 055 acme.sql-0.1.win32.zip               1 File(s)          16 055 bytes               2 Dir(s)   22 239 752 192 bytes free If a package contains C code, apart from a source distribution, it's important to release as many different binary distributions as possible. At the very least, a Windows binary distribution is important for those who don't have a C compiler installed. A binary release contains a tree that can be copied directly into the Python tree. It mainly contains a folder that is copied into Python's site-packages folder. bdist_egg The bdist_egg command is an extra command provided by setuptools. It basically creates a binary distribution like bdist, but with a tree comparable to the one found in the source distribution. In other words, the archive can be downloaded, uncompressed, and used as it is by adding the folder to the Python search path (sys.path). These days, this distribution mode should be used instead of the bdist-generated one. install The install command installs the package into Python. It will try to build the package if no previous build was made and then inject the result into the Python tree. When a source distribution is provided, it can be uncompressed in a temporary folder and then installed with this command. The install command will also install dependencies that are defined in the install_requires metadata. This is done by looking at the packages in the Python Package Index (PyPI). For instance, to install pysqlite and SQLAlchemy together with acme.sql, the setup call can be changed to: from setuptools import setupsetup(name='acme.sql', version='0.1.1',      install_requires=['pysqlite', 'SQLAlchemy']) When we run the command, both dependencies will be installed. How to Uninstall a Package The command to uninstall a previously installed package is missing in setup.py. This feature was proposed earlier too. This is not trivial at all because an installer might change files that are used by other elements of the system. The best way would be to create a snapshot of all elements that are being changed, and a record of all files and directories created. A record option exists in install to record all files that have been created in a text file: $ python setup.py install --record installation.txtrunning install...writing list of installed files to 'installation.txt' This will not create any backup on any existing file, so removing the file mentioned might break the system. There are platform-specific solutions to deal with this. For example, distutils allow you to distribute the package as an RPM package. But there's no universal way to handle it as yet. The simplest way to remove a package at this time is to erase the files created, and then remove any reference in the easy-install.pth file that is located in the sitepackages folder. develop setuptools added a useful command to work with the package. The develop command builds and installs the package in place, and then adds a simple link into the Python site-packages folder. This allows the user to work with a local copy of the code, even though it's available within Python's site-packages folder. All packages that are being created are linked with the develop command to the interpreter. When a package is installed this way, it can be removed specifically with the -u option, unlike the regular install: $ sudo python setup.py developrunning develop...Adding iw.recipe.fss 0.1.3dev-r7606 to easy-install.pth fileInstalled /Users/repos/ingeniweb.sourceforge.net/iw.recipe.fss/trunkProcessing dependencies ...$ sudo python setup.py develop -urunning developRemoving...Removing iw.recipe.fss 0.1.3dev-r7606 from easy-install.pth file Notice that a package installed with develop will always prevail over other versions of the same package installed. test Another useful command is test. It provides a way to run all tests contained in the package. It scans the folder and aggregates the test suites it finds. The test runner tries to collect tests in the package but is quite limited. A good practice is to hook an extended test runner such as zope.testing or Nose that provides more options. To hook Nose transparently to the test command, the test_suite metadata can be set to 'nose.collector' and Nose added in the test_requires list: setup(...test_suite='nose.collector',test_requires=['Nose'],...) register and upload To distribute a package to the world, two commands are available: register: This will upload all metadata to a server. upload: This will upload to the server all archives previously built in the dist folder. The main PyPI server, previously named the Cheeseshop, is located at http://pypi.python.org/pypi and contains over 3000 packages from the community. It is a default server used by the distutils package, and an initial call to the register command will generate a .pypirc file in your home directory. Since the PyPI server authenticates people, when changes are made to a package, you will be asked to create a user over there. This can also be done at the prompt: $ python setup.py registerrunning register...We need to know who you are, so please choose either: 1. use your existing login, 2. register as a new user, 3. have the server generate a new password for you (and email it toyou), or 4. quitYour selection [default 1]: Now, a .pypirc file will appear in your home directory containing the user and password you have entered. These will be used every time register or upload is called: [server-index]username: tarekpassword: secret There is a bug on Windows with Python 2.4 and 2.5. The home directory is not found by distutils unless a HOME environment variable is added. But, this has been fixed in 2.6. To add it, use the technique where we modify the PATH variable. Then add a HOME variable for your user that points to the directory returned by os.path.expanduser('~'). When the download_url metadata or the url is specified, and is a valid URL, the PyPI server will make it available to the users on the project web page as well. Using the upload command will make the archive directly available at PyPI, so the download_url can be omitted: Distutils defines a Trove categorization (see PEP 301: http://www.python.org/dev/peps/pep-0301/#distutils-trove-classification) to classify the packages, such as the one defined at Sourceforge. The trove is a static list that can be found at http://pypi.python.org/pypi?%3Aaction=list_classifiers, and that is augmented from time to time with a new entry. Each line is composed of levels separated by "::": ...Topic :: TerminalsTopic :: Terminals :: SerialTopic :: Terminals :: TelnetTopic :: Terminals :: Terminal Emulators/X TerminalsTopic :: Text Editors Topic :: Text Editors :: DocumentationTopic :: Text Editors :: Emacs... A package can be classified in several categories, which can be listed in the classifiers meta-data. A GPL package that deals with low-level Python code (for instance) can use: Programming Language :: PythonTopic :: Software Development :: Libraries :: Python ModulesLicense :: OSI Approved :: GNU General Public License (GPL) Python 2.6 .pypirc Format The .pypirc file has evolved under Python 2.6, so several users and their passwords can be managed along with several PyPI-like servers. A Python 2.6 configuration file will look somewhat like this: [distutils]index-servers =    pypi    alternative-server    alternative-account-on-pypi[pypi]username:tarekpassword:secret[alternative-server]username:tarekpassword:secretrepository:http://example.com/pypi The register and upload commands can pick a server with the help of the -r option, using the repository full URL or the section name: # upload to http://example.com/pypi$ python setup.py sdist upload -r   alternative-server#  registers with default account (tarek at pypi)$ python setup.py register#  registers to http://example.com$ python setup.py register -r http://example.com/pypi This feature allows interaction with servers other than PyPI. When dealing with a lot of packages that are not to be published at PyPI, a good practice is to run your own PyPI-like server. The Plone Software Center (see http://plone.org/products/plonesoftwarecenter) can be used, for example, to deploy a web server that can interact with distutils upload and register commands. Creating a New Command distutils allows you to create new commands, as described in http://docs.python.org/dist/node84.html. A new command can be registered with an entry point, which was introduced by setuptools as a simple way to define packages as plug-ins. An entry point is a named link to a class or a function that is made available through some APIs in setuptools. Any application can scan for all registered packages and use the linked code as a plug-in. To link the new command, the entry_points metadata can be used in the setup call: setup(name="my.command",          entry_points="""             [distutils.commands]             my_command = my.command.module.Class          """) All named links are gathered in named sections. When distutils is loaded, it scans for links that were registered under distutils.commands. This mechanism is used by numerous Python applications that provide extensibility. setup.py Usage Summary There are three main actions to take with setup.py: Build a package. Install it, possibly in develop mode. Register and upload it to PyPI. Since all the commands can be combined in the same call, some typical usage patterns are: # register the package with PyPI, creates a source and# an egg distribution, then upload them$ python setup.py register sdist bdist_egg upload# installs it in-place, for development purpose$ python setup.py develop# installs it$ python setup.py install The alias Command To make the command line work easily, a new command has been introduced by setuptools called alias. In a file called setup.cfg, it creates an alias for a given combination of commands. For instance, a release command can be created to perform all actions needed to upload a source and a binary distribution to PyPI: $ python setup.py alias release register sdist bdist_egg uploadrunning aliasWriting setup.cfg$ python setup.py release... Other Important Metadata Besides the name and the version of the package being distributed, the most important arguments setup can receive are: description: A few sentences to describe the package long_description: A full description that can be in reStructuredText keywords: A list of keywords that define the package author: The author's name or organization author_email: The contact email address url: The URL of the project license: The license (GPL, LGPL, and so on) packages: A list of all names in the package; setuptools provides a small function called find_packages that calculates this namespace_packages: A list of namespaced packages A completed setup.py file for acme.sql would be: import osfrom setuptools import setup, find_packagesversion = '0.1.0'README = os.path.join(os.path.dirname(__file__), 'README.txt')long_description = open(README).read() + 'nn'setup(name='acme.sql',      version=version,      description=("A package that deals with SQL, "                    "from ACME inc"),      long_description=long_description,      classifiers=[        "Programming Language :: Python",        ("Topic :: Software Development :: Libraries ::          "Python Modules"),        ],      keywords='acme sql',      author='Tarek',      author_email='tarek@ziade.org',      url='http://ziade.org',      license='GPL',      packages=find_packages(),      namespace_packages=['acme'],      install_requires=['pysqlite','SQLAchemy']      ) The two comprehensive guides to keep under your pillow are: The distutils guide at http://docs.python.org/dist/dist.html The setuptools guide at http://peak.telecommunity.com/DevCenter/setuptools 

The reader should note that the solution is only intended to explain how Python and STAF may be used. No claim is made that the solution presented here is the best one in any way, just that is one more option that the reader may consider in future developments. The Problem Let's imagine that we have a computer network in which a machine periodically generates some kind of file with information that is of interest to other machines in that network. For example, let's say that this file is a new software build of a product that must transferred to a group of remote machines, in which its functionality has to be tested to make sure it can be delivered to the client. The Python-only solution Sequential A simple solution to make the software build available to all the testing machines could be to copy it to a specific directory whenever a new file is available. For additional security, let's suppose that we're required to verify that the md5 sum for both original and destination files is equal to ensure that build file was copied correctly. If it is considered that /tmp is a good destination directory, then the following script will do the job: 1 #!/usr/bin/python 2 """ 3 Copy a given file to a list of destination machines sequentially 4 """ 5 6 import os, argparse 7 import subprocess 8 import logging 9 10 def main(args): 11 logging.basicConfig(level=logging.INFO, format="%(message)s") 12 13 # Calculate md5 sum before copyin the file 14 orig_md5 = run_command("md5sum %s" % args.file).split()[0] 15 16 # Copy the file to every requested machine and verify 17 # that md5 sum of the destination file is equal 18 # to the md5 sum of the original file 19 for machine in args.machines: 20 run_command("scp %s %s:/tmp/" % (args.file, machine)) 21 dest_md5 = run_command("ssh %s md5sum /tmp/%s" 22 % (machine, os.path.basename(args.file))).split()[0] 23 assert orig_md5 == dest_md5 24 25 def run_command(command_str): 26 """ 27 Run a given command and another process and return stdout 28 """ 29 logging.info(command_str) 30 return subprocess.Popen(command_str, stdout=subprocess.PIPE, 31 shell=True).communicate()[0] 32 33 if __name__ == "__main__": 34 parser = argparse.ArgumentParser(description=__doc__) 35 parser.add_argument("file", 36 help="File to copy") 37 parser.add_argument(metavar="machine", dest="machines", nargs="+", 38 help="List of machines to which file must be copied") 39 40 args = parser.parse_args() 41 args.file = os.path.realpath(args.file) 42 main(args) Here it is assumed that ssh keys have been exchanged between origin and destination machines for automatic authentication without human intervention. The script makes use of the Popen class in the subprocess python standard library. This powerful library provides the capability to launch new operating system processes and capture not only the result code, but also the standard output and error streams. However, it should be taken into account that the Popen class cannot be used to invoke commands on a remote machine by itself. However, as it can be seen in the code, ssh and related commands may be used to launch processes on remote machines when configured properly. For example, if the file of interest was STAF325-src.tar.gz (STAF 3.2.5 source) and the remote machines were 192.168.1.1 and 192.168.1.2, then the file would be copied using the copy.py script in the following way: $ ./copy.py STAF325-src.tar.gz 192.168.1.{1,2}md5sum STAF325-src.tar.gzscp STAF325-src.tar.gz 192.168.1.1:/tmp/ssh 192.168.1.1 md5sum /tmp/STAF325-src.tar.gzscp STAF325-src.tar.gz 192.168.1.2:/tmp/ssh 192.168.1.2 md5sum /tmp/STAF325-src.tar.gz Parallel What would happen if the files were copied in parallel? For this example, it might not make much sense given that probably the network is at bottleneck and there isn't any increase in performance. However, in the case of the md5sum operation, it's a waste of time waiting for the operation to complete on one machine while the other is essentially idle waiting for the next command. Clearly, it would be more interesting to make both machines do the job in parallel to take advantage of CPU cycles. A parallel implementation similar to the sequential one is displayed below: 1 #!/usr/bin/python 2 """ 3 Copy a given file to a list of destination machines in parallel 4 """ 5 6 import os, argparse 7 import subprocess 8 import logging 9 import threading 10 11 def main(args): 12 logging.basicConfig(level=logging.INFO, format="%(threadName)s: %(message)s") 13 orig_md5 = run_command("md5sum %s" % args.file).split()[0] 14 15 # Create one thread for machine 16 threads = [ WorkingThread(machine, args.file, orig_md5) 17 for machine in args.machines] 18 19 # Run all threads 20 for thread in threads: 21 thread.start() 22 23 # Wait for all threads to finish 24 for thread in threads: 25 thread.join() 26 27 class WorkingThread(threading.Thread): 28 """ 29 Thread that performs the copy operation for one machine 30 """ 31 def __init__(self, machine, orig_file, orig_md5): 32 threading.Thread.__init__(self) 33 34 self.machine = machine 35 self.file = orig_file 36 self.orig_md5 = orig_md5 37 38 def run(self): 39 # Copy file to remote machine 40 run_command("scp %s %s:/tmp/" % (self.file, self.machine)) 41 42 # Calculate md5 sum of the file copied at the remote machine 43 dest_md5 = run_command("ssh %s md5sum /tmp/%s" 44 % (self.machine, os.path.basename(self.file))).split()[0] 45 assert self.orig_md5 == dest_md5 46 47 def run_command(command_str): 48 """ 49 Run a given command and another process and return stdout 50 """ 51 logging.info(command_str) 52 return subprocess.Popen(command_str, stdout=subprocess.PIPE, 53 shell=True).communicate()[0] 54 55 if __name__ == "__main__": 56 parser = argparse.ArgumentParser(description=__doc__) 57 parser.add_argument("file", 58 help="File to copy") 59 parser.add_argument(metavar="machine", dest="machines", nargs="+", 60 help="List of machines to which file must be copied") 61 62 args = parser.parse_args() 63 args.file = os.path.realpath(args.file) 64 main(args) Here the same assumptions as in the sequential case are made. In this solution the work that was done inside the for loop is now implemented in the run method of a class that is inherited from threading.Thread class, which is a class that provides an easy way to create working threads such as the ones in the example. In this case, the output of the command, using the same arguments as in the previous example, is: $ ./copy_parallel.py STAF325-src.tar.gz 192.168.1.{1,2}MainThread: md5sum STAF325-src.tar.gzThread-1: scp STAF325-src.tar.gz 192.168.1.1:/tmp/Thread-2: scp STAF325-src.tar.gz 192.168.1.2:/tmp/Thread-2: ssh 192.168.1.2 md5sum /tmp/STAF325-src.tar.gzThread-1: ssh 192.168.1.1 md5sum /tmp/STAF325-src.tar.gz As it can be seen in the logs, md5sum command execution isn't necessarily executed in the same order as threads were created. This solution isn't much more complex than the sequential one, but it finishes earlier. Hence, in the case in which a CPU intensive task must be performed in every machine, the parallel solution will be more convenient since the small increment in coding complex will pay off in execution performance. The Python+STAF solution Sequential The solutions to the problem presented in the previous section are perfectly fine. However, some developers may find it cumbersome to write scripts from scratch using Popen class and desire to work with a platform with feature such as launching process on remote machines already implemented. That's were STAF (Software Testing Automation Framework) might be helpful. STAF is a framework that provides the ability to automate jobs specially, but not uniquely, for testing environments. STAF is implemented as a process which runs on every machine that provides services that may be used by clients to accomplish different tasks. For more information regarding STAF, please refer to the project homepage. The Python+STAF sequential version of the program that has been used as example throughout this article is below: 1 #!/usr/bin/python 2 """ 3 Copy a given file to a list of destination machines sequentially 4 """ 5 6 import os, argparse 7 import subprocess 8 import logging 9 import PySTAF 10 11 def main(args): 12 logging.basicConfig(level=logging.INFO, format="%(message)s") 13 handle = PySTAF.STAFHandle(__file__) 14 15 # Calculate md5 sum before copyin the file 16 orig_md5 = run_process_command(handle, "local", "md5sum %s" % args.file).split()[0] 17 18 # Copy the file to every requested machine and verify 19 # that md5 sum of the destination file is equal 20 # to the md5 sum of the original file 21 for machine in args.machines: 22 copy_file(handle, args.file, machine) 23 dest_md5 = run_process_command(handle, machine, "md5sum /tmp/%s" 24 % os.path.basename(args.file)).split()[0] 25 assert orig_md5 == dest_md5 26 27 handle.unregister() 28 29 def run_process_command(handle, location, command_str): 30 """ 31 Run a given command and another process and return stdout 32 """ 33 logging.info(command_str) 34 35 result = handle.submit(location, "PROCESS", "START SHELL COMMAND %s WAIT RETURNSTDOUT" 36 % PySTAF.STAFWrapData(command_str)) 37 assert result.rc == PySTAF.STAFResult.Ok 38 39 mc = PySTAF.unmarshall(result.result) 40 return mc.getRootObject()['fileList'][0]['data'] 41 42 def copy_file(handle, filename, destination): 43 """ 44 Run a given command and another process and return stdout 45 """ 46 logging.info("copying %s to %s" % (filename, destination)) 47 48 result = handle.submit("local", "FS", "COPY FILE %s TODIRECTORY /tmp TOMACHINE %s" 49 % (PySTAF.STAFWrapData(filename), 50 PySTAF.STAFWrapData(destination))) 51 assert result.rc == PySTAF.STAFResult.Ok 52 53 if __name__ == "__main__": 54 parser = argparse.ArgumentParser(description=__doc__) 55 parser.add_argument("file", 56 help="File to copy") 57 parser.add_argument(metavar="machine", dest="machines", nargs="+", 58 help="List of machines to which file must be copied") 59 60 args = parser.parse_args() 61 args.file = os.path.realpath(args.file) 62 main(args) The code makes use of PySTAF, a python library, which is shipped with the STAF software that provides the ability to interact with the framework as a client. The typical usage of the library may summarized as follows: Register a handle in STAF (line 13): The communication with the server process is managed using handles. A client must have a handle to be able to send requests to local and/or remote machines. Submit requests (lines 35 and 48): Once the handle is available at the client, the client can use it to submit requests to any location and service. The two basic services that are used in this example are PROCESS, which is used to launch processes on a machine the same way ssh was used in the python-only version of the example; and FS, which is used to copy files between different machines as scp was used in the python-only solution. Check result code (lines 37 and 51): After a request has been submitted, result code should be checked to make sure that there wasn't any communication or syntax problem. Unmarshall results (lines 39-40): When the standard output is captured, it must be unmarshalled before using it in python since responses are encoded in a language independent format. Unregister handle (line 27): When STAF isn't needed anymore, it's advisable to unregister the handle to free resources allocated to the client in the server. Compared with the python-only solution, the advantages of STAF aren't appreciable at first sight. The handler syntax isn't easier than creating Popen objects and we have to deal with marshalling when we previously were just parsing text. However, as a framework, if has to be taken into account that it is has a learning curve and has much more functionality to offer than this one that makes it worthwhile. Please bear with me until section 5, in which the STAX solution we'll be shown, with an example with a completely different approach to the problem. Using the script in this section, the output would be pretty much the same as the previous sequential example: $ ./staf_copy.py STAF325-src.tar.gz 192.168.1.{1,2}md5sum STAF325-src.tar.gzcopying STAF325-src.tar.gz to 192.168.1.1md5sum /tmp/STAF325-src.tar.gzcopying STAF325-src.tar.gz to 192.168.1.2md5sum /tmp/STAF325-src.tar.gz As in the previous section, the sequential solution suffers the same problems when CPU intensive tasks are to be performed. Hence, the same comments apply. Parallel When using STAF, the parallel solution requires the same changes that were explained before. That is, create a new class that inherits from threading.Thread and implement the working threads. The code below shows how this might be implemented: 1 #!/usr/bin/python 2 """ 3 Copy a given file to a list of destination machines in parallel 4 """ 5 6 import os, argparse 7 import subprocess 8 import logging 9 import threading 10 import PySTAF 11 12 def main(args): 13 logging.basicConfig(level=logging.INFO, format="%(threadName)s %(message)s") 14 handle = PySTAF.STAFHandle(__file__) 15 orig_md5 = run_process_command(handle, "local", "md5sum %s" % args.file).split()[0] 16 17 # Create one thread for machine 18 threads = [ WorkingThread(machine, args.file, orig_md5) 19 for machine in args.machines] 20 21 # Run all threads 22 for thread in threads: 23 thread.start() 24 25 # Wait for all threads to finish 26 for thread in threads: 27 thread.join() 28 29 handle.unregister() 30 31 class WorkingThread(threading.Thread): 32 """ 33 Thread that performs the copy operation for one machine 34 """ 35 def __init__(self, machine, orig_file, orig_md5): 36 threading.Thread.__init__(self) 37 38 self.machine = machine 39 self.file = orig_file 40 self.orig_md5 = orig_md5 41 self.handle = PySTAF.STAFHandle("%s:%s" % (__file__, self.getName())) 42 43 def run(self): 44 # Copy file to remote machine 45 copy_file(self.handle, self.file, self.machine) 46 47 # Calculate md5 sum of the file copied at the remote machine 48 dest_md5 = run_process_command(self.handle, self.machine, "md5sum /tmp/%s" 49 % os.path.basename(self.file)).split()[0] 50 assert self.orig_md5 == dest_md5 51 self.handle.unregister() 52 53 def run_process_command(handle, location, command_str): 54 """ 55 Run a given command and another process and return stdout 56 """ 57 logging.info(command_str) 58 59 result = handle.submit(location, "PROCESS", "START SHELL COMMAND %s WAIT RETURNSTDOUT" 60 % PySTAF.STAFWrapData(command_str)) 61 assert result.rc == PySTAF.STAFResult.Ok 62 63 mc = PySTAF.unmarshall(result.result) 64 return mc.getRootObject()['fileList'][0]['data'] 65 66 def copy_file(handle, filename, destination): 67 """ 68 Run a given command and another process and return stdout 69 """ 70 logging.info("copying %s to %s" % (filename, destination)) 71 72 result = handle.submit("local", "FS", "COPY FILE %s TODIRECTORY /tmp TOMACHINE %s" 73 % (PySTAF.STAFWrapData(filename), 74 PySTAF.STAFWrapData(destination))) 75 assert result.rc == PySTAF.STAFResult.Ok 76 77 if __name__ == "__main__": 78 parser = argparse.ArgumentParser(description=__doc__) 79 parser.add_argument("file", 80 help="File to copy") 81 parser.add_argument(metavar="machine", dest="machines", nargs="+", 82 help="List of machines to which file must be copied") 83 84 args = parser.parse_args() 85 args.file = os.path.realpath(args.file) 86 main(args) As it happened before, this solution is faster since it takes advantage of having multiple CPUs working on md5sum calculation instead of just one at a time. The output we get invoking the script could be: $ ./staf_copy_parallel.py STAF325-src.tar.gz 192.168.1.{1,2}MainThread md5sum STAF325-src.tar.gzThread-1 copying STAF325-src.tar.gz to 192.168.1.1Thread-2 copying STAF325-src.tar.gz to 192.168.1.2Thread-2 md5sum /tmp/STAF325-src.tar.gzThread-1 md5sum /tmp/STAF325-src.tar.gz This time it can be seen that md5sum calculation mustn't necessarily start in the same order as file copy operation. Once again, this solution is slightly more complex, but the gain in performance makes it convenient when dealing with tasks with high computational cost.    

Introduction SOA enablement wraps key application interfaces in services, and integrates it into the SOA. This largely leaves the existing application logic intact, minimizing changes and adding risk only to those components that needed restructuring work to become SOA-ready. While the interfaces are modernized, without subjecting the core application components to a lot of change, the high costs and the various legacy risks associated with the mainframe platform remain. In addition, the performance and scalability of the new interfaces needs to be well-specified and tested, and the additional load they place on the system should be included in any planned capacity upgrades, potentially increasing the overall costs. Reducing or eliminating the legacy mainframe costs and risks via re-host based modernization also helps customers to fund SOA enablement, and the re-architecture phases of legacy modernization, and lay the groundwork for these steps. SOA-enabling a re-hosted application is a much easier process on an open-systems-based, SOA-ready software stack, and a more efficient one as well in terms of system resource utilization and cost. Re-architecting selected components of a re-hosted application based on specific business needs is a lower risk approach than re-architecting the entire applications en masse, and the risk can be further reduced by ensuring that target re-hosting stack provides rugged and transparent integration between re-hosted services and new components. Keeping It Real: Selective re-architecture is all about maximizing ROI by focusing re-architecture investment in the areas with the best pay-off. Undertaking a change from one language or development paradigm to another shouldn't be undertaken lightly—the investment and risks need to be well understood and justified. It is the right investment for components that require frequent maintenance changes but are difficult to maintain, because of poor /structure and layered changes. The payback on re-architecture investment will come from reducing the cost of future maintenance. Similarly, components that need significant functional changes to meet new business requirements can benefit from substantial productivity increase after re-architecture to a more modern development framework with richer tools to support future changes. The payback comes from greater business agility and time-to-market improvements. On the other hand, well-structured and maintainable COBOL components that do not need extensive changes to meet business needs will have very little return to show for the significant re-architecture investment. Leaving them in COBOL on a modern, extensible platform saves significant re-architecture costs that can be invested elsewhere, reduces risk, and shortens payback time. These considerations can help to optimize ROI for medium to large modernization projects where components measure in hundreds or thousands and contain millions or tens of millions lines of code. Re-Hosting Based Modernization For many organizations, mainframe modernization has become a matter of 'how', and not 'if'. Numerous enterprises and public sector organizations choose re-hosting as the first tangible step in their legacy modernization program precisely because it delivers the best ROI in the fastest possible manner, and accelerates the move to SOA enablement and selective re-architecture. Oracle together with our services partners provides a comprehensive re-hosting-based modernization solution that many customers have leveraged for a successful migration of selected applications or complete mainframe environments ranging from a few hundred MIPS to well over 10,000 MIPS. Two key pillars support successful re-hosting projects: Optimal target environment that lowers the Total Cost of Ownership (TCO) by 50–80 percent and maintains mainframe-class Quality of Service (QoS) using open, extensible, SOA-ready, future-proof architecture Predictable, efficient projects delivered by our SI partners with proven methodologies and automated tools Optimal target environment provided by Oracle is powered by proven open systems software stack leveraging Oracle Database and Oracle Tuxedo for a rock-solid, mainframe-class transaction processing (TP) infrastructure closely matching mainframe requirements for online applications. Mainframe-compatible Transaction Processing: Support for IBM CICS or IMS TM applications in native COBOL or C/C++ language containers with mainframe-compatible TP features. RASP: Mainframe-class performance, reliability, and scalability provided by Oracle Real Application Clusters (RAC) and Tuxedo multi-node and multi-domain clustering for load-balancing and high availability despite failure of individual nodes or network links. Workload and System Management: End-to-end transaction and service monitoring to support 24X7 operations management provided by Oracle's Enterprise Manager Grid Control and Tuxedo System and Application Monitor. SOA Enablement and Integration: Extensibility with Web services using Oracle Services Architecture Leveraging Tuxedo (SALT), J2EE integration (using WebLogic-Tuxedo Connector (WTC), Enterprise Service Bus (ESB), Portal, and BPM technologies to enable easy integration of re-hosted applications into modern Service-Oriented Architectures (SOAs). Scalable Platforms and Commodity Hardware: Scalable, Linux/UNIX-based open systems from HP, Dell, Sun, and IBM, providing: Performance on a par with mainframe systems for most workloads at significantly reduced TCO Reliability and workload management similar to mainframe installations, including physical and logical partitioning Robust clustering technologies for high availability and fail-over capabilities within a data center or across the world The diagram below shows conceptual mapping of mainframe environment to compatible open systems infrastructure: Predictable, efficient projects delivered by leading SIs and key modernization specialists use risk-mitigation methodologies, and automated tools honed over numerous projects to address a complete range of Online, Batch, and Data architectures, and the various technologies used in them. These project methodologies and automated tools that support them encompass all phases of a migration project: Preliminary Assessment Study Application Asset Discovery and Analysis Application and Data Conversion (pilot or entire application portfolio) System and Application Integration Test Engineering Regression and Performance Testing Education and Training Operations Migration Switch-Over Combining a proven target architecture stack that is well-matched to the needs of mainframe applications with mature methodologies supported by automated tools has led to a large and growing number of successful re-hosting projects. There is a rising interest to leverage the re-hosting approach to mainframe application modernization, as a way to get off a mainframe fast, and with minimal risk, in a more predictable manner for large, business-critical applications evolved over a long term and multiple development teams. Re-hosting based modernization approach preserves an organizations long term investment in critical business logic and data without risking business operations or sacrificing the QoS, while enabling customers to: Reduce or eliminate mainframe maintenance costs, and/or defer upgrade costs, saving customers 50–80 percent of their annual maintenance and operations budget Increase productivity and flexibility in IT development and operations, protecting long-term investment through application modernization Speed up and simplify application integration via SOA, without losing transactional integrity and the high performance expected by the users The rest of this article explores the critical success factors and proven transformation architecture for re-hosting legacy applications and data, describes SOA integration options and considerations when SOA-enabling re-hosted applications, highlights key risk mitigation methodologies, and provides a foundation for the financial analysis and ROI model derived from over a hundred, mainframe re-hosting projects. Critical Success Factors in Mainframe Re-Hosting Companies considering a re-hosting-based modernization strategy that involves migrating some applications off the mainframe have to address a range of concerns, which can be summarized by the following questions: How to preserve the business logic of these applications and their valuable data? How to ensure that migrated applications continue to meet performance requirements? How to maintain scalability, reliability, transactional integrity, and other QoS attributes in an open system environment? How to migrate in phases, maintaining robust integration links between migrated and mainframe applications? How to achieve predictable, cost-effective results and ensure a low-risk project? Meeting these challenges requires a versatile and powerful application infrastructure—one that natively supports key mainframe languages and services, enables automated adaptation of application code, and delivers proven, mainframe-like QoS on open system platforms. For re-hosting to enable broader aspects of the modernization strategy, this infrastructure must also provide native Web services and ESB capabilities to rapidly integrate re-hosted applications as first-class services in an SOA. Equally important is a proven, risk-mitigation methodology, automated tools, and project services specifically honed to address automated conversion and adaptation of application code and data, supported by cross-platform test engineering and execution methodology, strong system and application integration expertise, and deep experience with operations migration and switch-over. Preserving Application Logic and Data The re-hosting approach depends on a mainframe-compatible transaction processing and application services platform supporting common mainframe languages such as COBOL and C, which preserves the original business logic and data for the majority of mainframe applications and avoids the risks and uncertainties of a re-write. A complete re-hosting solution provides native support for TP and Batch programs, leveraging an application server-based platform that provides container-based support for COBOL and C/C++ application services, and TP APIs similar to IBM CICS, IMS TM, or other mainframe TP monitors. Online Transaction Processing Environment Oracle Tuxedo is the most popular TP platform for open systems, as well as leading re-hosting platform that can run most of mainframe COBOL and C applications unchanged in container-based framework that combines common application server features, including health monitoring, fail-over, service virtualization, and dynamic load balancing critical to large-scale OLTP applications together with standard TP features, including transaction management and reliable coordination of distributed transactions (a.k.a. Two-Phase Commit or XA standard). It provides the highest possible performance and scalability, and has been recently benchmarked against a mainframe at over 100,000 transactions per second, with sub-second response time. Oracle Tuxedo supports common mainframe programming languages, that is, COBOL and C, and provides comprehensive TP features compatible with CICS and IMS TM, which makes it a preferred application platform choice for re-hosting CICS or IMS TM applications with minimal changes and risks. In the Tuxedo environment, COBOL or C business logic remains unchanged. The only adaptation required is automated mapping of CICS APIs (CICS EXEC calls) to equivalent Tuxedo API functions. This mapping typically leverages a pre-processor and a mapping library implemented on Tuxedo platform, and using a full range of Tuxedo APIs. The automated nature of pre-processing and comprehensive coverage provided by the library ensures that most CICS COBOL or C programs are easily transformed into Tuxedo services. Unlike other solutions that embed this transformation in their compiler coupled with a proprietary emulation run-time, Tuxedo-based solution provides this mapping as a compiler-independent source module, which can be easily extended as needed. The resultant code uses Tuxedo API at native speed, allowing it to reach tens of thousands of transactions per second, while taking advantage of all Tuxedo facilities. In a re-hosted application CICS transactions become Tuxedo services, registered for processing by Tuxedo server processes. These services can be deployed in a single machine or across multiple machines in a Tuxedo domain (SYSPLEX-like cluster.). The services are called by front-end Java, .Net, or Tuxedo/WS clients, or UI components (tn3270 or web-based converted 3270/BMS screens), or by other services in case of transaction linking. Deferred transactions are handled by Tuxedo's/Q component, which provides in-memory and persistent queuing services. The diagram below shows Oracle Tuxedo and its surrounding ecosystem of SOA, J2EE, ESB, CORBA, MQ, and Mainframe integration components:   User Interface Migration The UI elements in these programs are typically defined using CICS Basic Mapping Support (BMS) for 3270 "green screen" terminals. While it is possible to preserve these using tn3270 emulation, many customers in re-hosting projects choose to take advantage of automated conversion of BMS macros into JSP/HTML for Web UI. Supported by a specialized Javascript library, these Web screens mimic the appearance and the behavior of "green screens" in a web browser, including tab-based navigation and PF keys. These UI components can connect to re-hosted CICS transactions running as Tuxedo services using Oracle Jolt (Java client interface for Tuxedo), Weblogic-Tuxedo Connector (WTC), or Tuxedo's Web services gateway provided by Oracle Services Architecture Leveraging Tuxedo (SALT) product. The diagram on the next page depicts a target re-hosting architecture for a typical mainframe OLTP application. The architecture uses Tuxedo services to run re-hosted CICS programs and a web application server to run re-hosted BMS UI. The servlets or JSPs containing the HTML that defines the screens, connect with Tuxedo services via Oracle Jolt, WTC, or SALT. Customers using mainframe 4GLs or languages such as PL/I or Assembler frequently choose to convert these applications to COBOL or C/C++. The adaptation of CICS or IMS TM API calls is automated through a mapping layer, which minimizes overall changes for the development team and allows them to maintain the familiar applications. For more significant extensions and new capabilities, customers incrementally leverage Tuxedo's own APIs and facilities, or leverage a tightly-linked J2EE environment provided by the WebLogic Server, and even transparently make Web services calls. The optimal extensibility options depend on application needs, availability of Java or C/COBOL skills, and other factors.   Feature or Action CICS Verb Tuxedo API Communications Area DFHCOMMAREA Typed Buffer Transaction Request LINK tpcall Transaction Return RETURN tpreturn Transfer Control XCTL tpforward Allocate Storage GETMAIN tpalloc Queues READQ / WRITEQ TD,TS /Q tpenqueue / tpdequeue Begin new transaction START TRANID /Q and TMQFORWARD Abort transaction ISSUE ABEND tpreturn TPFAIL Commit or Rollback SYNCPOINT / SYNCPOINT ROLLBACK tpcommit / tpabort     Keeping it Real:For those familiar with CICS, this is a very short example of the CICS verbs. CICS has many functions, most of which either map natively to a similar Tuxedo API or are provided by migration specialists based on their extensive experience with such migrations. In summary, Tuxedo provides a popular platform for deploying, executing, and managing COBOL and C re-hosted transactional applications requiring any of the following OLTP and infrastructure services: Native, compiler-independent support for COBOL, C, or C++ Rich set of infrastructure services for managing and scaling diverse workloads Feature-set compatibility and inter-operability with IBM CICS and IMS/TM Two-Phase Commit (2PC) for managing transactions across multiple application domains and XA-compliant resource managers (databases, message queues) Guaranteed inter-application messaging and transactional queuing Transactional data access (using XA-compliant resource managers) with ACID qualities Services virtualization and dynamic load balancing Centralized management of multiple nodes in a domain, and across multiple domains Communications gateways for multiple traditional and modern communication protocols SOA Enablement through native Web services and ESB integration Workload Monitoring and Management An important aspect of the mainframe environment is workload monitoring and management, which provides information for effective performance analysis and capabilities that enable mainframe systems to achieve better throughput and responsiveness. Oracle's Tuxedo System and Application Monitor (TSAM) provides similar capabilities too. Define monitoring policies and patterns based on application requests, services, system servers such as gateways, bridges, and XA-defined stages of a distributed transaction Define SLA thresholds that can trigger a variety of events within Tuxedo event services including notifications, and instantiation of additional servers Monitor transactions on an end-to-end basis from a client call through all services across all domains involved in a client request Collect service statistics for all infrastructure components such as servers and gateways Detail time spent on IPC queues, waiting on network links, and time spent on subordinate services TSAM provides a built-in, central, web-based management and monitoring console, and an open framework for integration with third-party performance management tools. Batch Jobs Mainframe batch jobs are a response to a human 24-hour clock on which many businesses run. It includes beginning-of-period or end-of-period (day, week, month, quarter) processing for batched updates, reconciliation, reporting, statement generation, and similar applications. In some industries, external events tied to a fixed schedule such as intra-day, opening or closing trade in a stock exchange, drive specific processing needs. Batch applications are an equally important asset, and often need to be preserved and migrated as well. The batch environment uses Job Control Language (JCL) jobs managed and monitored by JES2 or JES3 (Job Entry System), which invoke one or more programs, access and manipulate large datasets and databases using sort and other specialized utilities, and often run under the control of a job scheduler such as CA-7/CA-11. JCL defines a series of job steps—a sequence of programs and utilities, specifies input and output files, and provides exception handling. Automated parsing and translation of JCL jobs to UNIX scripts such as Korn shell (ksh) or Perl, enables the overall structure of the job to remain the same, including job steps, classes, and exception handling. Standard shell processing is supplemented with required utilities such as SyncSort, and support for Generation Data Group (GDG) files. REXX/CLIST/PROC scripting environments on the mainframe are similarly converted to ksh or other scripting languages. Integration with Oracle Scheduler, or other job schedulers running in UNIX/Linux or Windows provides a rich set of calendar and event-based scheduling capabilities as well as dependency management similar to mainframe schedulers. In some cases, reporting done via batch jobs can be replaced using standard reporting packages such as Oracle BI Publisher. The diagram below shows a typical target re-hosting architecture for batch. It includes a scheduler to control and trigger batch jobs, scripting framework to support individual job scripts, and an application server execution framework for the batch COBOL or C programs. Unlike other solutions that run these programs directly as OS processes without the benefit of application server middleware, Oracle recommends using container-based middleware to provide higher reliability, availability, and monitoring to the batch programs. The target batch programs invoked by the scripts can also run directly as OS processes, but if mainframe-class management and monitoring similar to JES2 or JES3 environment is a requirement, these programs can run as services under Tuxedo, benefiting from the health monitoring, fail-over, load balancing, and other application server-like features it provides. Files and Databases When moving platforms (mainframe to open systems), the application and data have to be moved together. Data schemas and data stores need to be moved in a re-hosted mainframe modernization project just as with a re-architecture. The approach taken depends on the source data store. DB2 is the most straightforward, since DB2 and Oracle are both relational databases. In addition to migrating the data, customers sometimes choose to perform data cleansing, field extensions, merge columns, or other data maintenance practices leveraging the automated tooling that synchronizes all data changes with changes to the application's data access code. Mainframe DB2 DB2 is a predominant relational database on IBM mainframes. When migrating to Oracle Database, the migration approach is highly automated, and resolves all discrepancies between the two RDBMS in terms of field formats as well as error codes returned to applications, so as to maintain application behavior unchanged, including stored procedures if any. IMS IMS/DB (also known as DL/1) is a popular hierarchical database for older applications. Creating appropriate relational data schema for this data requires an understanding of the application access patterns so as to optimize the schema for best performance based on the most frequent access paths. To minimize code impact, a translation layer can be used at run-time to support IMS DB style data access from the application, and map it to appropriate SQL calls. This allows the applications to interface with the segments, now translated as DB2 UDB or ORACLE tables, without impacting application code and maintenance. VSAM VSAM files are used for keyed-sequential data access, and can be readily migrated to ISAM files or to Oracle Database tables wherever transactional integrity is required (XA features). Some customers also choose to migrate VSAM files to Oracle Database to provide accessibility from other distributed applications, or to simplify the re-engineering required to extend certain data fields or merge multiple data sources. Meeting Performance and Other QoS Requirements The mainframe's performance, reliability, scalability, manageability, and other QoS attributes have earned it pre-eminence for business-critical applications. How well do re-hosting solutions measure up against these characteristics? Earlier solutions based on IBM CICS emulators derived from development tools often did not measure up to the demands of mainframe workloads since they were never intended for true production environment and have not been exposed to large-scale applications. As a result, they have only been used for re-hosting small systems under 300 MIPS and not requiring any clustering or distributed workload handling. Oracle Tuxedo was built to scale ground up, to support high performance telecommunications operations. It has the distinction of being the only non-mainframe TP solution recognized for its mainframe-like performance, reliability, and QoS characteristics. Most large enterprise customers requiring such capabilities in distributed systems have traditionally relied on Tuxedo. Consistently rated by IDC and Gartner as the market leader, and predominant in non-mainframe OLTP applications, it has also become the preferred COBOL/C application platform and transaction engine for re-hosted mainframe applications requiring high performance and/or mission-critical availability and reliability. Reasons for the broad recognition of Tuxedo as the only mainframe-class application platform and transaction engine for distributed systems are based on mainframe-class performance, scalability, reliability, availability, and other QoS attributes proven in multiple customer deployments. The following table highlights some of these capabilities:   Reliability Availability Guaranteed messaging and transactional integrity Hardened code from 25 years of use in the world's largest transaction applications Transaction integrity across systems and domains through a two phase commit (XA) for all resources such as databases, queues, and so on. Proven in mainframe-to-mainframe transactions and messaging No single point of failure, 99.999% uptime with N+1/N+2 clusters Application services upgradeable in operation Self-monitoring, automated fail-over, datadriven routing for super high availability Centralized monitoring and management with clustered domains; automated, lights-out operations     Workload Management   Performance and Scalability   Resource management and prioritization across Tuxedo services Dynamic load balancing across domains based on load conditions Data-driven routing enables horizontally distributed database grids and differentiated QoS End-to-end monitoring of Tuxedo system and application services enables SLA enforcement Virtualization support enables spawning of Tuxedo servers on demand Parallel processing to maximize resource utilization with low latency code paths that provide sub-second response at any load Horizontal and vertical scaling of system resources yields linear performance increases Request multiplexing (synchronous and asynchronous) maximizes CPU utilization Proven in credit card authorizations at over 13.5K tps, and in telco billing at over 56K tps. Middleware of choice in HP, Fujitsu, Sun, IBM, and NEC TPC-C benchmarks    

Following are some of the components, we'll examine in Tapestry 5: The Grid component allows us to display different data in a fairly sophisticated table. We are going to use it to display our collection of celebrities. The BeanEditForm component greatly simplifies creating forms for accepting user input. We shall use it for adding a new celebrity to our collection. The DateField component provides an easy and attractive way to enter or edit the date. The FCKEditor component is a rich text editor, and it is as easy to incorporate into a Tapestry 5 web application, just as a basic TextField is. This is a third party component, and the main point here is to show that using a library of custom components in a Tapestry 5 application requires no extra effort. It is likely that a similar core component will appear in a future version of the framework. Grid Component It is possible to display our collection of celebrities with the help of the Loop component. It isn't difficult, and in many cases, that will be exactly the solution you need for the task at hand. But, as the number of displayed items grow (our collection grows) different problems may arise. We might not want to display the whole collection on one page, so we'll need some kind of pagination mechanism and some controls to enable navigation from page to page. Also, it would be convenient to be able to sort celebrities by first name, last name, occupation, and so on. All this can be achieved by adding more controls and more code to finally achieve the result that we want, but a table with pagination and sorted columns is a very common part of a user interface, and recreating it each time wouldn't be efficient. Thankfully, the Grid component brings with it plenty of ready to use functionality, and it is very easy to deal with. Open the ShowAll.tml template in an IDE of your choice and remove the Loop component and all its content, together with the surrounding table: <table width="100%"> <tr t_type="loop" t_source="allCelebrities" t:value="celebrity"> <td> <a href="#" t_type="PageLink" t_page="Details" t:context="celebrity.id"> ${celebrity.lastName} </a> </td> <td>${celebrity.firstName}</td> <td> <t:output t_format="dateFormat" t:value="celebrity.dateOfBirth"/> </td> <td>${celebrity.occupation}</td> </tr> </table> In place of this code, add the following line: <t:grid t_source="allCelebrities"/> Run the application, log in to be able to view the collection, and you should see the following result: Quite an impressive result for a single short line of code, isn't it? Not only are our celebrities now displayed in a neatly formatted table, but also, we can sort the collection by clicking on the columns' headers. Also note that occupation now has only the first character capitalized—much better than the fully capitalized version we had before. Here, we see the results of some clever guesses on Tapestry's side. The only required parameter of the Grid component is source, the same as the required parameter of the Loop component. Through this parameter, Grid receives a number of objects of the same class. It takes the first object of this collection and finds out its properties. It tries to create a column for each property, transforming the property's name for the column's header (for example, lastName property name gives Last Name column header) and makes some additional sensible adjustments like changing the case of the occupation property values in our example. All this is quite impressive, but the table, as it is displayed now, has a number of deficiencies: All celebrities are displayed on one page, while we wanted to see how pagination works. This is because the default number of records per page for  Grid component is 25—more than we have in our collection at the moment. The last name of the celebrities does not provide a link to the Details page anymore. It doesn't make sense to show the Id column. The order of the columns is wrong. It would be more sensible to have the Last Name in the first column, then First Name, and finally the Date of Birth. By default, to define the display of the order of columns in the table, Tapestry will use the order in which getter methods are defined in the displayed class. In the Celebrity class, the getFirstName method is the first of the getters and so the First Name column will go first, and so on. There are also some other issues we might want to take care of, but let's first deal with these four. Tweaking the Grid First of all let's change the number of records per page. Just add the following parameter to the component's declaration: <t:grid t_source="allCelebrities" rowsPerPage="5"/> Run the application, and here is what you should see: You can now easily page through the records using the attractive pager control that appeared at the bottom of the table. If you would rather have the pager at the top, add another parameter to the Grid declaration: <t:grid t_source="allCelebrities" rowsPerPage="5" pagerPosition="top"/> You can even have two pagers, at the top and at the bottom, by specifying pagerPosition="both", or no pagers at all (pagerPosition="none"). In the latter case however, you will have to provide some custom way of paging through records. The next enhancement will be a link surrounding the celebrity's last name and linking to the Details page. We'll be adding an ActionLink and will need to know which Celebrity to link to, so we have the Grid store using the row parameter. This is how the Grid declaration will look: <t:grid t_source="allCelebrities" rowsPerPage="5" row="celebrity"/> As for the page class, we already have the celebrity property in it. It should have been left from our experiments with the Loop component. It will also be used in exactly the same way as with Loop, while iterating through the objects provided by its source parameter, Grid will assign the object that is used to display the current row to the celebrity property. The next thing to do is to tell Tapestry that when it comes to the contents of the Last Name column, we do not want Grid to display it in a default way. Instead, we shall provide our own way of displaying the cells of the table that contain the last name. Here is how we do this: <t:grid t_source="allCelebrities" rowsPerPage="5" row="celebrity"> <t:parameter name="lastNameCell"> <t:pagelink t_page="details" t_context="celebrity.id"> ${celebrity.lastName} </t:pagelink> </t:parameter> </t:grid> Here, the Grid component contains a special Tapestry element , similar to the one that we used in the previous chapter, inside the If component. As before, it serves to provide an alternative content to display, in this case, the content which will fill in the cells of the Last Name column. How does Tapestry know this? By the name of the element, lastNameCell. The first part of this name, lastName, is the name of one of the properties of the displayed objects. The last part, Cell, tells Tapestry that it is about the content of the table cells displaying the specified property. Finally, inside , you can see an expansion displaying the name of the current celebrity and surrounding it with the PageLink component that has for its context the ID of the current celebrity. Run the application, and you should see that we have achieved what we wanted: Click on the last name of a celebrity, and you should see the Details page with the appropriate details on it. All that is left now is to remove the unwanted Id column and to change the order of the remaining columns. For this, we'll use two properties of the Grid—remove and reorder. Modify the component's definition in the page template to look like this: <t:grid t_source="celebritySource" rowsPerPage="5" row="celebrity" remove="id" reorder="lastName,firstName,occupation,dateOfBirth"> <t:parameter name="lastNameCell"> <t:pagelink t_page="details" t_context="celebrity.id"> ${celebrity.lastName} </t:pagelink> </t:parameter> </t:grid> Please note that re-ordering doesn't delete columns. If you omit some columns while specifying their order, they will simply end up last in the table. Now, if you run the application, you should see that the table with a collection of celebrities is displayed exactly as we wanted: Changing the Column Titles Column titles are currently generated by Tapestry automatically. What if we want to have different titles? Say we want to have the title, Birth Date, instead of Date Of Birth. The easiest and the most efficient way to do this is to use the message catalog, the same one that we used while working with the Select component in the previous chapter. Add the following line to the app.properties file: dateOfBirth-label=Birth Date Run the application, and you will see that the column title has changed appropriately. This way, appending -label to the name of the property displayed by the column, you can create the key for a message catalog entry, and thus change the title of any column. Now you should be able to adjust the Grid component to most of the possible requirements and to display with its help many different kinds of objects. However, one scenario can still raise a problem. Add an output statement to the getAllCelebrities method in the ShowAll page class, like this: public List<Celebrity> getAllCelebrities() { System.out.println("Getting all celebrities..."); return dataSource.getAllCelebrities(); } The purpose of this is simply to be aware when the method is called. Run the application, log in, and as soon as the table with celebrities is shown, you will see the output, as follows: Getting all celebrities... The Grid component has the allCelebrities property defined as its source, so it invokes the getAllCelebrities method to obtain the content to display. Note however that Grid, after invoking this method, receives a list containing all 15 celebrities in collection, but displays only the first five. Click on the pager to view the second page—the same output will appear again. Grid requested for the whole collection again, and this time displayed only the second portion of five celebrities from it. Whenever we view another page, the whole collection is requested from the data source, but only one page of data is displayed. This is not too efficient but works for our purpose. Imagine, however, that our collection contains as many as 10,000 celebrities, and it's stored in a remote database. Requesting for the whole collection would put a lot of strain on our resources, especially if we are going to have 2,000 pages. We need to have the ability to request the celebrities, page-by-page—only the first five for the first page, only the second five for the second page and so on. This ability is supported by Tapestry. All we need to do is to provide an implementation of the GridDataSource interface. Here is a somewhat simplified example of such an implementation.

In this article, we will discuss the following topics: Need for Java Business Integration (JBI) Enterprise Service Bus Normalized Message Router Service Engines in NetBeans Need for Java Business Integration (JBI) To have a good understanding of Service Engines (a specific type of JBI component), we need to first understand the reason for Java Business Integration. In the business world, not all systems talk the same language. They use different protocols and different forms of communications. Legacy systems in particular can use proprietary protocols for external communication. The advent and acceptance of XML has been greatly beneficial in allowing systems to be easily integrated, but XML itself is not the complete solution. When some systems were first developed, they were not envisioned to be able to communicate with many other systems; they were developed with closed interfaces using closed protocols. This, of course, is fine for the system developer, but makes system integration very difficult. This closed and proprietary nature of enterprise systems makes integration between enterprise applications very difficult. To allow enterprise systems to effectively communicate between each other, system integrators would use vendor-supplied APIs and data formats or agree on common exchange mechanisms between their systems. This is fine for small short term integration, but quickly becomes unproductive as the number of enterprise applications to integrate gets larger. The following figure shows the problems with traditional integration. As we can see in the figure, each third party system that we want to integrate with uses a different protocol. As a system integrator, we potentially have to learn new technologies and new APIs for each system we wish to integrate with. If there are only two or three systems to integrate with, this is not really too much of a problem. However, the more systems we wish to integrate with, the more proprietary code we have to learn and integration with other systems quickly becomes a large problem. To try and overcome these problems, the Enterprise Application Integration (EAI) server was introduced. This concept has an integration server acting as a central hub. The EAI server traditionally has proprietary links to third party systems, so the application integrator only has to learn one API (the EAI server vendors). With this architecture however, there are still several drawbacks. The central hub can quickly become a bottleneck, and because of the hub-and-spoke architecture, any problems at the hub are rapidly manifested at all the clients. Enterprise Service Bus To help solve this problem, leading companies in the integration community (led by Sun Microsystems) proposed the Java Business Integration Specification Request (JSR 208) (Full details of the JSR can be found at http://jcp.org/en/jsr/detail?id=208). JSR 208 proposed a standard framework for business integration by providing a standard set of service provider interfaces (SPIs) to help alleviate the problems experienced with Enterprise Application Integration. The standard framework described in JSR 208 allows pluggable components to be added into a standard architecture and provides a standard common mechanism for each of these components to communicate with each other based upon WSDL. The pluggable nature of the framework described by JSR 208 is depicted in the following figure. It shows us the concept of an Enterprise Service Bus and introduces us to the Service Engine (SE) component: JSR 208 describes a service engine as a component, which provides business logic and transformation services to other components, as well as consuming such services. SEs can integrate Java-based applications (and other resources), or applications with available Java APIs. Service Engine is a component which provides (and consumes) business logic and transformation services to other components. There are various Service Engines available, such as the BPEL service engine for orchestrating business processes, or the Java EE service engine for consuming Java EE Web Services. The Normalized Message Router As we can see from the previous figure, SE's don't communicate directly with each other or with the clients, instead they communicate via the NMR. This is one of the key concepts of JBI, in that it promotes loose coupling of services. So, what is NMR and what is its purpose? NMR is responsible for taking messages from clients and routing them to the appropriate Service Engines for processing. (This is not strictly true as there is another standard JBI component called the Binding Component responsible for receiving client messages. Again, this further enhances the support for loose coupling within JBI, as Service Engines are decoupled from their transport infrastructure). NMR is responsible for passing normalized (that is based upon WSDL) messages between JBI components. Messages typically consist of a payload and a message header which contains any other message data required for the Service Engine to understand and process the message (for example, security information). Again, we can see that this provides a loosely coupled model in which Service Engines have no prior knowledge of other Service Engines. This therefore allows the JBI architecture to be flexible, and allows different component vendors to develop standard based components. Normalized Message Router enables technology for allowing messages to be passed between loosely coupled services such as Service Engines. The figure below gives an overview of the message routing between a client application and two service engines, in this case the EE and SQL service engines. In this figure, a request is made from the client to the JBI Container. This request is passed via NMR to the EE Service Engine. The EE Service Engine then makes a request to the SQL Service Engine via NMR. The SQL Service Engine returns a message to the EE Service Engine again via NMR. Finally, the message is routed back to the client through NMR and JBI framework. The important concept here is that NMR is a message routing hub not only between clients and service engines, but also for intra-communication between different service engines. The entire architecture we have discussed is typically referred to as an Enterprise Service Bus. Enterprise Service Bus (ESB) is a standard-based middleware architecture that allows pluggable components to communicate with each other via a messaging subsystem.

Starting the Project and Configuration We start by creating a new project for our chat room, with the project name DWRChatRoom. We also need to add the dwr.jar file to the lib directory and enable DWR in the web.xml file. The following is the source code of the dwr.xml file. <?xml ver sion="1.0" encoding="UTF-8"?> <!DOCTYPE dwr PUBLIC "-//GetAhead Limited//DTD Direct Web Remoting 2.0//EN" "http://getahead.org/dwr/dwr20.dtd"> <dwr> <allow> <create creator="new" javascript="Login"> <param name="class" value="chatroom.Login" /> </create> <create creator="new" javascript="ChatRoomDatabase"> <param name="class" value="chatroom.ChatRoomDatabase" /> </create> </allow> </dwr> The source code for web.xml is as follows: <?xml ver sion="1.0" encoding="UTF-8"?> <web-app xsi_schemaLocation="http://java. sun.com/xml/ns/javaee http://java.sun.com/xml/ns/javaee/web-app_2_ 5.xsd" id="WebApp_ID" version="2.5"> <display-name>DWRChatRoom</display-name> <servlet> <display-name>DWR Servlet</display-name> <servlet-name>dwr-invoker</servlet-name> <servlet-class> org.directwebremoting.servlet.DwrServlet </servlet-class> <init-param> <param-name>debug</param-name> <param-value>true</param-value> </init-param> <init-param> <param-name>activeReverseAjaxEnabled</param-name> <param-value>true</param-value> </init-param> </servlet> <servlet-mapping> <servlet-name>dwr-invoker</servlet-name> <url-pattern>/dwr/*</url-pattern> </servlet-mapping> <welcome-file-list> <welcome-file>index.html</welcome-file> <welcome-file>index.htm</welcome-file> <welcome-file>index.jsp</welcome-file> <welcome-file>default.html</welcome-file> <welcome-file>default.htm</welcome-file> <welcome-file>default.jsp</welcome-file> </welcome-file-list></web-app> Developing the User Interface The next step we do is to create files for presentation: style sheet and HTML/JSP files. The style sheet, loginFailed.html, and index.jsp files are required for the application. The source code of the style sheet is as follows: body{margin:0;padding:0;line-height: 1.5em;}b{font-size: 110%;}em{color: red;}#topsection{background: #EAEAEA;height: 90px; /*Height of top section*/}#topsection h1{margin: 0;padding-top: 15px;}#contentwrapper{float: left;width: 100%;}#contentcolumn{margin-left: 200px; /*Set left margin to LeftColumnWidth*/}#leftcolumn{float: left;width: 200px; /*Width of left column*/margin-left: -100%;background: #C8FC98;}#footer{clear: left;width: 100%;background: black;color: #FFF;text-align: center;padding: 4px 0;}#footer a{color: #FFFF80;}.innertube{margin: 10px; /*Margins for inner DIV inside each column (to provide padding)*/margin-top: 0;} Our first page is the login page. It is located in the WebContent directory and it is named index.jsp. The source code for the page is given as follows: <%@ page language="java" contentType="text/html; charset=ISO-8859-1"    pageEncoding="ISO-8859-1"%><!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd"><html><head><meta http-equiv="Content-Type" content="text/html; charset=ISO-8859-1"><title>Book Authoring</title><script type='text/javascript' src='/DWRChatroom/dwr/interface/Login.js'></script><script type='text/javascript' src='/DWRChatroom/dwr/engine.js'></script><script type='text/javascript' src='/DWRChatroom/dwr/util.js'></script>  <script type="text/javascript">function login(){  var userNameInput=dwr.util.byId('userName');  var userName=userNameInput.value;  Login.doLogin(userName,loginResult);}function loginResult(newPage){  window.location.href=newPage;}</script></head><body><h1>Book Authoring Sample</h1><table cellpadding="0" cellspacing="0"><tr><td>User name:</td><td><input id="userName" type="text" size="30"></td></tr><tr><td>&nbsp;</td><td><input type="button" value="Login" onclick="login();return false;"></td></tr></table></body></html> The login screen uses the DWR functionality to process the user login (the Java classes are presented after the web pages). The loginResults function opens either the failure page or the main page based on the result of the login operation. If the login was unsuccessful, a very simple loginFailed.html page is shown to the user, the source code for which is as follows: <!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd"><html><head><meta http-equiv="Content-Type" content="text/html;                                         charset=ISO-8859-1"><title>Login failed</title></head><body><h2>Login failed.</h2></body></html> The main page, mainpage.jsp, includes all the client-side logic of our ChatRoom application. The source code for the page is as follows: <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN""http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"><html lang="en" xml_lang="en"><head><meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1" /><title>Chatroom</title><link href="styles.css" rel="stylesheet" type="text/css" /><%   if (session.getAttribute("username") == null         || session.getAttribute("username").equals("")) {      //if not logged in and trying to access this page      //do nothing, browser shows empty page      return;   }%><script type='text/javascript' src='/DWRChatRoom/dwr/interface/Login.js'></script><script type='text/javascript' src='/DWRChatRoom/dwr/interface/ChatRoomDatabase.js'></script><script type='text/javascript' src='/DWRChatRoom/dwr/engine.js'></script><script type='text/javascript' src='/DWRChatRoom/dwr/util.js'></script>  <script type="text/javascript">dwr.engine.setActiveReverseAjax(true);function logout(){  Login.doLogout(showLoginScreen);}function showLoginScreen(){  window.location.href='index.jsp';}function showUsersOnline(){  var cellFuncs = [          function(user) {             return '<i>'+user+'</i>';          }          ];    Login.getUsersOnline({    callback:function(users)     {      dwr.util.removeAllRows('usersOnline');      dwr.util.addRows( "usersOnline",users, cellFuncs,                                    { escapeHtml:false });    }    });}function getPreviousMessages(){    ChatRoomDatabase.getChatContent({    callback:function(messages)     {      var chatArea=dwr.util.byId('chatArea');      var html="";      for(index in messages)      {         var msg=messages[index];         html+=msg;      }      chatArea.innerHTML=html;      var chatAreaHeight = chatArea.scrollHeight;      chatArea.scrollTop = chatAreaHeight;    }    });}function newMessage(message){  var chatArea=dwr.util.byId('chatArea');  var oldMessages=chatArea.innerHTML;  chatArea.innerHTML=oldMessages+message;    var chatAreaHeight = chatArea.scrollHeight;  chatArea.scrollTop = chatAreaHeight;}function sendMessageIfEnter(event){  if(event.keyCode == 13)  {    sendMessage();  }}function sendMessage(){    var message=dwr.util.byId('messageText');    var messageText=message.value;    ChatRoomDatabase.postMessage(messageText);    message.value='';}</script></head><body onload="showUsersOnline();"><div id="maincontainer"><div id="topsection"><div class="innertube"><h1>Chatroom</h1><h4>Welcome <i><%=(String) session.getAttribute("username")%></i></h4></div></div><div id="contentwrapper"><div id="contentcolumn"><div id="chatArea" style="width: 600px; height: 300px; overflow: auto"></div><div id="inputArea"><h4>Send message</h4><input id="messageText" type="text" size="50"  onkeyup="sendMessageIfEnter(event);"><input type="button" value="Send msg"                                                      onclick="sendMessage();"></div></div></div><div id="leftcolumn"><div class="innertube"><table cellpadding="0" cellspacing="0">  <thead>    <tr>      <td><b>Users online</b></td>    </tr>  </thead>  <tbody id="usersOnline">  </tbody></table><input id="logoutButton" type="button" value="Logout"  onclick="logout();return false;"></div></div><div id="footer">Stylesheet by <a  href="http://www.dynamicdrive.com/style/">Dynamic Drive CSSLibrary</a></div></div><script type="text/javascript">getPreviousMessages();</script></body></html> The first chat-room-specific JavaScript function is getPreviousMessages(). This function is called at the end of mainpage.jsp, and it retrieves previous chat messages for this chat room. The newMessage() function is called by the server-side Java code when a new message is posted to the chat room. The function also scrolls the chat area automatically to show the latest message. The sendMessageIfEnter() and sendMessage() functions are used to send user messages to the server. There is the input field for the message text in the HTML code, and the sendMessageIfEnter() function listens to onkeyup events in the input field. If the user presses enter, the sendMessage() function is called to send the message to the server. The HTML code includes the chat area of specified size and with automatic scrolling. Developing the Java Code There are several Java classes in the application. The Login class handles the user login and logout and also keeps track of the logged-in users. The source code of the Login class is as follows: package chatroom;import java.util.Collection;import java.util.List;import javax.servlet.ServletContext;import javax.servlet.http.HttpServletRequest;import javax.servlet.http.HttpSession;import org.directwebremoting.ScriptSession;import org.directwebremoting.ServerContext;import org.directwebremoting.ServerContextFactory;import org.directwebremoting.WebContext;import org.directwebremoting.WebContextFactory;import org.directwebremoting.proxy.ScriptProxy;public class Login {   public Login() {   }      public String doLogin(String userName) {      UserDatabase userDb=UserDatabase.getInstance();      if(!userDb.isUserLogged(userName)) {         userDb.login(userName);         WebContext webContext= WebContextFactory.get();         HttpServletRequest request = webContext.getHttpServletRequest();         HttpSession session=request.getSession();         session.setAttribute("username", userName);         String scriptId = webContext.getScriptSession().getId();         session.setAttribute("scriptSessionId", scriptId);         updateUsersOnline();         return "mainpage.jsp";      }      else {         return "loginFailed.html";      }   }      public void doLogout() {      try {         WebContext ctx = WebContextFactory.get();         HttpServletRequest request = ctx.getHttpServletRequest();         HttpSession session = request.getSession();         Util util = new Util();         String userName = util.getCurrentUserName(session);         UserDatabase.getInstance().logout(userName);         session.removeAttribute("username");         session.removeAttribute("scriptSessionId");         session.invalidate();      } catch (Exception e) {         System.out.println(e.toString());      }      updateUsersOnline();   }      private void updateUsersOnline() {      WebContext webContext= WebContextFactory.get();      ServletContext servletContext = webContext.getServletContext();      ServerContext serverContext = ServerContextFactory.get(servletContext);      webContext.getScriptSessionsByPage("");      String contextPath = servletContext.getContextPath();      if (contextPath != null) {         Collection<ScriptSession> sessions =                            serverContext.getScriptSessionsByPage                                            (contextPath + "/mainpage.jsp");         ScriptProxy proxy = new ScriptProxy(sessions);         proxy.addFunctionCall("showUsersOnline");      }   }      public List<String> getUsersOnline() {      UserDatabase userDb=UserDatabase.getInstance();      return userDb.getLoggedInUsers();   }} The following is the source code of the UserDatabase class package chatroom;import java.util.List;import java.util.Vector;//this class holds currently logged in users//there is no persistencepublic class UserDatabase {      private static UserDatabase userDatabase=new UserDatabase();      private List<String> loggedInUsers=new Vector<String>();      private UserDatabase() {}      public static UserDatabase getInstance() {      return userDatabase;   }      public List<String> getLoggedInUsers() {      return loggedInUsers;   }      public boolean isUserLogged(String userName) {      return loggedInUsers.contains(userName);    }      public void login(String userName) {      loggedInUsers.add(userName);   }      public void logout(String userName) {      loggedInUsers.remove(userName);   }} The Util class is used by the Login class, and it provides helper methods for the sample application. The source code for the Util class is as follows: package chatroom;import java.util.Hashtable;import java.util.Map;import javax.servlet.http.HttpServletRequest;import javax.servlet.http.HttpSession;import org.directwebremoting.WebContext;import org.directwebremoting.WebContextFactory;public class Util {      public Util() {         }      public String getCurrentUserName() {      //get user name from session      WebContext ctx = WebContextFactory.get();      HttpServletRequest request = ctx.getHttpServletRequest();      HttpSession session=request.getSession();      return getCurrentUserName(session);   }   public String getCurrentUserName(HttpSession session) {      String userName=(String)session.getAttribute("username");      return userName;   }} The logic for the server-side chat room functionality is in the ChatRoomDatabase class. The source code for the ChatRoomDatabase is as follows: package chatroom;import java.util.Collection;import java.util.Date;import java.util.List;import java.util.Vector;import javax.servlet.ServletContext;import org.directwebremoting.ScriptSession;import org.directwebremoting.ServerContext;import org.directwebremoting.ServerContextFactory;import org.directwebremoting.WebContext;import org.directwebremoting.WebContextFactory;import org.directwebremoting.proxy.ScriptProxy;public class ChatRoomDatabase {   private static List<String> chatContent = new Vector<String>();   public ChatRoomDatabase() {   }   public void postMessage(String message) {      String user = (new Util()).getCurrentUserName();      if (user != null) {         Date time = new Date();         StringBuffer sb = new StringBuffer();         sb.append(time.toString());         sb.append(" <b><i>");         sb.append(user);         sb.append("</i></b>:  ");         sb.append(message);         sb.append("<br/>");         String newMessage=sb.toString();         chatContent.add(newMessage);         postNewMessage(newMessage);      }   }   public List<String> getChatContent() {      return chatContent;   }   private ScriptProxy getScriptProxyForSessions() {      WebContext webContext = WebContextFactory.get();      ServletContext servletContext = webContext.getServletContext();      ServerContext serverContext = ServerContextFactory.get(servletContext);      webContext.getScriptSessionsByPage("");      String contextPath = servletContext.getContextPath();      if (contextPath != null) {         Collection<ScriptSession> sessions = serverContext               .getScriptSessionsByPage(contextPath + "/mainpage.jsp");         ScriptProxy proxy = new ScriptProxy(sessions);         return proxy;      }      return null;   }   public void postNewMessage(String newMessage) {      ScriptProxy proxy = getScriptProxyForSessions();      if (proxy != null) {         proxy.addFunctionCall("newMessage",newMessage);      }   }} The Chatroom code is surprisingly simple. The chat content is stored in a Vector of Strings. The getChatContent()method just returns the chat content Vector to the browser. The postMessage()method is called when the user sends a new chat message. The method verifies whether the user is logged in, and adds the current time and username to the chat message and then appends the message to the chat content. The method also calls the postNewMessage() method that is used to show new chat content to all logged-in users. Note that the postMessage() method does not return any value. We let DWR and reverse AJAX functionality show the chat message to all users, including the user who sent the message. The getScriptProxyForSessions() and postNewMessage() methods use reverse AJAX to update the chat areas of all logged-in users with the new message. And that is it! The chat room sample is very straightforward and basic functionality is already in place, and the application is ready for further development. Testing the Chat We test the chat room application with three users: Smith, Brown, and Jones. We have given some screenshots of a typical scenario in a chat room here. Both Smith and Brown log into the system and exchange some messages. Both users see empty chat rooms when they log in and start chatting. The empty area that is above the send message input field is reserved for chat content. Smith and Brown exchange some messages as is seen in the following screenshot: The third user, Jones, joins the chat and sees all the previous messages in the chat room. Jones then exchanges some messages with Smith and Brown. Smith and Brown log out from the system leaving Jones alone in the chat room (until she also logs out). This is visible in the following screenshot: Summary This sample application showed how to use DWR in a chat room application. This application makes it clear that DWR makes development of these kind of collaborative applications very easy. DWR itself does not even play a big part in the applications. DWR is just a transparent feature of the application. So developers can concentrate on the actual project and aspects such as persistence of data and a neat user interface, instead of the low-level details of AJAX.    

Modeling Business Processes The transparency of the process flow is crucial, as this gives the process owners, process analysts, and all others involved an insight into what is going on. An understanding of the as-is process flow also ensures that we can judge the efficiency and the quality of the process. The main objective of process modeling is the definition of the as-is process flow. Process modeling needs to answer the following questions: What is the outcome of the business process? What activities are performed within the business process? What is the order of activities? Who performs the activities? Which business documents are exchanged within the process? How foolproof is the process, and how can it be extended in the future? After answering these and some other questions, we get a good insight into how the process works. We can also identify structural, organizational, and technological weak points and even bottlenecks, and identify potential improvements to the process. We will model business process to satisfy the following objectives: To specify the exact result of the business process, and to understand the business value of this result. To understand the activities of the business process. Knowing the exact tasks and activities that have to be performed is crucial to understanding the details of the process. To understand the order of activities. Activities can be performed in sequence or in parallel, which can help improve the overall time required to fulfill a business process. Activities can be short-running or long-running. To understand the responsibilities, to identify (and later supervise) who is responsible for which activities and tasks. To understand the utilization of resources consumed in the business process. Knowing who uses which resources can help improve the utilization of resources as resource requirements can be planned for and optimized. To understand the relationship between people involved in the processes, and their communication. Knowing exactly who communicates with whom is important and can help to organize and optimize communications. To understand the document flow. Business processes produce and consume documents (regardless of whether these are paper or electronic documents). Understanding where the documents are going, and where they are coming from is important. A good overview of the documents also gives us the opportunity to identify whether all of the documents are really necessary. To identify potential bottlenecks and points of improvements, which can be used later in the process optimization phase. To introduce quality standards such as ISO 9001 more successfully, and to better pass certification. To improve the understandability of quality regulations that can be supplemented with process diagrams. To use business process models as work guidelines for new employees who can introduce themselves to the business processes faster and more efficiently. To understand business processes, which will enable us to understand and describe the company as a whole. A good understanding of business processes is very important for developing IT support. Applications that provide end-to-end support for business processes, can be developed efficiently only if we understand the business processes in details. Modeling Method and Notation Efficient process modeling requires a modeling method that provides a structured and controlled approach to process modeling. Several modeling methods have been developed over the years. Examples include IDS Sheer's the ARIS methodology, CSC's Catalyst, Business Genetics, SCOR and the extensions PCOR and VCOR, POEM, and so on. The ARIS methodology has been the most popular methodology, and has been adopted by many software vendors. In the next section, we will describe the basics of the ARIS methodology, which has lately been adapted to be conformant with SOA. ARIS ARIS is both a BPM methodology, and an architectural framework for designing enterprise architectures. Enterprise architecture combines business models (process models, organizational models, and so on) with IT models (IT architecture, data model, and so on). ARIS stands for Architecture of Integrated Information Systems and comprises of two things, the methodology and framework, and the software that supports both. Here, we will give a brief introduction to ARIS methodology and framework, which dates back to 1992. The objective of ARIS is to narrow the gap between business requirements and IT. The ARIS framework is not only about process models (describing business processes), although process models are one of the most important things of ARIS. As enterprise architecture is complex, ARIS defines several views that focus on specific aspects such as business, technology, information, and so on, to reduce the complexity. The ARIS framework describes the following: Business processes Products and services related to the processes The structure of the organization Business objectives and strategies Information flows IT architecture and applications The data model Resources (people and hardware resources) Costs Skills and knowledge These views are gathered under the concept of ARIS House, which provides a structured view on all information on business processes. ARIS House offers five views: The process view (also called the control view) is the central view that shows the behavior of the processes, how the processes relate to the products and services, organization, functions, and data. The process view includes the process models in the selected notation, and other diagrams such as information flow, material flow, value chains, communication diagrams, and so on. The product and service view shows the products and services, their structures, relations, and product/service trees. The organizational view shows the organizational structure of the company, including departments, roles, and employees. It shows these in hierarchical organizational charts. The organization view also shows technical resources and communication networks. The function view defines process tasks and describes business objectives, function hierarchies, and application software. The data view shows business data and information. This view includes data models, information maps, database models, and knowledge structures. The ARIS House is illustrated in the following figure: In ARIS House, the process view is the central view of the dynamic behavior of the business processes and brings together the other four static views, the organizational view, data view, function view and product/service view. In this book, we will focus primarily on the process view. Each ARIS view is divided further into phases. The translation of business requirements into IT applications requires that we follow certain phases. Globally, three general phases are likely to be used: Requirements phase Design specification phase Implementation phase ARIS is particularly strong in the requirements phase, while other phases may differ depending on the implementation method and the architecture we use. We will talk about these later in this article. Let us now look at the other important aspect, the business process modeling notations. Modeling Notation Process modeling also requires a notation In the past, several notations were used to model processes. Flow diagrams and block diagrams were representatives of the first-generation notations. Then, more sophisticated notations were defined, such as EPC (Event Process Chain) and eEPC (Extended Event Process Chain). UML activity diagrams, XPDL, and IDEF 3 were also used, in addition to some other less-known notations. A few years ago a new notation, called Business Process Modeling Notation (BPMN) was developed. BPMN was developed particularly for modeling business processes in accordance with SOA. In this article, we will use BPMN for modeling processes. BPMN BPMN is the most comprehensive notation for process modeling so far. It has been developed under the hood of OMG (Object Management Group). Let us look into the brief introduction of the most important BPMN elements so that we can read the diagrams presented later in this article. The most important goals while designing BPMN have been: To develop a notation, which will be understandable at all levels: In business process modeling different people are involved, from business users, business analysts, and process owners, to the technical architects and developers. The management reviews business processes at periodic intervals. Therefore, the goal of BPMN has been to provide a graphical notation the is simple to understand, yet powerful enough to model business processes at the required level of detail. To enable automatic transformation into executable code, that is, BPEL, and vice-versa: The gap between the business process models and the information technology (application software) has been quite large in existing technologies. There is no clear definition on how one relates to the other. Therefore, BPMN has been designed specifically to provide such transformations. To model the diagrams, BPMN defines four categories of elements: Flow objects, which are activities, events, and gateways. Activities can be tasks or sub-processes. Events can be triggers or results. Three types of events are supported: start, intermediate, and end. Gateways control the divergence of sequential flows into concurrent flows, and their convergence back to sequential flow. Connecting objects are used to connect flow objects together. Connectors are sequence flows, message flows, and associations. Swim lanes are used to organize activities into visual categories in order to illustrate different responsibilities or functional capabilities. Pools and lanes can be used for swim lanes. Artifacts are used to add specific context to the business processes that are being modeled. Data objects are used to show how data is produced or required by the process. Groups are used to group together similar activities or other elements. Annotations are used to add text information to the diagram. We can also define custom artifacts. The following diagrams show the various notations used in BPMN: Activities are the basic elements of BPMN and are represented by rectangles with rounded corners. A plus sign denotes that the activity can be further decomposed: Decisions are shown as diamonds. A plus sign inside the diamond denotes a logical AND, while an x denotes a logical OR: Events are shown as double circles: Roles are shown as pools and swim-lanes within pools: A Document is shown as follows: The order of activities is indicated by an arrow: The flow of a document or information is shown with a dashed line: BPMN can be used to model parts of processes or whole processes. Processes can be modeled at different levels of fidelity. BPMN is equally suitable for internal (private) business processes, and for public (collaborative) business-to-business processes. Internal business processes focus on the point of view of a single company, and define activities that are internal to the company. Such processes might also define interactions with external partners. Public collaborative processes show the interaction between all involved businesses and organizations. Such processes models should be modeled from the general point of view, and should show interactions between the participants. Process Design The main activity in process design is the recording of the actual processes. The objective is to develop the as-is process model. To develop the as-is model, it is necessary to gather all knowledge about the process. This knowledge often exists only in the heads of the employees, who are involved in the process. Therefore, it is necessary to perform detailed interviews with all involved people. Often, process supervisors might think that they know exactly how the process is performed. However, after talking with those employees who really carry out the work, they see that the actual situation differs considerably. It is very important to gather all this information about the process, otherwise it will not be possible to develop a sound process model, that reflects the as-is state of the process. The first question related to the as-is model is the business result that the process generates. Understanding the business result is crucial, as sometimes it may not be clearly articulated. After the business result is identified, we should understand the process flow. The process flow consists of activities (or tasks) that are performed in a certain order. The process flow is modeled at various levels of abstraction. At the highest level of abstraction, the process flow shows only the most important activities (usually up to ten). Each of the top-level activities are then decomposed into detailed flows. The process complexity, and the required level of detail, are the criteria that instruct us how deep we should decompose. To understand the process behavior completely, it makes sense to decompose until atomic activities (that is, activities that cannot be further decomposed) are reached. When developing the as-is process model, one of the most important things to consider is the level of detail. In order to provide end-to-end support for business processes using SOA, detailed process modeling should be done. The difficulties often hide in the details! In the process design, we should understand the detailed structure of the business process. Therefore, we should identify at least the following: Process activities at various levels of detail Roles responsible for carrying out each process activity Events that trigger the process execution and events that interrupt the process flow Documents exchanged within the process. This includes input documents and output documents Business rules that are part of the process We should design the usual (also called optimal) process flow and identify possible exception scenarios. Exceptions interrupt the usual process flow. Therefore, we need to specify how the exceptions will be handled. The usual approach to the process design includes the following steps: Identifying the roles Identifying the activities Connecting activities to roles Defining the order of activities Adding events Adding documents We should also understand the efficiency of the business process. This includes resource utilization, the time taken by involved employees, possible bottlenecks, and inefficiencies. This is the reason why we should also identify metrics that are used to measure the efficiency of the process. While some of these metrics may be KPIs, other metrics relevant to the process should also be identified. We should identify if the process is compliant with standards or reference processes. In some industry domains, reference processes have been defined. An example is the telecommunications industry where the TMF (Telecom Management Forum) has defined NGOSS. Part of NGOSS is eTom (Enhanced Telecom Operations Map), which specifies compliant business processes for telecom companies. Other industries have also started to develop similar reference processes. We should also identify the business goals to which the process contributes to. Business goals are the same as the process results. A business process should not only have at least one result, but should also contribute to at least one (preferably more than one) business goal. Here, we can look into the company strategy to identify the business goals. We should also identify the events that can interrupt the process flow. Each process can be interrupted, and we should understand how this happens. If a process is interrupted, we might need to compensate those activities of the process that have already been successfully completed. Therefore, we should also specify the compensation logic related to different interruption events. Finally, we should also understand the current software support for the business process. This is important because existing software may hide the details of process behavior. This information can also be re-used for end-to-end process support. Once we have identified all of these artifacts, we will have gathered a good understanding of the process. Therefore, let us now look at the results of the process modeling.

This application supports the single document interface, which implies that we have one document class object and one view class object. The other applications support the multiple document interface, they have one document class object and zero or more view class objects. The following screenshot depicts a classic example of the Tetris Application: We start by generating the application's skeleton code with The Application Wizard. The process is similar to the Ring application code. There is a small class Square holding the position of one square and a class ColorGrid managing the game grid. The document class manages the data of the game and handles the active (falling down) figure and the next (shown to the right of the game grid) figure. The view class accepts input from the keyboard and draws the figures and the game grid. The Figure class manages a single figure. It is responsible for movements and rotations. There are seven kinds of figures. The Figure Info files store information pertaining to their colors and shapes. The Tetris Files We start by creating a MFC application with the name Tetris and follow the steps of the Ring application. The classes CTetrisApp, CMainFrame, CTetrisDoc, CTetrisView, and CAboutDlg are then created and added to the project. There are only two differences. We need to state that we are dealing with a "Single Document Application Type", that the file extension is "Trs" and that the file type long name is "A Game of Tetris". Otherwise, we just accept the default settings. Note that in this application we accept the CView base class instead of the CScrollView like we did in the Ring application.     We add the marked lines below. In all other respects, we leave the file unmodified. We will not need to modify the files Tetris.h, MainFrm.h, MainFrm.cpp, StdAfx.h, StdAfx.cpp, Resource.h, and Tetris.rc. #include"stdafx.h"#include "Square.h"#include"Figure.h"#include "ColorGrid.h"#include"Tetris.h"#include "MainFrm.h"#include"TetrisDoc.h"#include "TetrisView.h"//... The Color Grid Class The ColorGrid handles the background game grid of twenty rows and twenty columns. Each square can have a color. At the beginning, every square is initialized to the default color white. The Index method is overloaded with a constant version that returns the color of the given square, and a non-constant version that returns a reference to the color. The latter version makes it possible to change the color of a square. ColorGrid.h classSquare{public:Square();Square(int iRow, int iCol);int Row() const {return m_iRow;}int Col() const {return m_iCol;}private:int m_iRow, m_iCol;}; There are two Index methods, the second one is intended to be called on a constant object. Both methods check that the given row and position have valid values. The checks are, however, for debugging purposes only. The methods are always called with valid values. Do not forget to include the file StdAfx.h. ColorGrid.cpp const int ROWS = 20;const int COLS = 10;classColorGrid{public:ColorGrid();void Clear();COLORREF&Index(int iRow, int iCol);const COLORREF Index(int iRow, int iCol)const;void Serialize(CArchive&archive);private:COLORREF m_buffer[ROWS * COLS];}; The Document Class CTetrisDoc is the document class of this application. When created, it overrides OnNewDocument and Serialize from its base class CDocument. We add to the CTetrisDoc class a number of fields and methods. The field m_activeFigure is active figure, that is the one falling down during the game. The field m_nextFigure is the next figure, that is the one showed in the right part of the game view. They both are copies of the objects in the m_figureArray, which is an array figure object. There is one figure object of each kind (one figure of each color). The integer list m_scoreList holds the ten top list of the game. It is loaded from the file ScoreList.txt by the constructor and saved by the destructor. The integer field m_iScore holds the score of the current game. GetScore, GetScoreList, GetActiveFigure, GetNextFigure, and GetGrid are called by the view class in order to draw the game grid. They simply return the values of the corresponding fields. The field m_colorGrid is an object of the class ColorGrid, which we defined in the previous section. It is actually just a matrix holding the colors of the squares of the game grid. Each square is intialized to the color white and a square is considered to be empty as long as it is white. When the application starts, the constructor calls the C standard library function srand. The name is an abbreviation for sowing a random seed. By calling srand with an integer seed, it will generate a series of random number. In order to find a new seed every time the application starts, the C standard library function time is called, which returns the number of seconds elapsed since January 1, 1970. In order to obtain the actual random number, we call rand that returns a number in the interval from zero to the predefined constant RAND_MAX. The prototypes for these functions are defined in time.h (time) and stdlib.h (rand and srand), respectively. #include"StdAfx.h"COLORREF& ColorGrid::Index(int iRow, int iCol){check((iRow >= 0) && (iRow < ROWS));check((iCol >= 0) && (iCol < COLS));return m_buffer[iRow * COLS + iCol];}const COLORREF ColorGrid::Index(int iRow, int iCol)const{check((iRow >= 0) && (iRow < ROWS));check((iCol >= 0) && (iCol < COLS));return m_buffer[iRow * COLS + iCol];} When the user presses the space key and the active figure falls down or when a row is filled and is flashed, we have to slow down the process in order for the user to apprehand the event. There is a Win32 API function Sleep that pauses the application for the given amount of milliseconds. #include <time.h>#include <stdlib.h>time_ttime(time_t *pTimer);void srand(unsigned int uSeed);intrand(); The user can control the horizontal movement and rotation of the falling figures by pressing the arrow keys. Left and right arrow keys move the figure to the left or right. The up and down arrow key rotates the figure clockwise or counter clockwise, respectively. Every time the user presses one of those keys, a message is sent to the view class object and caught by the method OnKeyDown, which in turn calls one of the methods LeftArrowKey, RightArrowKey, UpArrowKey, DownArrowKey to deal with the message. They all work in a similar fashion. They try to execute the movement or rotation in question. If it works, both the old and new area of the figure is repainted by making calls to UpdateAllViews. The view class also handles a timer that sends a message every second the view is in focus. The message is caught by the view class method OnTimer that in turn calls Timer. It tries to move the active figure one step downwards. If that is possible, the area of the figure is repainted in the same way as in the methods above. However, if it is not possible, the squares of the figure are added to the game grid. The active figure is assigned to the next figure, and the next figure is assigned a copy of a randomly selected figure in m_figureArray. We also check whether any row has been filled. In that case, it will be removed and we will check to see if the game is over. The user can speed up the game by pressing the space key. The message is caught and sent to SpaceKey. It simply calls OnTimer as many times as possible at intervals of twenty milliseconds in order to make the movement visible to the user. When a figure has reached its end position and any full rows have been removed, the figure must be valid. That is, its squares are not allowed to occupy any already colored position. If it does, the game is over and GameOver is called. It starts by making the game grid gray and asks the users whether they want to play another game. If they do, the game grid is cleared and set back to colored mode and a new game starts. If they do not, the application exits. NewGame informs the players whether they made to the top ten list and inquires about another game by displaying a message box. AddToScore examines whether the player has made to the ten top list. If so, the score is added to the list and the ranking is returned, if not, zero is returned. DeleteFullRows traverses the game grid from top to bottom flashing and removing every full row. IsRowFull traverses the given row and returns true if no square has the default color (white). FlashRow flashes the row by showing it three times in grayscale and color at intervals of twenty milliseconds. DeleteRow removes the row by moving all rows above one step downwards and inserting an empty row (all white squares) at top. The next figure and the current high score are painted at specific positions on the client area, the rectangle constants NEXT_AREA and SCORE_AREA keep track of those positions. TetrisDoc.h void Sleep(int iMilliSeconds); The field m_figureArray holds seven figure objects, one of each color. When we need a new figure, we just randomly copy one of them. TetrisDoc.cpp typedef CList<int>IntList;const int FIGURE_ARRAY_SIZE = 7;class CTetrisDoc :publicCDocument{protected:CTetrisDoc();public:virtual ~CTetrisDoc();void SaveScoreList();protected:DECLARE_MESSAGE_MAP()DECLARE_DYNCREATE(CTetrisDoc)public:virtual void Serialize(CArchive& archive);int GetScore() const {return m_iScore;}const IntList* GetScoreList() {return &m_scoreList;}const ColorGrid* GetGrid() {return &m_colorGrid;}const Figure& GetActiveFigure() const{return m_activeFigure;}const Figure& GetNextFigure() const {return m_nextFigure;}public:void LeftArrowKey();void RightArrowKey();void UpArrowKey();void DownArrowKey();BOOL Timer();void SpaceKey();private:void GameOver();BOOL NewGame();int AddScoreToList();void DeleteFullRows();BOOL IsRowFull(int iRow);void FlashRow(int iFlashRow);void DeleteRow(int iDeleteRow);private:ColorGrid m_colorGrid;Figure m_activeFigure, m_nextFigure;int m_iScore;IntList m_scoreList;const CRect NEXT_AREA, SCORE_AREA;static Figure m_figureArray[FIGURE_ARRAY_SIZE];}; When the user presses the left arrow key, the view class object catches the message and calls LeftArrowKey in the document class object. We try to move the active figure one step to the left. It is not for sure that we succeed. The figure may already be located at the left part of the game grid. However, if the movement succeeds, the figure's position is repainted and true is returned. In that case, we repaint the figure's old and new graphic areas in order to repaint the figure. Finally, we set the modified flag since the figure has been moved. The method RightArrowKey works in a similar way. Figure redFigure(NORTH, RED, RedInfo);Figure brownFigure(EAST, BROWN, BrownInfo);Figure turquoiseFigure(EAST, TURQUOISE, TurquoiseInfo);Figure greenFigure(EAST, GREEN, GreenInfo);Figure blueFigure(SOUTH, BLUE, BlueInfo);Figure purpleFigure(SOUTH, PURPLE, PurpleInfo);Figure yellowFigure(SOUTH, YELLOW, YellowInfo);Figure CTetrisDoc::m_figureArray[] = {redFigure, brownFigure, turquoiseFigure, greenFigure, yellowFigure, blueFigure, purpleFigure}; Timer is called every time the active figure is to moved one step downwards. That is,each second when the application has focus. If the downwards movement succeeds, then the figure is repainted in a way similar to LeftArrowKey above. However, if the movement does not succeed, the movement of the active figure has come to an end. We call AddToGrid to color the squares of the figure. Then we copy the next figure to the active figure and randomly copy a new next figure. The next figure is the one shown to the right of the game grid. However, the case may occur that the game grid is full. That is the case if the new active figure is not valid, that is, the squares occupied by the figure are not free. If so, the game is over, and the user is asked whether he wants a new game. void CTetrisDoc::LeftArrowKey(){CRectrcOldArea = m_activeFigure.GetArea();if (m_activeFigure.MoveLeft()){CRectrcNewArea = m_activeFigure.GetArea();UpdateAllViews(NULL, COLOR, (CObject*) &rcOldArea);UpdateAllViews(NULL, COLOR, (CObject*) &rcNewArea);SetModifiedFlag();}} If the user presses the space key, the active figure falling will fall faster. The Timer method is called every 20 milliseconds.   BOOLCTetrisDoc::Timer(){SetModifiedFlag();CRectrcOldArea = m_activeFigure.GetArea();if (m_activeFigure.MoveDown()){CRectrcNewArea = m_activeFigure.GetArea();UpdateAllViews(NULL, COLOR, (CObject*) &rcOldArea);UpdateAllViews(NULL, COLOR, (CObject*) &rcNewArea);returnTRUE;}else{m_activeFigure.AddToGrid();m_activeFigure = m_nextFigure;CRect rcActiveArea = m_activeFigure.GetArea();UpdateAllViews(NULL, COLOR, (CObject*) &rcActiveArea);m_nextFigure = m_figureArray[rand() % FIGURE_ARRAY_SIZE];UpdateAllViews(NULL, COLOR, (CObject*) &NEXT_AREA);DeleteFullRows();if (!m_activeFigure.IsFigureValid()){GameOver();}returnFALSE;}} When the game is over, the users are asked whether they want a new game. If so, we clear the grid, randomly select the the next active and next figure, and repaint the whole client area. void CTetrisDoc::SpaceKey(){while(Timer()){Sleep(20);}} Each time a figure is moved, one or more rows may be filled. We start by checking the top row and then go through the rows downwards. For each full row, we first flash it and then remove it. voidCTetrisDoc::GameOver(){UpdateAllViews(NULL, GRAY);if (NewGame()){m_colorGrid.Clear();m_activeFigure = m_figureArray[rand() %FIGURE_ARRAY_SIZE];m_nextFigure = m_figureArray[rand() % FIGURE_ARRAY_SIZE];UpdateAllViews(NULL, COLOR);else{SaveScoreList();exit(0);}} When a row is completely filled, it will flash before it is removed. The flash effect is executed by redrawing the row in color and in grayscale three times with an interval of 50 milliseconds. void CTetrisDoc::DeleteFullRows(){int iRow = ROWS - 1;while (iRow >= 0){if(IsRowFull(iRow)){FlashRow(iRow);DeleteRow(iRow);++m_iScore;UpdateAllViews(NULL, COLOR, (CObject*) &SCORE_AREA);}else{--iRow;}}} When a row is removed, we do not really remove it. If we did, the game grid would shrink. Instead, we copy the squares above it and clear the top row. voidCTetrisDoc::FlashRow(int iRow){for (int iCount = 0; iCount < 3; ++iCount){CRect rcRowArea(0, iRow, COLS, iRow + 1);UpdateAllViews(NULL, GRAY, (CObject*) &rcRowArea);Sleep(50);CRect rcRowArea2(0, iRow, COLS, iRow + 1);UpdateAllViews(NULL, COLOR, (CObject*) &rcRowArea2);Sleep(50);}} The View Class CTetrisView is the view class of the application. It receives system messages and (completely or partly) redraws the client area. The field m_iColorStatus holds the painting status of the view. Its status can be either color or grayscale. The color status is the normal mode, m_iColorStatus is initialized to color in the constructor. The grayscale is used to flash rows and to set the game grid in grayscale while asking the user for another game. OnCreate is called after the view has been created but before it is shown. The field m_pTetrisDoc is set to point at the document class object. It is also confirmed to be valid. OnSize is called each time the size of the view is changed. It sets the global variables g_iRowHeight and g_iColWidth (defi ned in Figure.h), which are used by method of the Figure and ColorGrid classes to paint the squares of the figures and the grid. OnSetFocus and OnKillFocus are called when the view receives and loses the input focus. Its task is to handle the timer. The idea is that the timer shall continue to send timer messages every second as long as the view has the input focus. Therefore, OnSetFocus sets the timer and OnKillFocus kills it. This arrangement implies that OnTimer is called each second the view has input focus. In Windows, the timer cannot be turned off temporarily; instead, we have to set and kill it. The base class of the view, CWnd, has two methods: SetTimer that initializes a timer and KillTimer that stops the timer. The first parameter is a unique identifier to distinguish this particular timer from any other one. The second parameter gives the time interval of the timer, in milliseconds. When we send a null pointer as the third parameter, the timer message will be sent to the view and caught by OnTimer. KillTimer simply takes the identity of the timer to finish. void CTetrisDoc::DeleteRow(int iMarkedRow){for (int iRow = iMarkedRow; iRow > 0; --iRow){for (int iCol = 0; iCol < COLS; ++iCol){m_colorGrid.Index(iRow, iCol) = m_colorGrid.Index(iRow - 1, iCol);}}for (int iCol = 0; iCol < COLS; ++iCol){m_colorGrid.Index(0, iCol) = WHITE;}CRect rcArea(0, 0, COLS, iMarkedRow + 1);UpdateAllViews(NULL, COLOR, (CObject*) &rcArea);} OnKeyDown is called every time the user presses a key on the keyboard. It analizes the pressed key and calls suitable methods in the document class if the left, right, up, or down arrow key or the space key is pressed. When a method of the document class calls UpdateAllViews, OnUpdate of the view class object connected to the document object is called. As this is a single view application, the application has only one view object on which OnUpdate is called. UpdateAllViews takes two extra parameters, hints, which are sent to OnUpdate. The first hint tells us whether the next repainting shall be done in color or in grayscale, the second hint is a pointer to a rectangle holding the area that is to be repainted. If the pointer is not null, we calculate the area and repaint it. If it is null, the whole client area is repainted. OnUpdate is also called by OnInitialUpdate of the base class CView with both hints set to zero. That is not a problem because the COLOR constant is set to zero. The effect of this call is that the whole view is painted in color. OnUpdate calls UpdateWindow in CView that in turn calls OnPaint and OnDraw with a device context. OnPaint is also called by the system when the view (partly or completely) needs to be repainted. OnDraw loads the device context with a black pen and then draws the grid, the score list, and´the active and next figures. TetrisView.h UINT_PTR SetTimer(UINT_PTR iIDEvent, UINT iElapse, void (CALLBACK* lpfnTimer)(HWND, UINT, UINT_PTR, DWORD));BOOL KillTimer(UINT_PTR nIDEvent); TetrisView.cpp This application catches the messsages WM_CREATE, WM_SIZE, WM_SETFOCUS, WM_KILLFOCUS, WM_TIMER, and WM_KEYDOWN. const intTIMER_ID = 0;enum {COLOR = 0, GRAY = 1};class CTetrisDoc;COLORREF GrayScale(COLORREF rfColor);class CTetrisView : public CView{protected: CTetrisView();DECLARE_DYNCREATE(CTetrisView)DECLARE_MESSAGE_MAP()public: afx_msgint OnCreate(LPCREATESTRUCT lpCreateStruct);afx_msg void OnSize(UINT nType, int iClientWidth, int iClientHeight);afx_msg void OnSetFocus(CWnd* pOldWnd);afx_msg void OnKillFocus(CWnd* pNewWnd);afx_msg void OnKeyDown(UINT nChar, UINT nRepCnt, UINT nFlags);afx_msg void OnTimer(UINT nIDEvent);void OnUpdate(CView* /* pSender */, LPARAM lHint, CObject* pHint);void OnDraw(CDC* pDC); private: void DrawGrid(CDC* pDC);void DrawScoreAndScoreList(CDC* pDC);void DrawActiveAndNextFigure(CDC* pDC);private: CTetrisDoc* m_pTetrisDoc;int m_iColorStatus;}; When the view object is created, is connected to the document object by the pointer m_pTetrisDoc. BEGIN_MESSAGE_MAP(CTetrisView, CView)ON_WM_CREATE()ON_WM_SIZE()ON_WM_SETFOCUS()ON_WM_KILLFOCUS()ON_WM_TIMER()ON_WM_KEYDOWN()END_MESSAGE_MAP() The game grid is dimensioned by the constants ROWS and COLS. Each time the user changes the size of the application window, the global variables g_iRowHeight and g_iColWidth, which are defined in Figure.h, store the height and width of one square in pixels. int CTetrisView::OnCreate(LPCREATESTRUCT lpCreateStruct){// We check that the view has been correctly created.if (CView::OnCreate(lpCreateStruct) == -1){return -1;}m_pTetrisDoc = (CTetrisDoc*) m_pDocument;check(m_pTetrisDoc != NULL);ASSERT_VALID(m_pTetrisDoc);return 0;} OnUpdate is called by the system when the window needs to be (partly or completely) repainted. In that case, the parameter pHint is zero and the whole client area is repainted. However, this method is also indirectly called when the document class calls UpdateAllView. In that case, lHint has the value color or gray, depending on whether the client area shall be repainted in color or in a grayscale. If pHint is non-zero, it stores the coordinates of the area to be repainted. The coordinates are given in grid coordinates that have to be translated into pixel coordinates before the area is invalidated. The method first calls Invalidate or InvalidateRect to define the area to be repainted, then the call to UpdateWindow does the actual repainting by calling OnPaint in CView, which in turn calls OnDraw below. void CTetrisView::OnSize(UINT /* uType */,int iClientWidth, int iClientHeight){g_iRowHeight = iClientHeight / ROWS;g_iColWidth = (iClientWidth / 2) / COLS;} OnDraw is called when the client area needs to be repainted, by the system or by UpdateWindow in OnUpdate. It draws a vertical line in the middle of the client area, and then draws the game grid, the high score list, and the current figures. voidCTetrisView::OnUpdate(CView* /* pSender */, LPARAM lHint, CObject*pHint){m_iColorStatus = (int) lHint;if (pHint != NULL){CRect rcArea = *(CRect*) pHint;rcArea.left *= g_iColWidth;rcArea.right *= g_iColWidth;rcArea.top *= g_iRowHeight;rcArea.bottom *= g_iRowHeight;InvalidateRect(&rcArea);}else{Invalidate();}UpdateWindow();} DrawGrid traverses through the game grid and paints each non-white square. If a square is not occupied, it has the color white and it not painted. The field m_iColorStatus decides whether the game grid shall be painted in color or in grayscale. void CTetrisView::OnDraw(CDC* pDC){CPen pen(PS_SOLID, 0, BLACK);CPen* pOldPen = pDC->SelectObject(&pen);pDC->MoveTo(COLS * g_iColWidth, 0);pDC->LineTo(COLS * g_iColWidth, ROWS * g_iRowHeight);DrawGrid(pDC);DrawScoreAndScoreList(pDC);DrawActiveAndNextFigure(pDC);pDC->SelectObject(&pOldPen);} GrayScale returns the grayscale of the given color, which is obtained by mixing the average of the red, blue, and green component of the color. voidCTetrisView::DrawGrid(CDC* pDC){const ColorGrid* pGrid = m_pTetrisDoc->GetGrid();for (int iRow = 0; iRow < ROWS; ++iRow){for (int iCol = 0; iCol < COLS; ++iCol){ COLORREF rfColor = pGrid->Index(iRow, iCol);if (rfColor != WHITE){CBrushbrush((m_iColorStatus == COLOR) ? rfColor:GrayScale(rfColor));CBrush* pOldBrush = pDC->SelectObject(&brush);DrawSquare(iRow, iCol, pDC);pDC->SelectObject(pOldBrush);}}}} The active figure (m_activeFigure) is the figure falling down on the game grid.The next figure (m_nextFigure) is the figure announced at the right side of the client area. In order for it to be painted at the right-hand side, we alter the origin to the middle of the client area, and one row under the upper border by calling SetWindowOrg.

In this article, we recommend that you use the JBoss AS (version 4.2), which is a free J2EE Application Server that can be downloaded from http://www.jboss.org/jbossas/downloads/ (complete documentation can be downloaded from http://www.jboss.org/jbossas/docs/). JBoss AS plug-in and the Eclipse WTP JBoss AS plug-in can be treated as an elegant method of connecting a J2EE Application Server to the Eclipse IDE. It's important to know that JBoss AS plug-in does this by using the WTP support, which is a project included by default in the Eclipse IDE. WTP is a major project that extends the Eclipse platform with a strong support for Web and J2EE applications. In this case, WTP will sustain important operations, like starting the server in run/debug mode, stopping the server, and delegating WTP projects to their runtimes. For now, keep in mind that Eclipse supports a set of WTP servers and for every WTP server you may have one WTP runtime. Now, we will see how to install and configure the JBoss 4.2.2 runtime and server. Adding a WTP runtime in Eclipse In case of JBoss Tools, the main scope of Server Runtimes is to point to a server installation somewhere on your machine. By runtimes, we can use different configurations of the same server installed in different physical locations. Now, we will create a JBoss AS Runtime (you can extrapolate the steps shown below for any supported server): From the Window menu, select Preferences. In the Preferences window, expand the Server node and select the Runtime Environments child-node. On the right side of the window, you can see a list of currently installed runtimes, as shown in the following screenshot, where you can see that an Apache Tomcat runtime is reported (this is just an example, the Apache Tomcat runtime is not a default one). Now, if you want to install a new runtime, you should click the Add button from the top-right corner. This will bring in front the New Server Runtime Environment window as you can see in the following screenshot. Because we want to add a JBoss 4.2.2 runtime, we will select the JBoss 4.2 Runtime option (for other adapters proceed accordingly). After that, click Next for setting the runtime parameters. In the runtimes list, we have runtimes provided by WTP and runtimes provided by JBoss Tools (see the section marked in red on the previous screenshot). Because this article is about JBoss Tools, we will further discuss only the runtimes from this category. Here, we have five types of runtimes with the mention that the JBoss Deploy-Only Runtime type is for developers who start/stop/debug applications outside Eclipse. In this step, you will configure the JBoss runtime by indicating the runtime's name (in the Name field), the runtime's home directory (in the Home Directory field), the Java Runtime Environment associated with this runtime (in the JRE field), and the configuration type (in the Configuration field).In the following screenshot, we have done all these settings for our JBoss 4.2 Runtime. The official documentation of JBoss AS 4.2.2 recommends using JDK version 5. If you don't have this version in the JRE list, you can add it like this: Display the Preferences window by clicking the JRE button. In this window, click the Add button to display the Add JRE window. Continue by selecting the Standard VM option and click on the Next button. On the next page, use the Browse button to navigate to the JRE 5 home directory. Click on the Finish button and you should see a new entry in the Installed JREs field of the Preferences window (as shown in the following screenshot). Just check the checkbox of this new entry and click OK. Now, JRE 5 should be available in the JRE list of the New Server Runtime Environment window. After this, just click on the Finish button and the new runtime will be added, as shown in the following screenshot: From this window, you can also edit or remove a runtime by using the Edit and Remove buttons. These are automatically activated when you select a runtime from the list. As a final step, it is recommended to restart the Eclipse IDE.

This article by Javier Collado expands the set of web scraping techniques shown in his previous article by looking closely into a more complex problem that cannot be solved with the tools that were explained there. For those who missed out on that article, here's the link. Web Scraping with Python This article will show how to extract the desired information using the same three steps when the web page is not written directly using HTML, but is auto-generated using JavaScript to update the DOM tree. As you may remember from that article, web scraping is the ability to extract information automatically from a set of web pages that were designed only to display information nicely to humans; but that might not be suitable when a machine needs to retrieve that information. The three basic steps that were recommended to be followed when performing a scraping task were the following: Explore the website to find out where the desired information is located in the HTML DOM tree Download as many web pages as needed Parse downloaded web pages and extract the information from the places found in the exploration step What should be taken into account when the content is not directly coded in the HTML DOM tree? The main difference, as you probably have already noted, is that using the downloading methods that were suggested in the previous article (urllib2 or mechanize) just don't work. This is because they generate an HTTP request to get the web page and deliver the received HTML directly to the scraping script. However, the pieces of information that are auto-generated by the JavaScript code are not yet in the HTML file because the code is not executed in any virtual machine as it happens when the page is displayed in a web browser. Hence, instead of relying on a library that generates HTTP requests, we need a library that behaves as a real web browser, or even better, a library that interacts with a real web browser. So that we are sure that we obtain the same data as we see when manually opening a page in a web browser. Please remember that the aim of web scraping is actually parsing the data that a human user sees, so interacting with a real web browser would be a really nice feature. Is there any tool out there to perform that? Fortunately, the answer is yes. In particular, there are a couple of tools used for web testing automation that can be used to solve the JavaScript execution problem: Selenium and Windmill . For the code samples in the sections below, Windmill is used. Any choice would be fine as both of them are well documented and stable tools ready to be used for production. Let's now follow the same three steps that were suggested in the previous article to solve the scraping of the contents of a web page that is partly generated using JavaScript code. Explore Imagine that you are a fan of NASA Image of the day gallery. You want to get a list of the names of all the images in the gallery together with the link to the whole resolution picture just in case you decide to download it later to use as a desktop wallpaper. The first thing to do is to locate the data that has to be extracted on the desired web page. In the case of the Image of the day gallery (see screenshot below), there are three elements that are important to note: Title of the image that is being currently displayed Link to the image full resolution file Next link to make it possible navigate through all the images To find out the location of each piece of interesting information, as it was already suggested in the previous article, it's better to use a tool such as Firebug whose inspect functionality can be really useful. The following picture, for example, shows the location of the image title inside an h3 tag: The other two fields can be located as easily as the title, so no further explanation will be given here. Please refer to the previous article for further information. Download As explained in the introduction, to download the content of the web page, we will use Windmill as it allows the JavaScript code to execute in the web browser before getting the page content. Because Windmill is mostly a testing library, instead of writing a script that calls the Windmill API, I will write a test case for Windmill to navigate through all the image web pages. The code for the test should be as follows: 1 def test_scrape_iotd_gallery(): 2 """ 3 Scrape NASA Image of the Day Gallery 4 """ 5 # Extra data massage for BeautifulSoup 6 my_massage = get_massage() 7 8 # Open main gallery page 9 client = WindmillTestClient(__name__) 10 client.open(url='http://www.nasa.gov/multimedia/imagegallery/iotd.html') 11 12 # Page isn't completely loaded until image gallery data 13 # has been updated by javascript code 14 client.waits.forElement(xpath=u"//div[@id='gallery_image_area']/img", 15 timeout=30000) 16 17 # Scrape all images information 18 images_info = {} 19 while True: 20 image_info = get_image_info(client, my_massage) 21 22 # Break if image has been already scrapped 23 # (that means that all images have been parsed 24 # since they are ordered in a circular ring) 25 if image_info['link'] in images_info: 26 break 27 28 images_info[image_info['link']] = image_info 29 30 # Click to get the information for the next image 31 client.click(xpath=u"//div[@class='btn_image_next']") 32 33 # Print results to stdout ordered by image name 34 for image_info in sorted(images_info.values(), 35 key=lambda image_info: image_info['name']): 36 print ("Name: %(name)sn" 37 "Link: %(link)sn" % image_info) As it can be seen, the usage of Windmill is similar to other libraries such as mechanize. For example, first of all a client object has to be created to interact with the browser, (line 9) and later, the main web page, that is going to be used to navigate through all the information, has to be opened (line 10). Nevertheless, it also includes some facilities that take into account JavaScript code as shown at line 14. In this line, the waits.forElement method has been used to look for DOM element that is filled by the JavaScript code so when that element, in this case the big image in the image gallery, is displayed, the rest of the script can proceed. It is important to note here that the web page processing doesn't start when the page is downloaded (this happens after line 10), but when there's some evidence that JavaScript code has finished the DOM tree manipulation. For navigating through all the pages that contain the information needed, this is just a matter of pressing over the next arrow (line 30). As the images are ordered in a circular buffer, the point when it is decided to stop is when the same image link has been parsed twice (line 25). To execute the script, instead of launching it as we would normally do for a python script, we should call it through the Windmill script to properly initialize the environment: $ windmill firefox test=nasa_iotd.py As it can be seen in the following screenshot, Windmill takes care of opening a browser (Firefox in this case) window and a controller window in which it's possible to see the commands that the script is executing (several clicks on next in the example): The controller window is really interesting because not only does it display the progress of the test cases, but also allows to enter/record actions interactively, which is a nice feature when trying things out. In particular, the recording may be used under some situations to replace Firebug in the exploration step. This is because the captured actions may be stored in a script without spending much time in xpath expressions. For more information about how to use Windmill and the complete API, please refer to the Windmill documentation.

In this article by Stoyan Stefanov, you'll learn about the prototype property of the function objects. Understanding how the prototype works is an important part of learning the JavaScript language. After all, JavaScript is classified as having a prototype-based object model. There's nothing particularly difficult about the prototype, but it is a new concept and as such may sometimes take some time to sink in. It's one of these things in JavaScript (closures are another) which, once you "get" them, they seem so obvious and make perfect sense. As with the rest of the article, you're strongly encouraged to type in and play around with the examples; this makes it much easier to learn and remember the concepts. The following topics are discussed in this article: Every function has a prototype property and it contains an object Adding properties to the prototype object Using the properties added to the prototype The difference between own properties and properties of the prototype __proto__, the secret link every object keeps to its prototype Methods such as isPrototypeOf(), hasOwnProperty(), and propertyIsEnumerable() The prototype Property The functions in JavaScript are objects and they contain methods and properties. Some of the common methods are apply() and call() and some of the common properties are length and constructor. Another property of the function objects is prototype. If you define a simple function foo() you can access its properties as you would do with any other object: >>>function foo(a, b){return a * b;}>>>foo.length 2 >>>foo.constructor Function() prototype is a property that gets created as soon as you define the function. Its initial value is an empty object. >>>typeof foo.prototype "object" It's as if you added this property yourself like this: >>>foo.prototype = {} You can augment this empty object with properties and methods. They won't have any effect of the foo() function itself; they'll only be used when you use foo()as a constructor. Adding Methods and Properties Using the Prototype Constructor functions can be used to create (construct) new objects. The main idea is that inside a function invoked with new you have access to the value this, which contains the object to be returned by the constructor. Augmenting (adding methods and properties to) this object is the way to add functionality to the object being created. Let's take a look at the constructor function Gadget() which uses this to add two properties and one method to the objects it creates. function Gadget(name, color) {   this.name = name;   this.color = color;   this.whatAreYou = function(){    return 'I am a ' + this.color + ' ' + this.name;   }} Adding methods and properties to the prototype property of the constructor function is another way to add functionality to the objects this constructor produces. Let's add two more properties, price and rating, and a getInfo() method. Since prototype contains an object, you can just keep adding to it like this: Gadget.prototype.price = 100;Gadget.prototype.rating = 3;Gadget.prototype.getInfo = function() {   return 'Rating: ' + this.rating + ', price: ' + this.price;}; Instead of adding to the prototype object, another way to achieve the above result is to overwrite the prototype completely, replacing it with an object of your choice: Gadget.prototype = {   price: 100,   rating: 3,   getInfo: function() {    return 'Rating: ' + this.rating + ', price: ' + this.price;   }}; Using the Prototype's Methods and Properties All the methods and properties you have added to the prototype are directly available as soon as you create a new object using the constructor. If you create a newtoy object using the Gadget() constructor, you can access all the methods and properties already defined. >>> var newtoy = new Gadget('webcam', 'black');>>> newtoy.name; "webcam" >>> newtoy.color; "black" >>> newtoy.whatAreYou(); "I am a black webcam" >>> newtoy.price; 100 >>> newtoy.rating; 3 >>> newtoy.getInfo(); "Rating: 3, price: 100" It's important to note that the prototype is "live". Objects are passed by reference in JavaScript, and therefore the prototype is not copied with every new object instance. What does this mean in practice? It means that you can modify the prototype at any time and all objects (even those created before the modification) will inherit the changes. Let's continue the example, adding a new method to the prototype: Gadget.prototype.get = function(what) {   return this[what];}; Even though newtoy was created before the get() method was defined, newtoy will still have access to the new method: >>> newtoy.get('price'); 100 >>> newtoy.get('color'); "black" Own Properties versus prototype Properties In the example above getInfo() used this internally to address the object. It could've also used Gadget.prototype to achieve the same result: Gadget.prototype.getInfo = function() {   return 'Rating: ' + Gadget.prototype.rating + ', price: ' + Gadget.prototype.price;}; What's is the difference? To answer this question, let's examine how the prototype works in more detail. Let's again take our newtoy object: >>> var newtoy = new Gadget('webcam', 'black'); When you try to access a property of newtoy, say newtoy.name the JavaScript engine will look through all of the properties of the object searching for one called name and, if it finds it, will return its value. >>> newtoy.name "webcam" What if you try to access the rating property? The JavaScript engine will examine all of the properties of newtoy and will not find the one called rating. Then the script engine will identify the prototype of the constructor function used to create this object (same as if you do newtoy.constructor.prototype). If the property is found in the prototype, this property is used. >>> newtoy.rating 3 This would be the same as if you accessed the prototype directly. Every object has a constructor property, which is a reference to the function that created the object, so in our case: >>> newtoy.constructor Gadget(name, color) >>> newtoy.constructor.prototype.rating 3 Now let's take this lookup one step further. Every object has a constructor. The prototype is an object, so it must have a constructor too. Which in turn has a prototype. In other words you can do: >>> newtoy.constructor.prototype.constructor Gadget(name, color) >>> newtoy.constructor.prototype.constructor.prototype Object price=100 rating=3 This might go on for a while, depending on how long the prototype chain is, but you eventually end up with the built-in Object() object, which is the highest-level parent. In practice, this means that if you try newtoy.toString() and newtoy doesn't have an own toString() method and its prototype doesn't either, in the end you'll get the Object's toString() >>> newtoy.toString() "[object Object]" Overwriting Prototype's Property withOwn Property As the above discussion demonstrates, if one of your objects doesn't have a certain property of its own, it can use one (if exists) somewhere up the prototype chain. What if the object does have its own property and the prototype also has one with the same name? The own property takes precedence over the prototype's. Let's have a scenario where a property name exists both as an own property and as a property of the prototype object: function Gadget(name) {   this.name = name;}Gadget.prototype.name = 'foo'; "foo" Creating a new object and accessing its name property gives you the object's ownname property. >>> var toy = new Gadget('camera');>>> toy.name; "camera" If you delete this property, the prototype's property with the same name"shines through": >>> delete toy.name; true >>> toy.name; "foo" Of course, you can always re-create the object's own property: >>> toy.name = 'camera';>>> toy.name; "camera"  

To keep things simple though, the article doesn’t discuss how to implement database-backed web pages with Python, concentrating only on how to connect Python with MySQL. Sample Application The best way to learn new programming techniques is to write an application that exercises them. This article will walk you through the process of building a simple Python application that interacts with a MySQL database. In a nutshell, the application picks up some live data from a web site and then persists it to an underlying MySQL database. For the sake of simplicity, it doesn’t deal with a large dataset. Rather, it picks up a small subset of data, storing it as a few rows in the underlying database. In particular, the application gets the latest post from the Packt Book Feed page available at http://feeds.feedburner.com/packtpub/sDsa?format=xml. Then, it analyzes the post’s title, finding appropriate tags for the article associated with the post, and finally inserts information about the post into the posts and posttags underlying database tables. As you might guess, a single post may be associated with more than one tag, meaning a record in the posts table may be related to several records in the posttags table. Diagrammatically, the sample application components and their interactions might look like this: Note the use of appsample.py. This script file will contain all the application code written in Python. In particular, it will contain the list of tags, as well as several Python functions packaging application logic. Software Components To build the sample discussed in the article you’re going to need the following software components installed on your computer: Python 2.5.x MySQLdb 1.2.x MySQL 5.1 All these software components can be downloaded and used for free. Although you may already have these pieces of software installed on your computer, here’s a brief overview of where you can obtain them. You can download an appropriate Python release from the Downloads page at Python’s web site at http://python.org/download/. You may be tempted to download the most recent release. Before you choose the release, however, it is recommended that you visit the Python for MySQL page at http://sourceforge.net/projects/mysql-python/ to check what Python releases are supported by the current MySQLdb module that will be used to connect your Python installation with MySQL. MySQLdb is the Python DB API-2.0 interface for MySQL. You can pick up the latest MySQLdb package (version 1.2.2 at the time of writing) from the sourceforge.net’s Python for MySQL page at http://sourceforge.net/projects/mysql-python/. Before you can install it, though, make sure you have Python installed in your system. You can obtain the MySQL 5.1 distribution from the mysql.com web site at http://dev.mysql.com/downloads/mysql/5.1.html, picking up the package designed for your operating system. Setting up the Database Assuming you have all the software components that were outlined in the preceding section installed in your system, you can now start building the sample application. The first step is to create the posts and posttags tables in your underlying MySQL database. As mentioned earlier, a single post may be associated with more than one tag. What this means in practice is that the posts and posttags tables should have a foreign key relationship. In particular, you might create these tables as follows: CREATE TABLE posts ( title VARCHAR(256) PRIMARY KEY, guid VARCHAR(1000), pubDate VARCHAR(50) ) ENGINE = InnoDB; CREATE TABLE posttags ( title VARCHAR(256), tag VARCHAR(20), PRIMARY KEY(title,tag), FOREIGN KEY(title) REFERENCES posts(title) ) ENGINE = InnoDB; As you might guess, you don’t need to populate above tables with data now. This will be automatically done later when you launch the sample. Developing the Script Now that you have the underlying database ready, you can move on and develop the Python code to complete the sample. In particular, you’re going to need to write the following components in Python: tags nested list of tags that will be used to describe the posts obtained from the Packt Book Feed page. obtainPost function that will be used to obtain the information about the latest post from the Packt Book Feed page. determineTags function that will determine appropriate tags to be applied to the latest post obtained from the Packt Book Feed page. insertPost function that will insert the information about the post obtained into the underlying database tables: posts and posttags. execPr function that will make calls to the other, described above functions. You will call this function to launch the application. All the above components will reside in a single file, say, appsample.py that you can create in your favorite text editor, such as vi or Notepad. First, add the following import declarations to appsample.py: import MySQLdb import urllib2 import xml.dom.minidom As you might guess, the first module is required to connect Python with MySQL, providing the Python DB API-2.0 interface for MySQL. The other two are needed to obtain and then parse the Packt Book Feed page’s data. You will see them in action in the obtainPost function in a moment. But first let’s create a nested list of tags that will be used by the determineTags function that determines the tags appropriate for the post being analyzed. To save space here, the following list contains just a few tags. You may and should include more tags to this list, of course. tags=["Python","Java","Drupal","MySQL","Oracle","Open Source"] The next step is to add the obtainPost function responsible for getting the data from the Packt Book Feed page and generating the post dictionary that will be utilized in further processing: def obtainPost(): addr = "http://feeds.feedburner.com/packtpub/sDsa?format=xml" xmldoc = xml.dom.minidom.parseString(urllib2.urlopen(addr).read()) item = xmldoc.getElementsByTagName("item")[0] title = item.getElementsByTagName("title")[0].firstChild.data guid = item.getElementsByTagName("guid")[0].firstChild.data pubDate = item.getElementsByTagName("pubDate")[0].firstChild.data post ={"title": title, "guid": guid, "pubDate": pubDate} return post Now that you have obtained all the required information about the latest post on the Packt Book Feed page, you can analyze the post’s title to determine appropriate tags. For that, add the determineTags function to appsample.py: def determineTags(title, tagslist): curtags=[] for curtag in tagslist: if title.find(curtag)>-1:curtags.append(curtag) return curtags By now, you have both the post and tags to be persisted to the database. So, add the insertPost function that will handle this task (don’t forget to change the parameters specified to the MySQLdb.connect function for the actual ones): def insertPost(title, guid, pubDate, curtags): db=MySQLdb.connect(host="localhost",user="usrsample",passwd="pswd",db="dbsample") c=db.cursor() c.execute("""INSERT INTO posts (title, guid, pubDate) VALUES(%s, %s,%s)""", (title, guid, pubDate)) db.commit() for tag in curtags: c.execute("""INSERT INTO posttags (title, tag) VALUES(%s,%s)""", (title, tag)) db.commit() db.close() All that is left to do is add the execPr function that brings all the pieces together, calling the above functions in the proper order: def execPr(): p = obtainPost() t = determineTags(p["title"],tags) insertPost(p["title"], p["guid"], p["pubDate"], t) Now let’s test the code we just wrote. The simplest way to do this is through Python’s interactive command line. To start an interactive Python session, you can type python at your system shell prompt. It’s important to realize that since the sample discussed here is going to obtain some data from the web, you must connect to the Internet before you launch the application. Once you’re connected, you can launch the execPr function in your Python session, as follows: >>>import appsample >>>appsample.execPr() If everything is okay, you should see no messages. To make sure that everything really went as planned, you can check the posts and posttags tables. To do this, you might connect to the database with the MySQL command-line tool and then issue the following SQL commands: SELECT * FROM posts; The above should generate the output that might look like this: |title |guid |pubDate ------------------------------------------------------------------ Open Source CMS Award Voting Now Closed | http://www.packtpub.com/ article/2008-award-voting-closed | Tue, 21 Oct 2008 09:29:54 +0100 Then, you might want to check out the posttags table: SELECT * FROM posttags; This might generate the following output: |title |tag Open Source CMS Award Voting Now Closed | Open Source Please note that you may see different results since you are working with live data. Another thing to note here is that if you want to re-run the sample, you first need to empty the posts and posttags tables. Otherwise, you will encounter the problem related to the primary key constraints. However, that won’t be a problem at all if you re-run the sample in a few days, when a new post or posts appear on the Packt Book Feed page. Conclusion In this article you looked at a simple Python application persisting data to an underlying MySQL database. Although, for the sake of simplicity, the sample discussed here doesn’t offer a web interface, it illustrates how you can obtain data from the Internet, and then utilize it within your application, and finally store that data in the database.

As you know, Eclipse offers an ingenious system of perspectives that helps us to switch between different technologies and to keep the main-screen as clean as possible. Every perspective is made of a set of components that can be added/removed by the user. These components are known as views. The JBoss AS Perspective has a set of specific views as follows: JBoss Server View Project Archives View Console View Properties View For launching the JBoss AS Perspective (or any other perspective), follow these two simple steps: From the Window menu, select Open Perspective | Other article. In the Open Perspective window, select the JBoss AS option and click on OK button (as shown in the following screenshot). If everything works fine, you should see the JBoss AS perspective as shown in the following screenshot: If any of these views is not available by default in your JBoss AS perspective, then you can add it manually by selecting from the Window menu the Show View | Other option. In the Show View window (shown in the following screenshot), you just select the desired view and click on the OK button. JBoss Server View This view contains a simple toolbar known as JBoss Server View Toolbar and two panels that separate the list of servers (top part) from the list of additional information about the selected server (bottom part). Note that the quantity of additional information is directly related to the server type. Top part of JBoss Server View In the top part of the JBoss Server View, we can see a list of our servers, their states, and if they are running or if they have stopped. Starting the JBoss AS The simplest ways to start our JBoss AS server are: Select the JBoss 4.2 Server from the server list and click the Start the server button from the JBoss Server View Toolbar (as shown in the following screenshot). Select the JBoss 4.2 Server from the server list and right-click on it. From the context menu, select the Start option (as shown in the following screenshot). In both cases, a detailed evolution of the startup process will be displayed in the Console View, as you can see in the following screenshot. Stopping the JBoss AS The simplest ways to stop JBoss AS server are: Select the JBoss 4.2 Server from the server list and click the Stop the server button from the JBoss Server View Toolbar. Select the JBoss 4.2 Server from the server list and right-click on it. From the context menu, select the Stop option. In both cases, a detailed evolution of the stopping process will be displayed in the Console View, as you can see in the following screenshot. Additional operations on JBoss AS Beside Start and Stop operations, JBoss Server View allows us to: Add a new server (the New Server option from the contextual menu) Remove an existing server (the Delete option from the contextual menu) Start the server in debug mode (first button on the JBoss Server View Toolbar) Start the server in profiling mode (third button on the JBoss Server View Toolbar) Publish to the server or synching the publish information between the server and the workspace (the Publish option from the contextual menu or the last button on the JBoss Server View Toolbar) Discard all publish state and republish from scratch (the Clean option from the contextual menu) Twiddle server (the Twiddle Server option from the contextual menu) Edit launch configuration (the Edit Launch Configuration option from the contextual menu as shown in the following screenshot). Add/remove projects (the Add and Remove Projects option from the contextual menu) Double-click the server name and modify parts of that server in the Server Editor—if you have a username and a password to start the server, then you can specify those credentials here (as shown in the following screenshot). Twiddle is a JMX library that comes with JBoss, and it is used to access (any) variables that are exposed via the JBoss JMX interfaces. Server publish status A server may have one of the following statuses: Synchronized: Allows you to see if changes are sync (as shown in the follo wing screenshot) Publishing: Allows you to see if changes are being updated Republish: Allows you to see if changes are waiting