How-To Tutorials - Programming

In this article by Angel Marquez author of the book PostGIS Essentials see how to insert GIS objects. Now is the time to fill our tables with data. It's very important to understand some of the theoretical concepts about spatial data before we can properly work with it. We will cover this concept through the real estate company example, used previously. Basically, we will insert two kinds of data: firstly, all the data that belongs to our own scope of interest. By this, I mean the spatial data that was generated by us (the positions of properties in the case of the example of the real estate company) for our specific problem, so as to save this data in a way that can be easily exploited. Secondly, we will import data of a more general use, which was provided by a third party. Another important feature that we will cover in this article are the spatial data files that we could use to share, import, and export spatial data within a standardized and popular format called shp or Shape files. In this article, we will cover the following topics: Developing insertion queries that include GIS objects Obtaining useful spatial data from a public third-party Filling our spatial tables with the help of spatial data files using a command line tool Filling our spatial tables with the help of spatial data files using a GUI tool provided by PostGIS (For more resources related to this topic, see here.) Developing insertion queries with GIS objects Developing an insertion query is a very common task for someone who works with databases. Basically, we follow the SQL language syntax of the insertion, by first listing all the fields involved and then listing all the data that will be saved in each one: INSERT INTO tbl_properties( id, town, postal_code, street, "number) VALUES (1, 'London', 'N7 6PA', 'Holloway Road', 32); If the field is of a numerical value, we simply write the number; if it's a string-like data type, we have to enclose the text in two single quotes. Now, if we wish to include a spatial value in the insertion query, we must first find a way to represent this value. This is where the Well-Known Text (WKT) notation enters. WKT is a notation that represents a geometry object that can be easily read by humans; following is an example of this: POINT(-0.116190 51.556173) Here, we defined a geographic point by using a list of two real values, the latitude (y-axis) and the longitude (x-axis). Additionally, if we need to specify the elevation of some point, we will have to specify a third value for the z-axis; this value will be defined in meters by default, as shown in the following code snippet: POINT(-0.116190 51.556173 100) Some of the other basic geometry types defined by the WKT notation are: MULTILINESTRING: This is used to define one or more lines POLYGON: This is used to define only one polygon MULTIPOLYGON: This is used to define several polygons in the same row So, as an example, an SQL insertion query to add the first row to the table, tbl_properties, of our real estate database using the WKT notation, should be as follows: INSERT INTO tbl_properties (id, town, postal_code, street, "number", the_geom) VALUES (1, 'London', 'N7 6PA', 'Holloway Road', 32, ST_GeomFromText('POINT(-0.116190 51.556173)')); The special function provided by PostGIS, ST_GeomFromText, parses the text given as a parameter and converts it into a GIS object that can be inserted in the_geom field. Now, we could think this is everything and, therefore, start to develop all the insertion queries that we need. It could be true if we just want to work with the data generated by us and there isn't a need to share this information with other entities. However, if we want to have a better understanding of GIS (believe me, it could help you a lot and prevent a lot of unnecessary headache when working with data from several sources), it would be better to specify another piece of information as part of our GIS object representation to establish its Spatial Reference System (SRS). In the next section, we will explain this concept. What is a spatial reference system? We could think about Earth as a perfect sphere that will float forever in space and never change its shape, but it is not. Earth is alive and in a state of constant change, and it's certainly not a perfect circle; it is more like an ellipse (though not a perfect ellipse) with a lot of small variations, which have taken place over the years. If we want to represent a specific position inside this irregular shape called Earth, we must first make some abstractions: First we have to choose a method to represent Earth's surface into a regular form (such as a sphere, ellipsoid, and so on). After this, we must take this abstract three-dimensional form and represent it into a two-dimensional plane. This process is commonly called map projection, also known as projection. There are a lot of ways to make a projection; some of them are more precise than others. This depends on the usefulness that we want to give to the data, and the kind of projection that we choose. The SRS defines which projection will be used and the transformation that will be used to translate a position from a given projection to another. This leads us to another important point. Maybe it has occurred to you that a geographic position was unique, but it is not. By this, I mean that there could be two different positions with the same latitude and longitude values but be in different physical places on Earth. For a position to be unique, it is necessary to specify the SRS that was used to obtain this position. To explain this concept, let's consider Earth as a perfect sphere; how can you represent it as a two-dimensional square? Well, to do this, you will have to make a projection, as shown in the following figure:   A projection implies that you will have to make a spherical 3D image fit into a 2D figure, as shown in the preceding image; there are several ways to achieve this. We applied an azimuthal projection, which is a result of projecting a spherical surface onto a plane. However, as I told you earlier, there are several other ways to do this, as we can see in the following image:   These are examples of cylindrical and conical projections. Each one produces a different kind of 2D image of the terrain. Each has its own peculiarities and is used for several distinct purposes. If we put all the resultant images of these projections one above the other, we must get an image similar to the following figure:   As you can see, the terrain positions, which are not necessary, are the same between two projections, so you must clearly specify which projection you are using in your project in order to avoid possible mistakes and errors when you establish a position. There are a lot of SRS defined around the world. They could be grouped by their reach, that is, they could be local (state or province), national (an entire country), regional (several countries from the same area), or global (worldwide). The International Association of Oil and Gas Producers has defined a collection of Coordinate Reference System (CRS) known as the European Petroleum Survey Group (EPSG) dataset and has assigned a unique ID to each of these SRSs; this ID is called SRID. For uniquely defining a position, you must establish the SRS that it belongs to, using its particular ID; this is the SRID. There are literally hundreds of SRSs defined; to avoid any possible error, we must standardize which SRS we will use. A very common SRS, widely used around the globe is the WGS84 SRS with the SRID 4326. It is very important that you store the spatial data on your database, using EPSG: 4326 as much as possible, or almost use one equal projection on your database; this way you will avoid problems when you analyze your data. The WKT notation doesn't support the SRID specification as part of the text, since this was developed at the EWKT notation that allows us to include this information as part of our input string, as we will see in the following example: 'SRID=4326;POINT(51.556173 -0.116190)' When you create a spatial field, you must specify the SRID that will be used. Including SRS information in our spatial tables The matter that was discussed in the previous section is very important to develop our spatial tables. Taking into account the SRS that they will use from the beginning, we will follow a procedure to recreate our tables by adding this feature. This procedure must be applied to all the tables that we have created on both databases. Perform the following steps: Open a command session on pgSQL in your command line tool or by using the graphical GUI, PGAdmin III. We will open the Real_Estate database. Drop the spatial fields of your tables using the following instruction: SELECT DropGeometryColumn('tbl_properties', 'the_geom') Add the spatial field using this command: SELECT AddGeometryColumn('tbl_properties', 'the_geom', 4326, 'POINT', 2); Repeat these steps for the rest of the spatial tables. Now that we can specify the SRS that was used to obtain this position, we will develop an insertion query using the Extended WKT (EWKT) notation: INSERT INTO tbl_properties ( id, town, postal_code, street, "number", the_geom)VALUES (1, 'London', 'N7 6PA', 'Holloway Road', 32, ST_GeomFromEWKT('SRID=4326;POINT(51.556173 -0.116190)')); The ST_GeomFromEWKT function works exactly as ST_GeomFromText, but it implements the extended functionality of the WKT notation. Now that you know how to represent a GIS object as text, it is up to you to choose the most convenient way to generate a SQL script that inserts existing data into the spatial data tables. As an example, you could develop a macro in Excel, a desktop application in C#, a PHP script on your server, and so on. Getting data from external sources In this section, we will learn how to obtain data from third-party sources. Most often, this data interchange is achieved through a spatial data file. There are many data formats for this file (such as KML, geoJSON, and so on). We will choose to work with the *.shp files, because they are widely used and supported in practically all the GIS tools available in the market. There are dozens of sites where you could get useful spatial data from practically any city, state, or country in the world. Much of this data is public and freely available. In this case, we will use data from a fabulous website called http://www.openstreetmap.org/. The following is a series of steps that you could follow if you want to obtain spatial data from this particular provider. I'm pretty sure you can easily adapt this procedure to obtain data from another provider on the Internet. Using the example of the real estate company, we will get data from the English county of Buckinghamshire. The idea is that you, as a member of the IT department, import data from the cities where the company has activities: Open your favorite Internet browser and go to http://www.openstreetmap.org/, as shown in the following screenshot: Click on the Export tab. Click on the Geofabrik Downloads link; you will be taken to http://download.geofabrik.de/, as shown in the following screenshot: There, you will find a list of sub regions of the world; select Europe: Next is a list of all countries in Europe; notice a new column called .shp.zip. This is the file format that we need to download. Select Great Britain: In the next list, select England, you can see your selection on the map located at the right-hand side of the web page, as shown in the following screenshot: Now, you will see a list of all the counties. Select the .shp.zip column from the county of Buckinghamshire: A download will start. When it finishes, you will get a file called buckinghamshire-latest.shp.zip. Unzip it. At this point, we have just obtained the data (several shp files). The next procedure will show us how to convert this file into SQL insertion scripts. Extracting spatial data from an shp file In the unzipped folder are shp files; each of them stores a particular feature of the geography of this county. We will focus on the shp named buildings.shp. Now, we will extract this data stored in the shp file. We will convert this data to a sql script so that we can insert its data into the tbl_buildings table. For this, we will use a Postgis tool called shp2pgSQL. Perform the following steps for extracting spatial data from an shp file: Open a command window with the cmd command. Go to the unzipped folder. Type the following command: shp2pgsql -g the_geom buildings.shp tbl_buildings > buildings.sql Open the script with Notepad. Delete the following lines from the script: CREATE TABLE "tbl_buildings"(gid serial, "osm_id" varchar(20), "name" varchar(50), "type" varchar(20), "timestamp" varchar (30) ); ALTER TABLE "tbl_buildings" ADD PRIMARY KEY (gid); SELECT AddGeometryColumn('','tbl_buildings','geom','0','MULTIPOLYGON',2); Save the script. Open and run it with the pgAdmin query editor. Open the table; you must have at least 13363 new registers. Keep in mind that this number can change when new updates come. Importing shp files with a graphical tool There is another way to import an shp file into our table; we could use a graphical tool called postgisgui for this. To use this tool, perform the following steps: In the file explorer, open the folder: C:Program FilesPostgreSQL9.3binpostgisgui. Execute the shp2pgsql-gui application. Once this is done, we will see the following window: Configure the connection with the server. Click on the View Connections Details... button. Set the data to connect to the server, as shown in the following screenshot: Click the Add File button. Select the points.shp file. Once selected, type the following parameters in the Import List section:     Mode: In this field, type Append     SRID: In this field, type 4326     Geo column: In this field, type the_geom     Table: In this field, type tbl_landmarks   Click on the Import button. The import process will fail and show you the following message: This is because the structure is not the same as shown in the shp and in our table. There is no way to indicate to the tool which field we don't want to import. So, the only way for us to solve this problem is let the tool create a new table and after this, change the structure. This can be done by following these steps: Go to pgAdmin and drop the tbl_landmarks table. Change the mode to Create in the Import list. Click on the Import button. Now, the import process is successful, but the table structure has changed. Go to the PGAdmin again, refresh the data, and edit the table structure to be the same as it was before:     Change the name of the geom field to the_geom.     Change the name of the osm_id field to id.     Drop the Timestamp field.     Drop the primary key constraint and add a new one attached to the id field. For that, right-click on Constraints in the left panel.     Navigate to New Object | New Primary Key and type pk_landmarks_id. In the Columns tab, add the id field.   Now, we have two spatial tables, one with data that contains positions represented as the PostGIS type, POINT (tbl_landmarks), and the other with polygons, represented by PostGIS with the type, MULTIPOLYGON(tbl_buildings). Now, I would like you to import the data contained in the roads.shp file, using one of the two previously viewed methods. The following table has data that represents the path of different highways, streets, roads, and so on, which belong to this area in the form of lines, represented by PostGIS with the MULTILINESTRING type. When it's imported, change its name to tbl_roads and adjust the columns to the structure used for the other tables in this article. Here's an example of how the imported data must look like, as you can see the spatial data is show in its binary form in the following table: Summary In this article, you learned some basic concepts of GIS (such as WKT, EWKT, and SRS), which are fundamental for working with the GIS data. Now, you are able to craft your own spatial insertion queries or import this data into your own data tables. Resources for Article: Further resources on this subject: Improving proximity filtering with KNN [article] Installing PostgreSQL [article] Securing the WAL Stream [article]

(For more resources related to this topic, see here.) Qt offers a better way: signals and slots. Like an event, the sending component generates a signal—in Qt parlance, the object emits a signal—which recipient objects may receive in a slot for the purpose. Qt objects may emit more than one signal, and signals may carry arguments; in addition, multiple Qt objects can have slots connected to the same signal, making it easy to arrange one-to-many notifications. Equally important, if no object is interested in a signal, it can be safely ignored, and no slots connected to the signal. Any object that inherits from QObject, Qt's base class for objects, can emit signals or provide slots for connection to signals. Under the hood, Qt provides extensions to C++ syntax for declaring signals and slots. A simple example will help make this clear. The classic example you find in the Qt documentation is an excellent one, and we'll use it again it here, with some extension's. Imagine you have the need for a counter, that is, a container that holds an integer. In C++, you might write: class Counter { public: Counter() { m_value = 0; } int value() const { return m_value; } void setValue(int value); private: int m_value; }; The Counter class has a single private member, m_value, bearing its value. Clients can invoke the value to obtain the counter's value, or set its value by invoking setValue with a new value. In Qt, using signals and slots, we write the class this way: #include <QObject> class Counter : public QObject { Q_OBJECT public: Counter() { m_value = 0; } int value() const { return m_value; } public slots: void setValue(int value); void increment(); void decrement(); signals: void valueChanged(int newValue); private: int m_value; }; This Counter class inherits from QObject, the base class for all Qt objects. All QObject subclasses must include the declaration Q_OBJECT as the first element of their definition; this macro expands to Qt code implementing the subclass-specific glue necessary for the Qt object and signal-slot mechanism. The constructor remains the same, initializing our private member to zero. Similarly, the accessor method value remains the same, returning the current value for the counter. An object's slots must be public, and are declared using the Qt extension to C++ public slots. This code defines three slots: a setValue slot, which accepts a new value for the counter, and the increment and decrement slots, which increment and decrement the value of the counter. Slots may take arguments, but do not return them; the communication between a signal and its slots is one way, initiating with the signal and terminating with the slot(s) connected to the signal. The counter offers a single signal. Like slots, signals are also declared using a Qt extension to C++, signals. In the example above, a Counter object emits the signal valueChanged with a single argument, which is the new value of the counter. A signal is a function signature, not a method; Qt's extensions to C++ use the type signature of signals and slots to ensure type safety between signal-slot connections, a key advantage signals and slots have over other decoupled messaging schemes. As the developers, it's our responsibility to implement each slot in our class with whatever application logic makes sense. The Counter class's slots look like this: void Counter::setValue(int newValue) { if (newValue != m_value) { m_value = newValue; emit valueChanged(newValue); } } void Counter::increment() { setValue(value() + 1); } void Counter::decrement() { setValue(value() – 1); } We use the implementation of the setValue slot as a method, which is what all slots are at their heart. The setValue slot takes a new value and assigns the new value to the Counter class's private member variable if they aren't the same. Then, the signal emits the valueChanged signal, using the Qt extension emit, which triggers an invocation to the slots connected to the signal. This is a common pattern for signals that handle object properties: testing the property to be set for equality with the new value, and only assigning and emitting a signal if the values are unequal. If we had a button, say QPushButton, we could connect its clicked signal to the increment or decrement slot, so that a click on the button incremented or decremented the counter. I'd do that using the QObject::connect method, like this: QPushButton* button = new QPushButton(tr("Increment"), this); Counter* counter = new Counter(this); QObject::connect(button, SIGNAL(clicked(void)), Counter, SLOT(increment(void)); We first create the QPushButton and Counter objects. The QPushButton constructor takes a string, the label for the button, which we denote to be the string Increment or its localized counterpart. Why do we pass this to each constructor? Qt provides a parent-child memory management between QObjects and their descendants, easing clean-up when you're done using an object. When you free an object, Qt also frees any children of the parent object, so you don't have to. The parent-child relationship is set at construction time; I'm signaling to the constructors that when the object invoking |this code is freed, the push button and counter may be freed as well. (Of course, the invoking method must also be a subclass of QObject for this to work.) Next, I call QObject::connect, passing first the source object and the signal to be connected, and then the receiver object and the slot to which the signal should be sent. The types of the signal and the slot must match, and the signals and slots must be wrapped in the SIGNAL and SLOT macros, respectively. Signals can also be connected to signals, and when that happens, the signals are chained and trigger any slots connected to the downstream signals. For example, I could write: Counter a, b; QObject::connect(&a, SIGNAL(valueChanged(int)), &b, SLOT(setValue(int))); This connects the counter b with the counter a, so that any change in value to the counter a also changes the value of the counter b. Signals and slots are used throughout Qt, both for user interface elements and to handle asynchronous operations, such as the presence of data on network sockets and HTTP transaction results. Under the hood, signals and slots are very efficient, boiling down to function dispatch operations, so you shouldn't hesitate to use the abstraction in your own designs. Qt provides a special build tool, the meta-object compiler, which compiles the extensions to C++ that signals and slots require and generates the additional code necessary to implement the mechanism. Summary In this article we learned the usage events for the purpose of coupling different objects; components offering data encapsulate that data in an event, and an event loop (or, more recently, an event listener) catches the event and performs some action. Resources for Article: Further resources on this subject: One-page Application Development [Article] Android 3.0 Application Development: Multimedia Management [Article] Creating and configuring a basic mobile application [Article]

For the past decade or so we have begun to realize that computers are an indispensable necessity. They're around us everywhere, starting from our comfortable households to rovers from other planets. Currently, it is not uncommon at all to have more than a few dozen of office computers and other IT equipment in the infrastructure of a small company that does nothing directly related to that specific area. It should not surprise anyone that in case of business environments there has to be some streamlined inventory. Especially, when we consider that the network might have a total of several hundreds, if not thousands, of workstation computers, servers, portable devices, other office equipment such as printers, scanners, and other networking components. Resource management, in its essence, when viewed from an IT perspective, is providing a method to gather and store all kinds of information about items in our infrastructure. Later on supporting means to further maintain the said inventory. Also, performing routine tasks based on the collected data such as generating reports, locating relevant information easily (like where is a specific memory module with the model number you're looking for), auditing type of software installed on workstation computers, and more. Our plan of action is going to be pretty straightforward; we analyze the IT inventorying needs and some general requisites when it comes to managing those assets. What's more we'll be presenting the client-sever model that is the underlying foundation on which most centralized management solutions are working. This is when OCS Inventory NG pops into the picture saving the day. Soon we will see why. We will get to know more about OCS Inventory NG soon, for now it's enough to realize that it's an open source project. No matter how successful a company is, open source solutions are always appreciated by the IT staff and management. As long as the project is actively developed, it's fairly popular, well documented, provides community support, and meets their needs. Among others, open source projects end up modular and flexible. Inventorying requirements in the real world One of the general requirements of an IT inventory is to be efficient and practical. The entire process should be seamless to the clients and requires limited (or none at all) user interaction. Once set up, it just needs to be automated to update the inventory database based on the latest changes without manually being required to do so. Thereafter, the collage of data gathered is ought to be organized and labeled the way we want. Businesses everywhere have come to realize that process integration is the best method for querying, standardizing, and organizing information about the infrastructure. The age of hi-tech computing made this possible by speeding up routine tasks and saving up employee time, eliminating bureaucracy and unnecessary filing of papers that all lead to frustration and waste of resources. Implementing integrated processes can change the structure and behavior of an organization. But finding the correct integration often becomes a dilemma. Feasible solution to avoid inevitable havoc Drifting back to the case of IT department, the necessity of having an integrated and centralized solution to manage numerous systems and other hardware equipment becomes obvious. The higher the number of systems, the bigger the volume to be managed, the easier the situation can get out of control, thus leading to crisis. Everyone runs around in panic like headless zombies trying to figure out who can be held responsible and what's there to do in order to avoid such scenarios. Taking a rational approach soon enough can improve the stability of entire organizations. Chances are you already know this, but usually system administrators tend to dislike working with papers. Filling in forms, storing them purely for archiving needs, and then when they least expect it, finding relevant information. A system like that won't make anyone happy. A centralized repository in some shape or form of a database gives almost instant access to results whenever such a query happens. Its actual state of always being up-to-date and reflecting the actual state of the infrastructure can be guaranteed by implementing updating mechanism. Later on, once the database is in healthy state and the process is integrated, tried and proven, it won't make any significant difference between managing dozens of computers or thousands. A well-designed integrated process is future proof and scalable. This way it won't become a setback if and when the company decides to expand. Streamlining software auditing and license management As mentioned earlier, it is important to understand that auditing workstation machines cannot be neglected. In certain environments, the users or employees have limited access and work within a sort of enclosed program area and they can do little to nothing outside of their specialization. But there are situations when the employees are supposed to have administrative access and full permissions. It is for the good of both the user and company to monitor and pay attention to what happens within each and every computer. Having an up to par auditing mechanism can integrate the system of license management as well. The persons responsible for this can track the total amount of licenses used and owned by the company, can calculate balance, notify when this number is about to run out, and so forth. It isn't all that uncommon to automate the purchasing of licenses either. The license management process description varies from firm to firm, but usually it's something similar to the following: user requests for a license, supervisor agrees, and the request arrives to the relevant IT staff. After this step, the license request gets analyzed and based on the result it is either handed out or ordered/acquired if necessary. If the process is not automated, all this would involve paperwork. And soon you will see frustrated employees running back-and-forth through departments asking who else needs to sign this paper. The process of automating and printing the end result is elegant and takes no trouble. The responsible department can then store the printed document for archiving purposes, if required. But the key of the process lies in integration. Inventorying can help here too. More uses of an integrated IT inventory solution The count of office consumables can also be tracked and maintained. This is a trickier process because it cannot be made unattended totally. Unless by installing some sort of sensor to track the count of printer cartridges inside an office furniture or the warehouse. However, you can update this field each time the said item gets restocked. A centralized method for consumables means the responsible parties can get notified before running out of stock. Once again, this step eliminates unexpected scenarios and unnecessary tasks. The beauty of centralized management solutions in the IT world is that if it is done right, they can open doors to numerous other activities as well. For example, in case of workstation PCs, the integrated process can be expanded into providing remote administration and similar other activities to be carried out remotely on the client machine. Package deployment and execution of scripts are just few distinctive examples. Think of it like, license is granted, the package is deployed, and the script is run to ensure proper registration of the application, if required. System administrators can usually help fixing common issues of employees via remote execution of scripts. Surely there are other means to administer the machines, but we're focusing on all-in-one integrated solutions. Another possibility is integrating the help-desk and ticketing system within the centralized inventory's management control panel as well. This way when an employee asks for help or reports a hardware issue, the system administrator can take a look at what's inside that system (hardware specifications, software installed, and so on.).Therefore, the system administrator gets to know the situation beforehand and thus use the right tools to troubleshoot the said issue. Gathering relevant inventory information We can conclude that in order to have a complete inventory on top of which we can build and implement other IT-related and administrative tasks, we need at least the following: Collecting relevant hardware information in case of workstation computers Manufacturer, serial number, model number of every component When applicable,some of the following: revision number, size, speed, memory, type, description, designation, connection port, interface, slot number, driver, MAC and IP address, and so on Collecting installed software/OS (licensing) information Operating system: Name, version, and registration information Application name, publisher, version, location Custom-queries from the Windows registry (if applicable) Collecting information about networking equipment and office peripherals Manufacturer, serial number, model, type of component, and so on MAC and IP address When applicable: revision number, firmware, total uptime, and so on

In this article written by Feroz Pearl Louis and Gaurav Gupta, author of the book Mastering Mobile Test Automation, we will learn that the star rating system on mobile marketplaces, such as Google Play and Application Store, is a source of positive as well as negative feedback for the applications deployed by any organization. This system is used to measure various aspects of the application, such as like functionality, usability, and is a way to quantify the all-elusive measurement-defying factor that organizations yearn to measure called "user experience", besides the obvious ones, such as the appeal and aesthetics of an application's graphical user interface (GUI). If an organization does not spend time in testing the functionality adequately, then it may suffer the consequences and lose the market share to competitors. The challenge to enable different channels such as web applications through mobile browsers, as well as providing different native or hybrid applications to service the customers as per their preferences, often leads to a situation where organizations have to develop both a web version and a hybrid version of the application. (For more resources related to this topic, see here.) At any given point of time, it is almost impossible to test an application completely, and to cover various permutations and combinations of operating systems, their versions, device manufacturers, device specifications with various screen sizes, and application types, with solely employed manual testing techniques. This is where automation comes to the rescue. However, mobile automation in itself is very complex because of the previously explained fragmentation issue. In this article, you will learn how not to fall into the trap of using different tools, frameworks, and techniques to address these differences. In this article, we will cover the following topics: Introduction to mobile test automation Types of mobile application packages Mobile test automation overview Some common factors to be considered during mobile testing, including Interrupt testing, form factor testing, layout testing, and more Overview of different types of mobile automation testing approaches Selection of the best mobile testing approach depending on the project Troubleshooting and best practices Introduction to mobile test automation Before we start learning about mobile test automation, let's understand what functional test automation is. Test automation has always been a fundamental part of the software testing lifecycle for any project. Organizations invariably look to automate the repetitive testing actions in order to utilize the manual effort thus saved for more dynamic and productive tasks. Use of automation tools also allows utilization of system idle time more effectively. To address these needs, there are a plethora of tools available in the market along with various frameworks and implementation techniques. There are both open source and licensed tools available in the market. Tools such as HP's Unified Functional Testing (UFT), formerly known as QuickTest Professional (QTP), TestComplete, Selenium, eggPlant, Ranorex, SilkTest, IBM Functional tester, and numerous others, provide various capabilities for functional automation. However, almost all of these tools are designed to support only a single operating system (predominantly Windows—owing to its popularity and the coverage it enjoys across industry verticals), although a few provide support for other lesser-used operating systems, such as Unix, Linux, Sun Solaris, and Apple Macintosh. As far as functional automation is concerned, you don't need to even consider the implications of supporting multiple operating systems in most cases. With Windows as the only operating system that is supported, there aren't any considerations for different operating systems. If the application is a web application, then there may be a need to do cross-browser testing, that is, testing automation on various browser types (Chrome, Firefox, and Safari besides Internet Explorer) and their respective versions. Also, as far as functional automation is considered, there is a very clear demarcation between nonfunctional and functional requirements. So, an automated solution for functional testing is not required to consider factors such as how others processes running on the machine would impact it, or any of the hardware aspects, such as the screen resolution of monitors and the make of the machines (IBM, Lenovo, and others). When it comes to mobile automation, there is an impact on the test suite design due to various other aspects, such as operating systems (Android, iOS, Blackberry, Windows) on which the application is supposed to be accessed, the mode of access (Wi-Fi, 3G, LTE, and so on), the form factor of the devices (tablets, phones, phablets, and so on), and the behavior of the application in various orientation modes (portrait, landscape, and so on). So, apart from normal automation challenges, a robust mobile automation suite should be able to address all these challenges in a reliable way. Fragmentation of the mobile ecosystem is an aspect that compounds this manifold problem. An application should be able to service different operating systems and their flavors provided by original equipment manufacturers (OEMs), such as Apple with iOS, Google's Android with Samsung, HTC, Xiaomi, and numerous others, Windows with Nokia and HTC, and even Blackberry and other lesser-used operating systems and devices. Add to this the complexity of dealing with various form factors, such as phones, tablets, phablets, and their various hybrids. The following figure is a visualization of the Android market fragmentation over various equipment manufacturers, form factors, and OS versions: As we know, test automation is the use of software to automate and control the setting up of test preconditions, execution of tests, test control, and test reporting functions with minimum, or ideally zero, user intervention. Automating the testing for any mobile application is the best way to ensure quality, and to achieve the quick and precise results that are needed to accommodate fast development cycles. Organizations look toward functional test automation primarily to reduce the total cost of ownership over a period of time, and to ensure the quality of the product or application being developed. These advantages are compounded many times for mobile test automation and hence it provides the same advantages, but to a much greater degree. The following are the various advantages of mobile test automation for any project: Improved testing efficiency: The same scripts can be used to run uniform validations across different devices, operating systems, and application types (of the same application), thereby reducing the test execution effort considerably. This also means that the return on investment (RoI), which typically takes about 3-5 cycles of executing the conventional functional automation to achieve breakeven, is viable in most cases within the first release itself, as mobile testing is typically repeated on many devices. So, in this case, fragmentation acts as a positive factor if the automation is employed properly, whereas, with pure manual testing, it greatly increases the costs. Consistent and repeatable testing process: Human beings tend to get bored with repetitive tasks and this makes such a test prone to errors. Due to the effect of fragmentation in the mobile world, the same application functionality needs to be validated across various combinations of operating systems, application types, device manufacturers, network conditions, and many more. Hence, the use of automation, which is basically a program, ensures that the same scripts run without any modifications every time. Improved regression testing coverage: The use of automation scripts allows the regression tests to be iterated over multiple combinations of test data. Such data-driven scripts allow the same flow to be validated against different test data combinations. For example, if an application allows users to search for the nearest ATMs in a given area, basically, the same flow would need to be tested with various zip codes as inputs. Hence, the use of automated scripts would instantly allow the test coverage to be increased dramatically. More tests can be run in less time: Since automated scripts can be run in parallel over various devices, the same amount of testing can be compacted inside a much smaller time window in comparison to the manually executed functional testing. With the use of automation scripts that include device setups as preconditions, the execution window can be exponentially reduced, which otherwise would take a manual tester considerable time to complete. 24/7 operation: Although any functional automation suite can lead to better resource utilization in terms of executing more number of scripts in lesser time, with respect to mobile automation, the resources are often expensive mobile devices. If functional testing is done manually, then more of the same devices need to be procured to allow manual testers to carry out tests, and especially, more so in the case of geographically distributed testing teams. Mobile automation scripts, on the other hand, can be triggered remotely and can run unattended, reducing the overall cost of ownership and allowing 24/7 utilization of devices and tools. Human resources are free to perform advanced manual tests: Having automation scripts perform repetitive regression testing tasks frees up the bandwidth of manual testing teams for exploratory tests that are expensive to automate and cumbersome to manage. Hence, the use of automation leads to a balanced approach, where testers can perform more meaningful work and thereby improve the quality of delivered applications. In mobiles, since regression is more repetitive on account of the fragmentation problem, the amount of effort saved is manifold, and hence, testers can generally focus on testing aspects such as user interface (UI) testing and user experience testing. Simple reproduction of found defects: Since automation scripts can be executed multiple times on demand and are usually accompanied with reports and screenshots, defect triangulation is easy and is just a matter of re-execution of automation scripts. With pure manual testing, a tester would have to spend effort on manually recreating the defect, capturing all the required details, and then reporting it for defect tracking. With mobile automation, the same flow can be triggered multiple times on a multitude of devices hence, the same defect can be replicated and isolated if it occurs only on a specific set of devices. Accurate and realistic real-life mobile scenarios: Since a mobile requires tests to be specifically designed for variable network conditions and other considerations, such as device screen sizes, orientation, and more, which are difficult to recreate accurately with pure manual testing effort, automation scripts can be developed that accurately to recreate these real-world scenarios in a reliable way. These types of tests are mainly not required to be developed for functional automation suites, and hence, this is one of the major differences. For the most realistic results, conventional wisdom is to test automation on actual devices—without optical recognition, emulation, jailbreaking, or tethering. It is impractical to try to automate everything, especially for mobile devices. However, leveraging commercial off-the-shelf (COTS) tools can vastly reduce the cost of automation and thereby enhance the benefits of the automation process. In the following section, we will discuss in detail the challenges that make mobile automation vastly different from conventional functional automation. The following are some of the issues that make the effective testing automation of mobile applications challenging: Restricted access to native methods to enable automation tools: Traditional functional automation tools utilize native operating system methods to emulate user interactions. This is comparatively easy to do as the operating system allows access. However, the same level of access is not available with a mobile operating system. Also, inter-application interactions are restricted in a mobile operating system and each application is treated as an individual thread. This is normally only allowed when a phone is rooted or when the application under test is modified to allow instrumentation access. So, using other software (the test automation tool) to control user inputs in a mobile application is much more difficult to achieve and consequently slower or more error prone. For example, if an Android application under test makes a call to the photo gallery, then the automated test would not be able to continue because a new application comes to the foreground. Lack of prediction techniques for UI synchronization in a Mobile environment: In addition to the restricted access mentioned in the previous point, mobile application user interface response times are dependent on many variables, such as connection speed and device configuration other than the server response times. Hence, it is much harder to predict the synchronization response in a mobile application. Due to this automation of mobile, the application is more prone to be unstable unless hardcoded wait times are included in the automation scripts. Handling location-specific changes in the application behavior: Many mobile applications are designed to interact with the user location, and behave differently as per the change in GPS coordinates. Since network strengths cannot be controlled externally, it is very difficult to predict the application behavior and to replicate the preconditions of a network strength-specific use case through the use of automation. So, this is another aspect that every automation solution has to address appropriately. Some automation tools allow the simulation of such network conditions that should be specifically handled while developing the automation suite. Supporting application behavior changes for varied form factors: As explained earlier, since there are different screen sizes, available for mobile devices, the behavior of the application is often specific to the screen size owing to responsive design techniques that are now quite widely used. Even with the change in the orientation of the devices, application use cases have alternative behavior. For example, an application interface loaded in the portrait mode would appear different, with objects in different locations than they would appear in the landscape mode. Hence, automation solutions would need to factor this in and ensure that such changes are handled in a robust and scalable way. Scripting complexity due to diversity in OS: Since many applications are developed to support various OSes, especially mobile web applications, it is a key challenge to handle application differences, such as mobile device input methods for various devices, as devices differ in keystrokes, input methods, menu structures, and display properties. With different mobile operating systems in the market, such as Android, iOS, Brew Symbian, Tizen, Windows, and BlackBerry (RIM), each having its own limitations and variations, creation of a single script for every device is a challenge that needs to be adequately tackled in order to make the automation solution more robust, maintainable, and scalable to support newer devices in future. Mobile application packages With the advancement in wireless technology, big technology companies, such as Apple, Amazon, Google, and so on, came out with a solution that provides users with a more realistic approach to finding information, making decisions, shopping, and other countless things at their fingertips by developing mobile applications for their products. The main purpose of developing mobile applications was actually to retrieve information using various productivity tools, which includes calculator, e-mail, calendar, contacts, and many more. However, with more demand for and the availability of resources, there was a rapid growth and expansion in other categories, such as mobile games, shopping, GPS and location-based services, banking, order tracking, ticket purchases, and recently, mobile medical applications. The distribution platforms, such as Apple App Store, Google Play, Windows Phone Store, Nokia Store, and BlackBerry Application World, are operated by the owners of the mobile operating systems, and mobile applications are made available to users by them. We usually hear about the terms such as a native application, hybrid application, or web application, so, did you ever wonder what they are and what is the difference is between them? Moving ahead, we will discuss the different mobile packages available for use and their salient features that make an impact on the selection of a strategy and testing tool for automation. The different mobile packages available are: Native applications Web applications Hybrid applications Native applications Any mobile application needs to be installed through various distribution systems, such as Application Store and Google Play. Native applications are the applications developed specifically for one platform, such as iOS, Android, Windows, and many more. They can interact and take full advantage of operating system features and other software that is typically installed on that platform. They have the ability to use device-specific hardware and software, such as the GPS, compass, camera, contact book, and so on. These types of applications can also incorporate gestures such as standard operating system gestures or new application-defined gestures. Native applications have their entire code developed for a particular operating system and hence have no reusability across operating systems. A native application for iOS would thus have its application handles built specifically for Objective-C or Swift and hence would not work on an Android device. If the same application needs to be used across different operating systems, which is a very logical requirement for any successful application, then developers would have to write a whole new repository of code for another mobile operating system. This makes the application maintenance cumbersome and the uniformity of features is another challenge that becomes difficult to manage. However, having different code bases for different operating systems allows the flexibility to have operating-system-specific customizations that are easy to build and deploy. Also, today, there is a need to follow very strict "look and feel" guidelines for each operating system. Using a native application might be the best way to keep this presentation correct one for each OS. Also, testing native applications is usually limited to the operating system in question and hence, the fragmentation is usually limited in impact. Only manufactures and operating system versions need to be considered. Mobile web applications A mobile web application is actually not an application but in essence only websites that are accessed via a mobile interface, and it has design features specific to the smaller screen interface and it has user interactions such as swipe, scroll, pinch, and zoom built in. These mobile web applications are accessed via a mobile browser and are typically developed using HTML or HTML5. Users first access them as they would access any web page. They navigate to a special URL and then have the option of installing them on their home screen by creating a bookmark for that page. So, in many ways, a web application is hard to differentiate from a native application, as in mobile screens, usually there are no visible browser buttons or bars, although it runs in mobile browsers. A user can perform various native application functionalities, such as swiping to move on to new sections of the application. Most of the native application features are available in the HTML5 web application, for example, they can use the tap-to-call feature, GPS, compass, camera, contact book, and so on. However, there are still some native features that are inaccessible (at least for now) in a browser, such as the push notifications, running an application in the background, accelerometer information (other than detecting landscape or portrait orientations), complex gestures, and more. While web applications are generally very quick to develop with a lot of ready-to-use libraries and tools, such as AngularJS, Sencha, and JQuery, and also provide a unique code base for all operating systems, there is an added complexity of testing that adds to the fragmentation problem discussed earlier. There is no dearth of good mobile browsers and on a mobile device, there is very limited control that application developers can have, so users are free to use any mobile browser of their choice, such as Chrome, Safari, UC Browser, Opera Mobile, Opera Mini, Firefox, and many more. Consequently, these applications are generally development-light and testing-heavy. Hence, while developing automation scripts, the solution has to consider this impact, and the tool and technique selected should have the facility to run scripts on all these different browsers. Of course, it could be argued that many applications (native or otherwise) do not take advantage of the extra features provided by native applications. However, if an application really requires native features, you will have to create a native application or, at least, a hybrid application. Hybrid applications Hybrid applications are combinations of both native applications and web applications, and because of that, many people incorrectly call them web applications. Like native applications, they are installed in a device through an Application Store and can take advantage of the many device features available. Just like web applications, hybrid applications are dependent on HTML being rendered in a browser, with the caveat that the browser is embedded within the application. So, for an existing web page, companies build hybrid applications as wrappers without spending significant effort and resources, and they can get their existence known in Application Store and have a star rating! Web applications usually do not have one and hence have this added disadvantage of lacking the automatic publicity that a five-star rating provides in the mobile stores. Because of cross-platform development and significantly low development costs, hybrid applications are becoming popular, as the same HTML code components are reusable on different mobile operating systems. The other added advantage is that hybrid applications can have the same code base wrapped inside an operating-system-specific shell thereby making it development-light. By removing the problem posed by various device browsers, hybrid applications can be more tightly controlled, making them less prone to fragmentation, at least on the browser side. However, since they are hybrid applications, any automation testing solution should have the ability to test across different operating system and version combinations, with the ability to differentiate between various operating-system-specific functionality differences. Various tools such as PhoneGap and Sencha allow developers to code and design an application across various platforms just by using the power of HTML. Factors to be considered during mobile testing In many aspects, an approach to perform any type of testing is not so different from mobile automation testing. From methodology and experience, while working with the actual testing tools, what testers have learned in testing can be applied to mobile automation testing. So, a question might come to your minds that then, where does the difference lie and how should you accommodate these differences? So, following this topic, we will see some of the factors that are highly relevant to mobile automation testing and require particular attention, but if handled correctly, then we can ensure a successful mobile testing effort. Some of the factors that need to be taken care of in testing mobile applications are as follows: Testing for cross device and platform coverage: It is not feasible to test an application on each and every available device because of the plethora of devices that support the application across different platforms, which means you have to strategically choose only a limited, but sufficient set of physical devices. You need to remember that testing on one device, irrespective of whether it is of the same make, same operating system version, or uses the same platform cannot ensure that it would work on any other device. So, it is important that, at the very least, most of the critical features, if not all, are tested on a physical device. Otherwise, the application always runs a risk of potential failure on an untested device, especially when the target audience for the application is widespread, such as for a game or banking application. Use of emulated devices is one of the common ways to overcome the issues of testing on numerous physical devices. Although this approach is generally less expensive, we cannot rely completely on the emulated devices for the results they present, and with emulators, it may be quite possible that test conditions are not close enough to the real-life scenarios. So, an adequate coverage of different physical devices is required to test these following variations, providing sufficient coverage in order to negate the effects of fragmentation and have sufficient representation of these various factors: Varying screen sizes Different form factors Different pixel densities and resolutions Different input methods, such as QWERTY, touch screen, and more Different user input methods, such as swipes, gestures, scrolling, and many more Testing different versions of an operating system of the same platform: For thorough testing, we need to test the application on all major platforms, such as Android, iOS, Windows, and others, for the target customer base, but each one of them has numerous versions available that keep on growing regularly. Most commonly, testing automation on the latest version of any operating system can be sufficient, as the operating systems are generally backward compatible. However, due to fragmentation of the Android OS, the application would still need to be tested on at least the most commonly used versions besides the latest ones, which in some cases may be significantly behind the latest version. This is because there may be many Android devices that are on an earlier version of Android and are not supported by the latest versions of Android. Testing of various network types and network providers: Most of the mobile applications, such as banking- or information-search-related applications require network connectivity, such as CDMA or GSM, at least partially, if not completely. If the application talks to a server about the flow of information to and fro, testing on various (at least all major) network providers is important. The network infrastructure used by network providers may affect data communication between application and the backend. Apart from the different network providers, an application needs to be tested on other modes of network communication, such as Wi-Fi network as well. Testing for mobile-environment-specific constraints: The mobile environment is very dynamic and has constraints, such as limited computing resources, available memory, in-between calls or messages, network switching, battery life, and a lot of other sensors and features, such as accelerometer, gyroscope, GPS, memory cards, camera, and others, present in the device, as an application's behavior depends on these factors. An application should integrate or interact (if required) with these features gracefully, and sufficient testing needs to be carried out in various situations to ensure this. However, oftentimes, it is not practically feasible to recreate all permutations and combinations of these factors, and hence a strategic approach needs to be taken to ensure sufficient coverage. Testing for the unpredictability of a mobile user: A tester has to be more cautious and should expand the horizon while testing the applications. They should make sure that an application provides an overall good response to all users and a good user experience; hence, User Experience (UX) testing invariably needs to be performed to a certain degree for all mobile applications. Since, a mobile application's audience comprises of various people ranging from nontech people to skilled technical users and from children to middle-aged users. Each of the users have their own style of using the application and have set their own expectations of it. A middle-aged or an aged user will be much calmer while using any application than someone who is young when it comes to the performance of the application. In general, we can say that mobile users have set incredibly high expectations of the applications available in the marketplace. Mobile automation testing approaches In this section, you will understand the different approaches used for automation of a mobile application and their salient points. There are, broadly speaking, four different approaches or techniques available for mobile application testing automation: Test automation using physically present real devices Test automation using emulators and simulators Mobile web application test automation through the user agent simulation technique Cloud-solutions-based test automation Automation using real devices As the name suggests, this technique is based on the usage of real devices that are physically present with the testing automation team. Since this technique is based on the usage of real devices, it is a natural consequence that the Application Under Test (AUT) is also tested over a real network (GSM, CDMA, or Wi-Fi). To establish connectivity of the automation tool with the devices, any of the communication mechanisms, such as USB, Bluetooth, or Wi-Fi can be used; however, the most commonly used and the most reliable one is the USB connection. After the connection is established between the machines on which the automation tool is installed and the Device Under Test (DUT), the automation scripts can capture object properties of the AUT and later, the developed scripts can be executed on other devices as well, but with minor modifications. There are numerous automation tools, both licensed as well as open source freeware, available for mobile automation. Some commonly used licensed tools are: Experitest SeeTest TestPlant eggPlant Mobile /eggOn Jamo Solutions M-eux Test ZAP-fiX Prominent tools for Android and iOS automation are: Selenium with solutions such as Selendroid and Appium along with iOS and Android drivers MonkeyTalk (formerly FoneMonkey) The following are the salient features of this approach: The AUT is accessed on devices either by using a real mobile network or Wi-Fi network and can also be accessed by the Intranet network of the machine to which it is connected The automation testing tool is installed on the desktop that uses the USB or Wi-Fi connectivity to control devices under test Steps to set up automation For automation on real devices, scripts are required to be executed on the devices with a USB or Wi-Fi connection to send commands via the execution tool to the devices. The following is a step-by-step description of how to perform the automation on real devices: Determine the device connectivity solution (USB or Wi-Fi connectivity) based on the available setup. In some cases, USB connectivity is not enabled due to security policies and only in those cases is a Wi-Fi connection utilized. Identify the tool to be used for the automation based on the tool feasibility study of the application. Procure the required licenses (seat or concurrent) if a licensed tool is selected. License procurement might mean that lengthy agreements need to be signed by both parties, besides arranging for the payment of services such as support. So, this step should be well planned with enough buffer time. If the existing automation setup is to be leveraged, then an additional license needs to be acquired that corresponds to the tool (such as Quick Test Professional, Quality Center, and more). In some cases, you might also have to integrate the existing automation scripts developed with tools such as Quick Test Professional/Unified Functional Testing along with the automation scripts developed for the mobile. In such a case, the framework already in place needs to be modified. Install the tools on the automation computer and establish the connectivity with the real devices. Installation may not be as simple as just running an executable file when it comes to mobile automation. There are various network-level settings and additional drivers that are needed to connect the computer and to control various mobile devices from the computer. Hence, all this should be done and planned well in advance. Script the test cases and execute them on real devices. Limitations of this automation This approach has the following limitations: The overall cost can be high as multiple devices are required to be procured for different teams and testers Maintenance and physical security can be an overhead Script maintenance can be delayed if testing cycles are overlapping with functional and automation teams Emulators-based automation Emulators are programs that replicate the behavior of a mobile operating system and, to some extent, the device features on a computer. So, in essence, these programs are used to create virtual devices. So, any mobile application can be deployed on such virtual devices and then tested without the use of a real device. Ideally speaking, there are two types of mobile device virtualization programs: emulators and simulators. From a purely theoretical standpoint, the following are the differences between an emulator and a simulator. A device emulator is a desktop application that emulates both the mobile device hardware and its operating systems; thus, it allows us to test the applications to a lesser degree of tolerance and better accuracy. There are also operating system emulators that don't represent any real device hardware, but rather the operating system as a whole. These exist for Windows Mobile and Android, but a simulator is a simpler application that simulates some of the behavior of a device, does not emulate hardware, and does not work over the real operating system. These tools are simpler and less useful than emulators. A simulator may be created by the device manufacturer or by some other company that offers a simulation environment for developers. Thus, simulator programs have lesser accuracy than emulator programs. For the sake of keeping the discussion simple, we will refer to both as emulators in this article. Since this technique does not use real devices, it is a natural consequence that the AUT is not tested over a real network (GSM, CDMA, or Wi-Fi), and the network connection of the machine is utilized to make a connection with the application server (if it connects to a server, which around 90 percent of mobile applications do). Since the virtual devices are available on the computer, there is no external connection required between the device's operating system and automation tool. However, an emulator is not as simple as automating any other program because the actual AUT runs inside the shell of the virtual device. So, a special configuration needs to be enabled with the automation tools to enable the automation on the virtual device. The following is a diagram depicting an Android emulator running on a Windows 7 computer: In most projects, this technique is used for prelaunch testing of the application, but there are cases where emulators are automated to a great extent. However, since the emulator is essentially more limited in scope than the real devices, mobile-network-specific and certain other features such as memory utilization cannot be relied upon while testing automation with emulators. There are numerous automation tools, both licensed as well as of an open source freeware available for mobile automation on these virtual devices, and ideally, emulators for various mobile platforms can be automated with most of the tools that support real device automation. The prominent licensed tools are: ExperiTest SeeTest TestPlant eggPlant Mobile /eggOn Jamo Solutions M-eux Test Tools such as Selenium and ExperiTest SeeTest can be used to launch device platform emulators and execute scripts on the AUT. The prominent free-to-use tools for emulator automation are: Selenium WebDriver Appium MonkeyTalk (formerly FoneMonkey) Since emulators are also software that run on other machines, device-specific configurations need to be performed prior to test automation and have to be handled in the scripts. The following is the conceptual depiction of this technique. The emulator and simulator programs are installed on a computer with a given operating system, such as Windows, Linux, or Mac, which then virtualizes the mobile operating system, such as Android, iOS, RIM, or Windows, and subsequently, which can be used to run scripts that emulate the behavior of an application on the real devices. Steps to set up automation The following are the steps to set up the automation process for this approach: Identify the various platforms for which the AUT needs to be automated. Establish the connectivity to AUT by enabling the firewall access in the required network for mobile applications. Identify the various devices, platforms, emulators, and device configurations, according to which test needs to be carried out. Install emulators/simulators for the various platforms. Create scripts and execute them across multiple emulators/simulators. Advantages This approach has the following advantages: Standalone emulators that don't have real devices can be utilized No additional connectivity is required for automation This provides support for iOS and Android with freeware This provides support to all platforms and types of applications with licensed tools, such as Jamo Solutions M-eux and ExperiTest SeeTest Limitations This approach has the following limitations: This can be difficult to automate as the emulators and simulators are themselves not thoroughly tested software and might have unknown bugs. Selenium WebDriver cannot be used to automate Android applications in some versions due to a bug in the Android emulator. It might sometimes be difficult to triangulate a defect that is detected on a virtual device and it might be needed that you recreate it on a real device first. In many cases, it has been observed that defects caught on emulators are not reproduced on real devices. For iOS simulators, access to a Mac machine with Xcode is required, which can be difficult to set up in a secure Offshore Development Center (ODC) due to security restrictions. User agent-simulation-based automation The third technique is the simplest of all. However, it is also very limited in its scope of applicability. It can be used only for mobile web applications and only to a very limited extent. Hence, it is generally only used to automate the functional regression testing of mobile web applications and rarely used for GUI validations. User agent is the string that web servers use to identify information, such as the operating system of the requester and the browser that is accessing it. This string is normally sent with the HTTP/HTTPS request to identify the requester details to the server. Based on this information, a server presents the required interface to the requesting browser. This approach utilizes the browser user agent manipulation technique. This is depicted in the following schematic diagram: In this approach, an external program or a browser add-on is used to override the user agent information that is sent to the web application server to identify the requestor system as a mobile instead of its real information. So, for example, when a web application URL such as https://www.yahoo.com is accessed from a mobile device, the application server detects the requester to be a mobile device and redirects it to https://mobile.yahoo.com/, thereby presenting the mobile view. If the user agent information is overridden to indicate that it is coming from a Safari browser on an iPhone, then it will be presented with the mobile view. The following screenshot depicts how the application server has responded to a request when it detects that the request is from an iPhone 4 and is presented the mobile view: Since the mobile web application is accessed entirely from the computer, automation can be done using traditional web browser automation tools, such as Quick Test Professional/Unified Functional Testing or Selenium. The salient features of this technique are as follows: With browser user agent manipulation, any mobile platform can be simulated Browser uder agent manipulation is limited to only mobile web applications and is not extended to native and hybrid applications Browser simulation can be done using freeware files that are available for all leading web browsers The common user agent switching tools are: Bayden UAPick for IE User agent switcher add-on for Firefox Fiddler for IE Modify Headers for Firefox UA Spoofer add-on for Chrome Built-in device emulator with Chrome that can be accessed from developer tools Steps to set up the automation The following are the steps to set up the automation process for this approach: Identify the various platforms for which the AUT needs to be validated. Identify the user-agent switcher tool that corresponds to any browser that needs to be leveraged for testing. Identify the user-agent string for all platforms in scope and set up configuration in the user-agent switcher tool. Leverage any functional testing tool that offers testing capabilities using any web browser, for example, Quick Test Professional, RFT, SilkTest, and Selenium WebDriver. Advantages This approach has the following advantages: All platforms can be automated with little modification of scripts Quick implementation of automation solution Can leverage an open source software, such as Selenium for automation Existing automation set up can be leveraged Limitations This approach has the following limitations: Least representative of real device-based tests Device-specific issues cannot be captured through this approach This cannot be used for UI-related test cases This approach supports only web-based mobile applications Cloud-based automation This technique provides most of the capabilities for test automation, but is also one of the more expensive techniques. In this technique, automation is done on real devices connected to real networks that are accessed remotely through cloud-based solutions, such as Perfecto Mobile, Mobile Labs, Sauce Labs, and DeviceAnywhere. The salient features of this technique are as follows: Cloud-based tools, such as Perfecto mobile and Device Anywhere provide a WYSIWYG (What You See Is What You Get) solution for automation Both OCR (Optical Character Recognition) and native object recognition and analysis is utilized in these tools for automation These tools also provide simple high-level keywords, such as Launch Browser, Call Me, and many more that can be used to design test cases The scripts that are thus created need to be re-recorded for every new type of device due to differences between the interface and GUI objects Mobile devices are accessed via a web interface or thick-client, by teams in various regions The devices are connected to real networks that use Wi-Fi or various mobile network operators (AT&T, Vodafone, and more) The AUT is accessed via the Internet or through a secure intranet connection This approach provides offer integration with common automation tools such as Quick Test Professional/UFT and Selenium Steps to set up the automation The following are the steps to set up the automation process for this approach: Identify the various platforms and devices for which the AUT needs to be automated. Establish connectivity to AUT by enabling the firewall access for mobile web applications. Open an account with the chosen cloud solution provider and negotiate to get the licenses for automation or set up a private cloud infrastructure within your company premises. Install the cloud service provider client-side software setup along with the automation plugin for the relevant tool of choice (UFT or Selenium). Book the devices as per testing needs (this usage normally has a cost associated with it). Create scripts and execute across multiple devices. Advantages This approach has the following advantages: This allows us to test automation on multiple devices of various manufactures (hardware) For example: Samsung, Apple, Sony, Nokia A script can be executed on multiple mobile devices from the same manufactures (models) For example: Galaxy SII, Galaxy SIII, iPhone 4S, iPhone 5, iPad2 Scripts can be tested on different platforms (software) For example: Android 2.3 - 4.4, iOS 4-8, Symbian, Bada Limitations This approach has the following limitations: Network latency may be experienced Cost can be high as fees depends on device usage Setting up a private mobile lab is costly, but may be necessary due to an organization's security policies, particularly in legally regulated industries, such as BFSI organizations Types of mobile application tests Apart from the usual functional test, which ensures that the application is working as per the requirements, there are a few more types that need to be handled with an automation solution: Interrupt testing: A mobile application while functioning may face several interruptions that can affect the performance or functionality of an application. The different types of interruptions that can adversely affect the functionality of an application are: Incoming calls and SMS or MMS Receiving notifications, such as Push Notifications Sudden removal of battery Transfer of data through a data cable by inserting or removing the data cable Network/Data loss or recovery Turning a Media Player off or on Ideally, an application should be able to handle these interruptions, for example, whenever an interruption is there, an application can go into a suspended state and resuming afterwards. So, we should design automation scripts in such a way that they can not only test these interrupts, but they can reliably also reproduce them at the requisite step of the flow. UI testing: A user interface for a mobile application is designed to support various screen sizes and hence, the various components of a mobile application screen appear differently or in some cases, even behave differently as per the OS or device make. Hence, any automation script needs be able to work with varying components and also be able to verify the component's behavior. Use of automation ensures that the application is quickly tested and the fixes are regression tested across different applications. Since UI is where the end users interact with the application, use of a robust automation suite is the best way to ensure that the application is thoroughly tested so that it rolls out to the users in the most cost-effective manner. A properly tested application makes the end user experience more seamless and thereby, the application under test is more likely to get a better star rating and its key to commercial success. Installation testing: Installation testing ensures that the installation process goes smoothly without the user facing any difficulty. This type of a testing process includes not only installing an application but also updating and uninstalling an application. Use of automation to install and uninstall any application as per the defined process is one of the most cost-effective ways to do this type of testing. Form factor testing: Applications may behave differently (especially in terms of user interface) on smartphones and tablets. If the test application supports both smartphones and tablets, it should be tested on both form factors. This can be treated as an extension to the UI testing type. Selection of the best mobile testing approach While selecting a suitable mobile testing approach, you need to look at the following important considerations: Availability of automation tools: The availability of relevant mobile automation tool plays a big role in the selection and implementation of the mobile automation approach. Mode of connection of devices: This is one of the primary, if not the most important, aspect that plays a pivotal role in the selection of a mobile automation approach. There are different ways in which devices can be connected to the automation tools such as: Using a USB connection Using a Wi-Fi connection Using a Bluetooth connectivity (only for a very limited set of tools) Using localized hotspots, that is, having one device as a hotspot and other devices riding its network for access Cloud connection Use of emulators and simulators All these approaches need specific configurations on machines, and with the automation tools, which may sometimes be restricted, any automation solution should be able to work around the constraints in various setups. The key consideration is the degree of tolerance of the automation solution. The four different approaches that we discussed earlier in this article have each got a different level of accuracy. The least accurate is the user agent-based approach because it relies just on a web browser's rendering on a Windows machine rather than a real device. The most accurate approach, in terms of closeness to the real-world situation, is the use of real devices. However, this approach suffers from restrictions in terms of scalability of the solution, that is, supporting multiple devices simultaneously. Use of emulators and simulators is also prone to inaccuracies with respect to the real-device features, such as RAM, screen resolutions, pixel sizes, and many more. While working with cloud-based solutions, a remote connection is established with the devices, but there can be unwanted signal delays and screen refresh issues due to network bandwidth issues. So, any approach that is selected for automation should factor in the degree of tolerance that is acceptable with any automation suite. For example, for a mobile application that makes heavy usage of graphics and advanced HTML 5 controls, such as embedded videos and music, automation should not be carried out with an emulator solution, as the degree of accuracy would suffer adversely and usually beyond the acceptable tolerance limit. Consider another application that is a simple mobile web application with no complex controls and that doesn't rely on any mobile-device-specific controls, such as camera controls, or touch screen sensitive controls, such as pinch and zoom. Such an application can easily be automated with the user agent-based approach without any significant impact on the degree of accuracy. If an application uses network bandwidth very heavily, then it is not recommended to use the cloud-based approach, as it will suffer from network issues more severely and would have unhandled exceptions in the automation suite. Conversely, the cloud-based approach is most suitable for organizations that have geographically and logically dispersed teams that can use remotely connected devices from a single web interface. This approach is also very suitable when there are restrictions on the usage of other device connection approaches, such as USB, Wi-Fi, or Bluetooth. Although this approach does need additional tools to enable cloud access, it is a worthwhile investment for organizations that have a high need for system and network security, such as banking and financial organizations. Troubleshooting and best practices The following best practices should ideally be followed for any mobile automation project: The mode of connectivity between the AUT, DUT, and computer on which the automation tool is installed should be clearly established with all the considerations of any organization's security policies. In most cases, there is no way to workaround to the absence of USB connectivity, other than to use cloud-based automation solutions. So, before starting a project, the physical setup should be thoroughly vetted. The various operating systems and versions, mobile equipment manufacturers, and different form factors that need to be supported with the application, and consequently, the automation solution should be designed to support all of them. However, if you start automating before identifying all the supported devices, then there would invariably be a lot of rework required to make the scripts work with other devices. Hence, automation scripts should be made for all supported OSes and devices right from the design stage. A user agent-based automation can only be implemented for mobile web applications. It is a cost-effective and quick way to implement solutions since it involves automation of just a few basic features. However, this technique should not be relied upon for validating GUI components and should always be accompanied with a round of device testing. If any simulation or emulation technique (user agent or emulators/simulators) is used for automation, then it should strictly be used for functional regression testing on different device configurations. Ideally, projects utilizing these solutions should also have a GUI testing round with real devices, at least for the first release. If a geographically-distributed team is to utilize the automation solution, for example, an offshore-onsite team that needs to use the same devices, then the most cost-effective solution in the long run is the cloud-based automation. Even though the initial setup cost of the cloud solution generally is the highest of the four techniques, since different teams can multiplex and use devices from different locations and so the overall cost is offset by using fewer devices overall. During the use of emulators/simulators, the automation scripts should be designed to trigger the virtualization program with the required settings for memory, RAM, and the requisite version of the operating system, so that there is no manual intervention required to start the programs before you trigger the execution. Also, this way, scripts can be triggered remotely and in an unmonitored way. Irrespective of the technique utilized, a proper framework should be implemented with the automation solution. Summary In this article, we learned what mobile test automation is, what are the different mobile packages that are available, and what factors should be considered during mobile automation testing. We then moved on to learn the different types of approaches and selection of the best approach according to any specific project requirements. So, it is evident that with the use of automation to test any mobile application, a good user experience can be ensured with a defect-free software, with which a good star rating can be expected for the AUT. Resources for Article: Further resources on this subject: DOM and QTP [article] Automated testing using Robotium [article] Managing Test Structure with Robot Framework [article]

(For more resources related to this topic, see here.) The progression of testing Way back when testing used to be primarily manual, test cases were created and executed by developers or quality assurance team members. These test cases would comprise of anything from simple unit tests (testing single methods or classes of code) or integration tests (testing multiple components within code) or even functional tests (tests that ensure the system performs as required). As we began to develop differently, whether it was from the agile project methodology or extreme programming methodologies, we needed more robust tools to support our needs. This led to the advent of automated testing. Instead of a tester working with your application and running tests against it, they could simply press a few buttons, hit a few key strokes, and execute a 100 or 200 test case suite against your application to see the functionality. In some realms, something called a test harness was used. Test harnesses usually included running the compiled application in some kind of a sandbox environment that was probably something like production (this is after all the final binary that would be rolled out) that may or may not have pointed to a database (if it did, and it was smart, it probably pointed to a completely non-discreet database instance) to perform some level of testing. User input would be simulated and a report (possibly in some cryptic format that only few understood) would be generated indicating whether the application did what was expected or not. Since then, new tools such as JUnit, Selenium, SoapUI to name a few, have been introduced to add more functionality to your test cases. These are meant to drive both unit and functional testing of your application. They are meant to be standard tools, easy to use and reuse, and overall a platform that many developers can work with, and can be a desirable skill set for employers. Standardizing of tools also allows for more integrations to occur; it may be difficult to get leverage to build an integration with your own built tools, with many developers wanting an integration with widely used frameworks A and B. What is Arquillian If you haven't heard of Arquillian before (or are very new to it), this may be the section for you. Arquillian is a testing framework for Java that leverages JUnit and TestNG to execute test cases against a Java container. The Arquillian framework is broken up into three major sections: test runners (JUnit or TestNG), containers (Weld, OpenWebBeans, Tomcat, Glassfish, and so on), and test enrichers (integration of your test case into the container that your code is running in). ShrinkWrap is an external dependency for you to use with Arquillian; they are almost sibling projects. ShrinkWrap helps you define your deployments, and your descriptors to be loaded to the Java container you are testing against. The JUnit test container is used throughout. If you'd like to use TestNG with your code, you simply need to replace the JUnit container with the TestNG container, and have your test classes extend the Arquillian class found there. The JUnit test cases use a JUnit Runner to start Arquillian. The containers used will vary with each case. The Arquillian difference Arquillian can be considered a standardized test harness for JVM-based applications. It abstracts the container or application start-up logic away from your unit tests and instead drives a deployment runtime paradigm with your application, allowing you to deploy your program, both via command line and to a Java EE application server. Arquillian allows you to deploy your application to your targeted runtime to execute test cases. Your targeted runtime can be an embedded application server (or series of libraries), a managed application server (where Arquillian performs the calls necessary to start and stop the JVM), or even a remote application server (which can be local to your machine, remote in your corporate infrastructure, or even the cloud). Arquillian fits in to certain areas of testing, which can vary based on testing strategies for your application. If you are using Java EE 6, you may want to use an embedded CDI container (such as Weld) to unit test parts of your application. These tests could happen hourly or every time someone commits a code change. You could also use Arquillian to automate your integration test cases, where you use a managed or embedded application server to run your application, or even just parts of your application. You can even use Arquillian to perform automated acceptance testing of your application, using other tools such as Selenium to drive requests through the user interface. This can also be used to smoke test deployments of applications. Arquillian can be used with a wide variety of containers that support everything from JBoss 4.2 (slightly pre-Java EE 5) through Java EE 6 and can control these containers in what Arquillian considers types – embedded, managed, and remote. Embedded application servers run within the same JVM as your test cases always do. Managed run within a separate JVM and are started and stopped by Arquillian. A managed container will start on the first test that requires a deployment and stop once all tests have been executed; there is no restart. Remote containers are as the name implies, remote JVMs. This could be on the same physical hardware that your test runs on or a remote piece of hardware that the application server runs on. The application server must be running in order to deploy. Note that if there is a problem deploying, such as the managed application server will not start or the remote application server will not start, Arquillian will fail once and assume that deployments will fail afterwards for the remaining test cases: Do not mix your unit test application servers that are used for automated testing and those that you use for manual testing. Whether it's a separate instance required, a distinct domain, profile, whichever your application server vendor supports, avoid mixing them. One of your biggest blockers may be from your manually deployed application interfering with your automated testing application. Even though remote application servers can be physically separated from your testing, they typically require the binaries to be locally available. Plan to have a copy of your application server available on your CI server(s) for this purpose. Prepare your application for this kind of testing. Whether it's the automatic deployment or undeployment of resources (JMS queues, JDBC connections, users, and so on) or ensuring that the server is up and running (for example, prebuild, kick off a kill, and restart process) make sure this can all happen from your build, either in your CI server or using scripts within your source repository. Do not try to reuse application servers across applications. If two test cases are running in parallel, you can run into inconsistent results. The fundamentals of a test case As our software has evolved, our strategy for testing it must evolve as well. We have become more dependent on techniques such as dependency injection (or inversion of control – IoC). When we take this in to consideration, we realize that our testing has to change. Take a look at the following example: @Testpublic void testCalculationOfBusinessData() {CalculatorData cd = new CalculatorData(1, 3, 5);CalculatorService ms = new CalculatorServiceImpl();ms.calculateSum(cd);assertEquals(1 + 3 + 5, cd.getResult());} We can assume that the method calculateSum takes the int values passed in to MyDataObject and sums up the values. As a result, when I construct it using 1, 3, 5 the total should come out to 9. This is a valid test case for our service layer, since our service layer knows what implementations exist out there and how they should be tested. If there was another implementation of CalculatorService that multiplied all results by 2, a separate test case or test class would exist which tested that object. Let's say we look at the business layer that invokes this service object: @Modelpublic class CalculatorController {@Injectprivate CalculatorService service;@Injectprivate CalculatorForm form;/**For the injected form, calculates the total of the input**/public void sum() {CalculatorData data = new CalculatorData(form.getX(),form.getY(),form.getZ());service.calculateSum(data);form.setSum(data.getCalculatedResult());}} This example uses JSR-330 annotations to inject references to CalculatorService, a service layer object that can perform basic calculator functions and CalculatorForm, some sort of UI component that has form input and output that can be read or returned. If we want to test this class, we will immediately run into a problem. Any invocation of the sum method outside of a JSR-330 (dependency injection for Java) container will result in a NullPointerException. So what does Arquillian do to make our test case more legitimate? Let's take a look at the test case and review the anatomy to understand that better: @RunWith(Arquillian.class)public class CalculatorTest {@Deploymentpublic static JavaArchive createArchive() {return ShrinkWrap.create(JavaArchive.class,"foo.jar").addAsManifestResource(EmptyAsset.INSTANCE,"beans.xml").addPackage(CalculatorData.class.getPackage());}@Inject CalculatorForm form;@Inject CalculatorController controller;@Testpublic void testInjectedCalculator() {form.setX(1);form.setY(3);form.setZ(5);controller.sum();assertEquals(9,form.getSum());}} There are a few pieces that make up this test case, each of which we'll need to review. These are given as follows: The @RunWith annotation: It tells JUnit to use the Arquillian runner for running this class. The @Deployment annotation: It tells Arquillian to use the specified archive for deployment purposes and testing purposes. The injection points: In this case, CDI injection points represent the objects under test. The Test: This is where we process the actual test case. Using the injected objects, we simulate form input by inserting values in X, Y, and Z in the form, then invoke the controller's sum method, which would be called from your user interface. We then validate that the resulting sum matches our expectations. What we gained was leveraging Arquillian to perform the same IoC injection that we would expect to see in our application. In addition, we have the following dependencies within our Maven pom file: <dependencyManagement> <dependencies> <dependency> <groupId>org.jboss.shrinkwrap.resolver</groupId> <artifactId>shrinkwrap-resolver-bom</artifactId> <version>2.0.0-alpha-1</version> <scope>import</scope> <type>pom</type> </dependency> <dependency> <groupId>org.jboss.arquillian</groupId> <artifactId>arquillian-bom</artifactId> <version>${org.arquillian.bom.version}</version> <scope>import</scope> <type>pom</type> </dependency> </dependencies> </dependencyManagement> <dependencies> <dependency> <groupId>junit</groupId> <artifactId>junit</artifactId> <version>4.10</version> <scope>test</scope> </dependency> <dependency> <groupId>org.jboss.spec</groupId> <artifactId>jboss-javaee-6.0</artifactId> <version>1.0.0.Final</version> <type>pom</type> <scope>provided</scope> </dependency> <dependency> <groupId>org.jboss.shrinkwrap.resolver</groupId> <artifactId>shrinkwrap-resolver-impl-maven</artifactId> <scope>test</scope> </dependency> <dependency> <groupId>org.jboss.arquillian.junit</groupId> <artifactId>arquillian-junit-container</artifactId> <scope>test</scope> </dependency> <dependency> <groupId>org.jboss.arquillian.container</groupId> <artifactId>arquillian-weld-ee-embedded-1.1</artifactId> <version>1.0.0.CR3</version> <scope>test</scope> </dependency> <dependency> <groupId>org.jboss.weld</groupId> <artifactId>weld-core</artifactId> <version>1.1.8.Final</version> <scope>test</scope> </dependency> <dependency> <groupId>org.slf4j</groupId> <artifactId>slf4j-simple</artifactId> <version>1.6.4</version> <scope>test</scope> </dependency> </dependencies> Our dependencies are as follows: Arquillian BOM: It's an overall list of all artifacts for running Arquillian ShrinkWrap: It's a separate API for creating deployments JUnit and Arquillian JUnit Container: Arquillian support for working with JUnit Arquillian Weld EE: Arquillian containers control what your deployment runtime should be Testing profiles One of the benefits of using Arquillian with Maven is that you can control how your tests execute using Maven profiles. With this approach, you can define a profile per container to test against. In some cases, you may want to test your application or library against multiple containers. This allows you to reuse deployments against different libraries. We still remain with the core limitation: you can only have one container on your classpath at a time. Let's suppose we take this same application, but want to run it against Apache OpenWebBeans as well as Weld. Each Arquillian container has a default Maven profile configuration that can be used for testing. These containers are covered within the Arquillian documentation, found at https://docs.jboss.org/author/display/ARQ/Container+adapters. Steps to try out containers are given as follows: Import the configuration defined by the container definition. Run a mvn clean install Pconfiguration_name. You can choose to set a default profile as well if you like. If you don't choose a default profile, you will need to specify one every time you want to run tests. Running the tests for this project, for both the weld-ee-embedded-1.1 container and the openwebbeans-embedded-1 profile should result in the same thing – a working test suite that is valid in both implementations. At the time of writing, I used Weld 1.1.8.Final and OpenWebBeans 1.1.3. It is important to point out at this time that these profile names are only useful if your application is designed purely for cross-platform testing and you want to run all test cases against each individual platform. If your application only targets a single platform, you may want to derive test cases that run on that platform as well as any subcomponents of that platform (for example, if you are a WebSphere v8 user, you may want your unit tests against OpenWebBeans and integration against WebSphere; however, if you are a WebLogic user, you would want to use Weld and WebLogic for your testing). Typically, when it comes to testing, you will use a distinct Maven profile to cover your stages of testing. You should set up a default Maven profile that runs only your basic tests (your unit tests); this will be set as activeByDefault. This should include any testing dependencies needed to run only these unit tests. You may optionally choose to only run certain parts of your test suite, which could be distinct packages under src/test/java or even standalone projects that are children to your parent that are only run under certain circumstances. I prefer the former approach, since the usage of conditional child projects can become confusing for developers. Profiles are useful for conditional dependencies, since they do include a dependency and dependencyManagement section in their pom files. You can also avoid dependency leaking. For example, most applications require the use of the full Java EE 6 APIs, but including these APIs with your Glassfish build will cause your tests to fail. Likewise, deploying to a managed application server may require different APIs than deploying to a remote application server. Categorizing your test cases One thing that Arquillian ultimately derives is that names mean everything. There are two naming schemes that you should follow always, they relate to the questions "what" and "how". What components are you testing? What phase of testing are you under? How does this class relate to your test cases? How is this code being deployed? These questions really pop up due to the nature of Arquillian, and really show off its robustness. Considering some of the main testing phases, unit test, integration test, system test, compatibility test, smoke test, acceptance test, performance test, usability test should all relate to specific packages in your test code. Here's a proposed naming scheme. I will assume that your code starts with com.mycompany.fooapp where com.mycompany is your company's domain, fooapp is the name of your application. You may have packages below this such as com.mycompany.fooapp.model or com.mycompany.fooapp.controller or even com.mycompany.fooapp.modulea.controller all of which represent varying testing scenarios that you may consider. com.mycompany.fooapp.test: This is the parent package for all test classes. There may or may not be any classes in this package. com.mycompany.fooapp.test.utils: This is where all of your test utility code goes. This would be where any deployments are actually build, but invoked from your test classes. com.mycompany.fooapp.test.unit: This is where all unit tests should exist. The packages/classes under test should fall relative under here. For example, com.mycompany.fooapp.test.unit.controller should test your controller logic. com.mycompany.fooapp.test.integration: Likewise, all integration test cases should fall under here. Following this pattern, you can derive your functional, acceptance, usability, and so on test case names. Following this pattern, you can easily define Maven profiles within your projects to test out your various layers. Let's suppose you want to define a unittest profile where the build includes running all unit tests (usually light weight tests, that maybe use an embedded container with Arquillian), you could do something like this: <profile><activation><activeByDefault>true</activeByDefault></activation><id>unittest</id><build><plugins><plugin><groupId>org.apache.maven.plugins</groupId><artifactId>maven-surefire-plugin</artifactId><configuration><includes><include>**/unit/**</include></includes></configuration></plugin></plugins></build></profile> This will tell Maven to only run the tests in the unit test packages by default. This allows your builds to go by quickly and test key components of your code when needed. Since it's active by default, you would get these tests run anytime you kick off mvn install or mvn test from the command line, or your continuous integration server. Likewise, you may want to run more tests during your integration testing or system testing phases. These may or may not overlap with one another, but would likely include your unit tests as well. You could use the following, very similar Maven profile to achieve that: <profile><id>integrationtest</id><build><plugins><plugin><groupId>org.apache.maven.plugins</groupId><artifactId>maven-surefire-plugin</artifactId><configuration><includes><include>**/unit/**</include><include>**/integration/**</include><include>**/system/**</include><include>**/regression/**</include></includes></configuration></plugin></plugins></build></profile> With the way this profile is configured, any time you invoke mvn test – integrationtest you will run all unit, integration, system, and regression test cases. These tests will probably take a while. If you're lucky and your application server supports an embedded mode it probably has a quick start up/shutdown. These tests may be running against a remote container though, so the deployment and execution time may take longer. Enriching your tests One of the core principles noted in the demo test case is that we injected a reference to the business object being tested, which itself had injection points as well that were satisfied. There are three core injection points supported by default with Arquillian, though others are supported by extensions as well as some custom injection points defined by Arquillian itself. They are as follows: @Resource: It defines references to any JNDI entry as per standard naming and injection schemes. @EJB: As long as you are including a deployed EJB in your archive, you can refer to it as long as your target container includes EJB support. EJB JNDI entry conventions remain intact. It is also worthwhile to note that Local and Remote interfaces matter. When you deploy to a same JVM application server, or a remote application server you must ensure that your injected test cases can see the EJB reference appropriately. @Inject: This is available as long as you have deployed a valid bean archive (requires beans.xml in META-INF or WEB-INF). Running out of the container Sometimes you will need to mix your Arquillian tests with non Arquillian tests, or sometimes you have special test cases that require a deployment but should not be run with that deployment (simulating a remote client to that deployment, for example). This may include a set of unit tests that have deployments to the container and some that do not require this. You can run the test outside of the container JVM by using the @RunAsClient annotation. This can be done at the class level or at the method level, allowing you to have multiple run-types within a single test class. This approach would be useful if you want to test a web service client, either SOAP or a REST API, ensuring that you are executing outside of the container JVM for your client. One of the custom injection points that Arquillian supports is a @ArquillianResource URL baseUrl; this represents the base URL of a remote container deployment that you can use in your client-level tests. The ArquillianResource annotation supports an additional value attribute that can be the class of a Servlet deployed in case you have multiple deployments occurring as a part of your test. Once you have the URL for your application deployment, you can build the location of your web services to be able to use your client application for testing purposes. This would also be used if you wanted to functionally test your application via HTTP, perhaps using Selenium as the driver for the application. The other types of objects available for injection using this annotation are the Deployer used and the InitialContext of the remote application server. The deployer gives you access to the underlying deployment while InitialContext would allow you to look up remote objects – for example, EJBs, JMS Queues/Topics/ConnectionFactories, or any other remotely accessible resource. Efficient test authoring Now that we have a test that can run against multiple containers, we need to start adding more test cases to our suite. When we do this, we have to keep two key elements in mind. Don't repeat yourself, and don't bloat your software. In many of our applications, we have a number of components that make up various layers. Some of them are dependent on one another, others are more generic. One of the key things to think about when planning your Arquillian testing is how should your JAR files look. Let's suppose we have an entire data object layer that has dependencies all throughout your application. We must have those classes in every test case. However, we can usually skip to only certain controllers and business beans within their specific test cases. Remember to create utilities to define your object structure, this gives you a single entry point for creating your deployment archive and allows for better extensibility. Here is a prototype class that can be used to start, it supports creating both a JAR file as well as a full web application: public class TestUtils {public static JavaArchive createBasicJar() {return ShrinkWrap.create(JavaArchive.class,"test.jar").addAsManifestResource(EmptyAsset.INSTANCE,"beans.xml").addPackages(false,getCorePackages());}public static WebArchive createWebApp() {return ShrinkWrap.create(WebArchive.class,"test.war").addAsWebInfResource(EmptyAsset.INSTANCE,"beans.xml").addPackages(false,getCorePackages());}public static Package[] getCorePackages() {return new Package[]{CalculatorData.class.getPackage()};}} What these methods add is a significant reduction in code that is impacted by a package change, or code refactoring in your primary code base. Let's suppose that we want to extend getCorePackages to also take in a number of classes, which when added, add the entire package to the deployment: public static Package[] getCorePackages(Class<?>...classes) {List<Package> packages = new ArrayList<Package>();if(classes != null) {for(Class<?> c : classes) {packages.add(c.getPackage());}}packages.add(CalculatorData.class.getPackage());return packages.toArray(new Package[]{});} One benefit that we have using this approach is that anyone who was using getCorePackages() does not need to change their code, since the argument is not required. Note that the ShrinkWrap API has several addPackage/addPackages methods. This one used has a first argument, Boolean, whether to re-curse through child packages to find your code. Going back a few pages to some naming conventions I proposed, what would happen if you add the package com.mycompany.fooapp to your bundle? All of your application classes, including test classes, would be added to the deployment you are creating. This is probably not what you would have expected, as a result, the recommendation is to not re-curse into child packages and instead just list out each package you want added explicitly. Another option to consider is to have your test classes delegate their work. Commonly thought of as the façade programming paradigm, you can actually apply this to your test code as well. If you have code that should be tested distinctly but use different deployments you may want to use distinct deployment objects but reuse your test case. This may involve using a controller type test case that simply delegates its test calls to another object that is meant to purely handle the testing logic. Your controller would have methods annotated @Test and include your @Deployment(s) but would delegate logic to another class, potentially an injected test executor class. ShrinkWrap – building your own app One of the more curious things about Arquillian is that your test case is responsible for constructing the application to be deployed. When you're working with Maven it is especially odd, since all of the information to build the application is there, either implicitly based on the project structure or explicitly listed in your pom file. There are two realizations around this that are important: Your test cases are meant to test anything from a subset of your code to your entire application. Arquillian is flexible enough to handle both extremes. Arquillian works great with Maven, but also works with other build tools such as Ant and Gradle. In order to support the dynamic generation of your application, the ShrinkWrap project exists to help dynamically build Java archive files. There are four primary archive types supported in ShrinkWrap: Java Archive (plain JAR files), Web Archive (WAR files), Enterprise Archive (EAR files), and Resource Adapters (RARs). Your Arquillian test case can declare any number of these archive files to be created for testing purposes. Another place that ShrinkWrap helps with is the creation of deployment descriptors. These could be application.xml files, or persistence.xml files, or any of the standard deployment descriptors that you would use in your application. Likewise, it has extensibility built in to allow the creation of new descriptors in a programmatic fashion. This article assumes that you are using the ShrinkWrap 2.0 APIs; one of the key features added is support for resolving dependency files from reading a Maven pom file. Another key thing you need to do is modify your test classpath to include some files from your core application. Here is an example of what to do, from another project I was working on recently: <testResources><testResource><directory>src/test/resources</directory></testResource><testResource><directory>src/main/resources</directory></testResource><testResource><directory>src/main/webapp/WEB-INF</directory><targetPath>web</targetPath></testResource></testResources> This will make it easier to reference your main application resources without requiring full paths or file manipulation. ShrinkWrap also provides ways to build deployment descriptors. This is a programmatic approach to adding the descriptor to your deployment, including programmatically creating the descriptor. Because of the need to test using the same descriptors being built with the production application, I have found it easier to reference to an existing descriptor. However, in some cases it may make more sense to use one customized to your test applications. In this scenario, I would strongly recommend creating a utility method to build the descriptor. To do this, we will add the following to our Maven pom.xml in the dependencyManagement section: <dependency><groupId>org.jboss.shrinkwrap.descriptors</groupId><artifactId>shrinkwrap-descriptors-bom</artifactId><version>2.0.0-alpha-4</version><scope>import</scope><type>pom</type></dependency> Declare the following dependency: <dependency><groupId>org.jboss.shrinkwrap.descriptors</groupId><artifactId>shrinkwrap-descriptors-impl-javaee</artifactId></dependency> Then add the following method to our code: public static StringAsset createWebXml() {return new StringAsset(Descriptors.create(WebAppDescriptor.class).exportAsString());}public static WebArchive createWebApp() {return ShrinkWrap.create(WebArchive.class,"test.war").addAsWebInfResource(EmptyAsset.INSTANCE,"beans.xml").addAsWebInfResource(createWebXml(), "web.xml").addPackages(false,getCorePackages());} This will generate the web.xml file expected for your test case. One of the benefits of using the descriptors is that we can actually import the one from our application and make changes as needed. public static StringAsset createWebXml() {WebAppDescriptor descriptor = Descriptors.importAs(WebAppDescriptor.class).fromFile(new File("web/web.xml"));descriptor.createWelcomeFileList().welcomeFile("someFile.jsp");return new StringAsset(descriptor.exportAsString());} As a result, this will use your base web.xml file but change the welcome file to something else. You can do this with a number of other descriptors, including persistence.xml, beans.xml, web-fragment.xml, and so on. Getting the most from Arquillian Arquillian does take some understanding to work with. In order to get the most from it, you have to work with it and invest time. Your standard rules still apply. Arquillian does not use any special rules when it comes to processing deployments. The rules about deployment descriptors, archive contents, and so on and so forth still apply. You need to keep the rule of thumb – if it deploys to your application server, then you can deploy the same archive via Arquillian; just make sure that you are deploying the same archive. Note that you can use the Archive's toString method to print out the contents when in doubt. This supports a formatter as well, to make it easier to read the contents. Alternatively, you can export an Archive using archive.as(ZipExporter.class).exportTo(File) if you want to manually review the file. Run as many tests as you can with Arquillian. Due to Arquillian's nature, you're going to start to find inconsistencies in your code if you don't test it with Arquillian. This could include unexpected dependency injection expectations, which Arquillian will process for you. Arquillian, since it executes your code the way the application server would as requests come in, makes your tests more consistent with the real world of how the code works. Testing more in Arquillian, even if it is just using a basic CDI container or OpenEJB container, will allow you to test more effectively. You make the best use of Arquillian when you use it throughout 100 percent of your test cases. Finally, my last key advice to getting the most from Arquillian is to remember to not over-complicate your build to make use of Arquillian. Arquillian works fine as a part of your standard build. It has no requirement to create distinct projects for each build type or make overly complicated build structures on top of your application. If you are attempting to run Arquillian with your continuous integration server, then you must ensure that different test cases run either as different steps of the build or as separate build jobs. Arquillian extensions There are a number of extensions available for Arquillian; they are designed to extend Arquillian functionality to do some domain-specific testing: Persistence, using DBUnit and validating that results of interacting with the database. REST, invoke REST APIs from Arquillian the were deployed as a part of the test case. Spring, use a Spring Context and additional Spring libraries with your test cases. Drone/Selenium, functionally test your web applications using Arquillian. Summary My main goal in this article was to give you an overview of Arquillian and introduce you to some of its core concepts. Resources for Article: Further resources on this subject: So, what is Spring for Android? [Article] Web Services Testing and soapUI [Article] SOAP and PHP 5 [Article]

Class loader architecture This section covers the class loader architecture for Apache Geronimo. The following image shows the class loader hierarchy for an application that is deployed in Apache Geronimo: The BootStrap class loader of the JVM is followed by the Extensions class loader and then the System class loader. The j2ee-system class loader is the primary class loader of Apache Geronimo. After the j2ee-system class loader, there are multiple other layers of class loaders before reaching the application class loaders. Applications have an application class loader, which loads any required application-level libraries and EJB modules. However, the web application packaged in the EAR will have its own class loader. The Administration Console has a ClassLoader Viewer portlet that can be used to view the class loader hierarchy as well as the classes loaded by each class loader. Modifying default class loading behavior In certain situations, we will need to follow a class loading strategy that is different from the default one that is provided by Apache Geronimo. A common situation where we need this functionality is when a parent configuration uses a library that is also used by the child and the library used by the parent is a different version, which is incompatible with the child's version of the library. In this case, if we follow the default class loading behavior, then we will always get the classes loaded by the parent configuration and will never be able to reference the classes in the library present in the child configuration. Apache Geronimo provides you with the ability to modify the default class loading behavior at the configuration level to handle such scenarios. This is done by providing certain elements in the deployment plan which, if present, will change the class loading behavior. These elements and the changes in class loading behavior that they represent, are explained as follows: hidden-classes: This tag is used to hide classes that are loaded in parent class loaders, so that the child class loader loads its own copy. Similarly, we can use this tag to specify the resources that should be loaded from the configuration class loader. For example, consider the case where you have a module that needs to load its copy of log4j. The server also has its own copy used for logging that is loaded in the parent class loader. We can add the hidden-classes element in the deployment plan for that module so that it loads its own copy of log4j, and the server loaded version of log4j is hidden from it. non-overridable-classes: This element specifies the list of classes that can be loaded only from the parent configurations of this configuration. In other words, the classes specified in this element cannot be loaded by the current configuration's class loader. The non-overridable-classes element is for preventing applications from loading their own copies of classes that should always be loaded from the parent class loaders, such as the Java EE API classes. private-classes: The classes that are defined by this tag will not be visible to class loaders that are the children of the current class loader. These classes will be loaded either from the current class loader or from its parents. The same class loading behavior can be achieved by using the hidden-classes tag in all of the child class loaders. inverse-classloading: If this element is specified, then the standard class loading strategy will be reversed for this module. This in effect means that a class is first looked up from the current class loader and then from its parent. Thus, the class loader hierarchy is inverted. suppress-default-environment: This will suppress the environment that is created by the builder for this module or configuration. This is a rarely-used element and can have nasty side effects if it is used carelessly. Important modules In this section, we will list the important configurations in Apache Geronimo. We will group them according to the Apache or other open source projects that they wrap. Configurations that do not wrap any other open source project will be listed under the Geronimo section. Apache ActiveMQ   org.apache.geronimo.configs/activemqbroker/2.1.4/car Apache Axis org.apache.geronimo.configs/axis/2.1.4/car org.apache.geronimo.configs/axis-deployer/2.1.4/car Apache Axis2 org.apache.geronimo.configs/axis2-deployer/2.1.4/car org.apache.geronimo.configs/axis2-ejb/2.1.4/car org.apache.geronimo.configs/axis2-ejb-deployer/2.1.4/car Apache CXF org.apache.geronimo.configs/cxf/2.1.4/car org.apache.geronimo.configs/cxf-deployer/2.1.4/car org.apache.geronimo.configs/cxf-ejb/2.1.4/car org.apache.geronimo.configs/cxf-ejb-deployer/2.1.4/car Apache Derby org.apache.geronimo.configs/derby/2.1.4/car Apache Geronimo org.apache.geronimo.configs/client/2.1.4/car org.apache.geronimo.configs/client-deployer/2.1.4/car org.apache.geronimo.configs/client-security/2.1.4/car org.apache.geronimo.configs/client-transaction/2.1.4/car org.apache.geronimo.configs/clustering/2.1.4/car org.apache.geronimo.configs/connector-deployer/2.1.4/car org.apache.geronimo.configs/farming/2.1.4/car org.apache.geronimo.configs/hot-deployer/2.1.4/car org.apache.geronimo.configs/j2ee-deployer/2.1.4/car org.apache.geronimo.configs/j2ee-server/2.1.4/car org.apache.geronimo.configs/javamail/2.1.4/car org.apache.geronimo.configs/persistence-jpa10-deployer/2.1.4/car org.apache.geronimo.configs/sharedlib/2.1.4/car org.apache.geronimo.configs/transaction/2.1.4/car org.apache.geronimo.configs/webservices-common/2.1.4/car org.apache.geronimo.framework/client-system/2.1.4/car org.apache.geronimo.framework/geronimo-gbeandeployer/2.1.4/car org.apache.geronimo.framework/j2ee-security/2.1.4/car org.apache.geronimo.framework/j2ee-system/2.1.4/car org.apache.geronimo.framework/jee-specs/2.1.4/car org.apache.geronimo.framework/jmx-security/2.1.4/car org.apache.geronimo.framework/jsr88-cli/2.1.4/car org.apache.geronimo.framework/jsr88-deploymentfactory/2.1.4/car org.apache.geronimo.framework/offline-deployer/2.1.4/car org.apache.geronimo.framework/online-deployer/2.1.4/car org.apache.geronimo.framework/plugin/2.1.4/car org.apache.geronimo.framework/rmi-naming/2.1.4/car org.apache.geronimo.framework/server-securityconfig/2.1.4/car org.apache.geronimo.framework/shutdown/2.1.4/car org.apache.geronimo.framework/transformeragent/2.1.4/car org.apache.geronimo.framework/upgrade-cli/2.1.4/car Apache Yoko org.apache.geronimo.configs/j2ee-corba-yoko/2.1.4/car org.apache.geronimo.configs/client-corba-yoko/2.1.4/car Apache Jasper org.apache.geronimo.configs/jasper/2.1.4/car org.apache.geronimo.configs/jasper-deployer/2.1.4/car JaxWS org.apache.geronimo.configs/jaxws-deployer/2.1.4/car org.apache.geronimo.configs/jaxws-ejb-deployer/2.1.4/car JSR 88 org.apache.geronimo.configs/jsr88-earconfigurer/2.1.4/car org.apache.geronimo.configs/jsr88-jarconfigurer/2.1.4/car org.apache.geronimo.configs/jsr88-rarconfigurer/2.1.4/car org.apache.geronimo.configs/jsr88-warconfigurer/2.1.4/car Apache MyFaces org.apache.geronimo.configs/myfaces/2.1.4/car org.apache.geronimo.configs/myfaces-deployer/2.1.4/car Apache OpenEJB org.apache.geronimo.configs/openejb/2.1.4/car org.apache.geronimo.configs/openejb-corbadeployer/2.1.4/car org.apache.geronimo.configs/openejb-deployer/2.1.4/car Apache OpenJPA org.apache.geronimo.configs/openjpa/2.1.4/car Spring org.apache.geronimo.configs/spring/2.1.4/car Apache Tomcat6 org.apache.geronimo.configs/tomcat6/2.1.4/car org.apache.geronimo.configs/tomcat6-clusteringbuilder-wadi/2.1.4/car org.apache.geronimo.configs/tomcat6-clusteringwadi/2.1.4/car org.apache.geronimo.configs/tomcat6-deployer/2.1.4/car org.apache.geronimo.configs/tomcat6-no-ha/2.1.4/car Apache WADI org.apache.geronimo.configs/wadi-clustering/2.1.4/car GShell org.apache.geronimo.framework/gshell-framework/2.1.4/car org.apache.geronimo.framework/gshell-geronimo/2.1.4/car Apache XmlBeans org.apache.geronimo.framework/xmlbeans/2.1.4/car Apache Pluto org.apache.geronimo.plugins/pluto-support/2.1.4/car     If you check the configurations, then you will see that most of the components that make up Geronimo have a deployer configuration and a main configuration. The deployer configuration contains the GBeans that will deploy modules onto that component. For example, the openejb-deployer contains GBeans that implement the functionality to deploy an EJB module onto Apache Geronimo. For accomplishing this, the EJB JAR file and its corresponding deployment plan are parsed by the deployer and then converted into a format that can be understood by the OpenEJB subsystem. This is then deployed on the OpenEJB container. The main configuration will usually contain the GBeans that configure the container and also manage its lifecycle. Server directory structure It is important for a user or an administrator to understand the directory structure of a Geronimo server installation. The directory structure of a v2.1.4 server is shown in the following screenshot: Please note that the directory that we will be referring to as <GERONIMO_HOME> is the geronimo-tomcat6-javaee5-2.1.4 directory shown in the screenshot. The following are some important directories that you should be familiar with: The bin directory contains the command scripts and the JAR files required to start the server, stop the server, invoke the deployer, and start the GShell. The etc directory contains the configuration files for GShell. The schema directory contains Geronimo schemas. The var/config directory contains Geronimo configurations files. A Geronimo administrator or user can find most of the configuration information about the server here. The var/derby directory contains the database files for the embedded Derby database server. The var/log directory contains logging configuration and logfiles. The var/security directory contains user credential and grouping files. The var/security/keystores directory contains the cryptographic keystore files used for server SSL configuration. The following are some important configuration files under the Geronimo directory structure: config.xml: This file is located under the &ltGERONIMO_HOME>/var/config directory. This file preserves the information regarding GBean attributes and references that were overridden from the default values used at deployment time. config-substitutions.properties: This file is located under the &ltGERONIMO_HOME>/var/config directory. The property values specified in this file are used in expressions in config.xml. The property values in this file can be overridden by using a system property or environment variable with a property name that is prefixed with org.apache.geronimo.config.substitution. artifact_aliases.properties: This file is located under the &ltGERONIMO_HOME>/var/config directory. This file is used to substitute one module or configuration ID for another module or configuration ID. The entries in this file are of the form oldArtifactId=newArtifactId, for example default/mylib//jar=default/mylib/2.0/jar. Note that the version number in the old artifact ID may be omitted, but the version number in the new artifact ID must be specified. client_artifact_aliases.properties: This file is located under the &ltGERONIMO_HOME>/var/config directory. This file is used for artifact aliasing with application clients. server-log4j.properties: This file is located under the &ltGERONIMO_HOME>/var/log directory. This file contains the logging configuration for the server. deployer-log4j.properties: This file is located under the &ltGERONIMO_HOME>/var/log directory. This file contains the logging configuration for the deployer. client-log4j.properties: This file is located under the &ltGERONIMO_HOME>/var/log directory. This file contains the logging configuration for application clients. users.properties: This file is located under the &ltGERONIMO_HOME>/var/security directory. This file contains the authentication credentials for the server. groups.properties: This file is located under the &ltGERONIMO_HOME>/var/security directory. This file contains the grouping information for the users defined in users.properties Among the directories that contain sensitive information, such as user passwords, are var/security, var/derby, and var/config. These directories should be protected using operating system provided directory and file security.

Bootstrap your app There are two ways to bootstrap your ZF2 app. The default is less flexible but handles the entire configuration, and the manual is really flexible but you have to take care of everything. The goal of the bootstrap is to provide to the application, ZendMvcApplication, with all the components and dependencies needed to successfully handle a request. A Zend Framework 2 application relies on the following six components: Configuration array ServiceManager instance EventManager instance ModuleManager instance Request object Response object As these are the pillars of a ZF2 application, we will take a look at how these components are configured to bootstrap the app. To begin with, we will see how the components interact from a high perspective and then we will jump into details of how each one works. When a new request arrives to our application, ZF2 needs to set up the environment to be able to fulfill it. This process implies reading configuration files and creating the required objects and services; attach them to the events that are going to be used and finally create the request object based on the request data. Once we have the request object, ZF2 will tell the router to do his job and will inspect the request object to determine who is responsible for processing the data. Once a controller and action has been identified as the one in charge of the request, ZF2 dispatches it and gives the controller/action the control of the program in order to execute the code that will interpret the request and will do something with it. This can be from accepting an uploaded image to showing a sign-up form and also changing data on an external database. When the controller processes the data, sometimes a view object is generated to encapsulate the data that we should send to the client who made the request, and a response object is created. After we have a response object, ZF2 sends it to the browser and the request ends. Now that we have seen a very simple overview of the lifecycle of a request we will jump into the details of how each object works, the options available and some examples of each one. Configuration array Let's dissect the first component of the list by taking a look at the index.php file: chdir(dirname(__DIR__)); // Setup autoloading require 'init_autoloader.php'; // Run the application! ZendMvcApplication::init(require 'config/application.config.php')->run(); As you can see we only do three things. The first thing is we change the current folder for the convenience of making everything relative to the root folder. Then we require the autoloader file; we will examine this file later. Finally, we initialize a ZendMvcApplication object by passing a configuration file and only then does the run method get called. The configuration file looks like the following code snippet: return array( 'modules' => array( 'Application', ), 'module_listener_options' => array( 'config_glob_paths' => array( 'config/autoload/{,*.}{global,local}.php', ), 'module_paths' => array( './module', './vendor', ), ), ); This file will return an array containing the configuration options for the application. Two options are used: modules and module_listener_options. As ZF2 uses a module organization approach, we should add the modules that we want to use on the application here. The second option we are using is passed as configuration to the ModuleManager object. The config_glob_path array is used when scanning the folders in search of config files and the module_paths array is used to tell ModuleManager a set of paths where the module resides. ZF2 uses a module approach to organize files. A module can contain almost anything, simple PHP files, view scripts, images, CSS, JavaScript, and so on. This approach will allow us to build reusable blocks of functionality and we will adhere to this while developing our project. PSR-0 and autoloaders Before continuing with the key components, let's take a closer look at the init_autoloader.php file used in the index.php file. As is stated on the first block comment, this file is more complicated than it's supposed to be. This is because ZF2 will try to set up different loading mechanisms and configurations. if (file_exists('vendor/autoload.php')) { $loader = include 'vendor/autoload.php'; } The first thing is to check if there is an autoload.php file inside the vendor folder; if it's found, we will load it. This is because the user might be using composer, in which case composer will provide a PSR-0 class loader. Also, this will register the namespaces defined by composer on the loader. PSR-0 is an autoloading standard proposed by the PHP Framework Interop Group (http://www.php-fig.org/) that describes the mandatory requirements for autoloader interoperability between frameworks. Zend Framework 2 is one of the projects that adheres to it. if (getenv('ZF2_PATH')) { $zf2Path = getenv('ZF2_PATH'); } elseif (get_cfg_var('zf2_path')) { $zf2Path = get_cfg_var('zf2_path'); } elseif (is_dir('vendor/ZF2/library')) { $zf2Path = 'vendor/ZF2/library'; } In the next section we will try to get the path of the ZF2 files from different sources. We will first try to get it from the environment, if not, we'll try from a directive value in the php.ini file. Finally, if the previous methods fail the code, we will try to check whether a specific folder exists inside the vendor folder. if ($zf2Path) { if (isset($loader)) { $loader->add('Zend', $zf2Path); } else { include $zf2Path . '/Zend/Loader/AutoloaderFactory.php'; ZendLoaderAutoloaderFactory::factory(array( 'ZendLoaderStandardAutoloader' => array( 'autoregister_zf' => true ) )); } } Finally, if the framework is found by any of these methods, based on the existence of the composer autoloader, the code will just add the Zend namespace or will instantiate an internal autoloader, ZendLoaderAutoloader, and use it as a default. As you can see, there are multiple ways to set up the autoloading mechanism on ZF2 and at the end what matters is which one you prefer, as all of them in essence will behave the same. ServiceManager After all this execution of code, we arrive at the last section of the index.php file where we actually instantiate the ZendMvcApplication object. As we said, there are two methods of creating an instance of ZendMvcApplication. In the default approach, we call the static method init of the class by passing an optional configuration as the first parameter. This method will take care of instantiating a new ServiceManager object, storing the configuration inside, loading the modules specified in the configuration, and getting a configured ZendMvcApplication. ServiceManager is a service/object locator that implements the Service Locator design pattern; its responsibility is to retrieve other objects. $serviceManager = new ServiceManager( new ServiceServiceManagerConfig($smConfig) ); $serviceManager->setService('ApplicationConfig', $configuration); $serviceManager->get('ModuleManager')->loadModules(); return $serviceManager->get('Application')->bootstrap(); As you can see, the init method calls the bootstrap() method of the ZendMvcApplication instance. Service Locator is a design pattern used in software development to encapsulate the process of obtaining other objects. The concept is based on a central repository that stores the objects and also knows how to create them if required. EventManager This component is designed to provide multiple functionalities. It can be used to implement simple observer patterns, and also can be used to do aspect-oriented design or even create event-driven architectures. The basic operations you can do over these components is attaching and detaching listeners to named events, trigger events, and interrupting the execution of listeners when an event is fired. Let's see a couple of examples on how to attach to an event and how to fire them: //Registering an event listener $events = new EventManager(); $events->attach(array('EVENT_NAME'), $callback); //Triggering an event $events->trigger('EVENT_NAME', $this, $params); Inside the bootstrap method of ZendMvcApplication, we are registering the events of RouteListener, DispatchListener, and ViewManager. After that, the code is instantiating a new custom event called MvcEvent that will be used as the target when firing events. Finally, this piece of code will fire the bootstrap event. ModuleManager Zend Framework 2 introduces a completely redesigned ModuleManager. This new module has been built with simplicity, flexibility, and reuse in mind. These modules can hold everything from PHP to images, CSS, library code, views, and so on. The responsibility of this component in the bootstrap process of an app is loading the available modules specified by the config file. This is accomplished by the following code line located in the init method of ZendMvcApplication: $serviceManager->get('ModuleManager')->loadModules(); This line, when executed, will retrieve the list of modules located at the config file and will load each module. Each module has to contain a file called Module.php with the initialization of the components of the module if needed. This will allow the module manager to retrieve the configuration of the module. Let's see the usual content of this file: namespace MyModule; class Module { public function getAutoloaderConfig() { return array( 'ZendLoaderClassMapAutoloader' => array( __DIR__ . '/autoload_classmap.php', ), 'ZendLoaderStandardAutoloader' => array( 'namespaces' => array( __NAMESPACE__ => __DIR__ . '/src/' . __NAMESPACE__, ), ), ); } public function getConfig() { return include __DIR__ . '/config/module.config.php'; } } As you can see we are defining a method called getAutoloaderConfig() that provides the configuration for the autoloader to ModuleManager. The last method getConfig() is used to provide the configuration of the module to ModuleManager; for example, this will contain the routes handled by the module. Request object This object encapsulates all data related to a request and allows the developer to interact with the different parts of a request. This object is used in the constructor of ZendMvcApplication and also is set inside MvcEvent to be able to retrieve when some events are fired. Response object This object encapsulates all the parts of an HTTP response and provides the developer with a fluent interface to set all the response data. This object is used in the same way as the request object. Basically it is instantiated on the constructor and added to MvcEvent to be able to interact with it across all the events and classes. The request object As we said, the request object will encapsulate all the data related to a request and provide the developer with a fluent API to access the data. Let's take a look at the details of the request object in order to understand how to use it and what it can offer to us: use ZendHttpRequest; $string = "GET /foo HTTP/1.1rnrnSome Content"; $request = Request::fromString($string); $request->getMethod(); $request->getUri(); $request->getUriString(); $request->getVersion(); $request->getContent(); This example comes directly from the documentation and shows how a request object can be created from a string, and then access some data related with the request using the methods provided. So, every time we need to know something related to the request, we will access this object to get the data we need. If we check the code on ZendHttpPhpEnvironmentRequest.php, the first thing we can notice is that the data is populated on the constructor using the superglobal arrays. All this data is processed and then populated inside the object to be able to expose it in a standard way using methods. To manipulate the URI of the request you can get/set the data with three methods, two getters and one setter. The only difference with the getters is that one returns a plain string and the other returns an HttpUri object. getUri() and getUriString() setUri() To retrieve the data passed in the request, there are a few specialized methods depending on the data you want to get: getQuery() getPost() getFiles() getHeader() and getHeaders() About the request method, the object has a general way to know the method used, returning a string or nine specialized functions that will test specific methods based on the RFC 2616, which defines the standard methods for an HTTP request. getMethod() isOptions() isGet() isHead() isPost() isPut() isDelete() isTrace() isConnect() isPatch() Finally, two more methods are available in this object that will test special requests such as AJAX and requests made by a flash object. isXmlHttpRequest() isFlashRequest() Notice that the data stored on the superglobal arrays when populated on the object are converted from an Array to a Parameters object. The Parameters object lives in the Stdlib section of ZF2, a folder where common objects can be found and used across the framework. In this case, the Parameters class is an extension of ArrayObject and implements ParametersInterface that will bring ArrayAccess, Countable, Serializable, and Traversable functionality to the parameters stored inside the object. The goal with this object is to provide a common interface to access data stored in the superglobal arrays. This expands the ways you can interact with the data in an object-oriented approach.

 In this article by Paul Goody, the author of the book Salesforce CRM - The Definitive Admin Handbook - Fourth Edition, we will describe in detail the Salesforce CRM record storage features and user interface that can be customized, such as objects, fields, and page layouts. In addition, we will see an overview of the relationship that exists between the profile and these customizable features that the profile controls. This article looks at the methods to configure and tailor the application to suit the way your company information can be best represented within the Salesforce CRM application. We will look at the mechanisms to store data in Salesforce and the concepts of objects and fields. The features that allow this data to be grouped and arranged presented within the application are then considered by looking at apps, tabs, page layouts, and record types. Finally, we will take a look at some of the features that allow views of data to be presented and customized by looking in detail at related types, related lists, and list views. Finally, you will be presented with a number of questions about the key features of Salesforce CRM administration in the area of Standard and Custom Objects, which are covered in this article. We will cover the following topics in this article: Objects Fields Object relationships Apps Tabs Renaming labels for standard tabs, standard objects, and standard fields Creating custom objects Object limits Creating custom object relationships Creating custom fields Dependent picklists Building relationship fields Lookup relationship options Master detail relationship options Lookup filters Building formulas Basic formula Advanced formula Building formulas--best practices Building formula text and compiled character size limits Custom field governance Page layouts Freed-based page layouts Record types Related lists (For more resources related to this topic, see here.) The relationship between a profile and the features that it controls The following diagram describes the relationship that exists between a profile and the features that it controls: The profile is used to: Control access to the type of license specified for the user and any login hours or IP address restrictions that are set. Control access to objects and records using the role and sharing model. If the appropriate object-level permission is not set on the user's profile, then the user will be unable to gain access to the records of that object type in the application. In this article, we will look at the configurable elements that are set in conjunction with a profile. These are used to control the structure and the user interface for the Salesforce CRM application. Objects Objects are a key element in Salesforce CRM as they provide a structure to store data and are incorporated in the interface, allowing users to interact with the data. Similar in nature to a database table, objects have the following properties: Fields, which are similar in concept to a database column Records, which are similar in concept to a database row Relationships with other objects Optional tabs, which are user-interface components to display the object data Standard objects Salesforce provides standard objects in the application when you sign up; these include Account, Contact, Opportunity, and so on. These are the tables that contain the data records in any standard tab, such as Accounts, Contacts, and Opportunities. In addition to the standard objects, you can create custom objects and tabs. Custom objects Custom objects are the tables you create to store your data. You can create a custom object to store data specific to your organization. Once you have the custom objects, and have created records for these objects, you can also create reports and dashboards based on the record data in your custom object. Fields Fields in Salesforce are similar in concept to a database column: they store the data for the object records. An object record is analogous to a row in a database table. Standard fields Standard fields are predefined fields that are included as standard within the Salesforce CRM application. Standard fields cannot be deleted but non-required standard fields can be removed from page layouts, whenever necessary. With standard fields, you can customize visual elements that are associated to the field, such as field labels and field-level help, as well as certain data definitions, such as picklist values, the formatting of auto-number fields (which are used as unique identifiers for the records), and setting of field history tracking. Some aspects, however, such as the field name, cannot be customized, and some standard fields, such as Opportunity Probability, do not allow the changing of the field label. Custom fields Custom fields are unique to your business needs and can not only be added and amended, but also deleted. Creating custom fields allow you to store the information that is necessary for your organization. Both standard and custom fields can be customized to include custom help text to help users understand how to use the field as shown in following screenshot Object relationships Object relationships can be set on both standard and custom objects and are used to define how records in one object relate to records in another object. Accounts, for example, can have a one-to-many relationship with opportunities; these relationships are presented in the application as related lists. Apps An app in Salesforce is a container for all the objects, tabs, processes, and services associated with a business function. There are standard and custom apps that are accessed using the App menu located at the top-right corner of the Salesforce page, as shown in the following screenshot: When users select an app from the App menu, their screen changes to present the objects associated with that app. For example, when switching from an app that contains the Campaign tab to one that does not, the Campaign tab no longer appears. This feature is applied to both standard and custom apps. Standard apps Salesforce provides standard apps such as Call Center, Community, Content, MarketingSales, Salesforce Chatter, and Site.com. Custom apps A custom app can optionally include a custom logo. Both standard and custom apps consist of a name, a description, and an ordered list of tabs. Subtab apps A subtab app is used to specify the tabs that appear on the Chatter profile page. Subtab apps can include both default and custom tabs that you can set. Tabs A tab is a user-interface element that, when clicked, displays the record data on a page specific to that object. Hiding and showing tabs To customize your personal tab settings, navigate to Setup | My Personal Settings | Change My Display | Customize My Tabs. Now, choose the tabs that will display in each of your apps by moving the tab name between the Available Tabs and the Selected Tabs sections and click on Save. The following screenshot shows the section of tabs for the Sales app: To customize the tab settings of your users, navigate to Setup | Manage Users | Profiles. Now, select a profile and click on Edit. Scroll down to the Tab Settings section of the page, as shown in the following screenshot: Standard tabs Salesforce provides tabs for each of the standard objects that are provided in the application when you sign up. For example, there are standard tabs for Accounts, Contacts, Opportunities, and so on: Visibility of the tab depends on the setting on the Tab Display setting for the app. Custom tabs You can create three different types of custom tabs: Custom Object Tabs, Web Tabs, and Visualforce Tabs. Custom Object Tabs allow you to create, read, update, and delete the data records in your custom objects. Web Tabs display any web URL in a tab within your Salesforce application. Visualforce Tabs display custom user-interface pages created using Visualforce. Creating custom tabs: The text displayed on the Custom tab is set using the Plural Label of the custom object, which is entered when creating the custom object. If the tab text needs to be changed, this can be done by changing the Plural Label stored in the custom object. Salesforce.com recommends selecting the Append tab to a user’s existing personal customization checkbox. This benefits your users as they will automatically be presented with the new tab and can immediately access the corresponding functionality without having to first customize their personal settings themselves. It is recommended that you do not show tabs by setting appropriate permissions so that the users in your organization cannot see any of your changes until you are ready to make them available. You can create up to 25 custom tabs in the Enterprise Edition, and as many as you require in the Unlimited and Performance Editions. To create custom tabs for a custom object, navigate to Setup | Create | Tabs. Now, select the appropriate tab type and/or object from the available selections, as shown in the following screenshot: Renaming labels for standard tabs, standard objects, and standard fields Labels generally reflect the text that is displayed and presented to your users in the user interface and in reports within the Salesforce application. You can change the display labels of standard tabs, objects, fields, and other related user interface labels so they can reflect your company's terminology and business requirements better. For example, the Accounts tab and object can be changed to Clients; similarly, Opportunities to Deals, and Leads to Prospects. Once changed, the new label is displayed on all user pages. The Setup Pages and Setup Menu sections cannot be modified and do not include any renamed labels and continue. Here, the standard tab, object, and field reference continues to use the default, original labels. Also, the standard report names and views continue to use the default labels and are not renamed. To change standard tab, objects, and field labels, navigate to Setup | Customize | Tabs Names and Labels | Rename Tabs and Labels. Now, select a language, and then click on Edit to modify the tab names and standard field labels, as shown in the following screenshot: Click on Edit to select the tab that you wish to rename. Although the screen indicates that this is a change for the tab's name, this selection will also allow you to change the labels for the object and fields, in addition to the tab name. To change field labels, click through to step 2. Enter the new field labels. Here, we will rename Accounts tab to Clients. Enter the Singular and Plural names and then click on Next, as shown in the following screenshot: Only the following standard tabs and objects can be renamed: Accounts, Activities, Articles, Assets, Campaigns, Cases, Contacts, Contracts, Documents, Events, Ideas, Leads, Libraries, Opportunities, Opportunity Products, Partners, Price Books, Products, Quote Line Items, Quotes, Solutions, Tasks. Tabs such as HomeChatter, Forecasts, Reports, and Dashboards cannot be renamed. The following screenshot shows Standard Field available: Salesforce looks for the occurrence of the Account label and displays an auto-populated screen showing where the Account text will be replaced with Client. This auto-population of text is carried out for the standard tab, the standard object, and the standard fields. Review the replaced text, amend as necessary, and then click on Save, as shown in the following screenshot: After renaming, the new labels are automatically displayed on the tab, in reports, in dashboards, and so on. Some standard fields, such as Created By and Last Modified, are prevented from being renamed because they are audit fields that are used to track system information. You will, however, need to carry out the following additional steps to ensure the consistent renaming throughout the system as these may need manual updates: Check all list view names as they do not automatically update and will continue to show the original object name until you change them manually. Review standard report names and descriptions for any object that you have renamed. Check the titles and descriptions of any e-mail templates that contain the original object or field name, and update them as necessary. Review any other items that you have customized with the standard object or field name. For example, custom fields, page layouts, and record types may include the original tab or field name text that is no longer relevant. If you have renamed tabs, objects, or fields, you can also replace the Salesforce online help with a different URL. Your users can view this replaced URL whenever they click on any context-sensitive help link on an end-user page or from within their personal setup options. Creating custom objects Custom objects are database tables that allow you to store data specific to your organization on salesforce.com. You can use custom objects to extend Salesforce functionality or to build new application functionality. You can create up to 200 custom objects in the Enterprise Edition and 2000 in the Unlimited Edition. Once you have created a custom object, you can create a custom tab, custom-related lists, reports, and dashboards for users to interact with the custom object data. To create a custom object, navigate to Setup | Create | Objects. Now click on New Custom Object, or click on Edit to modify an existing custom object. The following screenshot shows the resulting screen:   On the Custom Object Definition Edit page, you can enter the following: Label: This is the visible name that is displayed for the object within the Salesforce CRM user interface and shown on pages, views, and reports, for example. Plural Label: This is the plural name specified for the object, which is used within the application in places such as reports and on tabs (if you create a tab for the object). Gender (language dependent): This field appears if your organization-wide default language expects gender. This is used for organizations where the default language settings are, for example, Spanish, French, Italian, and German, among many others. Your personal language preference setting does not affect whether the field appears or not. For example, if your organization's default language is English, but your personal language is French, you will not be prompted for gender when creating a custom object. Starts with a vowel sound: Use of this setting depends on your organization's default language and is a linguistic check to allow you to specify whether your label is to be preceded by an instead of a; for example, resulting in reference to the object as an Order instead of a Order. Object Name: This is a unique name used to refer to the object. Here, the Object Name field must be unique and can only contain underscores and alphanumeric characters. It must also begin with a letter, not contain spaces or two consecutive underscores, and not end with an underscore. Description: This is an optional description of the object. A meaningful description will help you explain the purpose of your custom objects when you are viewing them in a list. Context-Sensitive Help Setting: This defines what information is displayed when your users click on the Help for this Page context-sensitive help link from the custom object record home (overview), edit, and detail pages, as well as list views and related lists. The Help & Training link at the top of any page is not affected by this setting; it always opens Salesforce Help & Training window. Record Name: This is the name that is used in areas such as page layouts, search results, key lists, and related lists, as shown next. Data Type: This sets the type of field for the record name. Here, the data type can be either text or auto-number. If the data type is set to be Text, then when a record is created, users must enter a text value, which does not need to be unique. If the data type is set to be Auto Number, it becomes a read-only field, whereby new records are automatically assigned a unique number, as shown in the following screenshot: Display Format: This option, as shown in the preceding example, only appears when the Data Type field is set to Auto Number. It allows you to specify the structure and appearance of the Auto Number field. For example, {YYYY}{MM}-{000} is a display format that produces a four-digit year and a two-digit month prefix to a number with leading zeros padded to three digits. Example data output would include: 201203-001, 201203-066, 201203-999, 201203-1234. It is worth noting that although you can specify the number to be three digits, if the number of records created becomes over 999, the record will still be saved but the automatically incremented number becomes 1000, 1001, and so on. Starting Number: As described, Auto Number fields in Salesforce CRM are automatically incremented for each new record. Here, you must enter the starting number for the incremental count, which does not have to be set to start from one. Allow Reports: This setting is required if you want to include the record data from the custom object in any report or dashboard analytics. When a custom object has a relationship field associating it to a standard object, a new Report Type may appear in the standard report category. The new report type allows the user to create reports that relate the standard object to the custom object by selecting the standard object for the report type category instead of the custom object. Such relationships can be either a lookup or a master-detail. A new Report Type is created in the standard report category if the custom object is either the lookup object on the standard object or the custom object has a master-detail relationship with the standard object. Lookup relationships create a relationship between two records so that you can associate them with each other. Moreover, the master-detail relationship fields created are described in more detail later in this section. relationship between records where the master record controls certain behaviors of the detail record such as record deletion and security. When the custom object has a master-detail relationship with a standard object or is a lookup object on a standard object, a new report type will appear in the standard report category. The new report type allows the user to create reports that relate the standard object to the custom object, which is done by selecting the standard object for the report type category instead of the custom object. Allow Activities: This allows users to include tasks and events related to the custom object records, which appear as a related list on the custom object page. Track Field History: This enables the tracking of data-field changes on the custom object records, such as who changed the value of a field and when it was changed. Fields history tracking also stores the value of the field before and after the fields edit. This feature is useful for auditing and data-quality measurement, and is also available within the reporting tools. The field history data is retained for up to 18 months, and you can set field history tracking for a maximum of 20 fields for Enterprise, Unlimited, and Performance Editions. Allow in Chatter Groups: This setting allows your users to add records of this custom object type to Chatter groups. When enabled, records of this object type that are created using the group publisher are associated with the group, and also appear in the group record list. When disabled, records of this object type that are created using the group publisher are not associated with the group. Deployment Status: This indicates whether the custom object is now visible and available for use by other users. This is useful as you can easily set the status to In Development until you are happy for users to start working with the new object. Add Notes & Attachments: This setting allows your users to record notes and attach files to the custom object records. When this is specified, a related list with the New Note and Attach File buttons automatically appears on the custom object record page where your users can enter notes and attach documents. The Add Notes & Attachments option is only available when you create a new object. Launch the New Custom Tab Wizard: This starts the custom tab wizard after you save the custom object. The New Custom Tab Wizard option is only available when you create a new object. If you do not select Launch the New Custom Tab Wizard, you will not be able to create a tab in this step; however, you can create the tab later, as described in the section Custom tabs covered earlier in this article. When creating a custom object, a custom tab is not automatically created. Summary This article, thus, describes in detail the Salesforce CRM record storage features and user interface that can be customized, the mechanism to store data in Salesforce CRM, the relationship that exists between the profile, and the customizable features that the profile controls. Resources for Article: Further resources on this subject: Introducing Dynamics CRM [article] Understanding CRM Extendibility Architecture [article] Getting Dynamics CRM 2015 Data into Power BI [article]

(For more resources on Microsoft Sharepoint, see here.) Creating an InfoPath Form for a SharePoint List You can replace the default SharePoint list forms with an InfoPath form on any SharePoint list. This gives you much more flexibility and control over how you edit and display the data. Getting ready This recipe works for: SharePoint 2010 Enterprise Edition Office 365 (SharePoint Online) You will need a SharePoint List where you want to create an InfoPath form. You will need the Design or Full Control permission level to run this recipe. You will need InfoPath Designer 2010 installed on your client machine. This recipe uses a SharePoint 2010 Team Site with a contacts list added for illustration. How to do it... Open Internet Explorer and navigate to your SharePoint 2010 Team Site. Select the Contacts list from the Quick Launch menu. Select the List tab of the List Tools ribbon. Select the Customize Form icon. InfoPath Form Designer 2010 will open displaying an auto-generated InfoPath form for the contacts list. (Move the mouse over the image to enlarge.) Select the File tab in the ribbon to access the backstage view. Select the Quick Publish button. The InfoPath form is now published and replaces the default form on the SharePoint list. Click on the OK button in the Publish dialog displayed. How it works... When you create a SharePoint list, SharePoint automatically creates default edit, display, and new forms for you. While these forms are functional, they are somewhat limited in the presentation and customization options that they provide. If you have SharePoint Server Enterprise Edition installed, you replace these forms with an InfoPath 2010 form. InfoPath forms offer you many more options for creating and controlling how you edit and display your list data. This recipe demonstrates the mechanics of replacing the forms; once you have done this a whole range of new customization options is available to you. Every time you want to edit the form, just repeat this procedure. One gotcha that you may run into is if you have added a taxonomy field (that is, one that shows a term set) to your list. Unfortunately these fields are not supported in the current InfoPath release, and you will receive an error when you try to edit the list form. It's a big omission from the current version of SharePoint, and not one that there is an obvious workaround for. There's more... Having run this recipe, you may be left thinking "so what?" However, once you have created an InfoPath form for your list, you have all the power of the InfoPath form designer at your disposal. You can now remove columns, add graphics, text, and business rules to the form to fit your needs. Techniques for performing these customizations are described in the following sections. Removing columns It's quite common to have columns in a list that you don't want the user to fill in. When you have an InfoPath attached to the list, simply open the form, delete the field that you don't want the users to edit, and republish. If you have used the table-editing tools in Microsoft Word, you will find the InfoPath experience very familiar. The following screenshot shows the Attachments row being deleted from the form: It's important to realize that you are only removing columns from the form, not from the underlying list itself. Also, if you add a new column to your list after you have customized the form, SharePoint won't automatically add the new column to your form for you. It will prompt you that there are new columns available the next time you open the InfoPath form designer. Click on Yes to update the fields list. You can then select the new fields that you require and drag-and-drop them into the form. Adding images, explanatory text, and tooltips Normal SharePoint list forms are a bit dull. Now that you have an InfoPath form, you can start to brighten things up. You can change colors, fonts, add images, text, and tooltips. To add an image, use the Picture button on the Insert ribbon. Then simply browse to the picture that you want to add and republish the form. Adding text to your form can help guide your users when they are filling it in. You can add the text directly to the form or you can add it to the controls ScreenTip. The ScreenTip text will only be shown when the user's mouse hovers over the control in the browser. This helps your users without cluttering up your form. Adding rules to validate data InfoPath forms allow you to add rules. Rules can be added to the form to validate data, apply custom formatting, or perform custom actions. You can use actions to set field values, switch views, submit form data, and so on. InfoPath form rules are the way to implement and enforce custom business logic in your forms, so I advise you to invest some time learning how to build them up and exploring what they can do. The following screenshot shows how to apply InfoPath built in e-mail validation rule to the e-mail textbox field. This rule uses a regular expression to ensure that the value entered is in a valid e-mail format, and shows a validation error if it is not. InfoPath Designer versus InfoPath Filler You may notice two Microsoft Office InfoPath programs on your computer, Microsoft InfoPath Designer 2010 and Microsoft InfoPath Filler 2010. When creating forms for use in SharePoint, InfoPath Designer 2010 is the application you need to use. InfoPath is a standalone forms technology, while InfoPath Filler 2010 exists to allow users to fill in InfoPath forms without the use of SharePoint. That isn't something we cover in this book, though it's useful to know that you can use this if you need to.

Binding Components Service Engines are pluggable components which connect to the Normalized Message Router (NMR) to perform business logic for clients. Binding components are also standard JSR 208 components that plug in to NMR and provide transport independence to NMR and Service Engines. The role of binding components is to isolate communication protocols from JBI container so that Service Engines are completely decoupled from the communication infrastructure. For example, BPEL Service Engine can receive requests to initiate BPEL process while reading files on the local file system. It can receive these requests from SOAP messages, from a JMS message, or from any of the other binding components installed into JBI container. Binding Component is a JSR 208 component that provides protocol independent transport services to other JBI components. The following figure shows how binding components fit into the JBI Container architecture: In this figure, we can see that the role of BC is to send and receive messages both internally and externally from Normalized Message Router using protocols, specific to the binding component. We can also see that any number of binding components can be installed into the JBI container. This figure shows that like Service Engines (SE), binding components do not communicate directly with other binding components or with Service Engines. All communication between individual binding components and between binding components and Service Engines is performed via sending standard messages through the Normalized Message Router. NetBeans Support for Binding Components The following table lists which binding components are installed into the JBI container with NetBeans 5.5 and NetBeans 6.0:   As is the case with Service Engines, binding components can be managed within the NetBeans IDE. The list of Binding Components installed into the JBI container can be displayed by expanding the Servers | Sun Java System Application Server 9 | JBI | Binding Components node within the Services explorer. The lifecycle of binding components can be managed by right-clicking on a binding component and selecting a lifecycle process—Start, Stop, Shutdown, or Uninstall. The properties of an individual binding component can also be obtained by selecting the Properties menu option from the context menu as shown in the following figure. Now that we've discussed what binding components are, and how they communicate both internally and externally to the Normalized Message Router, let's take a closer look at some of the more common binding components and how they are accessed and managed from within the NetBeans IDE. File Binding Component The file binding component provides a communications mechanism for JBI components to interact with the file system. It can act as both a Provider by checking for new files to process, or as a Consumer by outputting files for other processes or components. The figure above shows the file binding component acting as a Provider of messages. In this scenario, a message has been sent to the JBI container, and picked up by a protocol-specific binding component (for example, a SOAP message has been received). A JBI Process then occurs within the JBI container which may include routing the message between many different binding components and Service Engines depending upon the process. Finally, after the JBI Process has completed, the results of the process are sent to File Binding Component which writes out the result to a file. The figure above shows the file binding component acting as a Consumer of messages. In this situation, the File Binding Component is periodically polling the file system looking for files with a specified filename pattern in a specified directory. When the binding component finds a file that matches its criteria, it reads in the file and starts the JBI Process, which may again cause the input message to be routed between many different binding components and Service Engines. Finally, in this example, the results of the JBI Process are output via a Binding Component. Of course, it is possible that a binding component can act as both a provider and a consumer within the same JBI process. In this case, the file binding component would be initially responsible for reading an input message from the file system. After any JBI processing has occurred, the file binding component would then write out the results of the process to a file. Within the NetBeans Enterprise Pack, the entire set of properties for the file binding component can be edited within the Properties window. The properties for the binding component are displayed when either the input or output messages are selected from the WSDL in a composite application as shown in the following figure.

Boost is a huge collection of libraries. Some of those libraries are small and meant for everyday use and others require a separate article to describe all of their features. This article is devoted to some of those big libraries and to give you some basics to start with. The first two recipes will explain the usage of Boost.Graph. It is a big library with an insane number of algorithms. We'll see some basics and probably the most important part of it visualization of graphs. We'll also see a very useful recipe for generating true random numbers. This is a very important requirement for writing secure cryptography systems. Some C++ standard libraries lack math functions. We'll see how that can be fixed using Boost. But the format of this article leaves no space to describe all of the functions. Writing test cases is described in the Writing test cases and Combining multiple test cases in one test module recipes. This is important for any production-quality system. The last recipe is about a library that helped me in many courses during my university days. Images can be created and modified using it. I personally used it to visualize different algorithms, hide data in images, sign images, and generate textures. Unfortunately, even this article cannot tell you about all of the Boost libraries. Maybe someday I'll write another book... and then a few more. Working with graphs Some tasks require a graphical representation of data. Boost.Graph is a library that was designed to provide a flexible way of constructing and representing graphs in memory. It also contains a lot of algorithms to work with graphs, such as topological sort, breadth first search, depth first search, and Dijkstra shortest paths. Well, let's perform some basic tasks with Boost.Graph! Getting ready Only basic knowledge of C++ and templates is required for this recipe. How to do it... In this recipe, we'll describe a graph type, create a graph of that type, add some vertexes and edges to the graph, and search for a specific vertex. That should be enough to start using Boost.Graph. We start with describing the graph type: #include <boost/graph/adjacency_list.hpp> #include <string> typedef std::string vertex_t; typedef boost::adjacency_list< boost::vecS , boost::vecS , boost::bidirectionalS , vertex_t > graph_type; Now we construct it: graph_type graph; Let's use a non portable trick that speeds up graph construction: static const std::size_t vertex_count = 5; graph.m_vertices.reserve(vertex_count); Now we are ready to add vertexes to the graph: typedef boost::graph_traits<graph_type> ::vertex_descriptor descriptor_t; descriptor_t cpp = boost::add_vertex(vertex_t("C++"), graph); descriptor_t stl = boost::add_vertex(vertex_t("STL"), graph); descriptor_t boost = boost::add_vertex(vertex_t("Boost"), graph); descriptor_t guru = boost::add_vertex(vertex_t("C++ guru"), graph); descriptor_t ansic = boost::add_vertex(vertex_t("C"), graph); It is time to connect vertexes with edges: boost::add_edge(cpp, stl, graph); boost::add_edge(stl, boost, graph); boost::add_edge(boost, guru, graph); boost::add_edge(ansic, guru, graph); We make a function that searches for a vertex: template <class GraphT> void find_and_print(const GraphT& g, boost::string_ref name) { Now we will write code that gets iterators to all vertexes: typedef typename boost::graph_traits<graph_type> ::vertex_iterator vert_it_t; vert_it_t it, end; boost::tie(it, end) = boost::vertices(g); It's time to run a search for the required vertex: typedef boost::graph_traits<graph_type>::vertex_descriptor desc_t; for (; it != end; ++ it) { desc_t desc = *it; if (boost::get(boost::vertex_bundle, g)[desc] == name.data()) { break; } } assert(it != end); std::cout << name << 'n'; } /* find_and_print */ How it works... In step 1, we are describing what our graph must look like and upon what types it must be based. boost::adjacency_list is a class that represents graphs as a two-dimensional structure, where the first dimension contains vertexes and the second dimension contains edges for that vertex. boost::adjacency_list must be the default choice for representing a graph; it suits most cases. The first template parameter, boost::adjacency_list, describes the structure used to represent the edge list for each of the vertexes; the second one describes a structure to store vertexes. We can choose different STL containers for those structures using specific selectors, as listed in the following table: Selector STL container boost::vecS std::vector boost::listS std::list boost::slistS std::slist boost::setS std::set boost::multisetS std::multiset boost::hash_setS std::hash_set The third template parameter is used to make an undirected, directed, or bidirectional graph. Use the boost::undirectedS, boost::directedS, and boost::bidirectionalS selectors respectively. The fifth template parameter describes the datatype that will be used as the vertex. In our example, we chose std::string. We can also support a datatype for edges and provide it as a template parameter. Steps 2 and 3 are trivial, but at step 4 you will see a non portable way to speed up graph construction. In our example, we use std::vector as a container for storing vertexes, so we can force it to reserve memory for the required amount of vertexes. This leads to less memory allocations/deallocations and copy operations during insertion of vertexes into the graph. This step is non-portable because it is highly dependent on the current implementation of boost::adjacency_list and on the chosen container type for storing vertexes. At step 4, we see how vertexes can be added to the graph. Note how boost::graph_traits<graph_type> has been used. The boost::graph_traits class is used to get types that are specific for a graph type. We'll see its usage and the description of some graph-specific types later in this article. Step 5 shows what we need do to connect vertexes with edges. If we had provided a datatype for the edges, adding an edge would look as follows: boost::add_edge(ansic, guru, edge_t(initialization_parameters), graph) Note that at step 6 the graph type is a template parameter. This is recommended to achieve better code reusability and make this function work with other graph types. At step 7, we see how to iterate over all of the vertexes of the graph. The type of vertex iterator is received from boost::graph_traits. The function boost::tie is a part of Boost.Tuple and is used for getting values from tuples to the variables. So calling boost::tie(it, end) = boost::vertices(g) will put the begin iterator into the it variable and the end iterator into the end variable. It may come as a surprise to you, but dereferencing a vertex iterator does not return vertex data. Instead, it returns the vertex descriptor desc, which can be used in boost::get(boost::vertex_bundle, g)[desc] to get vertex data, just as we have done in step 8. The vertex descriptor type is used in many of the Boost.Graph functions; we saw its use in the edge construction function in step 5. As already mentioned, the Boost.Graph library contains the implementation of many algorithms. You will find many search policies implemented, but we won't discuss them in this article. We will limit this recipe to only the basics of the graph library. There's more... The Boost.Graph library is not a part of C++11 and it won't be a part of C++1y. The current implementation does not support C++11 features. If we are using vertexes that are heavy to copy, we may gain speed using the following trick: vertex_descriptor desc = boost::add_vertex(graph); boost::get(boost::vertex_bundle, g_)[desc] = std::move(vertex_data); It avoids copy constructions of boost::add_vertex(vertex_data, graph) and uses the default construction with move assignment instead. The efficiency of Boost.Graph depends on multiple factors, such as the underlying containers types, graph representation, edge, and vertex datatypes. Visualizing graphs Making programs that manipulate graphs was never easy because of issues with visualization. When we work with STL containers such as std::map and std::vector, we can always print the container's contents and see what is going on inside. But when we work with complex graphs, it is hard to visualize the content in a clear way: too many vertexes and too many edges. In this recipe, we'll take a look at the visualization of Boost.Graph using the Graphviz tool. Getting ready To visualize graphs, you will need a Graphviz visualization tool. Knowledge of the preceding recipe is also required. How to do it... Visualization is done in two phases. In the first phase, we make our program output the graph's description in a text format; in the second phase, we import the output from the first step to some visualization tool. The numbered steps in this recipe are all about the first phase. Let's write the std::ostream operator for graph_type as done in the preceding recipe: #include <boost/graph/graphviz.hpp> std::ostream& operator<<(std::ostream& out, const graph_type& g) { detail::vertex_writer<graph_type> vw(g); boost::write_graphviz(out, g, vw); return out; } The detail::vertex_writer structure, used in the preceding step, must be defined as follows: namespace detail { template <class GraphT> class vertex_writer { const GraphT& g_; public: explicit vertex_writer(const GraphT& g) : g_(g) {} template <class VertexDescriptorT> void operator()(std::ostream& out, const VertexDescriptorT& d) const { out << " [label="" << boost::get(boost::vertex_bundle, g_)[d] << ""]"; } }; // vertex_writer } // namespace detail That's all. Now, if we visualize the graph from the previous recipe using the std::cout << graph; command, the output can be used to create graphical pictures using the dot command-line utility: $ dot -Tpng -o dot.png digraph G { 0 [label="C++"]; 1 [label="STL"]; 2 [label="Boost"]; 3 [label="C++ guru"]; 4 [label="C"]; 0->1 ; 1->2 ; 2->3 ; 4->3 ; }   The output of the preceding command is depicted in the following figure: We can also use the Gvedit or XDot programs for visualization if the command line frightens you. How it works... The Boost.Graph library contains function to output graphs in Graphviz (DOT) format. If we write boost::write_graphviz(out, g) with two parameters in step 1, the function will output a graph picture with vertexes numbered from 0. That's not very useful, so we provide an instance of the vertex_writer class that outputs vertex names. As we can see in step 2, the format of output must be DOT, which is understood by the Graphviz tool. You may need to read the Graphviz documentation for more info about the DOT format. If you wish to add some data to the edges during visualization, we need to provide an instance of the edge visualizer as a fourth parameter to boost::write_graphviz. There's more... C++11 does not contain Boost.Graph or the tools for graph visualization. But you do not need to worry—there are a lot of other graph formats and visualization tools and Boost. Graph can work with plenty of them. Using a true random number generator I know of many examples of commercial products that use incorrect methods for getting random numbers. It's a shame that some companies still use rand() in cryptography and banking software. Let's see how to get a fully random uniform distribution using Boost.Random that is suitable for banking software. Getting ready Basic knowledge of C++ is required for this recipe. Knowledge of different types of distributions will also be helpful. The code in this recipe requires linking against the boost_random library. How to do it... To create a true random number, we need some help from the operating system or processor. This is how it can be done using Boost: We'll need to include the following headers: #include <boost/config.hpp> #include <boost/random/random_device.hpp> #include <boost/random/uniform_int_distribution.hpp> Advanced random number providers have different names under different platforms: static const std::string provider = #ifdef BOOST_WINDOWS "Microsoft Strong Cryptographic Provider" #else "/dev/urandom" #endif ; Now we are ready to initialize the generator with Boost.Random: boost::random_device device(provider); Let's get a uniform distribution that returns a value between 1000 and 65535: boost::random::uniform_int_distribution<unsigned short> random(1000); That's it. Now we can get true random numbers using the random(device) call. How it works... Why does the rand() function not suit banking? Because it generates pseudo-random numbers, which means that the hacker could predict the next generated number. This is an issue with all pseudo-random number algorithms. Some algorithms are easier to predict and some harder, but it's still possible. That's why we are using boost::random_device in this example (see step 3). That device gathers information about random events from all around the operating system to construct an unpredictable hardware-generated number. The examples of such events are delays between pressed keys, delays between some of the hardware interruptions, and the internal CPU random number generator. Operating systems may have more than one such type of random number generators. In our example for POSIX systems, we used /dev/urandom instead of the more secure /dev/random because the latter remains in a blocked state until enough random events have been captured by the OS. Waiting for entropy could take seconds, which is usually unsuitable for applications. Use /dev/random to create long-lifetime GPG/SSL/SSH keys. Now that we are done with generators, it's time to move to step 4 and talk about distribution classes. If the generator just generates numbers (usually uniformly distributed), the distribution class maps one distribution to another. In step 4, we made a uniform distribution that returns a random number of unsigned short type. The parameter 1000 means that distribution must return numbers greater or equal to 1000. We can also provide the maximum number as a second parameter, which is by default equal to the maximum value storable in the return type. There's more... Boost.Random has a huge number of true/pseudo random generators and distributions for different needs. Avoid copying distributions and generators; this could turn out to be an expensive operation. C++11 has support for different distribution classes and generators. You will find all of the classes from this example in the <random> header in the std:: namespace. The Boost.Random libraries do not use C++11 features, and they are not really required for that library either. Should you use Boost implementation or STL? Boost provides better portability across systems; however, some STL implementations may have assembly-optimized implementations and might provide some useful extensions. Using portable math functions Some projects require specific trigonometric functions, a library for numerically solving ordinary differential equations, and working with distributions and constants. All of those parts of Boost.Math would be hard to fit into even a separate book. A single recipe definitely won't be enough. So let's focus on very basic everyday-use functions to work with float types. We'll write a portable function that checks an input value for infinity and not-a-number (NaN) values and changes the sign if the value is negative. Getting ready Basic knowledge of C++ is required for this recipe. Those who know C99 standard will find a lot in common in this recipe. How to do it... Perform the following steps to check the input value for infinity and NaN values and change the sign if the value is negative: We'll need the following headers: #include <boost/math/special_functions.hpp> #include <cassert> Asserting for infinity and NaN can be done like this: template <class T> void check_float_inputs(T value) { assert(!boost::math::isinf(value)); assert(!boost::math::isnan(value)); Use the following code to change the sign: if (boost::math::signbit(value)) { value = boost::math::changesign(value); } // ... } // check_float_inputs That's it! Now we can check that check_float_inputs(std::sqrt(-1.0)) and check_float_inputs(std::numeric_limits<double>::max() * 2.0) will cause asserts. How it works... Real types have specific values that cannot be checked using equality operators. For example, if the variable v contains NaN, assert(v!=v) may or may not pass depending on the compiler. For such cases, Boost.Math provides functions that can reliably check for infinity and NaN values. Step 3 contains the boost::math::signbit function, which requires clarification. This function returns a signed bit, which is 1 when the number is negative and 0 when the number is positive. In other words, it returns true if the value is negative. Looking at step 3 some readers might ask, "Why can't we just multiply by -1 instead of calling boost::math::changesign?". We can. But multiplication may work slower than boost::math::changesign and won't work for special values. For example, if your code can work with nan, the code in step 3 will be able to change the sign of -nan and write nan to the variable. The Boost.Math library maintainers recommend wrapping math functions from this example in round parenthesis to avoid collisions with C macros. It is better to write (boost::math::isinf)(value) instead of boost::math::isinf(value). There's more... C99 contains all of the functions described in this recipe. Why do we need them in Boost? Well, some compiler vendors think that programmers do not need them, so you won't find them in one very popular compiler. Another reason is that the Boost.Math functions can be used for classes that behave like numbers. Boost.Math is a very fast, portable, reliable library.

This article by Simon Chapple, author of the book Mastering Parallel Programming with R, helps us understand the intricacies of parallel computing. Here, we'll take a look into Delores' Crystal Ball at what the future holds for massively parallel computation that will likely have a significant impact on the world of R programming, particularly when applied to big data. (For more resources related to this topic, see here.) Three steps to successful parallelization The following three-step distilled guidance is intended to help you decide what form of parallelism might be best suited for your particular algorithm/problem and summarizes what you learned throughout this article. Necessarily, it applies a level of generalization, so approach these guidelines with due consideration: Determine the type of parallelism that may best apply to your algorithm. Is the problem you are solving more computationally bound or data bound? If the former, then your problem may be amenable to GPUs; if the latter, then your problem may be more amenable to cluster-based computing; and if your problem requires a complex processing chain, then consider using the Spark framework. Can you divide the problem data/space to achieve a balanced workload across all processes, or do you need to employ an adaptive load-balancing scheme—for example, a task farm-based approach? Does your problem/algorithm naturally divide spatially? If so, consider whether a grid-based parallel approach can be used. Perhaps your problem is on an epic scale? If so, maybe you can develop your message passing-based code and run it on a supercomputer. Is there an implied sequential dependency between tasks; that is, do processes need to cooperate and share data during their computation, or can each separate divided task be executed entirely independently of one another? A large proportion of parallel algorithms typically have a work distribution phase, a parallel computation phase, and a result aggregation phase. To reduce the overhead of the startup and close down phases, consider whether a Tree-based approach to work distribution and result aggregation may be appropriate in your case. Ensure that the basis of the compute in your algorithm has optimal implementation. Profile your code in serial to determine whether there are any bottlenecks, and target these for improvement. Is there an existing parallel implementation similar to your algorithm that you can use directly or adopt? Review CRAN Task View: High-Performance and Parallel Computing with R at https://cran.r-project.org/web/views/HighPerformanceComputing.html; in particular, take a look at the subsection entitled Parallel Computing: Applications, a snapshot of which at the time of writing can be seen in the following figure: Figure 1: CRAN provides various parallelized packages you can use in your own program. Test and evaluate the parallel efficiency of your implementation. Use the P-estimated form of Amdahl's Law to predict the level of scalability you can achieve. Test your algorithm at varying amounts of parallelism, particularly odd numbers that trigger edge-case behaviors. Don't forget to run with just a single process. Running with more processes than processors will lead to trigger-lurking deadlock/race conditions (this is most applicable to message passing-based implementations). Where possible, to reduce overhead, ensure that your method of deployment/initialization places the data being consumed locally to each parallel process. What does the future hold? Obviously, this final section is at a considerable risk of "crystal ball gazing" and getting it wrong. However, there are a number of clear directions in which we can see how both hardware and software will develop, which makes it clear that parallel programming will play an ever more important and increasing role in our computational future. Besides, it has now become critical for us to be able to process vast amounts of information within a short window of time in order to ensure our own individual and collective safety. For example, we are experiencing an increased momentum towards significant climate change and extreme weather events and will therefore require increasingly accurate weather prediction to help us deal with this; this will only be possible with highly efficient parallel algorithms. In order to gaze into the future, we need to look back at the past. The hardware technology available to parallel computing has evolved at a phenomenal pace through the years. The levels of performance that can be achieved today by single chip designs are truly staggering in terms of recent history. The history of HPC For an excellent infographic review of the development of computing performance, I would urge you to visit the following web page: http://pages.experts-exchange.com/processing-power-compared/ This beautifully illustrates how, for example, an iPhone 4 released in 2010 has near-equivalent performance to the Cray 2 supercomputer from 1985 of around 1.5 gigaflops, and the Apple Watch released in 2015 has around twice the performance of the iPhone 4 and Cray 2! While chip manufacturers have managed to maintain the famous Moore's law that predicts transistor count doubling every two years, we are now at 14 nanometers (nm) in chip production, giving us around 100 complex processing cores in a single chip. In July 2015, IBM announced a prototype chip at 7 nm (1/10,000th the width of a human hair). Some scientists suggest that quantum tunneling effects will start to impact at 5 nm (which Intel expects to bring to the market by 2020), although a number of research groups have shown individual transistor construction as small as 1 nm in the lab using materials such as graphene. What all of this suggests is that the placement of 1,000 independent high-performance computational cores, together with sufficient amounts of high-speed cache memory inside a single-chip package comparable to the size of today's chips, could potentially be possible within the next 10 years. NIVIDA and Intel are arguably at the forefront of the dedicated HPC chip development with their respective offerings used in the world's fastest supercomputers, which can also be embedded in your desktop computer. NVIDIA produces Tesla, the K80 GPU-based accelerator available that now peaks at 1.87 teraflops double precision and 5.6 teraflops single precision, utilizing 4,992 cores (dual processor) and 24 GB of on-board RAM. Intel produces Xeon Phi, the collective family brand name for its Many Integrated Core (MIC) architecture; Knights Landing, which is new, is expected to peak at 3 teraflops double precision and 6 teraflops single precision, utilizing 72 cores (single processor) and 16 GB of the highly integrated on-chip fast memory when it is released, likely in fall 2015. The successors to these chips, namely NVIDIA's Volta and Intel's Knights Hill, will be the foundation for the next generation of American $200 million dollar supercomputers in 2018, delivering around 150 to 300 petaflops peak performance (around 150 million iPhone 4s), as compared to China's TIANHE-2, ranked as the fastest supercomputer in the world in 2015, with a peak performance of around 50 petaflops from 3.1 million cores. At the other extreme, within the somewhat smaller and less expensive world of mobile devices, most currently use between two and four cores, though mixed multicore capability such as ARM's big.LITTLE octacore makes eight cores available. However, this is already on the increase with, for example, MediaTek's new SoC MT6797, which has 10 main processing cores split into a pair and two groups of four cores with different clock speeds and power requirements to serve as the basis for the next-generation mobile phones. Top-end mobile devices, therefore, exhibit a rich heterogeneous architecture with mixed power cores, separate sensor chips, GPUs, and Digital Signal Processors (DSP) to direct different aspects of workload to the most power-efficient component. Mobile phones increasingly act as the communications hub and signal a processing gateway for a plethora of additional devices, such as biometric wearables and the rapidly expanding number of ultra-low power Internet of Things (IoT) sensing devices smartening all aspects of our local environment. While we are a little bit away from running R itself natively on mobile devices, the time will come when we seek to harness the distributed computing power of all our mobile devices. In 2014 alone, around 1.25 billion smartphones were sold. That's a lot of crowd-sourced compute power and potentially far outstrips any dedicated supercomputer on the planet either existing or planned. The software that enables us to utilize parallel systems, which, as we noted, are increasingly heterogeneous, continues to evolve. In this book, we have examined how you can utilize OpenCL from R to gain access to both the GPU and CPU, making it possible to perform mixed computation across both components, exploiting the particular strengths of each for certain types of processing. Indeed, another related initiative, Heterogeneous System Architecture (HSA), which enables even lower-level access to the spectrum of processor capabilities, may well gain traction over the coming years and help promote the uptake of OpenCL and its counterparts. HSA Foundation HSA Foundation was founded by a cross-industry group led by AMD, ARM, Imagination, MediaTek, Qualcomm, Samsung, and Texas Instruments. Its stated goal is to help support the creation of applications that seamlessly blend scalar processing on the CPU, parallel processing on the GPU, and optimized processing on the DSP via high bandwidth shared memory access, enabling greater application performance at low power consumption. To enable this, HSA Foundation is defining key interfaces for parallel computation using CPUs, GPUs, DSPs, and other programmable and fixed-function devices, thus supporting a diverse set of high-level programming languages and creating the next generation in general-purpose computing. You can find the recently released version 1.0 of the HSA specification at the following link: http://www.hsafoundation.com/html/HSA_Library.htm Hybrid parallelism As a final wrapping up, I thought I would show how you can overcome some of the inherent single-threaded nature of R even further and demonstrate a hybrid approach to parallelism that combines two of the different techniques we covered previously within a single R program. We've also discussed how heterogeneous computing is potentially the way of the future. This example refers to the code we would develop to utilize MPI through pbdMPI together with ROpenCL to enable us to exploit both the CPU and GPU simultaneously. While this is a slightly contrived example and both the devices compute the same dist() function, the intention is to show you just how far you can take things with R to get the most out of all your available compute resource. Basically, all we need to do is to top and tail our implementation of the dist() function in OpenCL with the appropriate pbdMPI initialization and termination and run the script with mpiexec on two processes, as follows: # Initialise both ROpenCL and pdbMPI require(ROpenCL) library(pbdMPI, quietly = TRUE) init() # Select device based on my MPI rank r <- comm.rank() if (r == 0) { # use gpu device <- 1 } else { # use cpu device <- 2 } ... # Execute the OpenCL dist() function on my assigned device comm.print(sprintf("%d executing on device %s", r, getDeviceType(deviceID)), all.rank = TRUE) res <- teval(openclDist(kernel)) comm.print(sprintf("%d done in %f secs",r,res$Duration), all.rank = TRUE) finalize() This is simple and very effective! Summary In this article, we looked at Crystal Ball and saw the prospects for the combination of heterogeneous compute hardware that is here today and that will expand in capability even further in the future, not only in our supercomputers and laptops but also in our personal devices. Parallelism is the only way these systems can be utilized effectively. As the volume of new quantified self- and environmentally-derived data increases and the number of cores in our compute architectures continues to rise, so does the importance of being able to write parallel programs to make use of it all—job security for parallel programmers looks good for many years to come! Resources for Article: Further resources on this subject: Multiplying Performance with Parallel Computing [article] Training and Visualizing a neural network with R [article] Big Data Analysis (R and Hadoop) [article]

In this article by Unmesh Gundecha, author of Selenium Testing Tools Cookbook Second Edition, you will cover the following topics: Synchronizing a test with an implicit wait Synchronizing a test with an explicit wait Synchronizing a test with custom-expected conditions While building automated scripts for a complex web application using Selenium WebDriver, we need to ensure that the test flow is maintained for reliable test automation. When tests are run, the application may not always respond with the same speed. For example, it might take a few seconds for a progress bar to reach 100 percent, a status message to appear, a button to become enabled, and a window or pop-up message to open. You can handle these anticipated timing problems by synchronizing your test to ensure that Selenium WebDriver waits until your application is ready before performing the next step. There are several options that you can use to synchronize your test. In this article, we will see various features of Selenium WebDriver to implement synchronization in tests. (For more resources related to this topic, see here.) Synchronizing a test with an implicit wait The Selenium WebDriver provides an implicit wait for synchronizing tests. When an implicit wait is implemented in tests, if WebDriver cannot find an element in the Document Object Model (DOM), it will wait for a defined amount of time for the element to appear in the DOM. Once the specified wait time is over, it will try searching for the element once again. If the element is not found in specified time, it will throw NoSuchElement exception. In other terms, an implicit wait polls the DOM for a certain amount of time when trying to find an element or elements if they are not immediately available. The default setting is 0. Once set, the implicit wait is set for the life of the WebDriver object's instance. In this recipe, we will briefly explore the use of an implicit wait; however, it is recommended to avoid or minimize the use of an implicit wait. How to do it... Let's create a test on a demo AJAX-enabled application as follows: @Test public void testWithImplicitWait() { //Go to the Demo AjAX Application WebDriver driver = new FirefoxDriver(); driver.get("http://dl.dropbox.com/u/55228056/AjaxDemo.html"); //Set the Implicit Wait time Out to 10 Seconds driver.manage().timeouts().implicitlyWait(10, TimeUnit.SECONDS); try { //Get link for Page 4 and click on it WebElement page4button = driver.findElement(By.linkText("Page 4")); page4button.click(); //Get an element with id page4 and verify it's text WebElement message = driver.findElement(By.id("page4")); assertTrue(message.getText().contains("Nunc nibh tortor")); } catch (NoSuchElementException e) { fail("Element not found!!"); e.printStackTrace(); } finally { driver.quit(); } } How it works... The Selenium WebDriver provides the Timeouts interface for configuring the implicit wait. The Timeouts interface provides an implicitlyWait() method, which accepts the time the driver should wait when searching for an element. In this example, a test will wait for an element to appear in DOM for 10 seconds: driver.manage().timeouts().implicitlyWait(10, TimeUnit.SECONDS); Until the end of a test or an implicit wait is set back to 0, every time an element is searched using the findElement() method, the test will wait for 10 seconds for an element to appear. Using implicit wait may slow down tests when an application responds normally, as it will wait for each element appearing in the DOM and increase the overall execution time. Minimize or avoid using an implicit wait. Use Explicit wait, which provides more control when compared with an implicit wait. See also Synchronizing a test with an explicit wait Synchronizing a test with custom-expected conditions Synchronizing a test with an explicit wait The Selenium WebDriver provides an explicit wait for synchronizing tests, which provides a better way to wait over an implicit wait. Unlike an implicit wait, you can use predefined conditions or custom conditions or wait before proceeding further in the code. The Selenium WebDriver provides WebDriverWait and ExpectedConditions classes for implementing an explicit wait. The ExpectedConditions class provides a set of predefined conditions to wait before proceeding further in the code. The following table shows some common conditions that we frequently come across when automating web browsers supported by the ExpectedConditions class:   Predefined condition    Selenium method An element is visible and enabled elementToBeClickable(By locator) An element is selected elementToBeSelected(WebElement element) Presence of an element presenceOfElementLocated(By locator) Specific text present in an element textToBePresentInElement(By locator, java.lang.String text) Element value textToBePresentInElementValue(By locator, java.lang.String text) Title titleContains(java.lang.String title)  For more conditions, visit http://seleniumhq.github.io/selenium/docs/api/java/index.html. In this recipe, we will explore some of these conditions with the WebDriverWait class. How to do it... Let's implement a test that uses the ExpectedConditions.titleContains() method to implement an explicit wait as follows: @Test public void testExplicitWaitTitleContains() { //Go to the Google Home Page WebDriver driver = new FirefoxDriver(); driver.get("http://www.google.com"); //Enter a term to search and submit WebElement query = driver.findElement(By.name("q")); query.sendKeys("selenium"); query.click(); //Create Wait using WebDriverWait. //This will wait for 10 seconds for timeout before title is updated with search term //If title is updated in specified time limit test will move to the text step //instead of waiting for 10 seconds WebDriverWait wait = new WebDriverWait(driver, 10); wait.until(ExpectedConditions.titleContains("selenium")); //Verify Title assertTrue(driver.getTitle().toLowerCase().startsWith("selenium")); driver.quit(); } How it works... We can define an explicit wait for a set of common conditions using the ExpectedConditions class. First, we need to create an instance of the WebDriverWait class by passing the driver instance and timeout for a wait as follows: WebDriverWait wait = new WebDriverWait(driver, 10); Next, ExpectedCondition is passed to the wait.until() method as follows: wait.until(ExpectedConditions.titleContains("selenium")); The WebDriverWait object will call the ExpectedConditions class object every 500 milliseconds until it returns successfully. See also Synchronizing a test with an implicit wait Synchronizing a test with custom-expected conditions Synchronizing a test with custom-expected conditions With the explicit wait mechanism, we can also build custom-expected conditions along with common conditions using the ExpectedConditions class. This comes in handy when a wait cannot be handled with a common condition supported by the ExpectedConditions class. In this recipe, we will explore how to create a custom condition. How to do it... We will create a test that will create a wait until an element appears on the page using the ExpectedCondition class as follows: @Test public void testExplicitWait() { WebDriver driver = new FirefoxDriver(); driver.get("http://dl.dropbox.com/u/55228056/AjaxDemo.html"); try { WebElement page4button = driver.findElement(By.linkText("Page 4")); page4button.click(); WebElement message = new WebDriverWait(driver, 5) .until(new ExpectedCondition<WebElement>(){ public WebElement apply(WebDriver d) { return d.findElement(By.id("page4")); }}); assertTrue(message.getText().contains("Nunc nibh tortor")); } catch (NoSuchElementException e) { fail("Element not found!!"); e.printStackTrace(); } finally { driver.quit(); } } How it works... The Selenium WebDriver provides the ability to implement the custom ExpectedCondition interface along with the WebDriverWait class for creating a custom-wait condition, as needed by a test. In this example, we created a custom condition, which returns a WebElement object once the inner findElement() method locates the element within a specified timeout as follows: WebElement message = new WebDriverWait(driver, 5) .until(new ExpectedCondition<WebElement>(){ @Override public WebElement apply(WebDriver d) { return d.findElement(By.id("page4")); }}); There's more... A custom wait can be created in various ways. In the following section, we will explore some common examples for implementing a custom wait. Waiting for element's attribute value update Based on the events and actions performed, the value of an element's attribute might change at runtime. For example, a disabled textbox gets enabled based on the user's rights. A custom wait can be created on the attribute value of the element. In the following example, the ExpectedCondition waits for a Boolean return value, based on the attribute value of an element: new WebDriverWait(driver, 10).until(new ExpectedCondition<Boolean>() { public Boolean apply(WebDriver d) { return d.findElement(By.id("userName")).getAttribute("readonly").contains("true"); }}); Waiting for an element's visibility Developers hide or display elements based on the sequence of actions, user rights, and so on. The specific element might exist in the DOM, but are hidden from the user, and when the user performs a certain action, it appears on the page. A custom-wait condition can be created based on the element's visibility as follows: new WebDriverWait(driver, 10).until(new ExpectedCondition<Boolean>() { public Boolean apply(WebDriver d) { return d.findElement(By.id("page4")).isDisplayed(); }}); Waiting for DOM events The web application may be using a JavaScript framework such as jQuery for AJAX and content manipulation. For example, jQuery is used to load a big JSON file from the server asynchronously on the page. While jQuery is reading and processing this file, a test can check its status using the active attribute. A custom wait can be implemented by executing the JavaScript code and checking the return value as follows: new WebDriverWait(driver, 10).until(new ExpectedCondition<Boolean>() { public Boolean apply(WebDriver d) { JavascriptExecutor js = (JavascriptExecutor) d; return (Boolean)js.executeScript("return jQuery.active == 0"); }}); See also Synchronizing a test with an implicit wait Synchronizing a test with an explicit wait Summary In this article, you have learned how the Selenium WebDriver helps in maintaining a reliable automated test. Using the Selenium WebDriver, you also learned how you can synchronize a test using the implicit and the explicit wait methods. You also saw how to synchronize a test with custom-expected conditions. Resources for Article: Further resources on this subject: Javascript Execution With Selenium [article] Learning Selenium Testing Tools With Python [article] Cross-Browser Tests Using Selenium Webdriver [article]

As you know, Eclipse offers an ingenious system of perspectives that helps us to switch between different technologies and to keep the main-screen as clean as possible. Every perspective is made of a set of components that can be added/removed by the user. These components are known as views. The JBoss AS Perspective has a set of specific views as follows: JBoss Server View Project Archives View Console View Properties View For launching the JBoss AS Perspective (or any other perspective), follow these two simple steps: From the Window menu, select Open Perspective | Other article. In the Open Perspective window, select the JBoss AS option and click on OK button (as shown in the following screenshot). If everything works fine, you should see the JBoss AS perspective as shown in the following screenshot: If any of these views is not available by default in your JBoss AS perspective, then you can add it manually by selecting from the Window menu the Show View | Other option. In the Show View window (shown in the following screenshot), you just select the desired view and click on the OK button. JBoss Server View This view contains a simple toolbar known as JBoss Server View Toolbar and two panels that separate the list of servers (top part) from the list of additional information about the selected server (bottom part). Note that the quantity of additional information is directly related to the server type. Top part of JBoss Server View In the top part of the JBoss Server View, we can see a list of our servers, their states, and if they are running or if they have stopped. Starting the JBoss AS The simplest ways to start our JBoss AS server are: Select the JBoss 4.2 Server from the server list and click the Start the server button from the JBoss Server View Toolbar (as shown in the following screenshot). Select the JBoss 4.2 Server from the server list and right-click on it. From the context menu, select the Start option (as shown in the following screenshot). In both cases, a detailed evolution of the startup process will be displayed in the Console View, as you can see in the following screenshot. Stopping the JBoss AS The simplest ways to stop JBoss AS server are: Select the JBoss 4.2 Server from the server list and click the Stop the server button from the JBoss Server View Toolbar. Select the JBoss 4.2 Server from the server list and right-click on it. From the context menu, select the Stop option. In both cases, a detailed evolution of the stopping process will be displayed in the Console View, as you can see in the following screenshot. Additional operations on JBoss AS Beside Start and Stop operations, JBoss Server View allows us to: Add a new server (the New Server option from the contextual menu) Remove an existing server (the Delete option from the contextual menu) Start the server in debug mode (first button on the JBoss Server View Toolbar) Start the server in profiling mode (third button on the JBoss Server View Toolbar) Publish to the server or synching the publish information between the server and the workspace (the Publish option from the contextual menu or the last button on the JBoss Server View Toolbar) Discard all publish state and republish from scratch (the Clean option from the contextual menu) Twiddle server (the Twiddle Server option from the contextual menu) Edit launch configuration (the Edit Launch Configuration option from the contextual menu as shown in the following screenshot). Add/remove projects (the Add and Remove Projects option from the contextual menu) Double-click the server name and modify parts of that server in the Server Editor—if you have a username and a password to start the server, then you can specify those credentials here (as shown in the following screenshot). Twiddle is a JMX library that comes with JBoss, and it is used to access (any) variables that are exposed via the JBoss JMX interfaces. Server publish status A server may have one of the following statuses: Synchronized: Allows you to see if changes are sync (as shown in the follo wing screenshot) Publishing: Allows you to see if changes are being updated Republish: Allows you to see if changes are waiting

Who Should Read This Read on if you are new to change control, or believe that change control only applies to software, or that it is only meant for large projects. If you are a software pro working with large software projects, you can still read this if you want a gentle introduction to Subversion or svn as it is called. Introduction We have all heard those trite remarks about change -- “... change is the only constant ...”, or similar ones, especially before an unpleasant corporate announcement. These overused remarks about change are unfortunately true. During the course of a day, we make numerous (hopefully!) interrelated changes, updates, or transformations to our work products to reach specific project goals. Needless to say, these changes need to be tracked along with the rationale behind each if we are to prevent ourselves from repeating mistakes, or simply want to recall why we did what we did one month ago! Note that we are not talking about only code or documents here; your work products could be a portfolios of photographs, animations, or some arbitrary binary format. A change control discipline also gives you additional advantages such as being able to develop simultaneous versions of work products for different purposes or clients, rolling back to a previous arbitrary version, or setting up trial development in a so-called branch to bring it back to the main work stream after due review. You also have a running history of how your work product has evolved over time and features. Fetching from a change managed repository also prevents you from creating those fancifully named multiple copies of a file just to keep track of its versions. To reiterate: we use the words 'work product' and 'development' in the broadest sense and not just as applied to software. You might as well be creating a banner ad for your client as much as a Firefox plugin. In the rest of this article we will see how to build a simple personal change control discipline for your day-to-day work using a version control tool. As you will note, 'control' and 'management' have been used interchangeably, though a little hair splitting will yield rich dividends in terms of how different these terms are. Subversion Subversion is version control system available on the Linux (and similar) platforms. If you are trapped in a proprietary world by choice, circumstance, or compulsion, you should try TortoiseSVN. Here, we confine ourselves to the Linux platform. Subversion works by creating a time line of your work products from their inception (or from the point they are brought under version control) to the present point in time, by capturing snapshots of your work products at discrete points that you decide. Each snapshot is a version. You can traverse this time line and extract specific versions for use. How does subversion do it? It versions entire directories. A new version of your directory will be created even if you change one file in it. Don't worry; this does not lead to an explosion of file size with each version. Explaining some terminology, albeit informally, should make the going easier from here. Subversion stores your project(s) in a repository. For the purpose of this article, our repository will stay on the local machine. A revision is nothing but a particular snapshot of the project directory. A working directory is your sandbox. This is where you check out a particular version of your project directory from the repository, make any modifications to it, and then do a check in back into the repository. Revision numbers are bumped up with each check in. You can revert a configuration item, which is like undoing any changes you made. If all this sounds a little abstruse, don't worry, because we will shortly set up our repository so that you can try things out. A commit is when you...., well commit a change done to a file into the repository. Subversion is mostly bundled with a Linux distribution. Find out if you have yours with a 'man svn' or 'svn -h' or a 'whereis svn' command. Setting up Your Repository You can set up your repository in your home directory if you are working on a shared environment. If you have a machine to yourself, you might want to create an 'svn' account with /sbin/nologin (politely refuses logins) as the shell. Your repository might then be '/home/svn/repos'. Subversion is a command line tool. But the only command you will ever issue for the purpose of this article will be to set up your repository: $ svnadmin create /path/to/your/repository The rest, as they say, is GUI! Let Us Get Visual A GUI for subversion is a great tool for learning and working even if you decide to settle for the command line once you get more proficient. eSvn (http://zoneit.free.fr/esvn/) is a Qt-based graphical front end for Subversion. Follow the instructions with the download to compile and install eSvn. Run esvn and this is how it will look with the File | Options... dialog open. Make sure you enter the correct path to svn if not for the other items.