Search icon CANCEL
Subscription
0
Cart icon
Your Cart (0 item)
Close icon
You have no products in your basket yet
Save more on your purchases! discount-offer-chevron-icon
Savings automatically calculated. No voucher code required.
Arrow left icon
Explore Products
Best Sellers
New Releases
Books
Events
Videos
Audiobooks
Packt Hub
Free Learning
Arrow right icon
timer SALE ENDS IN
0 Days
:
00 Hours
:
00 Minutes
:
00 Seconds

How-To Tutorials - Programming

1081 Articles
article-image-watching-multiple-threads-c
Packt
15 Oct 2009
6 min read
Save for later

Watching Multiple Threads in C#

Packt
15 Oct 2009
6 min read
We can use the BackgroundWorker component and then the Thread class to create new threads independent of the main application thread. The applications can respond to UI events, while the processing continues, and take full advantage of multiple cores, and can thus run faster. However, we are used to debugging applications that run in just one thread (the main thread), and there are many changes in the debugging process that generate great confusion when following the classic procedures running many concurrent threads. How can we successfully debug applications that are running many concurrent threads? Time for action – Understanding the difficulty in debugging concurrent threads Your cellular phone rings! The FBI agents have detected a problem with an encryption engine. When the application receives the same messages many times during a certain period, the encryption process generates exactly the same results, as shown in the following image: Thus, hackers could easily break the code once they discover this important bug. They ask for your help. Of course, you want to cooperate because you do not want the FBI agents to get angry with you. However, you need to debug the multithreaded encryption engine, and you have never done that! Let's create a solution for this problem! First, we are going to try to debug the multithreaded application the same way we do with a single-threaded application to understand the new problems we might face: Open the project, SMSEncryption. Define a breakpoint in the line int liThreadNumber = (int)poThreadParameter; in the ThreadEncryptProcedure procedure code. Press F5 or select Debug | Start Debugging in the main menu. Enter or copy and paste a long text (with more than 5,000 lines) in the Textbox labeled Original SMS Messages and click on the Run in a thread button. The line with the breakpoint defined is shown highlighted as the next statement that will be executed. Press F10 or select, Debug | Step Over in the main menu two or three times (depending on the number of cores you have in the computer). As you can see, the next statement that gets executed is the same even when you try to go on with the next one. It seems that the statement is not being executed. However, inspecting the value of poThreadParameter (the parameter passed to the ThreadEncryptProcedure procedure) shows that it changes each time you step over the statement, as shown in the following image: Stop the application and repeat the steps 1 to 5 to make sure you are not crazy because of parallelism, multithreading, and the FBI agents! What just happened? You are getting nervous about the debugging process! Do not worry. We will learn how to debug your encryption engine while the FBI agents kindly prepare a cup of fresh cappuccino for you. The debugger executed each new Thread class instance call to the Start method, with this line: prloThreadList[liThreadNumber].Start(liThreadNumber); Then, it entered in the ThreadEncryptProcedure method (we have used the same method for every created encryption thread) with different values for the poThreadParameter parameter. Therefore, you stayed in the same statement as many times as the threads were created (equivalent to the number of cores available in the computer) in the following line: int liThreadNumber = (int)poThreadParameter; As we can see, debugging this way is very confusing, because the IDE switches from one thread to another, and you loose control over the statements that are going to be executed next. In a debugging process, you need to know in which part of the application you are. As we tested our first attempt to debug a multithreaded application, we tried the same technique as with single-threaded applications. There are new subjects to learn and new techniques to use. Debugging concurrent threads When we need to inspect values, execute a procedure step-by-step, and find solutions to problems related to some specific code, the best way to achieve that with a multithreaded application is to work with it as a single-threaded application. But, how can we do that? It is very simple. We must run one thread at a time and freeze the other concurrent threads while we are debugging the thread in which we are interested and on which we are focusing. When we debug single-threaded applications, we are aware of the method in which we are positioned and its context. In multithreaded applications, we must also be aware of the thread in which we are positioned. If we do not know in which thread we are executing statements, we will be completely confused in just a few seconds, as happened in our previous activity. We must tailor our multithreaded applications to simplify the debugging process. If we do not do this, the debugging process will be a nightmare. Indeed, we do not want that to happen! Time for action – Finding the threads You wonder where the threads are. How can you guess in which thread you are working while executing the application step-by-step? You are an excellent C# programmer, but multithreaded debugging is very confusing. You do not want the FBI agents to realize that you are in trouble. However, you must hurry up, because they have a great training in detecting nervous people in the course of their usual interrogations. Now, we are going to use the IDE features to help us find the threads in a multithreaded application: Using the same project that we used in the previous example, with the same breakpoint defined, press F5 or select Debug | Start Debugging in the main menu. Enter or copy and paste a long text (with more than 5,000 lines) in the Textbox labeled Original SMS Messages and click on the Run in a thread button. The line with the breakpoint defined is shown highlighted as the next statement that will be executed. Select Debug | Windows | Threads in the main menu or press Ctrl + Alt + H. The Threads window will be shown, displaying all the threads created by the application process, as shown in the following image: The yellow arrow in the left of the thread list points out the current thread—the thread for which the IDE is showing the current statement. Press F10 or select Debug | Step Over in the main menu. As you can see, the next statement is the same again, but the current thread pointed out in the thread list changes, as shown in the following image: Go on running the application step-by-step and watch how the current thread changes. Observe the Threads window throughout your debugging process. What just happened? You found the threads in the debugging process. Now, you believe you will be able to make the necessary changes to the application if you learn a few debugging techniques quickly. The Threads window displays the list of threads created by the application process. Many of them are created automatically by the C# runtime. The others are created by the Thread class instances and the BackgroundWorker component we have in the application. Using the Threads window, we can easily determine in which thread we are executing when debugging a multithreaded application. It is indeed very helpful. Remember that each thread has its own stack.  
Read more
  • 0
  • 0
  • 3554

article-image-create-quick-application-cakephp-part-1
Packt
17 Nov 2009
9 min read
Save for later

Create a Quick Application in CakePHP: Part 1

Packt
17 Nov 2009
9 min read
The ingredients are fresh, sliced up, and in place. The oven is switched on, heated, and burning red. It is time for us to put on the cooking hat, and start making some delicious cake recipes. So, are you ready, baker? In this article, we are going to develop a small application that we'll call the "CakeTooDoo". It will be a simple to-do-list application, which will keep record of the things that we need to do. A shopping list, chapters to study for an exam, list of people you hate, and list of girls you had a crush on are all examples of lists. CakeTooDoo will allow us to keep an updated list. We will be able to view all the tasks, add new tasks, and tick the tasks that are done and much more. Here's another example of a to-do list, things that we are going to cover in this article: Make sure Cake is properly installed for CakeTooDoo Understand the features of CakeTooDoo Create and configure the CakeTooDoo database Write our first Cake model Write our first Cake controller Build a list that shows all the tasks in CakeTooDoo Create a form to add new tasks to CakeTooDoo Create another form to edit tasks in the to-do list Have a data validation rule to make sure users do not enter empty task title Add functionality to delete a task from the list Make separate lists for completed and pending Tasks Make the creation and modification time of a task look nicer Create a homepage for CakeTooDoo Making Sure the Oven is Ready Before we start with CakeTooDoo, let's make sure that our oven is ready. But just to make sure that we do not run into any problem later, here is a check list of things that should already be in place: Apache is properly installed and running in the local machine. MySQL database server is installed and running in the local machine. PHP, version 4.3.2 or higher, is installed and working with Apache. The latest 1.2 version of CakePHP is being used. Apache mod_rewrite module is switched on. AllowOverride is set to all for the web root directory in the Apache configuration file httpd.conf. CakePHP is extracted and placed in the web root directory of Apache. Apache has write access for the tmp directory of CakePHP. In this case, we are going to rename the Cake directory to it CakeTooDoo. CakeTooDoo: a Simple To-do List Application As we already know, CakeTooDoo will be a simple to-do list. The list will consist of many tasks that we want to do. Each task will consist of a title and a status. The title will indicate the thing that we need to do, and the status will keep record of whether the task has been completed or not. Along with the title and the status, each task will also record the time when the task has been created and last modified. Using CakeTooDoo, we will be able to add new tasks, change the status of a task, delete a task, and view all the tasks. Specifically, CakeTooDoo will allow us to do the following things: View all tasks in the list Add a new task to the list Edit a task to change its status View all completed tasks View all pen Delete a task A homepage that will allow access to all the features. You may think that there is a huge gap between knowing what to make and actually making it. But wait! With Cake, that's not true at all! We are just 10 minutes away from the fully functional and working CakeTooDoo. Don't believe me? Just keep reading and you will find it out yourself. Configuring Cake to Work with a Database The first thing we need to do is to create the database that our application will use. Creating database for Cake applications are no different than any other database that you may have created before. But, we just need to follow a few simple naming rules or conventions while creating tables for our database. Once the database is in place, the next step is to tell Cake to use the database. Time for Action: Creating and Configuring the Database Create a database named caketoodoo in the local machine's MySQL server. In your favourite MySQL client, execute the following code: CREATE DATABASE caketoodoo; In our newly created database, create a table named tasks, by running the following code in your MySQL client: USE caketoodoo; CREATE TABLE tasks ( id int(10) unsigned NOT NULL auto_increment, title varchar(255) NOT NULL, done tinyint(1) default NULL, created datetime default NULL, modified datetime default NULL, PRIMARY KEY (id) ); Rename the main cake directory to CakeTooDoo, if you haven't done that yet. Move inside the directory CakeTooDoo/app/config. In the config directory, there is a file named database.php.default. Rename this file to database.php. Open the database.php file with your favourite editor, and move to line number 73, where we will find an array named $default. This array contains database connection options. Assign login to the database user you will be using and password to the password of that user. Assign database to caketoodoo. If we are using the database user ahsan with password sims, the configuration will look like this: var $default = array( 'driver' => 'mysql', 'persistent' => false, 'host' => 'localhost', 'port' => '', 'login' => 'ahsan', 'password' => 'sims', 'database' => 'caketoodoo', 'schema' => '', 'prefix' => '', 'encoding' => '' ); Now, let us check if Cake is being able to connect to the database. Fire up a browser, and point to http://localhost/CakeTooDoo/. We should get the default Cake page that will have the following two lines: Your database configuration file is present and Cake is able to connect to the database, as shown in the following screen shot. If you get the lines, we have successfully configured Cake to use the caketoodoo database. What Just Happened? We just created our first database, following Cake convention, and configured Cake to use that database. Our database, which we named caketoodoo, has only one table named task. It is a convention in Cake to have plural words for table names. Tasks, users, posts, and comments are all valid names for database tables in Cake. Our table tasks has a primary key named id. All tables in Cake applications' database must have id as the primary key for the table. Conventions in CakePHPDatabase tables used with CakePHP should have plural names. All database tables should have a field named id as the primary key of the table. We then configured Cake to use the caketoodoo database. This was achieved by having a file named database.php in the configuration directory of the application. In database.php, we set the default database to caketoodoo. We also set the database username and password that Cake will use to connect to the database server. Lastly, we made sure that Cake was able to connect to our database, by checking the default Cake page. Conventions in Cake are what make the magic happen. By favoring convention over configuration, Cake makes productivity increase to a scary level without any loss to flexibility. We do not need to spend hours setting configuration values to just make the application run. Setting the database name is the only configuration that we will need, everything else will be figured out "automagically" by Cake. Throughout this article, we will get to know more conventions that Cake follows.   Writing our First Model Now that Cake is configured to work with the caketoodoo database, it's time to write our first model. In Cake, each database table should have a corresponding model. The model will be responsible for accessing and modifying data in the table. As we know, our database has only one table named tasks. So, we will need to define only one model. Here is how we will be doing it: Time for Action: Creating the Task Model Move into the directory CakeTooDoo/app/models. Here, create a file named task.php. In the file task.php, write the following code: <?php class Task extends AppModel { var $name = 'Task'; } ?> Make sure there are no white spaces or tabs before the <?php tag and after the ?> tag. Then save the file. What Just Happened? We just created our first Cake model for the database table tasks. All the models in a CakePHP application are placed in the directory named models in the app directory. Conventions in CakePHP: All model files are kept in the directory named models under the app directory. Normally, each database table will have a corresponding file (model) in this directory. The file name for a model has to be singular of the corresponding database table name followed by the .php extension. The model file for the tasks database table is therefore named task.php. Conventions in CakePHP: The model filename should be singular of the corresponding database table name. Models basically contain a PHP class. The name of the class is also singular of the database table name, but this time it is CamelCased. The name of our model is therefore Task. Conventions in CakePHP: A model class name is also singular of the name of the database table that it represents. You will notice that this class inherits another class named AppModel. All models in CakePHP must inherit this class. The AppModel class inherits another class called Model. Model is a core CakePHP class that has all the basic functions to add, modify, delete, and access data from the database. By inheriting this class, all the models will also be able to call these functions, thus we do not need to define them separately each time we have a new model. All we need to do is to inherit the AppModel class for all our models. We then defined a variable named $name in the Task'model, and assigned the name of the model to it. This is not mandatory, as Cake can figure out the name of the model automatically. But, it is a good practice to name it manually.
Read more
  • 0
  • 0
  • 3547

article-image-dwr-java-ajax-user-interface-basic-elements-part-1
Packt
20 Oct 2009
16 min read
Save for later

DWR Java AJAX User Interface: Basic Elements (Part 1)

Packt
20 Oct 2009
16 min read
  Creating a Dynamic User Interface The idea behind a dynamic user interface is to have a common "framework" for all samples. We will create a new web application and then add new features to the application as we go on. The user interface will look something like the following figure: The user interface has three main areas: the title/logo that is static, the tabs that are dynamic, and the content area that shows the actual content. The idea behind this implementation is to use DWR functionality to generate tabs and to get content for the tab pages. The tabbed user interface is created using a CSS template from the Dynamic Drive CSS Library (http://dynamicdrive.com/style/csslibrary/item/css-tabs-menu). Tabs are read from a properties file, so it is possible to dynamically add new tabs to the web page. The following screenshot shows the user interface. The following sequence diagram shows the application flow from the logical perspective. Because of the built-in DWR features we don't need to worry very much about how asynchronous AJAX "stuff" works. This is, of course, a Good Thing. Now we will develop the application using the Eclipse IDE and the Geronimo test environment Creating a New Web Project First, we will create a new web project. Using the Eclipse IDE we do the following: select the menu File | New | Dynamic Web Project. This opens the New Dynamic Web Project dialog; enter the project name DWREasyAjax and click Next, and accept the defaults on all the pages till the last page, where Geronimo Deployment Plan is created as shown in the following screenshot: Enter easyajax as Group Id and DWREasyAjax as Artifact Id. On clicking Finish, Eclipse creates a new web project. The following screen shot shows the generated project and the directory hierarchy. Before starting to do anything else, we need to copy DWR to our web application. All DWR functionality is present in the dwr.jar file, and we just copy that to the WEB-INF | lib directory. A couple of files are noteworthy: web.xml and geronimo-web.xml. The latter is generated for the Geronimo application server, and we can leave it as it is. Eclipse has an editor to show the contents of geronimo-web.xml when we double-click the file. Configuring the Web Application The context root is worth noting (visible in the screenshot above). We will need it when we test the application. The other XML file, web.xml, is very important as we all know. This XML will hold the DWR servlet definition and other possible initialization parameters. The following code shows the full contents of the web.xml file that we will use: <?xml version="1.0" encoding="UTF-8"?> <web-app xsi_schemaLocation="http://java.sun.com/xml/ns/javaee http://java.sun.com/xml/ns/javaee/web- app_2_5.xsd" id="WebApp_ID" version="2.5"> <display-name>DWREasyAjax</display-name> <servlet> <display-name>DWR Servlet</display-name> <servlet-name>dwr-invoker</servlet-name> <servlet-class> org.directwebremoting.servlet.DwrServlet </servlet-class> <init-param> <param-name>debug</param-name> <param-value>true</param-value> </init-param> </servlet> <servlet-mapping> <servlet-name>dwr-invoker</servlet-name> <url-pattern>/dwr/*</url-pattern> </servlet-mapping> <welcome-file-list> <welcome-file>index.html</welcome-file> <welcome-file>index.htm</welcome-file> <welcome-file>index.jsp</welcome-file> <welcome-file>default.html</welcome-file> <welcome-file>default.htm</welcome-file> <welcome-file>default.jsp</welcome-file> </welcome-file-list> </web-app> DWR cannot function without the dwr.xml configuration file. So we need to create the configuration file. We use Eclipse to create a new XML file in the WEB-INF directory. The following is required for the user interface skeleton. It already includes the allow-element for our DWR based menu. <?xml version="1.0" encoding="UTF-8"?> <!DOCTYPE dwr PUBLIC "-//GetAhead Limited//DTD Direct Web Remoting 2.0//EN" "http://getahead.org/dwr/dwr20.dtd"> <dwr> <allow> <create creator="new" javascript="HorizontalMenu"> <param name="class" value="samples.HorizontalMenu" /> </create> </allow> </dwr> In the allow element, there is a creator for the horizontal menu Java class that we are going to implement here. The creator that we use here is the new creator, which means that DWR will use an empty constructor to create Java objects for clients. The parameter named class holds the fully qualified class name. Developing the Web Application Since we have already defined the name of the Java class that will be used for creating the menu, the next thing we do is implement it. The idea behind the HorizontalMenu class is that it is used to read a properties file that holds the menus that are going to be on the web page. We add properties to a file named dwrapplication.properties, and we create it in the same samples-package as the HorizontalMenu-class. The properties file for the menu items is as follows: menu.1=Tables and lists,TablesAndLists menu.2=Field completion,FieldCompletion The syntax for the menu property is that it contains two elements separated by a comma. The first element is the name of the menu item. This is visible to user. The second is the name of HTML template file that will hold the page content of the menu item. The class contains just one method, which is used from JavaScript and via DWR to retrieve the menu items. The full class implementation is shown here: package samples; import java.io.IOException; import java.io.InputStream; import java.util.List; import java.util.Properties; import java.util.Vector; public class HorizontalMenu { public HorizontalMenu() { } public List<String> getMenuItems() throws IOException { List<String> menuItems = new Vector<String>(); InputStream is = this.getClass().getClassLoader().getResourceAsStream( "samples/dwrapplication.properties"); Properties appProps = new Properties(); appProps.load(is); is.close(); for (int menuCount = 1; true; menuCount++) { String menuItem = appProps.getProperty("menu." + menuCount); if (menuItem == null) { break; } menuItems.add(menuItem); } return menuItems; } } The implementation is straightforward. The getMenuItems() method loads properties using the ClassLoader.getResourceAsStream() method, which searches the class path for the specified resource. Then, after loading properties, a for loop is used to loop through menu items and then a List of String-objects is returned to the client. The client is the JavaScript callback function that we will see later. DWR automatically converts the List of String objects to JavaScript arrays, so we don't have to worry about that. Testing the Web Application We haven't completed any client-side code now, but let's test the code anyway. Testing uses the Geronimo test environment. The Project context menu has the Run As menu that we use to test the application as shown in the following screenshot: Run on Server opens a wizard to define a new server runtime. The following screenshot shows that the Geronimo test environment has already been set up, and we just click Finish to run the application. If the test environment is not set up, we can manually define a new one in this dialog: After we click Finish, Eclipse starts the Geronimo test environment and our application with it. When the server starts, the Console tab in Eclipse informs us that it's been started. The Servers tab shows that the server is started and all the code has been synchronized, that is, the code is the most recent (Synchronization happens whenever we save changes on some deployed file.) The Servers tab also has a list of deployed applications under the server. Just the one application that we are testing here is visible in the Servers tab. Now comes the interesting part—what are we going to test if we haven't really implemented anything? If we take a look at the web.xml file, we will find that we have defined one initialization parameter. The Debug parameter is true, which means that DWR generates test pages for our remoted Java classes. We just point the browser (Firefox in our case) to the URL http://127.0.0.1:8080/DWREasyAjax/dwr and the following page opens up: This page will show a list of all the classes that we allow to be remoted. When we click the class name, a test page opens as in the following screenshot: This is an interesting page. We see all the allowed methods, in this case, all public class methods since we didn't specifically include or exclude anything. The most important ones are the script elements, which we need to include in our HTML pages. DWR does not automatically know what we want in our web pages, so we must add the script includes in each page where we are using DWR and a remoted functionality. Then there is the possibility of testing remoted methods. When we test our own method, getMenuItems(), we see a response in an alert box: The array in the alert box in the screenshot is the JavaScript array that DWR returns from our method. Developing Web Pages The next step is to add the web pages. Note that we can leave the test environment running. Whenever we change the application code, it is automatically published to test the environment, so we don't need to stop and start the server each time we make some changes and want to test the application. The CSS style sheet is from the Dynamic Drive CSS Library. The file is named styles.css, and it is in the WebContent directory in Eclipse IDE. The CSS code is as shown: /*URL: http://www.dynamicdrive.com/style/ */ .basictab{ padding: 3px 0; margin-left: 0; font: bold 12px Verdana; border-bottom: 1px solid gray; list-style-type: none; text-align: left; /*set to left, center, or right to align the menu as desired*/ } .basictab li{ display: inline; margin: 0; } .basictab li a{ text-decoration: none; padding: 3px 7px; margin-right: 3px; border: 1px solid gray; border-bottom: none; background-color: #f6ffd5; color: #2d2b2b; } .basictab li a:visited{ color: #2d2b2b; } .basictab li a:hover{ background-color: #DBFF6C; color: black; } .basictab li a:active{ color: black; } .basictab li.selected a{ /*selected tab effect*/ position: relative; top: 1px; padding-top: 4px; background-color: #DBFF6C; color: black; } This CSS is shown for the sake of completion, and we will not go into details of CSS style sheets. It is sufficient to say that CSS provides an excellent method to create websites with good presentation. The next step is the actual web page. We create an index.jsp page, in the WebContent directory, which will have the menu and also the JavaScript functions for our samples. It should be noted that although all JavaScript code is added to a single JSP page here in this sample, in "real" projects it would probably be more useful to create a separate file for JavaScript functions and include the JavaScript file in the HTML/JSP page using a code snippet such as this: <script type="text/javascript" src="myjavascriptcode/HorizontalMenu.js"/>. We will add JavaScript functions later for each sample. The following is the JSP code that shows the menu using the remoted HorizontalMenu class. <%@ page language="java" contentType="text/html; charset=ISO-8859-1" pageEncoding="ISO-8859-1"%> <!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd"> <html> <head> <meta http-equiv="Content-Type" content="text/html; charset=ISO-8859-1"> <link href="styles.css" rel="stylesheet" type="text/css"/> <script type='text/javascript' src='/DWREasyAjax/dwr/engine.js'></script> <script type='text/javascript' src='/DWREasyAjax/dwr/util.js'></script> <script type='text/javascript' src='/DWREasyAjax/dwr/interface/HorizontalMenu.js'></script> <title>DWR samples</title> <script type="text/javascript"> function loadMenuItems() { HorizontalMenu.getMenuItems(setMenuItems); } function getContent(contentId) { AppContent.getContent(contentId,setContent); } function menuItemFormatter(item) { elements=item.split(','); return '<li><a href="#" onclick="getContent(''+elements[1]+'');return false;">'+elements[0]+'</a></li>'; } function setMenuItems(menuItems) { menu=dwr.util.byId("dwrMenu"); menuItemsHtml=''; for(var i=0;i<menuItems.length;i++) { menuItemsHtml=menuItemsHtml+menuItemFormatter(menuItems[i]); } menu.innerHTML=menuItemsHtml; } function setContent(htmlArray) { var contentFunctions=''; var scriptToBeEvaled=''; var contentHtml=''; for(var i=0;i<htmlArray.length;i++) { var html=htmlArray[i]; if(html.toLowerCase().indexOf('<script')>-1) { if(html.indexOf('TO BE EVALED')>-1) { scriptToBeEvaled=html.substring(html.indexOf('>')+1,html.indexOf('</')); } else { eval(html.substring(html.indexOf('>')+1,html.indexOf('</'))); contentFunctions+=html; } } else { contentHtml+=html; } } contentScriptArea=dwr.util.byId("contentAreaFunctions"); contentScriptArea.innerHTML=contentFunctions; contentArea=dwr.util.byId("contentArea"); contentArea.innerHTML=contentHtml; if(scriptToBeEvaled!='') { eval(scriptToBeEvaled); } } </script> </head> <body onload="loadMenuItems()"> <h1>DWR Easy Java Ajax Applications</h1> <ul class="basictab" id="dwrMenu"> </ul> <div id="contentAreaFunctions"> </div> <div id="contentArea"> </div> </body> </html> This JSP is our user interface. The HTML is just normal HTML with a head element and a body element. The head includes reference to a style sheet and to DWR JavaScript files, engine.js, util.js, and our own HorizontalMenu.js. The util.js file is optional, but as it contains very useful functions, it could be included in all the web pages where we use the functions in util.js. The body element has a contentArea place holder for the content pages just below the menu. It also contains the content area for JavaScript functions for a particular content. The body element onload-event executes the loadMenuItems() function when the page is loaded. The loadMenuItems() function calls the remoted method of the HorizontalMenu Java class. The parameter of the HorizontalMenu. getMenuItems() JavaScript function is the callback function that is called by DWR when the Java method has been executed and it returns menu items. The setMenuItems() function is a callback function for the loadMenuItems() function mentioned in the previous paragraph. While loading menu items, the Horizontal.getMenuItems() remoted method returns menu items as a List of Strings as a parameter to the setMenuItems() function. The menu items are formatted using the menuItemFormatter() helper function. The menuItemFormatter() function creates li elements of menu texts. Menus are formatted as links, (a href) and they have an onclick event that has a function call to the getContent-function, which in turn calls the AppContent.getContent() function. The AppContent is a remoted Java class, which we haven't implemented yet, and its purpose is to read the HTML from a file based on the menu item that the user clicked. Implementation of AppContent and the content pages are described in the next section. The setContent() function sets the HTML content to the content area and also evaluates JavaScript options that are within the content to be inserted in the content area (this is not used very much, but it is there for those who need it). Our dynamic user interface looks like this: Note the Firebug window at the bottom of the browser screen. The Firebug console in the screenshot shows one POST request to our HorizontalMenu.getMenuItems() method. Other Firebug features are extremely useful during development work, and we find it useful that Firebug has been enabled throughout the development work. Callback Functions We saw our first callback function as a parameter in the HorizontalMenu.getMenuItems(setMenuItems) function, and since callbacks are an important concept in DWR, it would be good to discuss a little more about them now that we have seen their first usage. Callbacks are used to operate on the data that was returned from a remoted method. As DWR and AJAX are asynchronous, typical return values in RPCs (Remote Procedure Calls), as in Java calls, do not work. DWR hides the details of calling the callback functions and handles everything internally from the moment we return a value from the remoted Java method to receiving the returned value to the callback function. Two methods are recommended while using callback functions. We have already seen the first method in the HorizontalMenu.getMenuItems(setMenuItems) function call. Remember that there are no parameters in the getMenuItems()Java method, but in the JavaScript call, we added the callback function name at the end of the parameter list. If the Java method has parameters, then the JavaScript call is similar to CountryDB.getCountries(selectedLetters,setCountryRows), where selectedLetters is the input parameter for the Java method and setCountryRows is the name of the callback function (we see the implementation later on). The second method to use callbacks is a meta-data object in the remote JavaScript call. An example (a full implementation is shown later in this article) is shown here: CountryDB.saveCountryNotes(ccode,newNotes, { callback:function(newNotes) { //function body here } }); Here, the function is anonymous and its implementation is included in the JavaScript call to the remoted Java method. One advantage here is that it is easy to read the code, and the code is executed immediately after we get the return value from the Java method. The other advantage is that we can add extra options to the call. Extra options include timeout and error handler as shown in the following example: CountryDB.saveCountryNotes(ccode,newNotes, { callback:function(newNotes) { //function body here }, timeout:10000, errorHandler:function(errorMsg) { alert(errorMsg);} }); It is also possible to add a callback function to those Java methods that do not return a value. Adding a callback to methods with no return values would be useful in getting a notification when a remote call has been completed. Afterword Our first sample is ready, and it is also the basis for the following samples. We also looked at how applications are tested in the Eclipse environment. Using DWR, we can look at JavaScript code on the browser and Java code on the server as one. It may take a while to get used to it, but it will change the way we develop web applications. Logically, there is no longer a client and a server but just a single run time platform that happens to be physically separate. But in practice, of course, applications using DWR, JavaScript on the client and Java in the server, are using the typical client-server interaction. This should be remembered when writing applications in the logically single run-time platform.
Read more
  • 0
  • 0
  • 3544

article-image-jboss-plug-and-eclipse-web-tools-platform
Packt
23 Oct 2009
4 min read
Save for later

JBoss AS plug-in and the Eclipse Web Tools Platform

Packt
23 Oct 2009
4 min read
In this article, we recommend that you use the JBoss AS (version 4.2), which is a free J2EE Application Server that can be downloaded from http://www.jboss.org/jbossas/downloads/ (complete documentation can be downloaded from http://www.jboss.org/jbossas/docs/). JBoss AS plug-in and the Eclipse WTP JBoss AS plug-in can be treated as an elegant method of connecting a J2EE Application Server to the Eclipse IDE. It's important to know that JBoss AS plug-in does this by using the WTP support, which is a project included by default in the Eclipse IDE. WTP is a major project that extends the Eclipse platform with a strong support for Web and J2EE applications. In this case, WTP will sustain important operations, like starting the server in run/debug mode, stopping the server, and delegating WTP projects to their runtimes. For now, keep in mind that Eclipse supports a set of WTP servers and for every WTP server you may have one WTP runtime. Now, we will see how to install and configure the JBoss 4.2.2 runtime and server. Adding a WTP runtime in Eclipse In case of JBoss Tools, the main scope of Server Runtimes is to point to a server installation somewhere on your machine. By runtimes, we can use different configurations of the same server installed in different physical locations. Now, we will create a JBoss AS Runtime (you can extrapolate the steps shown below for any supported server): From the Window menu, select Preferences. In the Preferences window, expand the Server node and select the Runtime Environments child-node. On the right side of the window, you can see a list of currently installed runtimes, as shown in the following screenshot, where you can see that an Apache Tomcat runtime is reported (this is just an example, the Apache Tomcat runtime is not a default one). Now, if you want to install a new runtime, you should click the Add button from the top-right corner. This will bring in front the New Server Runtime Environment window as you can see in the following screenshot. Because we want to add a JBoss 4.2.2 runtime, we will select the JBoss 4.2 Runtime option (for other adapters proceed accordingly). After that, click Next for setting the runtime parameters. In the runtimes list, we have runtimes provided by WTP and runtimes provided by JBoss Tools (see the section marked in red on the previous screenshot). Because this article is about JBoss Tools, we will further discuss only the runtimes from this category. Here, we have five types of runtimes with the mention that the JBoss Deploy-Only Runtime type is for developers who start/stop/debug applications outside Eclipse. In this step, you will configure the JBoss runtime by indicating the runtime's name (in the Name field), the runtime's home directory (in the Home Directory field), the Java Runtime Environment associated with this runtime (in the JRE field), and the configuration type (in the Configuration field).In the following screenshot, we have done all these settings for our JBoss 4.2 Runtime. The official documentation of JBoss AS 4.2.2 recommends using JDK version 5. If you don't have this version in the JRE list, you can add it like this: Display the Preferences window by clicking the JRE button. In this window, click the Add button to display the Add JRE window. Continue by selecting the Standard VM option and click on the Next button. On the next page, use the Browse button to navigate to the JRE 5 home directory. Click on the Finish button and you should see a new entry in the Installed JREs field of the Preferences window (as shown in the following screenshot). Just check the checkbox of this new entry and click OK. Now, JRE 5 should be available in the JRE list of the New Server Runtime Environment window. After this, just click on the Finish button and the new runtime will be added, as shown in the following screenshot: From this window, you can also edit or remove a runtime by using the Edit and Remove buttons. These are automatically activated when you select a runtime from the list. As a final step, it is recommended to restart the Eclipse IDE.
Read more
  • 0
  • 0
  • 3541

article-image-article-setting-development-environment
Packt
13 Jan 2012
14 min read
Save for later

Setting Up a Development Environment

Packt
13 Jan 2012
14 min read
Selecting your virtual environment Prisoners serving life sentences (in Canada) have what is known as a faint hope clause where you have a glimmer of a chance of getting parole after 15 years. However, those waiting for Microsoft to provide us a version of Virtual PC that can run Virtual Hard Drives (VHDs) hosting 64-bit operating systems (such as Windows Server 2008 R2), have no such hope of ever seeing that piece of software. But miracles do happen, and I hope that the release of a 64-bit capable Virtual PC renders this section of the article obsolete. If this has in fact happened, go with it and proceed to the following section.   Getting ready Head into your computer's BIOS settings and enable the virtualization setting. The exact setting you are looking for varies widely, so please consult with your manufacturer's documentation. This setting seems universally defaulted to off, so I am very sure you will need to perform this action.   How to do it... Since you are still reading, however, it is safe to say that a miracle has not yet happened. Your first task is to select a suitable virtualization technology that can support a 64-bit guest operating system. The recipe here is to consider the choices in this order, with the outcome of your virtual environment being selected: Microsoft Virtualization: Hyper-V certainly has the ability to create and run Virtual Hard Disks (VHDs) with 64-bit operating systems. It's free—that is, you can install the Hyper-V role , but it requires the base operating system to be Windows Server 2008 R2. It can be brutal to get it running properly on something like a laptop (for example, because of driver issues). It won't be a good idea to get Windows 2008 Server running on a laptop, primarily because of driver issues. I recommend that if your laptop is running Windows 7, look at creating a dual boot, and a boot to VHD where this other boot option / partition is Windows Server 2008 R2. The main disadvantage is coming up with an (preferably licensed) installation of Windows Server 2008 R2 as the main computer operating system (or as a dual boot option). Or perhaps your company runs Hyper-V on their server farm and would be willing to host your development environment for you? Either way, if you have managed to get access to a Hyper-V server, you are good to go! VMware Workstation: Go to http://www.vmware.com and download my absolute favorite virtualization technology—VMware Workstation—fully featured, powerful, and can run on Windows 7. I have used it for years and love it. You must of course pay for a license, but please believe me, it is a worthwhile investment. You can sign up for a 30 day trial to explore the benefits. Note that you only need one copy of VMware Workstation to create a virtual machine. Once you have created it, you can run it anywhere using the freely available VMware Player. Oracle Virtual Box: Go to http://www.virtualbox.org/ and download this free software that will run on Windows 7 and create and host 64-bit guest operating systems. The reason that this is at the bottom of the list is that I personally do not have experience using this software. However, I have colleagues who have used it and have had no problems with it. Give this a try and see if it works as equally well as a paid version of VMware. With your selected virtualization technology in hand, head to the next section to install and configure Windows Server 2008 R2, which is the base operating system required for an installation of SharePoint Server 2010. Installing and configuring Windows Server 2008 R2 SharePoint 2010 requires the Windows Server 2008 R2 operating system in order to run. In this recipe, we will confi gure the components of Windows Server 2008 necessary in order to get ready to install SQL Server 2008 and SharePoint 2010. Getting ready Download Windows Server 2008 R2 from your MSDN subscription, or type in windows server 2008 R2 trial download into your favorite search engine to download the 180-day trial from the Microsoft site. This article does not cover actually installing the base operating system. The specific instructions to do so will be dependent upon the virtualization software. Generally, it will be provided as an ISO image (the file extension will be .iso). ISO means a compressed disk image, and all virtualization software that I am aware of will let you mount (attach) an ISO image to the virtual machine as a CD Drive. This means that when you elect to create a new virtual machine, you will normally be prompted for the ISO image, and the installation of the operating system should proceed in a familiar and relatively automated fashion. So for this recipe, ready means that you have your virtualization software up and running, the Windows Server 2008 R2 base operating system is installed, and you are able to log in as the Administrator (and that you are effectively logging in for the first time). How to do it... Log in as the Administrator. You will be prompted to change the password the first time—I suggest choosing a very commonly used Microsoft password—Password1. However, feel free to select a password of your choice, but use it consistently throughout. The Initial configuration tasks screen will come up automatically. On this screen: Activate windows using your 180 day trial key or using your MSDN key. Select Provide computer name and domain. Change the computer name to a simpler one of your choice. In my case, I named the machine OPENHIGHWAY. Leave the Member of option as Workgroup. The computer will require a reboot. In the Update this server section, choose Download and install updates. Click on the Change settings link and select the option Never check for updates and click OK. Click the Check for updates link. The important updates will be selected. Click on Install Updates. Now is a good time for a coffee break! You will need to reboot the server when the updates complete. In the Customize this server section, click on Add Features. Select the Desktop Experience, Windows, PowerShell, Integrated, Scripting, and Environment options. Choose Add Required Features when prompted to do so. Reboot the server when prompted to do so. If the Initial configuration tasks screen appears now, or in the future, you may now select the checkbox for Do not show this window at logon. We will continue configuration from the Server Manager, which should be displayed on your screen. If not, launch the Server Manager using the icon on the taskbar. We return to Server Manager to continue the confi guration: OPTIONAL: Click on Configure Remote Desktop if you have a preference for accessing your virtual machine using Remote Desktop (RDP) instead of using the virtual machine's console software. In the Security Information section, click Go to Windows Firewall. Click on the Windows Firewall Properties link. From the dialog, go to each of the tabs, namely, Domain Profi le, Private Profi le, and Public Profi le and set the Firewall State to Off on each tab and click OK. Click on the Server Manager node, and from the main screen, click on the Configure IE ESC link. Set both options to Off and click OK. From the Server Manager, expand the Configuration node and then expand Local Users and Groups node, and then click on the Users folder. Right-click on the Administrator account and select Properties. Select the option for Password never expires and click OK. From the Server Manager, click the Roles node . Click the Add Roles link. Now, click on the Introductory screen and select the checkbox for Active Directory Domain Services. Click Next, again click on Next, and then click Install. After completion, click the Close this wizard and launch the Active Directory Domain Services Installation Wizard (dcpromo.exe) link. Now, carry out the following steps: From the new wizard that pops up, from the welcome screen, select the checkbox Use advanced mode installation, click Next, and again click on Next on the Operating System Compatibility screen. Select the option Create a new domain in a new forest and click Next. Choose your domain (FQDN)! This is completely internal to your development server and does not have to be real. For article purposes, I am using theopenhighway.net, as shown in the following screenshot. Then click Next: From the Set Forest Functional Level drop-down, choose Windows Server 2008 R2 and click Next. Click Next on the Additional Domain Controller Option screen. Select Yes on the Static IP assignment screen. Click Yes on the Dns Delegation Warning screen. Click Next on the Location for Database, Log Files, and SYSVOL screen. On the Directory Services Restore Mode Administrator Password screen, enter the same password that you used for the Administrator account, in my case, Password1. Click Next. Click Next on the Summary screen. Click on the Reboot On Completion screen. Otherwise reboot the server after the installation completes You will now confi gure a user account that will run the application pools for the SharePoint web applications in IIS. From the Server Manager, expand the Roles node. Keep expanding the Active Directory Domain Services until you see the Users folder. Click on the Users folder. Now carry out the following: Right-click on the Users folder and select New | User Enter SP_AppPool in the full name field and also enter SP_AppPool in the user logon field and click Next. Enter the password as Password1 (or the same as you had selected for the Administrator account). Deselect the option for User must change password at next logon and select the option for Password never expires. Click Next and then click Finish. A loopback check is a security feature to mitigate against reflection attacks, introduced in Windows Server 2003 SP1. You will likely encounter connection issues with your local websites and it is therefore universally recommended that you disable the loopback check on a development server. This is done from the registry editor: Click the Start menu button, choose Run…, enter Regedit, and click OK to bring up the registry editor. Navigate to HKEY_LOCAL_MACHINE | SYSTEM | CurrentControlSet | Control | Lsa Right-click the Lsa node and select New | DWORD (32-bit) Value In the place of New Value #1 type DisableLoopbackCheck. Right-click DisableLoopbackCheck, select Modify, change the value to 1, and click OK Congratulations! You have successfully confi gured Windows Server 2008 R2. There's more... The Windows Shutdown Event Tracker is simply annoying on a development machine. To turn this feature off, click the Start button, select Run…, enter gpedit.msc, and click OK. Scroll down, right-click on Display Shutdown Event Tracker, and select Edit. Select the Disabled option and click OK, as shown in the following screenshot: Installing and configuring SQL Server 2008 R2 SharePoint 2010 requires Microsoft SQL Server as a fundamental component of the overall SharePoint architecture. The content that you plan to manage in SharePoint, including web content and documents, literally is stored within and served from SQL Server databases. The SharePoint 2010 architecture itself relies on information stored in SQL Server databases, such as confi guration and the many service applications. In this recipe, we will install and configure the components of SQL Server 2008 necessary to install SharePoint 2010. Getting ready I do not recommend SQL Server Express for your development environment, although this is a possible, free, and valid choice for the installation of SharePoint 2010. In my personal experience, I have valued the full power and fl exibility of the full version of SQL Server as well as not having to live with the constraints and limitations of SQL Express. Besides, there is another little reason too! The Enterprise edition of SQL Server is either readily available with your MSDN subscription or downloadable as a trial from the Microsoft site. Download SQL Server 2008 R2 Enterprise from your MSDN subscription, or type in sql server 2008 enterprise R2 trial download into your favorite search engine to download the 180-day trial from the Microsoft site. For SQL Server 2008 R2 Enterprise, if you have MSDN software, then you will be provided with an ISO image that you can attach to the virtual machine. If you download your SQL Server from the Microsoft site as a trial, extract the software (it is a self-extracting EXE) on your local machine, and then share the folder with your virtual machine. Finallly, run the Setup.exe fi le. How to do it... Here is your recipe for installing SQL Server 2008 R2 Enterprise. Carry out the following steps to complete this recipe: You will be presented with the SQL Server Installation Center; on the left side of the screen, select Installation, as shown in the following screenshot: For the choices presented on the Installation screen, select New installation or add features to an existing installation. The Setup Support Rules (shown in the following screenshot) will run to identify any possible problems that might occur when installing SQL Server. All rules should pass. Click OK to continue: You will be presented with the SQL Server 2008 R2 Setup screen. On the fi rst screen, you can select an evaluation or use your product key (from, for example, MSDN) and then click Next. Accept the terms in the license, but do not check the Send feature usage data to Microsoft checkbox, and click Next. On the Setup Support Files screen, click Install. All tests will pass except for a warning that you can safely ignore (the one noting we are installing on a domain controller), and click Next, as shown in the following screenshot: On the Setup Role screen, select SQL Server Feature Installation and click Next. On the Feature Selection, as shown in the following screenshot, carry out the following tasks: In Instance Features, select Database Engine Services (and both SQL Server Replication and Full Text Search), Analysis Services, and Reporting Services In Shared Features, select Business Intelligence Development Studio, Management Tools Basic (and Management Tools Complete), and Microsoft Sync Framework Finally, click Next. On the Installation Rules screen, click Next On the Instance Confi guration screen, click Next. On the Disk Space Requirements screen, click Next On the Server Confi guration screen: Set the Startup Type for SQL Server Agent to be Automatic Click on the button Use the same account for all SQL Server services. Select the account NT AUTHORITYSYSTEM and click OK. Finally, click Next. On the Database Configuration Engine screen: Look for the Account Provisioning tab and click the Add Current User button under Specify SQL Server administrators. Finally, click Next On the Analysis Services Confi guration screen: Look for the Account Provisioning tab and click the Add Current User button under Specify which users have administrative permissions for Analysis Services. Finally, click Next. On the Reporting Services Configuration screen, select the option to Install but do not configure the report server. Now, click Next. On the Error Reporting Screen, click Next. On the Installation Confi guration Rules screen, click Next. On the Ready to Install screen, click Install. Your patience will be rewarded with the Complete screen! Finally, click Close. The Complete screen is shown in the following screenshot: You can close the SQL Server Installation Center. Confi gure SQL Server security for the SP_AppPool account: Click Start | All Programs | SQL Server 2008 R2 | SQL Server Management Studio. On Connect to server, type a period (.) in the Server Name field and click Connect. Expand the Security node. Right-click Logins and select New Login. Use the Search function and enter SP_AppPool in the box Enter object name to select. Click the check names button and then click OK. In my case, you see the properly formatted THEOPENHIGHWAYSP_AppPool in the login name text box. On the Server Roles tab, ensure that the dbcreator and securityadmin roles are selected (in addition to the already selected public role). Finally, click OK. Congratulations! You have successfully installed and confi gured SQL Server 2008 R2 Enterprise.
Read more
  • 0
  • 0
  • 3541

article-image-setting-and-cleaning
Packt
11 Apr 2016
34 min read
Save for later

Setting Up and Cleaning Up

Packt
11 Apr 2016
34 min read
This article, by Mani Tadayon, author of the book, RSpec Essentials, discusses support code to set tests up and clean up after them. Initialization, configuration, cleanup, and other support code related to RSpec specs are important in real-world RSpec usage. We will learn how to cleanly organize support code in real-world applications by learning about the following topics: Configuring RSpec with spec_helper.rb Initialization and configuration of resources Preventing tests from accessing the Internet with WebMock Maintaining clean test state Custom helper code Loading support code on demand with tags (For more resources related to this topic, see here.) Configuring RSpec with spec_helper.rb The RSpec specs that we've seen so far have functioned as standalone units. Specs in the real world, however, almost never work without supporting code to prepare the test environment before tests are run and ensure it is cleaned up afterwards. In fact, the first line of nearly every real-world RSpec spec file loads a file that takes care of initialization, configuration, and cleanup: require 'spec_helper' By convention, the entry point for all support code for specs is in a file called spec_helper.rb. Another convention is that specs are located in a folder called spec in the root folder of the project. The spec_helper.rb file is located in the root of this spec folder. Now that we know where it goes, what do we actually put in spec_helper.rb? Let's start with an example: # spec/spec_helper.rb require 'rspec'   RSpec.configure do |config|   config.order            = 'random'   config.profile_examples = 3    end To see what these two options do, let's create a couple of dummy spec files that include our spec_helper.rb. Here's the first spec file: # spec/first_spec.rb require 'spec_helper'   describe 'first spec' do   it 'sleeps for 1 second' do     sleep 1   end     it 'sleeps for 2 seconds' do     sleep 2   end      it 'sleeps for 3 seconds' do     sleep 3   end  end And here's our second spec file: # spec/second_spec.rb require 'spec_helper'   describe 'second spec' do   it 'sleeps for 4 second' do     sleep 4   end     it 'sleeps for 5 seconds' do     sleep 5   end      it 'sleeps for 6 seconds' do     sleep 6   end  end Now let's run our two spec files and see what happens: We note that we used --format documentation when running RSpec so that we see the order in which the tests were run (the default format just outputs a green dot for each passing test). From the output, we can see that the tests were run in a random order. We can also see the three slowest specs. Although this was a toy example, I would recommend using both of these configuration options for RSpec. Running examples in a random order is very important, as it is the only reliable way of detecting bad tests which sometimes pass and sometimes fail based on the order the in which overall test suite is run. Also, keeping tests running fast is very important for maintaining a productive development flow, and seeing which tests are slow on every test run is the most effective way of encouraging developers to make the slow tests fast, or remove them from the test run. We'll return to both test order and test speed later. For now, let us just note that RSpec configuration is very important to keeping our specs reliable and fast. Initialization and configuration of resources Real-world applications rely on resources, such as databases, and external services, such as HTTP APIs. These must be initialized and configured for the application to work properly. When writing tests, dealing with these resources and services can be a challenge because of two opposing fundamental interests. First, we would like the test environment to match as closely as possible the production environment so that tests that interact with resources and services are realistic. For example, we may use a powerful database system in production that runs on many servers to provide the best performance. Should we spend money and effort to create and maintain a second production-grade database environment just for testing purposes? Second, we would like the test environment to be simple and relatively easy to understand, so that we understand what we are actually testing. We would also like to keep our code modular so that components can be tested in isolation, or in simpler environments that are easier to create, maintain, and understand. If we think of the example of the system that relies on a database cluster in production, we may ask ourselves whether we are better off using a single-server setup for our test database. We could even go so far as to use an entirely different database for our tests, such as the file-based SQLite. As always, there are no easy answers to such trade-offs. The important thing is to understand the costs and benefits, and adjust where we are on the continuum between production faithfulness and test simplicity as our system evolves, along with the goals it serves. For example, for a small hobbyist application or a project with a limited budget, we may choose to completely favor test simplicity. As the same code grows to become a successful fan site or a big-budget project, we may have a much lower tolerance for failure, and have both the motivation and resources to shift towards production faithfulness for our test environment. Some rules of thumb to keep in mind: Unit tests are better places for test simplicity Integration tests are better places for production faithfulness Try to cleverly increase production faithfulness in unit tests Try to cleverly increase test simplicity in integration tests In between unit and integration tests, be clear what is and isn't faithful to the production environment A case study of test simplicity with an external service Let's put these ideas into practice. I haven't changed the application code, except to rename the module OldWeatherQuery. The test code is also slightly changed to require a spec_helper file and to use a subject block to define an alias for the module name, which makes it easier to rename the code without having to change many lines of test code. So let's look at our three files now. First, here's the application code: # old_weather_query.rb   require 'net/http' require 'json' require 'timeout'   module OldWeatherQuery   extend self     class NetworkError < StandardError   end     def forecast(place, use_cache=true)     add_to_history(place)       if use_cache       cache[place] ||= begin         @api_request_count += 1         JSON.parse( http(place) )       end     else       JSON.parse( http(place) )     end   rescue JSON::ParserError     raise NetworkError.new("Bad response")   end     def api_request_count     @api_request_count ||= 0   end     def history     (@history || []).dup   end     def clear!     @history           = []     @cache             = {}     @api_request_count = 0   end     private     def add_to_history(s)     @history ||= []     @history << s   end     def cache     @cache ||= {}   end     BASE_URI = 'http://api.openweathermap.org/data/2.5/weather?q='   def http(place)     uri = URI(BASE_URI + place)       Net::HTTP.get(uri)   rescue Timeout::Error     raise NetworkError.new("Request timed out")   rescue URI::InvalidURIError     raise NetworkError.new("Bad place name: #{place}")   rescue SocketError     raise NetworkError.new("Could not reach #{uri.to_s}")   end end Next is the spec file: # spec/old_weather_query_spec.rb   require_relative 'spec_helper' require_relative '../old_weather_query'   describe OldWeatherQuery do   subject(:weather_query) { described_class }     describe 'caching' do     let(:json_response) do       '{"weather" : { "description" : "Sky is Clear"}}'     end       around(:example) do |example|       actual = weather_query.send(:cache)       expect(actual).to eq({})         example.run         weather_query.clear!     end       it "stores results in local cache" do       weather_query.forecast('Malibu,US')         actual = weather_query.send(:cache)       expect(actual.keys).to eq(['Malibu,US'])       expect(actual['Malibu,US']).to be_a(Hash)     end       it "uses cached result in subsequent queries" do       weather_query.forecast('Malibu,US')       weather_query.forecast('Malibu,US')       weather_query.forecast('Malibu,US')     end   end     describe 'query history' do     before do       expect(weather_query.history).to eq([])       allow(weather_query).to receive(:http).and_return("{}")     end     after do       weather_query.clear!     end       it "stores every place requested" do       places = %w(         Malibu,US         Beijing,CN         Delhi,IN         Malibu,US         Malibu,US         Beijing,CN       )         places.each {|s| weather_query.forecast(s) }         expect(weather_query.history).to eq(places)     end       it "does not allow history to be modified" do       expect {         weather_query.history = ['Malibu,CN']       }.to raise_error         weather_query.history << 'Malibu,CN'       expect(weather_query.history).to eq([])     end   end     describe 'number of API requests' do     before do       expect(weather_query.api_request_count).to eq(0)       allow(weather_query).to receive(:http).and_return("{}")     end       after do       weather_query.clear!     end       it "stores every place requested" do       places = %w(         Malibu,US         Beijing,CN         Delhi,IN         Malibu,US         Malibu,US         Beijing,CN       )         places.each {|s| weather_query.forecast(s) }         expect(weather_query.api_request_count).to eq(3)     end       it "does not allow count to be modified" do       expect {         weather_query.api_request_count = 100       }.to raise_error         expect {         weather_query.api_request_count += 10       }.to raise_error         expect(weather_query.api_request_count).to eq(0)     end   end end And last but not least, our spec_helper file, which has also changed only slightly: we only configure RSpec to show one slow spec (to keep test results uncluttered) and use color in the output to distinguish passes and failures more easily: # spec/spec_helper.rb   require 'rspec'   RSpec.configure do |config|   config.order            = 'random'   config.profile_examples = 1   config.color            = true end When we run these specs, something unexpected happens. Most of the time the specs pass, but sometimes they fail. If we keep running the specs with the same command, we'll see the tests pass and fail apparently at random. These are flaky tests, and we have exposed them because of the random order configuration we chose. If our tests run in a certain order, they fail. The problem could be simply in our tests. For example, we could have forgotten to clear state before or after a test. However, there could also be a problem with our code. In any case, we need to get to the bottom of the situation: We first notice that at the end of the failing test run, RSpec tells us "Randomized with seed 318". We can use this information to run the tests in the order that caused the failure and start to debug and diagnose the problem. We do this by passing the --seed parameter with the value 318, as follows: $ rspec spec/old_weather_query_spec.rb --seed 318 The problem has to do with the way that we increment @api_request_count without ensuring it has been initialized. Looking at our code, we notice that the only place we initialize @api_request_count is in OldWeatherQuery.api_request_count and OldWeatherQuery.clear!. If we don't call either of these methods first, then OldWeatherQuery.forecast, the main method in this module, will always fail. Our tests sometimes pass because our setup code calls one of these methods first when tests are run in a certain order, but that is not at all how our code would likely be used in production. So basically, our code is completely broken, but our specs pass (sometimes). Based on this, we can create a simple spec that will always fail: describe 'api_request is not initialized' do   it "does not raise an error" do     weather_query.forecast('Malibu,US')   end    end At least now our tests fail deterministically. But this is not the end of our troubles with these specs. If we run our tests many times with the seed value of 318, we will start seeing a second failing test case that is even more random than the first. This is an OldWeatherQuery::NetworkError and it indicates that our tests are actually making HTTP requests to the Internet! Let's do an experiment to confirm this. We'll turn off our Wi-Fi access, unplug our Ethernet cables, and run our specs. When we run our tests without any Internet access, we will see three errors in total. One of them is the error with the uninitialized @api_request_count instance variable, and two of them are instances of OldWeatherQuery::NetworkError, which confirms that we are indeed making real HTTP requests in our code. What's so bad about making requests to the Internet? After all, the test failures are indeed very random and we had to purposely shut off our Internet access to replicate the errors. Flaky tests are actually the least of our problems. First, we could be performing destructive actions that affect real systems, accounts, and people! Imagine if we were testing an e-commerce application that charged customer credit cards by using a third-party payment API via HTTP. If our tests actually hit our payment provider's API endpoint over HTTP, we would get a lot of declined transactions (assuming we are not storing and using real credit card numbers), which could lead to our account being suspended due to suspicions of fraud, putting our e-commerce application out of service. Also, if we were running a continuous integration (CI) server such as Jenkins, which did not have access to the public Internet, we would get failures in our CI builds due to failing tests that attempted to access the Internet. There are a few approaches to solving this problem. In our tests, we attempted to mock our HTTP requests, but obviously failed to do so effectively. A second approach is to allow actual HTTP requests but to configure a special server for testing purposes. Let's focus on figuring out why our HTTP mocks were not successful. In a small set of tests like in this example, it is not hard to hunt down the places where we are sending actual HTTP requests. In larger code bases with a lot of test support code, it may be harder. Also, it would be nice to prevent access to the Internet altogether so we notice these issues as soon as we run the offending tests. Fortunately, Ruby has many excellent tools for testing, and there is one that addresses our needs exactly: WebMock (https://github.com/bblimke/webmock). We simply install the gem and add a couple of lines to our spec helper file to disable all network connections in our tests: require 'rspec'   # require the webmock gem require 'webmock/rspec'   RSpec.configure do |config|   # this is done by default, but let's make it clear   WebMock.disable_net_connect!     Config.order            = 'random'   config.profile_examples = 1   config.color            = true end When we run our tests again, we'll see one or more instances of WebMock::NetConnectNotAllowedError, along with a backtrace to lead us to the point in our tests where the HTTP request was made: If we examine our test code, we'll notice that we mock the OldWeatherQuery.http method in a few places. However, we forgot to set up the mock in the first describe block for caching where we defined a json_response object, but never mocked the OldWeatherQuery.http method to return json_response. We can solve the problem by mocking OldWeatherQuery.http throughout the entire test file. We'll also take this opportunity to clean up the initialization of @api_request_count in our code. Here's what we have now: # new_weather_query.rb   require 'net/http' require 'json' require 'timeout'   module NewWeatherQuery   extend self     class NetworkError < StandardError   end     def forecast(place, use_cache=true)     add_to_history(place)     if use_cache       cache[place] ||= begin         increment_api_request_count         JSON.parse( http(place) )       end     else       JSON.parse( http(place) )     end   rescue JSON::ParserError => e     raise NetworkError.new("Bad response: #{e.inspect}")   end     def increment_api_request_count     @api_request_count ||= 0     @api_request_count += 1   end     def api_request_count     @api_request_count ||= 0   end     def history     (@history || []).dup   end     def clear!     @history           = []     @cache             = {}     @api_request_count = 0   end     private     def add_to_history(s)     @history ||= []     @history << s   end     def cache     @cache ||= {}   end     BASE_URI = 'http://api.openweathermap.org/data/2.5/weather?q='   def http(place)     uri = URI(BASE_URI + place)       Net::HTTP.get(uri)   rescue Timeout::Error     raise NetworkError.new("Request timed out")   rescue URI::InvalidURIError     raise NetworkError.new("Bad place name: #{place}")   rescue SocketError     raise NetworkError.new("Could not reach #{uri.to_s}")   end end And here is the spec file to go with it: # spec/new_weather_query_spec.rb   require_relative 'spec_helper' require_relative '../new_weather_query'   describe NewWeatherQuery do   subject(:weather_query) { described_class }     after { weather_query.clear! }     let(:json_response) { '{}' }   before do     allow(weather_query).to receive(:http).and_return(json_response)        end     describe 'api_request is initialized' do     it "does not raise an error" do       weather_query.forecast('Malibu,US')     end      end   describe 'caching' do     let(:json_response) do       '{"weather" : { "description" : "Sky is Clear"}}'     end       around(:example) do |example|       actual = weather_query.send(:cache)       expect(actual).to eq({})             example.run     end       it "stores results in local cache" do       weather_query.forecast('Malibu,US')         actual = weather_query.send(:cache)       expect(actual.keys).to eq(['Malibu,US'])       expect(actual['Malibu,US']).to be_a(Hash)     end       it "uses cached result in subsequent queries" do       weather_query.forecast('Malibu,US')       weather_query.forecast('Malibu,US')       weather_query.forecast('Malibu,US')     end   end     describe 'query history' do     before do       expect(weather_query.history).to eq([])     end       it "stores every place requested" do       places = %w(         Malibu,US         Beijing,CN         Delhi,IN         Malibu,US         Malibu,US         Beijing,CN       )         places.each {|s| weather_query.forecast(s) }         expect(weather_query.history).to eq(places)     end       it "does not allow history to be modified" do       expect {         weather_query.history = ['Malibu,CN']       }.to raise_error         weather_query.history << 'Malibu,CN'       expect(weather_query.history).to eq([])     end   end     describe 'number of API requests' do     before do       expect(weather_query.api_request_count).to eq(0)     end       it "stores every place requested" do       places = %w(         Malibu,US         Beijing,CN         Delhi,IN         Malibu,US         Malibu,US         Beijing,CN       )         places.each {|s| weather_query.forecast(s) }         expect(weather_query.api_request_count).to eq(3)     end       it "does not allow count to be modified" do       expect {         weather_query.api_request_count = 100       }.to raise_error         expect {         weather_query.api_request_count += 10       }.to raise_error         expect(weather_query.api_request_count).to eq(0)     end   end end Now we've fixed a major bug with our code that slipped through our specs and used to pass randomly. We've made it so that our tests always pass, regardless of the order in which they are run, and without needing to access the Internet. Our test code and application code has also become clearer as we've reduced duplication in a few places. A case study of production faithfulness with a test resource instance We're not done with our WeatherQuery example just yet. Let's take a look at how we would add a simple database to store our cached values. There are some serious limitations to the way we are caching with instance variables, which persist only within the scope of a single Ruby process. As soon as we stop or restart our app, the entire cache will be lost. In a production app, we would likely have many processes running the same code in order to serve traffic effectively. With our current approach, each process would have a separate cache, which would be very inefficient. We could easily save many HTTP requests if we were able to share the cache between processes and across restarts. Economizing on these requests is not simply a matter of improved response time. We also need to consider that we cannot make unlimited requests to external services. For commercial services, we would pay for the number of requests we make. For free services, we are likely to get throttled if we exceed some threshold. Therefore, an effective caching scheme that reduces the number of HTTP requests we make to our external services is of vital importance to the function of a real-world app. Finally, our cache is very simplistic and has no expiration mechanism short of clearing all entries. For a cache to be effective, we need to be able to store entries for individual locations for some period of time within which we don't expect the weather forecast to change much. This will keep the cache small and up to date. We'll use Redis (http://redis.io) as our database since it is very fast, simple, and easy to set up. You can find instructions on the Redis website on how to install it, which is an easy process on any platform. Once you have Redis installed, you simply need to start the server locally, which you can do with the redis-server command. We'll also need to install the Redis Ruby client as a gem (https://github.com/redis/redis-rb). Let's start with a separate configuration file to set up our Redis client for our tests: # spec/config/redis.rb   require 'rspec' require 'redis'   ENV['WQ_REDIS_URL'] ||= 'redis://localhost:6379/15'   RSpec.configure do |config|   if ! ENV['WQ_REDIS_URL'].is_a?(String)     raise "WQ_REDIS_URL environment variable not set"   end   ::REDIS_CLIENT = Redis.new( :url => ENV['WQ_REDIS_URL'] )     config.after(:example) do         ::REDIS_CLIENT.flushdb   end end Note that we place this file in a new config folder under our main spec folder. The idea is to configure each resource separately in its own file to keep everything isolated and easy to understand. This will make maintenance easy and prevent problems with configuration management down the road. We don't do much in this file, but we do establish some important conventions. There is a single environment variable, which takes care of the Redis connection URL. By using an environment variable, we make it easy to change configuration and also allow flexibility in how these configurations are stored. Our code doesn't care if the Redis connection URL is stored in a simple .env file with key-value pairs or loaded from a configuration database. We can also easily override this value manually simply by setting it when we run RSpec, like so: $ WQ_REDIS_URL=redis://1.2.3.4:4321/0 rspec spec Note that we also set a sensible default value, which is to run on the default Redis port of 6379 on our local machine, on database number 15, which is less likely to be used for local development. This prevents our tests from relying on our development database, or from polluting or destroying it. It is also worth mentioning that we prefix our environment variable with WQ (short for weather query). Small details like this are very important for keeping our code easy to understand and to prevent dangerous clashes. We could imagine the kinds of confusion and clashes that could be caused if we relied on REDIS_URL and we had multiple apps running on the same server, all relying on Redis. It would be very easy to break many applications if we changed the value of REDIS_URL for a single app to point to a different instance of Redis. We set a global constant, ::REDIS_CLIENT, to point to a Redis client. We will use this in our code to connect to Redis. Note that in real-world code, we would likely have a global namespace for the entire app and we would define globals such as REDIS_CLIENT under that namespace rather than in the global Ruby namespace. Finally, we configure RSpec to call the flushdb command after every example tagged with :redis to empty the database and keep state clean across tests. In our code, all tests interact with Redis, so this tag seems pointless. However, it is very likely that we would add code that had nothing to do with Redis, and using tags helps us to constrain the scope of our configuration hooks only to where they are needed. This will also prevent confusion about multiple hooks running for the same example. In general, we want to prevent global hooks where possible and make configuration hooks explicitly triggered where possible. So what does our spec look like now? Actually, it is almost exactly the same. Only a few lines have changed to work with the new Redis cache. See if you can spot them! # spec/redis_weather_query_spec.rb   require_relative 'spec_helper' require_relative '../redis_weather_query'   describe RedisWeatherQuery, redis: true do   subject(:weather_query) { described_class }     after { weather_query.clear! }     let(:json_response) { '{}' }   before do     allow(weather_query).to receive(:http).and_return(json_response)        end     describe 'api_request is initialized' do     it "does not raise an error" do       weather_query.forecast('Malibu,US')     end      end           describe 'caching' do     let(:json_response) do       '{"weather" : { "description" : "Sky is Clear"}}'     end       around(:example) do |example|       actual = weather_query.send(:cache).all       expect(actual).to eq({})             example.run     end     it "stores results in local cache" do       weather_query.forecast('Malibu,US')         actual = weather_query.send(:cache).all       expect(actual.keys).to eq(['Malibu,US'])       expect(actual['Malibu,US']).to be_a(Hash)     end       it "uses cached result in subsequent queries" do       weather_query.forecast('Malibu,US')       weather_query.forecast('Malibu,US')       weather_query.forecast('Malibu,US')     end   end     describe 'query history' do     before do       expect(weather_query.history).to eq([])     end       it "stores every place requested" do       places = %w(         Malibu,US         Beijing,CN         Delhi,IN         Malibu,US         Malibu,US         Beijing,CN       )         places.each {|s| weather_query.forecast(s) }         expect(weather_query.history).to eq(places)     end       it "does not allow history to be modified" do       expect {         weather_query.history = ['Malibu,CN']       }.to raise_error         weather_query.history << 'Malibu,CN'       expect(weather_query.history).to eq([])     end   end   describe 'number of API requests' do     before do       expect(weather_query.api_request_count).to eq(0)     end       it "stores every place requested" do       places = %w(         Malibu,US         Beijing,CN         Delhi,IN         Malibu,US         Malibu,US         Beijing,CN       )         places.each {|s| weather_query.forecast(s) }         expect(weather_query.api_request_count).to eq(3)     end       it "does not allow count to be modified" do       expect {         weather_query.api_request_count = 100       }.to raise_error         expect {         weather_query.api_request_count += 10       }.to raise_error         expect(weather_query.api_request_count).to eq(0)     end   end end So what about the actual WeatherQuery code? It changes very little as well: # redis_weather_query.rb   require 'net/http' require 'json' require 'timeout'   # require the new cache module require_relative 'redis_weather_cache' module RedisWeatherQuery   extend self     class NetworkError < StandardError   end     # ... same as before ...     def clear!     @history           = []     @api_request_count = 0           # no more clearing of cache here   end     private     # ... same as before ...       # the new cache module has a Hash-like interface   def cache     RedisWeatherCache   end     # ... same as before ...     end We can see that we've preserved pretty much the same code and specs as before. Almost all of the new functionality is accomplished in a new module that caches with Redis. Here is what it looks like: # redis_weather_cache.rb   require 'redis'   module RedisWeatherCache   extend self     CACHE_KEY             = 'weather_query:cache'   EXPIRY_ZSET_KEY       = 'weather_query:expiry_tracker'   EXPIRE_FORECAST_AFTER = 300 # 5 minutes       def redis_client     if ! defined?(::REDIS_CLIENT)       raise("No REDIS_CLIENT defined!")     end         ::REDIS_CLIENT   end     def []=(location, forecast)     redis_client.hset(CACHE_KEY, location, JSON.generate(forecast))     redis_client.zadd(EXPIRY_ZSET_KEY, Time.now.to_i, location)   end     def [](location)     remove_expired_entries         raw_value = redis_client.hget(CACHE_KEY, location)         if raw_value       JSON.parse(raw_value)     else       nil     end   end     def all     redis_client.hgetall(CACHE_KEY).inject({}) do |memo, (location, forecast_json)|       memo[location] = JSON.parse(forecast_json)       memo     end   end     def clear!     redis_client.del(CACHE_KEY)   end     def remove_expired_entries     # expired locations have a score, i.e. creation timestamp, less than a certain threshold     expired_locations = redis_client.zrangebyscore(EXPIRY_ZSET_KEY, 0, Time.now.to_i - EXPIRE_FORECAST_AFTER)       if ! expired_locations.empty?       # remove the cache entry       redis_client.hdel(CACHE_KEY, expired_locations)                  # also clear the expiry entry       redis_client.zrem(EXPIRY_ZSET_KEY, expired_locations)      end   end end We'll avoid a detailed explanation of this code. We simply note that we accomplish all of the design goals we discussed at the beginning of the section: a persistent cache with expiration of individual values. We've accomplished this using some simple Redis functionality along with ZSET or sorted set functionality, which is a bit more complex, and which we needed because Redis does not allow individual entries in a Hash to be deleted. We can see that by using method names such as RedisWeatherCache.[] and RedisWeatherCache.[]=, we've maintained a Hash-like interface, which made it easy to use this cache instead of the simple in-memory Ruby Hash we had in our previous iteration. Our tests all pass and are still pretty simple, thanks to the modularity of this new cache code, the modular configuration file, and the previous fixes we made to our specs to remove Internet and run-order dependencies. Summary In this article, we delved into setting up and cleaning up state for real-world specs that interact with external services and local resources by extending our WeatherQuery example to address a big bug, isolate our specs from the Internet, and cleanly configure a Redis database to serve as a better cache. Resources for Article: Further resources on this subject: Creating your first heat map in R [article] Probability of R? [article] Programming on Raspbian [article]
Read more
  • 0
  • 0
  • 3541
Unlock access to the largest independent learning library in Tech for FREE!
Get unlimited access to 7500+ expert-authored eBooks and video courses covering every tech area you can think of.
Renews at €18.99/month. Cancel anytime
article-image-test-all-things-python
Packt
18 Feb 2016
20 min read
Save for later

Test all the things with Python

Packt
18 Feb 2016
20 min read
The first testing tool we're going to look at is called doctest. The name is short for "document testing" or perhaps a "testable document". Either way, it's a literate tool designed to make it easy to write tests in such a way that computers and humans both benefit from them. Ideally, doctest tests both, informs human readers, and tells the computer what to expect. Mixing tests and documentation helps us: Keeps the documentation up-to-date with reality Make sure that the tests express the intended behavior Reuse some of the efforts involved in the documentation and test creation (For more resources related to this topic, see here.) Where doctest performs best The design decisions that went into doctest make it particularly well suited to writing acceptance tests at the integration and system testing levels. This is because doctest mixes human-only text with examples that both humans and computers can read. This structure doesn't support or enforce any of the formalizations of testing, but it conveys information beautifully and it still provides the computer with the ability to say that works or that doesn't work. As an added bonus, it is about the easiest way to write tests you'll ever see. In other words, a doctest file is a truly excellent program specification that you can have the computer check against your actual code any time you want. API documentation also benefits from being written as doctests and checked alongside your other tests. You can even include doctests in your docstrings. The basic idea you should be getting from all this is that doctest is ideal for uses where humans and computers will both benefit from reading them. The doctest language Like program source code, doctest tests are written in plain text. The doctest module extracts the tests and ignores the rest of the text, which means that the tests can be embedded in human-readable explanations or discussions. This is the feature that makes doctest suitable for uses such as program specifications. Example – creating and running a simple doctest We are going to create a simple doctest file, to show the fundamentals of using the tool. Perform the following steps: Open a new text file in your editor, and name it test.txt. Insert the following text into the file: This is a simple doctest that checks some of Python's arithmetic >>> 2 + 2 4 >>> 3 * 3 10 We can now run the doctest. At the command prompt, change to the directory where you saved test.txt. Type the following command: $ python3 ‑m doctest test.txt When the test is run, you should see output like this: Result – three times three does not equal ten You just wrote a doctest file that describes a couple of arithmetic operations, and ran it to check whether Python behaved as the tests said it should. You ran the tests by telling Python to execute doctest on the file containing the tests. In this case, Python's behavior differed from the tests because, according to the tests, three times three equals ten. However, Python disagrees on that. As doctest expected one thing and Python did something different, doctest presented you with a nice little error report showing where to find the failed test, and how the actual result differed from the expected result. At the bottom of the report is a summary showing how many tests failed in each file tested, which is helpful when you have more than one file containing tests. The syntax of doctests You might have already figured it out from looking at the previous example: doctest recognizes tests by looking for sections of text that look like they've been copied and pasted from a Python interactive session. Anything that can be expressed in Python is valid within a doctest. Lines that start with a >>> prompt are sent to a Python interpreter. Lines that start with a ... prompt are sent as continuations of the code from the previous line, allowing you to embed complex block statements into your doctests. Finally, any lines that don't start with >>> or ..., up to the next blank line or >>> prompt, represent the output expected from the statement. The output appears as it would in an interactive Python session, including both the return value and anything printed to the console. If you don't have any output lines, doctest assumes it to mean that the statement is expected to have no visible result on the console, which usually means that it returns None. The doctest module ignores anything in the file that isn't part of a test, which means that you can put explanatory text, HTML, line-art diagrams, or whatever else strikes your fancy in between your tests. We took advantage of this in the previous doctest to add an explanatory sentence before the test itself. Example – a more complex test Add the following code to your test.txt file, separated from the existing code by at least one blank line: Now we're going to take some more of doctest's syntax for a spin.   >>> import sys >>> def test_write(): ...     sys.stdout.write("Hellon") ...     return True >>> test_write() Hello True Now take a moment to consider before running the test. Will it pass or fail? Should it pass or fail? Result – five tests run? Just as we discussed before, run the test using the following command: python3 -m doctest test.txt You should see a result like this: Because we added the new tests to the same file containing the tests from before, we still see the notification that three times three does not equal 10. Now, though, we also see that five tests were run, which means our new tests ran and were successful. Why five tests? As far as doctest is concerned, we added the following three tests to the file: The first one says that, when we import sys, nothing visible should happen The second test says that, when we define the test_write function, nothing visible should happen The third test says that, when we call the test_write function, Hello and True should appear on the console, in that order, on separate lines Since all three of these tests pass, doctest doesn't bother to say much about them. All it did was increase the number of tests reported at the bottom from two to five. Expecting exceptions That's all well and good for testing that things work as expected, but it is just as important to make sure that things fail when they're supposed to fail. Put another way: sometimes your code is supposed to raise an exception, and you need to be able to write tests that check that behavior as well. Fortunately, doctest follows nearly the same principle in dealing with exceptions as it does with everything else; it looks for text that looks like a Python interactive session. This means it looks for text that looks like a Python exception report and traceback, and matches it against any exception that gets raised. The doctest module does handle exceptions a little differently from the way it handles other things. It doesn't just match the text precisely and report a failure if it doesn't match. Exception tracebacks tend to contain many details that are not relevant to the test, but that can change unexpectedly. The doctest module deals with this by ignoring the traceback entirely: it's only concerned with the first line, Traceback (most recent call last):, which tells it that you expect an exception, and the part after the traceback, which tells it which exception you expect. The doctest module only reports a failure if one of these parts does not match. This is helpful for a second reason as well: manually figuring out what the traceback will look like, when you're writing your tests, would require a significant amount of effort and would gain you nothing. It's better to simply omit them. Example – checking for an exception This is yet another test that you can add to test.txt, this time testing some code that ought to raise an exception. Insert the following text into your doctest file, as always separated by at least one blank line: Here we use doctest's exception syntax to check that Python is correctly enforcing its grammar. The error is a missing ) on the def line. >>> def faulty(: ...     yield from [1, 2, 3, 4, 5] Traceback (most recent call last): SyntaxError: invalid syntax The test is supposed to raise an exception, so it will fail if it doesn't raise the exception or if it raises the wrong exception. Make sure that you have your mind wrapped around this: if the test code executes successfully, the test fails, because it expected an exception. Run the tests using the following doctest: python3 -m doctest test.txt Result – success at failing The code contains a syntax error, which means this raises a SyntaxError exception, which in turn means that the example behaves as expected; this signifies that the test passes. When dealing with exceptions, it is often desirable to be able to use a wildcard matching mechanism. The doctest provides this facility through its ellipsis directive that we'll discuss shortly. Expecting blank lines The doctest uses the first blank line after  >>> to identify the end of the expected output, so what do you do when the expected output actually contains a blank line? The doctest handles this situation by matching a line that contains only the text <BLANKLINE> in the expected output with a real blank line in the actual output. Controlling doctest behavior with directives Sometimes, the default behavior of doctest makes writing a particular test inconvenient. For example, doctest might look at a trivial difference between the expected and real outputs and wrongly conclude that the test has failed. This is where doctest directives come to the rescue. Directives are specially formatted comments that you can place after the source code of a test and that tell doctest to alter its default behavior in some way. A directive comment begins with # doctest:, after which comes a comma-separated list of options that either enable or disable various behaviors. To enable a behavior, write a + (plus symbol) followed by the behavior name. To disable a behavior, white a – (minus symbol) followed by the behavior name. We'll take a look at the several directives in the following sections. Ignoring part of the result It's fairly common that only part of the output of a test is actually relevant to determining whether the test passes. By using the +ELLIPSIS directive, you can make doctest treat the text ... (called an ellipsis) in the expected output as a wildcard that will match any text in the output. When you use an ellipsis, doctest will scan until it finds text matching whatever comes after the ellipsis in the expected output, and continue matching from there. This can lead to surprising results such as an ellipsis matching against a 0-length section of the actual output, or against multiple lines. For this reason, it needs to be used thoughtfully. Example – ellipsis test drive We're going to use the ellipsis in a few different tests to better get a feel of how it works. As an added bonus, these tests also show the use of doctest directives. Add the following code to your test.txt file: Next up, we're exploring the ellipsis. >>> sys.modules # doctest: +ELLIPSIS {...'sys': <module 'sys' (built-in)>...}   >>> 'This is an expression that evaluates to a string' ... # doctest: +ELLIPSIS 'This is ... a string'   >>> 'This is also a string' # doctest: +ELLIPSIS 'This is ... a string'   >>> import datetime >>> datetime.datetime.now().isoformat() # doctest: +ELLIPSIS '...-...-...T...:...:...' Result – ellipsis elides The tests all pass, where they would all fail without the ellipsis. The first and last tests, in which we checked for the presence of a specific module in sys.modules and confirmed a specific formatting while ignoring the contents of a string, demonstrate the kind of situation where ellipsis is really useful, because it lets you focus on the part of the output that is meaningful and ignore the rest of the test. The middle tests demonstrate how different outputs can match the same expected result when ellipsis is in play. Look at the last test. Can you imagine any output that wasn't an ISO-formatted time stamp, but that would match the example anyway? Remember that the ellipsis can match any amount of text. Ignoring white space Sometimes, white space (spaces, tabs, newlines, and their ilk) is more trouble than it's worth. Maybe you want to be able to break a single line of expected output across several lines in your test file, or maybe you're testing a system that uses lots of white space but doesn't convey any useful information with it. The doctest gives you a way to "normalize" white space, turning any sequence of white space characters, in both the expected output and in the actual output, into a single space. It then checks whether these normalized versions match. Example – invoking normality We're going to write a couple of tests that demonstrate how whitespace normalization works. Insert the following code into your doctest file: Next, a demonstration of whitespace normalization. >>> [1, 2, 3, 4, 5, 6, 7, 8, 9] ... # doctest: +NORMALIZE_WHITESPACE [1, 2, 3,  4, 5, 6,  7, 8, 9]   >>> sys.stdout.write("This textn contains weird     spacing.n") ... # doctest: +NORMALIZE_WHITESPACE This text contains weird spacing. 39 Result – white space matches any other white space Both of these tests pass, in spite of the fact that the result of the first one has been wrapped across multiple lines to make it easy for humans to read, and the result of the second one has had its strange newlines and indentations left out, also for human convenience. Notice how one of the tests inserts extra whitespace in the expected output, while the other one ignores extra whitespace in the actual output? When you use +NORMALIZE_WHITESPACE, you gain a lot of flexibility with regard to how things are formatted in the text file. You may have noted the value 39 on the last line of the last example. Why is that there? It's because the write() method returns the number of bytes that were written, which in this case happens to be 39. If you're trying this example in an environment that maps ASCII characters to more than one byte, you will see a different number here; this will cause the test to fail until you change the expected number of bytes. Skipping an example On some occasions, doctest will recognize some text as an example to be checked, when in truth you want it to be simply text. This situation is rarer than it might at first seem, because usually there's no harm in letting doctest check everything it can. In fact, usually it's very helpful to have doctest check everything it can. For those times when you want to limit what doctest checks, though, there's the +SKIP directive. Example – humans only Append the following code to your doctest file: Now we're telling doctest to skip a test   >>> 'This test would fail.' # doctest: +SKIP If it were allowed to run. Result – it looks like a test, but it's not Before we added this last example to the file, doctest reported thirteen tests when we ran the file through it. After adding this code, doctest still reports thirteen tests. Adding the skip directive to the code completely removed it from consideration by doctest. It's not a test that passes, nor a test that fails. It's not a test at all. The other directives There are a number of other directives that can be issued to doctest, should you find the need. They're not as broadly useful as the ones already mentioned, but the time might come when you require one or more of them. The full documentation for all of the doctest directives can be found at http://docs.python.org/3/library/doctest.html#doctest-options. The remaining directives of doctest in the Python 3.4 version are as follows: DONT_ACCEPT_TRUE_FOR_1: This makes doctest differentiate between boolean values and numbers DONT_ACCEPT_BLANKLINE: This removes support for the <BLANKLINE> feature IGNORE_EXCEPTION_DETAIL: This makes doctest only care that an exception is of the expected type Strictly speaking, doctest supports several other options that can be set using the directive syntax, but they don't make any sense as directives, so we'll ignore them here. The execution scope of doctest tests When doctest is running the tests from text files, all the tests from the same file are run in the same execution scope. This means that, if you import a module or bind a variable in one test, that module or variable is still available in later tests. We took advantage of this fact several times in the tests written so far in this article: the sys module was only imported once, for example, although it was used in several tests. This behavior is not necessarily beneficial, because tests need to be isolated from each other. We don't want them to contaminate each other because, if a test depends on something that another test does, or if it fails because of something that another test does, these two tests are in some sense combined into one test that covers a larger section of your code. You don't want that to happen, because then knowing which test has failed doesn't give you as much information about what went wrong and where it happened. So, how can we give each test its own execution scope? There are a few ways to do it. One would be to simply place each test in its own file, along with whatever explanatory text that is needed. This works well in terms of functionality, but running the tests can be a pain unless you have a tool to find and run all of them for you. Another problem with this approach is that this breaks the idea that the tests contribute to a human-readable document. Another way to give each test its own execution scope is to define each test within a function, as follows: >>> def test1(): ...     import frob ...     return frob.hash('qux') >>> test1() 77 By doing this, the only thing that ends up in the shared scope is the test function (named test1 here). The frob module and any other names bound inside the function are isolated with the caveat that things that happen inside imported modules are not isolated. If the frob.hash() method changes a state inside the frob module, that state will still be changed if a different test imports the frob module again. The third way is to exercise caution with the names you create, and be sure to set them to known values at the beginning of each test section. In many ways this is the easiest approach, but this is also the one that places the most burden on you, because you have to keep track of what's in the scope. Why does doctest behave in this way, instead of isolating tests from each other? The doctest files are intended not just for computers to read, but also for humans. They often form a sort of narrative, flowing from one thing to the next. It would break the narrative to be constantly repeating what came before. In other words, this approach is a compromise between being a document and being a test framework, a middle ground that works for both humans and computers. Check your understanding Once you've decided on your answers to these questions, check them by writing a test document and running it through doctest: How does doctest recognize the beginning of a test in a document? How does doctest know when a test continues to further lines? How does doctest recognize the beginning and end of the expected output of a test? How would you tell doctest that you want to break the expected output across several lines, even though that's not how the test actually outputs it? Which parts of an exception report are ignored by doctest? When you assign a variable in a test file, which parts of the file can actually see that variable? Why do we care what code can see the variables created by a test? How can we make doctest not care what a section of output contains? Exercise – English to doctest Time to stretch your wings a bit. I'm going to give you a description of a single function in English. Your job is to copy the description into a new text file, and then add tests that describe all the requirements in a way that the computer can understand and check. Try to make the doctests so that they're not just for the computer. Good doctests tend to clarify things for human readers as well. By and large, this means that you present them to human readers as examples interspersed with the text. Without further ado, here is the English description: The fib(N) function takes a single integer as its only parameter N. If N is 0 or 1, the function returns 1. If N is less than 0, the function raises a ValueError. Otherwise, the function returns the sum of fib(N – 1) and fib(N – 2). The returned value will never be less than 1. A naïve implementation of this function would get very slow as N increased. I'll give you a hint and point out that the last sentence about the function being slow, isn't really testable. As computers get faster, any test you write that depends on an arbitrary definition of "slow" will eventually fail. Also, there's no good way to test the difference between a slow function and a function stuck in an infinite loop, so there's not much point in trying. If you find yourself needing to do that, it's best to back off and try a different solution. Not being able to tell whether a function is stuck or just slow is called the halting problem by computer scientists. We know that it can't be solved unless we someday discover a fundamentally better kind of computer. Faster computers won't do the trick, and neither will quantum computers, so don't hold your breath. The next-to-last sentence also provides some difficulty, since to test it completely would require running every positive integer through the fib() function, which would take forever (except that the computer will eventually run out of memory and force Python to raise an exception). How do we deal with this sort of thing, then? The best solution is to check whether the condition holds true for a random sample of viable inputs. The random.randrange() and random.choice() functions in the Python standard library make that fairly easy to do. Summary We learned the syntax of doctest, and went through several examples describing how to use it. After that, we took a real-world specification for the AVL tree, and examined how to formalize it as a set of doctests, so that we could use it to automatically check the correctness of an implementation. Specifically, we covered doctest's default syntax and the directives that alter it, how to write doctests in text files, how to write doctests in Python docstrings, and what it feels like to use doctest to turn a specification into tests. If, you want to learn more about Python Testing then you can refer the following books: Expert Python Programming: https://www.packtpub.com/application-development/expert-python-programming Python Testing Cookbook: https://www.packtpub.com/application-development/python-testing-cookbook Resources for Article:   Further resources on this subject: Façade Pattern – Being Adaptive with Façade [article] Predicting Sports Winners with Decision Trees and pandas [article] Gradient Descent at Work [article]
Read more
  • 0
  • 0
  • 3540

article-image-symbolizers
Packt
18 Jun 2015
8 min read
Save for later

Symbolizers

Packt
18 Jun 2015
8 min read
In this article by Erik Westra, author of the book, Python Geospatial Analysis Essentials, we will see that symbolizers do the actual work of drawing a feature onto the map. Multiple symbolizers are often used to draw a single feature. There are many different types of symbolizers available within Mapnik, and many of the symbolizers have complex options associated with them. Rather than exhaustively listing all the symbolizers and their various options, we will instead just look at some of the more common types of symbolizers and how they can be used. (For more resources related to this topic, see here.) PointSymbolizer The PointSymbolizer class is used to draw an image centered over a Point geometry. By default, each point is displayed as a 4 x 4 pixel black square: To use a different image, you have to create a mapnik.PathExpression object to represent the path to the desired image file, and then pass that to the PointSymbolizer object when you instantiate it: path = mapnik.PathExpression("/path/to/image.png") point_symbol = PointSymbolizer(path) Note that PointSymbolizer draws the image centered on the desired point. To use a drop-pin image as shown in the preceding example, you will need to add extra transparent whitespace so that the tip of the pin is in the middle of the image, like this: You can control the opacity of the drawn image by setting the symbolizer's opacity attribute. You can also control whether labels will be drawn on top of the image by setting the allow_overlap attribute to True. Finally, you can apply an SVG transformation to the image by setting the transform attribute to a string containing a standard SVG transformation expression, for example point_symbol.transform = "rotate(45)". Documentation for the PointSymbolizer can be found at https://github.com/mapnik/mapnik/wiki/PointSymbolizer. LineSymbolizer A mapnik.LineSymbolizer is used to draw LineString geometries and the outlines of Polygon geometries. When you create a new LineSymbolizer, you would typically configure it using two parameters: the color to use to draw the line as a mapnik.Color object, and the width of the line, measured in pixels. For example: line_symbol = mapnik.LineSymbolizer(mapnik.Color("black"), 0.5) Notice that you can use fractional line widths; because Mapnik uses anti-aliasing, a line narrower than 1 pixel will often look better than a line with an integer width if you are drawing many lines close together. In addition to the color and the width, you can also make the line semi-transparent by setting the opacity attribute. This should be set to a number between 0.0 and 1.0, where 0.0 means the line will be completely transparent and 1.0 means the line will be completely opaque. You can also use the stroke attribute to get access to (or replace) the stroke object used by the line symbolizer. The stroke object, an instance of mapnik.Stroke, can be used for more complicated visual effects. For example, you can create a dashed line effect by calling the stroke's add_dash() method: line_symbol.stroke.add_dash(5, 7) Both numbers are measured in pixels; the first number is the length of the dash segment, while the second is the length of the gap between dashes. Note that you can create alternating dash patterns by calling add_dash() more than once. You can also set the stroke's line_cap attribute to control how the ends of the line should be drawn, and the stroke's line_join attribute to control how the joins between the individual line segments are drawn whenever the LineString changes direction. The line_cap attribute can be set to one of the following values: mapnik.line_cap.BUTT_CAP mapnik.line_cap.ROUND_CAP mapnik.line_cap.SQUARE_CAP The line_join attribute can be set to one of the following: mapnik.line_join.MITER_JOIN mapnik.line_join.ROUND_JOIN mapnik.line_join.BEVEL_JOIN Documentation for the LineSymbolizer class can be found at https://github.com/mapnik/mapnik/wiki/LineSymbolizer. PolygonSymbolizer The mapnik.PolygonSymbolizer class is used to fill the interior of a Polygon geometry with a given color. When you create a new PolygonSymbolizer, you would typically pass it a single parameter: the mapnik.Color object to use to fill the polygon. You can also change the opacity of the symbolizer by setting the fill_opacity attribute, for example: fill_symbol.fill_opacity = 0.8 Once again, the opacity is measured from 0.0 (completely transparent) to 1.0 (completely opaque). There is one other PolygonSymbolizer attribute which you might find useful: gamma. The gamma value can be set to a number between 0.0 and 1.0. The gamma value controls the amount of anti-aliasing used to draw the edge of the polygon; with the default gamma value of 1.0, the edges of the polygon will be fully anti-aliased. While this is usually a good thing, if you try to draw adjacent polygons with the same color, the antialiasing will cause the edges of the polygons to be visible rather than combining them into a single larger area. By turning down the gamma slightly (for example, fill_symbol.gamma = 0.6), the edges between adjacent polygons will disappear. Documentation for the PolygonSymbolizer class can be found at https://github.com/mapnik/mapnik/wiki/PolygonSymbolizer. TextSymbolizer The TextSymbolizer class is used to draw textual labels onto a map. This type of symbolizer can be used for point, LineString, and Polygon geometries. The following example shows how a TextSymbolizer can be used: text_symbol = mapnik.TextSymbolizer(mapnik.Expresion("[label]"), "DejaVu Sans Book", 10, mapnik.Color("black")) As you can see, four parameters are typically passed to the TextSymbolizer's initializer: A mapnik.Expression object defining the text to be displayed. In this case, the text to be displayed will come from the label attribute in the datasource. The name of the font to use for drawing the text. To see what fonts are available, type the following into the Python command line: import mapnik for font in mapnik.FontEngine.face_names():    print font The font size, measured in pixels. The color to use to draw the text. By default, the text will be drawn in the center of the geometry; for example: This positioning of the label is called point placement. The TextSymbolizer allows you to change this to use what is called line placement, where the label will be drawn along the lines: text_symbol.label_placement = mapnik.label_placement.LINE_PLACEMENT As you can see, this causes the label to be drawn along the length of a LineString geometry, or along the perimeter of a Polygon. The text won't be drawn at all for a Point geometry, since there are no lines within a point. The TextSymbolizer will normally just draw the label once, but you can tell the symbolizer to repeat the label if you wish by specifying a pixel gap to use between each label: text_symbol.label_spacing = 30 By default, Mapnik is smart enough to stop labels from overlapping each other. If possible, it moves the label slightly to avoid an overlap, and then hides the label completely if it would still overlap. For example: You can change this by setting the allow_overlap attribute: text_symbol.allow_overlap = True Finally, you can set a halo effect to draw a lighter-colored border around the text so that it is visible even against a dark background. For example, text_symbol.halo_fill = mapnik.Color("white") text_symbol.halo_radius = 1 There are many more labeling options, all of which are described at length in the documentation for the TextSymbolizer class. This can be found at https://github.com/mapnik/mapnik/wiki/TextSymbolizer. RasterSymbolizer The RasterSymbolizer class is used to draw raster-format data onto a map. This type of symbolizer is typically used in conjunction with a Raster or GDAL datasource. To create a new raster symbolizer, you instantiate a new mapnik.RasterSymbolizer object: raster_symbol = mapnik.RasterSymbolizer() The raster symbolizer will automatically draw any raster-format data provided by the map layer's datasource. This is often used to draw a basemap onto which the vector data is to be displayed; for example: While there are some advanced options to control the way the raster data is displayed, in most cases, the only option you might be interested in is the opacity attribute. As usual, this sets the opacity for the displayed image, allowing you to layer semi-transparent raster images one on top of the other. Documentation for the RasterSymbolizer can be found at https://github.com/mapnik/mapnik/wiki/RasterSymbolizer. Summary In this article, we covered different types of symbolizers, which are available in the Mapnik library. We also examined that symbolizers which can be used to display spatial features, how the visible extent is used to control the portion of the map to be displayed, and how to render a map as an image file. Resources for Article: Further resources on this subject: Python functions – Avoid repeating code [article] Preparing to Build Your Own GIS Application [article] Server Logs [article]
Read more
  • 0
  • 0
  • 3517

article-image-biztalk-server-standard-message-exchange-patterns-and-types-service
Packt
06 Apr 2010
4 min read
Save for later

BizTalk Server: Standard Message Exchange Patterns and Types of Service

Packt
06 Apr 2010
4 min read
Identifying Standard Message Exchange Patterns When we talk about Message Exchange Patterns, or MEPs, we're considering the direction and timing of data between the client and service. How do I get into the bus and what are the implications of those choices? Let's discuss the four primary options. Request/Response services This is probably the pattern that's most familiar to you. We're all comfortable making a function call to a component and waiting for a response. When a service uses this pattern, it's frequently performing a remote procedure call where the caller accesses functionality on the distant service and is blocked until either a timeout occurs or until the receiver sends a response that is expected by the caller. As we'll see below, while this pattern may set developers at ease, it may encourage bad behavior. Nevertheless, the cases where request/response services make the most sense are fine-grained functions and mashup services. If you need a list of active contracts that a hospital has with your company, then a request/response operation fits best. The client application should wait until that response is received before moving on to the next portion of the application. Or, let's say my web portal is calling an aggregate service, which takes contact data from five different systems and mashes them up into a single data entity that is then returned to the caller. This data is being requested for immediate presentation to an end user, and thus it's logical to solicit information from a service and wait to draw the screen until the completed result is loaded. BizTalk Server 2009 has full support for both consuming and publishing services adhering to a request/response pattern. When exposing request/response operations through BizTalk orchestrations, the orchestration port's Communication Pattern is set to Request-Response and the Port direction of communication is equal to I'll be receiving a request and sending a response. Once this orchestration port is bound to a physical request/response receive port, BizTalk takes care of correlating the response message with the appropriate thread that made the request. This is significant because by default, BizTalk is a purely asynchronous messaging engine. Even when you configure BizTalk Server to behave in a request/response fashion, it's only putting a facade on the standard underlying plumbing. A synchronous BizTalk service interface actually sits on top of a sophisticated mechanism of correlating MessageBox communication to simulate a request/response pattern. When consuming request/response services from BizTalk from an orchestration, the orchestration port's Communication Pattern is set to Request-Response and the Port direction of communication is equal to I'll be sending a request and receiving a response. The corresponding physical send port uses a solicit-response pattern and allows the user to set up both pipelines and maps for the inbound and outbound messages. One concern with either publishing or consuming request/response services is the issue of blocking and timeouts. From a BizTalk perspective, this means that whenever you publish an orchestration as a request/response service, you should always verify that the logic residing between inbound and outbound transmissions will either complete or fail within a relatively brief amount of time. This dictates wrapping this logic inside an orchestration Scope shape with a preset timeout that is longer than the standard web service timeout interval. For consuming services, a request/response pattern forces the orchestration to block and wait for the response to be returned. If the service response isn't necessary for processing to continue, consider using a Parallel shape that isolates the service interaction pattern on a dedicated branch. This way, the execution of unrelated workflow steps can proceed even though the downstream service is yet to respond.
Read more
  • 0
  • 0
  • 3516

article-image-sage-act-2011-working-act-dashboards
Packt
18 Feb 2011
6 min read
Save for later

Sage ACT! 2011: Working with the ACT! Dashboards

Packt
18 Feb 2011
6 min read
In ACT! CRM, dashboards allow you to access key information in the form of a graphical interface. You can filter a Dashboard so that it contains just the information you need, or you can tweak the various elements of a Dashboard to give it a different look. Administrators and Managers of your ACT! database can create brand new Dashboards if they're required. If you want more details about the information you see in a Dashboard, you can drill-down into the Dashboard with a simple double-click to access all the juicy details. At that point, you can edit or add to your information and the Dashboard will update automatically. You can even print out a hard copy of a Dashboard to preserve the contents for posterity. Because many of the Dashboards consist of pie charts and graphs, you might even want to copy one of them and paste it into other applications such as Word, Excel, or PowerPoint. Getting familiar with the Dashboard layouts Quite simply, a Dashboard is a graphical interface that gives you a visual snapshot of a part of your business. ACT! Dashboards let you view and work with the various information contained in your database in one easy-to-access location. Dashboards can take the form of charts, graphs, or even lists. Dashboards are associated with a database and not a user; therefore all users share the same Dashboards. However, you can select your own Dashboard view in much the same way that you can select a layout. You can also filter the information that is shown in your Dashboard. A Dashboard consists of two parts: The Dashboard layout: The Dashboard layout determines which Dashboard components you see and how the filters are set. ACT! comes with six Dashboard layouts. However, Managers and Administrators can create additional Dashboard layouts using the Dashboard Designer or make permanent changes to the existing ones. Dashboard components: Each Dashboard layout consists of one or more components. A component displays different types of data from the ACT! database. For example, a Dashboard layout might include a component that lists a user's top 10 current sales opportunities, another component that graphs the activities of a specific user, and a component that displays a pie chart of the company's current pipeline. A layout can have a maximum of six components. Getting ready In order to really take advantage of the ACT! Dashboard, you'll need to make sure that your database contains a variety of information. Specifically you'll need to make sure that your database contains a few Contacts, Activities, and Opportunities if you're going to view any of those Dashboards. How to do it... Click the Dashboard icon on ACT!'s navigation bar. The following screenshot shows you an example of the default Dashboard page: Choose the Dashboard layout you want from the Dashboard drop-down list, at the top-left side of the Dashboard. How it works... Many of the layouts consist of the exact same components. However, the components in each of the layouts are filtered differently, giving the layouts a bit of variety. There's more... Out of the box there are five Dashboard layouts: ACT! Activities Dashboard: Shows the activities of the user currently signed into ACT!, like the one you see in the following screenshot: ACT! Administration Dashboard: Lists the database users and shows when they've logged in and out of the database. Also lists any remote sync users, the date of their list sync, and how many days they have before their remote database expires if they don't synchronize. ACT! Contact Dashboard: Gives you a list of recently created contacts, recently edited contacts, and the number of fields that have changed. ACT! Default Dashboard: Includes three activity and three opportunity components. ACT! Opportunities Dashboard: Provides you with four different opportunity components including sales analysis by stage, value, and product. Each Dashboard layout is actually a file ending with the .dsh extension. You'll find them safely filed in the Dashboards sub-folder of the database files folder associated with your database. Accessing information from Dashboards Once you've become familiar with the various Dashboard layouts your next step is to start exploring the components found in each layout to see what data they contain. The components are generally arranged in a grid of two columns of three rows for a total of six components per Dashboard layout. The basic components include: My Schedule At-A-Glance: Found on the Default and Activities Dashboards, this component lists your activities of the current user for the current day. Activities by User: Found on the Default and Activities Dashboards, this component displays the activities for the current user for the current month, sorted by type, in a bar chart. The total numbers of activities are shown in the following screenshot. In addition you can hover your mouse over a bar section to see the number of activities for that activity type. Activity List: Found on the Activities Dashboard, this component is identical to the My Schedule At-A-Glance components, except that it lists all the activities for the current month. Opportunity Pipeline by Stage: Found on the Opportunities and Default Dashboards, this component displays your open opportunities in the ACT! Sales Cycle process for the current month sorted by stage, in a pie chart and includes a recap on the side like the one you see in the following screenshot: Contact History Count by User Type: Found on the Contacts Dashboard this component displays history items created by database users within a specified number of days similar to what's shown in the following screenshot: Opportunities - Open by Product: Found on the Opportunities Dashboard, the component displays a pie chart of the open opportunities by product and by user. Top 10 Opportunities: Found on the Opportunities and Default Dashboards, this component displays a list, by company and opportunity name, of the top ten open opportunities in the ACT! Sales Cycle process for the current month. Closed Sales to Date: Found on the Opportunities and Default Dashboards, this component displays the weighted and total value of opportunities in the ACT! Sales Cycle process, you closed and won in the form of a chart. Optionally you can customize the component to indicate a sales goal like the one shown in the following screenshot:
Read more
  • 0
  • 0
  • 3498
article-image-getting-started-scratch-14-part-1
Packt
16 Oct 2009
6 min read
Save for later

Getting Started with Scratch 1.4 (Part 1)

Packt
16 Oct 2009
6 min read
Before we create any code, let's make sure we speak the same language. The interface at a glance When we encounter software that's unfamiliar to us, we often wonder, "Where do I begin?" Together, we'll answer that question and click through some important sections of the Scratch interface so that we can quickly start creating our own projects. Now, open Scratch and let's begin. Time for action – first step When we open Scratch, we notice that the development environment roughly divides into three distinct sections, as seen in the following screenshot. Moving from left to right, we have the following sections in sequential order: Blocks palette Script editor Stage Let's see if we can get our cat moving: In the blocks palette, click on the Looks button. Drag the switch to costume block onto the scripts area. Now, in the blocks palette, click on the Control button. Drag the when flag clicked block to the scripts area and snap it on top of the switch to costume block, as illustrated in the following screenshot. How to snap two blocks together?As you drag a block onto another block, a white line displays to indicate that the block you are dragging can be added to the script. When you see the white line, release your mouse to snap the block in place. In the scripts area, click on the Costumes tab to display the sprite's costumes. Click on costume2 to change the sprite on the stage. Now, click back on costume1 to change how the sprite displays on the stage. Directly beneath the stage is a sprites list. The current list displays Sprite1 and Stage. Click on the sprite named Stage and notice that the scripts area changes. Click back on Sprite1 in the sprites list and again note the change to the scripts area. Click on the flag above the stage to set our first Scratch program in motion. Watch closely, or you might miss it. What just happened? Congratulations! You created your first Scratch project. Let's take a closer look at what we did just now. As we clicked through the blocks palette, we saw that the available blocks changed depending on whether we chose Motion, Looks, or Control. Each set of blocks is color-coded to help us easily identify them in our scripts. The first block we added to the script instructed the sprite to display costume2. The second block provided a way to control our script by clicking on the flag. Blocks with a smooth top are called hats in Scratch terminology because they can be placed only at the top of a stack of blocks. Did you look closely at the blocks as you snapped the control block into the looks block? The bottom of the when flag clicked block had a protrusion like a puzzle piece that fits the indent on the top of the switch to costume block. As children, most of us probably have played a game where we needed to put the round peg into the round hole. Building a Scratch program is just that simple. We see instantly how one block may or may not fit into another block. Stack blocks have indents on top and bumps on the bottom that allow blocks to lock together to form a sequence of actions that we call a script. A block depicting its indent and bump can be seen in the following screenshot: When we clicked on the Costumes tab, we learned that our cat had two costumes or appearances. Clicking on the costume caused the cat on the stage to change its appearance. As we clicked around the sprites list, we discovered our project had two sprites: a cat and a stage. And the script we created for the cat didn't transfer to the stage. We finished the exercise by clicking on the flag. The change was subtle, but our cat appeared to take its first step when it switched to costume2. Basics of a Scratch project Inside every Scratch project, we find the following ingredients: sprites, costumes, blocks, scripts, and a stage. It's how we mix the ingredients with our imagination that creates captivating stories, animations, and games. Sprites bring our program to life, and every project has at least one. Throughout the book, we'll learn how to add and customize sprites. A sprite wears a costume. Change the costume and you change the way the sprite looks. If the sprite happens to be the stage, the costume is known as a background. Blocks are just categories of instructions that include motion, looks, sound, pen, control, sensing, operators, and variables. Scripts define a set of blocks that tell a sprite exactly what to do. Each block represents an instruction or piece of information that affects the sprite in some way. We're all actors on Scratch's stage Think of each sprite in a Scratch program as an actor. Each actor walks onto the stage and recites a set of lines from the script. How each actor interacts with another actor depends on the words the director chooses. On Scratch's stage, every object, even the stone in the corner, is a sprite capable of contributing to the story. As directors, we have full creative control. Time for action – save your work It's a good practice to get in the habit of saving your work. Save your work early, and save it often: To save your new project, click the disk icon at the top of the Scratch window or click File | Save As. A Save Project dialog box opens and asks you for a location and a New Filename. Enter some descriptive information for your project by supplying the Project author and notes About this project in the fields provided. Set the cat in motion Even though our script contains only two blocks, we have a problem. When we click on the flag, the sprite switches to a different costume and stops. If we try to click on the flag again, nothing appears to happen, and we can't get back to the first costume unless we go to the Costumes tab and select costume1. That's not fun. In our next exercise, we're going to switch between both costumes and create a lively animation.
Read more
  • 0
  • 0
  • 3497

article-image-getting-started-openlayers
Packt
22 Mar 2011
9 min read
Save for later

Getting Started with OpenLayers

Packt
22 Mar 2011
9 min read
OpenLayers 2.10 Beginner's Guide Create, optimize, and deploy stunning cross-browser web maps with the OpenLayers JavaScript web mapping library What is OpenLayers? OpenLayers is an open source, client side JavaScript library for making interactive web maps, viewable in nearly any web browser. Since it is a client side library, it requires no special server side so beware or settings—you can use it without even downloading anything! Originally developed by Metacarta, as a response, in part, to Google Maps, it has grown into a mature, popular framework with many passionate developers and a very helpful community. Why use OpenLayers? OpenLayers makes creating powerful web-mapping applications easy and fun. It is very powerful but also easy to use—you don't even need to be a programmer to make a great map with it. It's open source, free, and has a strong community behind it. So if you want to dig into the internal code, or even improve it, you're encouraged to do so. Cross browser compatibility is handled for you—it even works in IE6. OpenLayers is not tied to any proprietary technology or company, so you don't have to worry so much about your application breaking (unless you break it). At the time of writing, support for modern mobile and touch devices is in the works (with many proof of concept examples), and should be in the official library in the near future—if they aren't by the time you're reading this. OpenLayers allows you to build entire mapping applications from the ground up, with the ability to customize every aspect of your map—layers, controls, events, etc. You can use a multitude of different map server backends together, including a powerful vector layer. It makes creating map 'mashups' extremely easy. What, technically, is OpenLayers? We said OpenLayers is a client side JavaScript library, but what does this mean? Client side When we say client side we are referring to the user's computer, specifically their web browser. The only thing you need to have to make OpenLayers work is the OpenLayers code itself and a web browser. You can either download it and use it on your computer locally, or download nothing and simply link to the JavaScript file served on the site that hosts the OpenLayers project (http://openlayers.org). OpenLayers works on nearly all browsers and can be served by any web server or your own computer. Using a modern, standard-based browser such as Firefox, Google Chrome, Safari, or Opera is recommended. Library When we say library we mean that OpenLayers is an API (Application Programmer Interface) that provides you with tools to develop your own web maps. Instead of building a mapping application from scratch, you can use OpenLayers for the mapping part, which is maintained and developed by a bunch of brilliant people For example, if you wanted to write a blog you could either write your own blog engine, or use an existing one such as WordPress or Blogger and build on top of it. Similarly, if you wanted to create a web map, you could write your own from scratch, or use so beware that has been developed and tested by a group of developers with a strong community behind it. By choosing to use OpenLayers, you do have to learn how to use the library, but the benefits greatly outweigh the costs. You get to use a rich, highly tested and maintained code base, and all you have to do is learn how to use it. OpenLayers is written in JavaScript, but don't fret if you don't know it very well. All you really need is some knowledge of the basic syntax, and we'll try to keep things as clear as possible in the code examples. If you are unfamiliar with JavaScript, Mozilla provides phenomenal JavaScript documentation at https://developer.mozilla. org/en/javascript. Anatomy of a web-mapping application First off —what is a 'web-mapping application. To put it bluntly, it's some type of Internet application that makes use of a map. This could be a site that displays the latest geo-tagged images from Flickr , a map that shows markers of locations you've traveled to, or an application that tracks invasive plant species and displays them. If it contains a map and it does something, you could argue that it is a web map application. The term can be used in a pretty broad sense. So where exactly does OpenLayers fit in? We know OpenLayers is a client side mapping library, but what does that mean? Let's take a look at the following screenshot: This is called the Client / Server Model and it is, essentially, the core of how all web applications operate. In the case of a web map application, some sort of map client (e.g., OpenLayers) communicates with some sort of web map server (e.g., a WMS server or the Google Maps backend). Web map client OpenLayers lives on the client side. One of the primary tasks the client performs is to get map images from a map server. Essentially, the client has to ask a map server for what you want to look at. Every time you navigate or zoom around on the map, the client has to make new requests to the server—because you're asking to look at something different. OpenLayers handles this all for you, and it is happening via asynchronous JavaScript (AJAX) calls to a map server. To reiterate—the basic concept is that OpenLayers sends requests to a map server for map images every time you interact with the map, then OpenLayers pieces together all the returned map images so it looks like one big, seamless map. Web map server A map server (or map service) provides the map itself. There are a myriad of different map server backends. A small sample includes WMS, Google Maps, Yahoo! Maps, ESRI ArcGIS, WFS, and OpenStreet Maps. If you are unfamiliar with those terms, don't sweat it. The basic principle behind all those services is that they allow you to specify the area of the map you want to look at (by sending a request), and then the map servers send back a response containing the map image. With OpenLayers, you can choose to use as many different backends in any sort of combination as you'd like. OpenLayers is not a web map server; it only consumes data from them. So, you will need to be able to access some type of web map service. Don't worry though. Fortunately, there are a myriad of free and/or open source web map servers available that are remotely hosted or easy to set up yourself, such as MapServer. Throughout this article, we'll often use a freely available web mapping service from OSGeo, so don't worry about having to provide your own. With many web map servers you do not have to do anything to use them—just supplying a URL to them in OpenLayers is enough. OSGeo, OpenStreet Maps, Google, Yahoo!, and Bing Maps, for instance, provide access to their map servers (although, some commercial restrictions may apply with various services in some situations). Relation to Google / Yahoo! / and other mapping APIs The Google, Yahoo!, Bing, and ESRI Mappings API allow you to connect with their map server backend. Their APIs also usually provide a client side interface (at least in the case of Google Maps). The Google Maps API , for instance, is fairly powerful. You have the ability to add markers, plot routes, and use KML data (things you can also do in OpenLayers)—but the main drawback is that your mapping application relies totally on Google. The map client and map server are provided by a third party. This is not inherently a bad thing, and for many projects, Google Maps and the like are a good fit However, there are quite a few drawbacks: You're not in control of the backend You can't really customize the map server backend, and it can change at any time There may be some commercial restrictions, or some costs involved These other APIs also cannot provide you with anything near the amount of flexibility and customization that an open source mapping application framework (i.e., OpenLayers) offers. Layers in OpenLayers So, what's with the Layer in OpenLayers? Well, OpenLayers allows you to have multiple different 'backend' servers that your map can use. To access a web map server, you create a layer object and add it to your map with OpenLayers. For instance, if you wanted to have a Google Maps and a WMS service displayed on your map, you would use OpenLayers to create a GoogleMaps layer object and a WMS layer object, and then add them to your OpenLayers map. We'll soon see an example with a WMS layer, so don't worry if you're a little confused. What is a Layer? Like layers of an onion, each layer is above and will cover up the previous one; the order that you add in the layers is important. With OpenLayers, you can arbitrarily set the overall transparency of any layer, so you are easily able to control how much layers cover each other up, and dynamically change the layer order at any time. For instance, you could have a Google map as your base layer, a layer with satellite imagery that is semi-transparent, and a vector layer all active on your map at once. A vector layer is a powerful layer that lets us add markers and various geometric objects to our maps. Thus, in this example, your map would have three separate layers. The OpenLayers website The website for OpenLayers is located at http://openlayers.org/. To begin, we need to download a copy of OpenLayers (or, we can directly link to the library—but we'll download a local copy). You can download the compressed library as either a .tar.gz or .zip, but both contain the same files. Let's go over the links: Link to the hosted version: If you do not want to actually download OpenLayers, you can instead link to the OpenLayers library by adding this script URL to your site in a <script> tag. 2.10 (Stable) .tar.gz or .zip: This should show the latest stable release (2.10 at the time of writing). You can download it as either a tar.gz or .zip; if you are unsure of which to get, you should download the .zip version. 2.10 Release Notes: This highlights things that have changed, bugs that have been fixed, etc. Class documentation on, more documentation on: These are links to the API documentation, which we will make heavy use of throughout the article. I recommend opening it up and keeping it up while working through the examples.
Read more
  • 0
  • 0
  • 3492

article-image-using-osgi-services
Packt
26 Aug 2014
14 min read
Save for later

Using OSGi Services

Packt
26 Aug 2014
14 min read
This article created by Dr Alex Blewitt the author of Mastering Eclipse Plug-in Development will present OSGi services as a means to communicate with and connect applications. Unlike the Eclipse extension point mechanism, OSGi services can have multiple versions available at runtime and can work in other OSGi environments, such as Felix or other commercial OSGi runtimes. (For more resources related to this topic, see here.) Overview of services In an Eclipse or OSGi runtime, each individual bundle is its own separate module, which has explicit dependencies on library code via Import-Package, Require-Bundle, or Require-Capability. These express static relationships and provide a way of configuring the bundle's classpath. However, this presents a problem. If services are independent, how can they use contributions provided by other bundles? In Eclipse's case, the extension registry provides a means for code to look up providers. In a standalone OSGi environment, OSGi services provide a similar mechanism. A service is an instance of a class that implements a service interface. When a service is created, it is registered with the services framework under one (or more) interfaces, along with a set of properties. Consumers can then get the service by asking the framework for implementers of that specific interface. Services can also be registered under an abstract class, but this is not recommended. Providing a service interface exposed as an abstract class can lead to unnecessary coupling of client to implementation. The following diagram gives an overview of services: This separation allows the consumer and producer to depend on a common API bundle, but otherwise be completely decoupled from one another. This allows both the consumer and producer to be mocked out or exchange with different implementations in the future. Registering a service programmatically To register a service, an instance of the implementation class needs to be created and registered with the framework. Interactions with the framework are performed with an instance of BundleContext—typically provided in the BundleActivator.start method and stored for later use. The *FeedParser classes will be extended to support registration as a service instead of the Equinox extension registry. Creating an activator A bundle's activator is a class that is instantiated and coupled to the lifetime of the bundle. When a bundle is started, if a manifest entry Bundle-Activator exists, then the corresponding class is instantiated. As long as it implements the BundleActivator interface, the start method will be called. This method is passed as an instance of BundleContext, which is the bundle's connection to the hosting OSGi framework. Create a class in the com.packtpub.e4.advanced.feeds project called com.packtpub.e4.advanced.feeds.internal.FeedsActivator, which implements the org.osgi.framework.BundleActivator interface. The quick fix may suggest adding org.osgi.framework as an imported package. Accept this, and modify the META-INF/MANIFEST.MF file as follows: Import-Package: org.osgi.framework Bundle-Activator: com.packtpub.e4.advanced.feeds.internal.FeedsActivator The framework will automatically invoke the start method of the FeedsActivator when the bundle is started, and correspondingly, the stop method when the bundle is stopped. Test this by inserting a pair of println calls: public class FeedsActivator implements BundleActivator { public void start(BundleContext context) throws Exception { System.out.println("Bundle started"); } public void stop(BundleContext context) throws Exception { System.out.println("Bundle stopped"); } } Now run the project as an OSGi framework with the feeds bundle, the Equinox console, and the Gogo shell. The required dependencies can be added by clicking on Add Required Bundles, although the Include optional dependencies checkbox does not need to be selected. Ensure that the other workspace and target bundles are deselected with the Deselect all button, as shown in the following screenshot: The required bundles are as follows: com.packtpub.e4.advanced.feeds org.apache.felix.gogo.command org.apache.felix.gogo.runtime org.apache.felix.gogo.shell org.eclipse.equinox.console org.eclipse.osgi On the console, when the bundle is started (which happens automatically if the Default Auto-Start is set to true), the Bundle started message should be seen. If the bundle does not start, ss in the console will print a list of bundles and start 2 will start the bundle with the ID 2. Afterwards, stop 2 can be used to stop bundle 2. Bundles can be stopped/started dynamically in an OSGi framework. Registering the service Once the FeedsActivator instance is created, a BundleContext instance will be available for interaction with the framework. This can be persisted for subsequent use in an instance field and can also be used directly to register a service. The BundleContext class provides a registerService method, which takes an interface, an instance, and an optional Dictionary instance of key/value pairs. This can be used to register instances of the feed parser at runtime. Modify the start method as follows: public void start(BundleContext context) throws Exception { context.registerService(IFeedParser.class, new RSSFeedParser(), null); context.registerService(IFeedParser.class, new AtomFeedParser(), null); context.registerService(IFeedParser.class, new MockFeedParser(), null); } Now start the framework again. In the console that is launched, look for the bundle corresponding to the feeds bundle: osgi> bundles | grep feeds com.packtpub.e4.advanced.feeds_1.0.0.qualifier [4] {com.packtpub.e4.advanced.feeds.IFeedParser}={service.id=56} {com.packtpub.e4.advanced.feeds.IFeedParser}={service.id=57} {com.packtpub.e4.advanced.feeds.IFeedParser}={service.id=58} This shows that bundle 4 has started three services, using the interface com.packtpub.e4.advanced.feeds.IFeedParser, and with service IDs 56, 57, and 58. It is also possible to query the runtime framework for services of a known interface type directly using the services command and an LDAP style filter: osgi> services (objectClass=com.packtpub.e4.advanced.feeds.IFeedParser) {com.packtpub.e4.advanced.feeds.IFeedParser}={service.id=56} "Registered by bundle:" com.packtpub.e4.advanced.feeds_1.0.0.qualifier [4] "No bundles using service." {com.packtpub.e4.advanced.feeds.IFeedParser}={service.id=57} "Registered by bundle:" com.packtpub.e4.advanced.feeds_1.0.0.qualifier [4] "No bundles using service." {com.packtpub.e4.advanced.feeds.IFeedParser}={service.id=58} "Registered by bundle:" com.packtpub.e4.advanced.feeds_1.0.0.qualifier [4] "No bundles using service." The results displayed represent the three services instantiated. They can be introspected using the service command passing the service.id: osgi> service 56 com.packtpub.e4.advanced.feeds.internal.RSSFeedParser@52ba638e osgi> service 57 com.packtpub.e4.advanced.feeds.internal.AtomFeedParser@3e64c3a osgi> service 58 com.packtpub.e4.advanced.feeds.internal.MockFeedParser@49d5e6da Priority of services Services have an implicit order, based on the order in which they were instantiated. Each time a service is registered, a global service.id is incremented. It is possible to define an explicit service ranking with an integer property. This is used to ensure relative priority between services, regardless of the order in which they are registered. For services with equal service.ranking values, the service.id values are compared. OSGi R6 adds an additional property, service.bundleid, which is used to denote the ID of the bundle that provides the service. This is not used to order services, and is for informational purposes only. Eclipse Luna uses OSGi R6. To pass a priority into the service registration, create a helper method called priority, which takes an int value and stores it in a Hashtable with the key service.ranking. This can be used to pass a priority to the service registration methods. The following code illustrates this: private Dictionary<String,Object> priority(int priority) { Hashtable<String, Object> dict = new Hashtable<String,Object>(); dict.put("service.ranking", new Integer(priority)); return dict; } public void start(BundleContext context) throws Exception { context.registerService(IFeedParser.class, new RSSFeedParser(), priority(1)); context.registerService(IFeedParser.class, new MockFeedParser(), priority(-1)); context.registerService(IFeedParser.class, new AtomFeedParser(), priority(2)); } Now when the framework starts, the services are displayed in order of priority: osgi> services | (objectClass=com.packtpub.e4.advanced.feeds.IFeedParser) {com.packtpub.e4.advanced.feeds.IFeedParser}={service.ranking=2, service.id=58} "Registered by bundle:" com.packtpub.e4.advanced.feeds_1.0.0.qualifier [4] "No bundles using service." {com.packtpub.e4.advanced.feeds.IFeedParser}={service.ranking=1, service.id=56} "Registered by bundle:" com.packtpub.e4.advanced.feeds_1.0.0.qualifier [4] "No bundles using service." {com.packtpub.e4.advanced.feeds.IFeedParser}={service.ranking=-1, service.id=57} "Registered by bundle:" com.packtpub.e4.advanced.feeds_1.0.0.qualifier [4] "No bundles using service." Dictionary was the original Java Map interface, and Hashtable the original HashMap implementation. They fell out of favor in Java 1.2 when Map and HashMap were introduced (mainly because they weren't synchronized by default) but OSGi was developed to run on early releases of Java (JSR 8 proposed adding OSGi as a standard for the Java platform). Not only that, early low-powered Java mobile devices didn't support the full Java platform, instead exposing the original Java 1.1 data structures. Because of this history, many APIs in OSGi refer to only Java 1.1 data structures so that low-powered devices can still run OSGi systems. Using the services The BundleContext instance can be used to acquire services as well as register them. FeedParserFactory, which originally used the extension registry, can be upgraded to refer to services instead. To obtain an instance of BundleContext, store it in the FeedsActivator.start method as a static variable. That way, classes elsewhere in the bundle will be able to acquire the context. An accessor method provides an easy way to do this: public class FeedsActivator implements BundleActivator { private static BundleContext bundleContext; public static BundleContext getContext() { return bundleContext; } public void start(BundleContext context) throws Exception { // register methods as before bundleContext = context; } public void stop(BundleContext context) throws Exception { bundleContext = null; } } Now the FeedParserFactory class can be updated to acquire the services. OSGi services are represented via a ServiceReference instance (which is a sharable object representing a handle to the service) and can be used to acquire a service instance: public class FeedParserFactory { public List<IFeedParser> getFeedParsers() { List<IFeedParser> parsers = new ArrayList<IFeedParser>(); BundleContext context = FeedsActivator.getContext(); try { Collection<ServiceReference<IFeedParser>> references = context.getServiceReferences(IFeedParser.class, null); for (ServiceReference<IFeedParser> reference : references) { parsers.add(context.getService(reference)); context.ungetService(reference); } } catch (InvalidSyntaxException e) { // ignore } return parsers; } } In this case, the service references are obtained from the bundle context with a call to context.getServiceReferences(IFeedParser.class,null). The service references can be used to access the service's properties, and to acquire the service. The service instance is acquired with the context.getService(ServiceReference) call. The contract is that the caller "borrows" the service, and when finished, should return it with an ungetService(ServiceReference) call. Technically, the service is only supposed to be used between the getService and ungetService calls as its lifetime may be invalid afterwards; instead of returning an array of service references, the common pattern is to pass in a unit of work that accepts the service and then call ungetService afterwards. However, to fit in with the existing API, the service is acquired, added to the list, and then released immediately afterwards. Lazy activation of bundles Now run the project as an Eclipse application, with the feeds and feeds.ui bundles installed. When a new feed is created by navigating to File | New | Other | Feeds | Feed, and a feed such as http://alblue.bandlem.com/atom.xml is entered, the feeds will be shown in the navigator view. When drilling down, a NullPointerException may be seen in the logs, as shown in the following: !MESSAGE An exception occurred invoking extension: com.packtpub.e4.advanced.feeds.ui.feedNavigatorContent for object com.packtpub.e4.advanced.feeds.Feed@770def59 !STACK 0 java.lang.NullPointerException at com.packtpub.e4.advanced.feeds.FeedParserFactory. getFeedParsers(FeedParserFactory.java:31) at com.packtpub.e4.advanced.feeds.ui.FeedContentProvider. getChildren(FeedContentProvider.java:80) at org.eclipse.ui.internal.navigator.extensions. SafeDelegateTreeContentProvider. getChildren(SafeDelegateTreeContentProvider.java:96) Tracing through the code indicates that the bundleContext is null, which implies that the feeds bundle has not yet been started. This can be seen in the console of the running Eclipse application by executing the following code: osgi> ss | grep feeds 866 ACTIVE com.packtpub.e4.advanced.feeds.ui_1.0.0.qualifier 992 RESOLVED com.packtpub.e4.advanced.feeds_1.0.0.qualifier While the feeds.ui bundle is active, the feeds bundle is not. Therefore, the services haven't been instantiated, and bundleContext has not been cached. By default, bundles are not started when they are accessed for the first time. If the bundle needs its activator to be called prior to using any of the classes in the package, it needs to be marked as having an activation policy of lazy. This is done by adding the following entry to the MANIFEST.MF file: Bundle-ActivationPolicy: lazy The manifest editor can be used to add this configuration line by selecting Activate this plug-in when one of its classes is loaded, as shown in the following screenshot: Now, when the application is run, the feeds will resolve appropriately. Comparison of services and extension points Both mechanisms (using the extension registry and using the services) allow for a list of feed parsers to be contributed and used by the application. What are the differences between them, and are there any advantages to one or the other? Both the registry and services approaches can be used outside of an Eclipse runtime. They work the same way when used in other OSGi implementations (such as Felix) and can be used interchangeably. The registry approach can also be used outside of OSGi, although that is far less common. The registry encodes its information in the plugin.xml file by default, which means that it is typically edited as part of a bundle's install (it is possible to create registry entries from alternative implementations if desired, but this rarely happens). The registry has a notification system, which can listen to contributions being added and removed. The services approach uses the OSGi framework to store and maintain a list of services. These services don't have an explicit configuration file and, in fact, can be contributed by code (such as the registerService calls) or by declarative representations. The separation of how the service is created versus how the service is registered is a key difference between the service and the registry approach. Like the registry, the OSGi services system can generate notifications when services come and go. One key difference in an OSGi runtime is that bundles depending on the Eclipse registry must be declared as singletons; that is, they have to use the ;singleton:=true directive on Bundle-SymbolicName. This means that there can only be one version of a bundle that exposes registry entries in a runtime, as opposed to multiple versions in the case of general services. While the registry does provide mechanisms to be able to instantiate extensions from factories, these typically involve simple configurations and/or properties that are hard-coded in the plugin.xml files themselves. They would not be appropriate to store sensitive details such as passwords. On the other hand, a service can be instantiated from whatever external configuration information is necessary and then registered, such as a JDBC connection for a database. Finally, extensions in the registry are declarative by default and are activated on demand. This allows Eclipse to start quickly because it does not need to build the full set of class loader objects or run code, and then bring up services on demand. Although the approach previously didn't use declarative services, it is possible to do this. Summary This article introduced OSGi services as a means to extend an application's functionality. It also shed light on how to register a service programmatically. Resources for Article: Further resources on this subject: Apache Maven and m2eclipse [article] Introducing an Android platform [article] Installing and Setting up JavaFX for NetBeans and Eclipse IDE [article]
Read more
  • 0
  • 0
  • 3488
article-image-human-interactions-bpel
Packt
08 Sep 2010
13 min read
Save for later

Human Interactions in BPEL

Packt
08 Sep 2010
13 min read
(For more resources on BPEL, SOA and Oracle see here.) Human interactions in business processes The main objective of BPEL has been to standardize the process automation. BPEL business processes make use of services and externalize their functionality as services. BPEL processes are defined as a collection of activities through which services are invoked. BPEL does not make a distinction between services provided by applications and other interactions, such as human interactions, which are particularly important. Real-world business processes namely often integrate not only systems and services, but also humans. Human interactions in business processes can be very simple, such as approval of certain tasks or decisions, or complex, such as delegation, renewal, escalation, nomination, chained execution, and so on. Human interactions are not limited to approvals and can include data entries, process monitoring and management, process initiation, exception handling, and so on. Task approval is the simplest and probably the most common human interaction. In a business process for opening a new account, a human interaction might be required to decide whether the user is allowed to open the account. In a travel approval process, a human might approve the decision from which airline to buy the ticket (as shown in the following figure). If the situation is more complex, a business process might require several users to make approvals, either in sequence or in parallel. In sequential scenarios, the next user often wants to see the decision made by the previous user. Sometimes, particularly in parallel human interactions, users are not allowed to see the decisions taken by other users. This improves the decision potential. Sometimes one user does not even know which other users are involved, or whether any other users are involved at all. A common scenario for involving more than one user is workflow with escalation. Escalation is typically used in situations where an activity does not fulfill a time constraint. In such a case, a notification is sent to one or more users. Escalations can be chained, going first to the first-line employees and advancing to senior staff if the activity is not fulfilled. Sometimes it is difficult or impossible to define in advance which user should perform an interaction. In this case, a supervisor might manually nominate the task to other employees; the nomination can also be made by a group of users or by a decision-support system. In other scenarios, a business process may require a single user to perform several steps that can be defined in advance or during the execution of the process instance. Even more complex processes might require that one workflow is continued with another workflow. Human interactions are not limited to only approvals; they may also include data entries or process management issues, such as process initiation, suspension, and exception management. This is particularly true for long-running business processes, where, for example, user exception handling can prevent costly process termination and related compensation for those activities that have already been successfully completed. As a best practice for human workflows, it is usually not wise to associate human interactions directly to specific users; it is better to connect tasks to roles and then associate those roles with individual users. This gives business processes greater flexibility, allowing any user with a certain role to interact with the process and enabling changes to users and roles to be made dynamically. To achieve this, the process has to gain access to users and roles, stored in the enterprise directory, such as LDAP (Lightweight Directory Access Protocol). Workflow theory has defined several workflow patterns, which specify the abovedescribed scenarios in detail. Examples of workflow patterns include sequential workflow, parallel workflow, workflow with escalation, workflow with nomination, ad-hoc workflow, workflow continuation, and so on. Human Tasks in BPEL So far we have seen that human interaction in business processes can get quite complex. Although BPEL specification does not specifically cover human interactions, BPEL is appropriate for human workflows. BPEL business processes are defined as collections of activities that invoke services. BPEL does not make a distinction between services provided by applications and other interactions, such as human interactions. There are mainly two approaches to support human interactions in BPEL. The first approach is to use a human workflow service. Several vendors today have created workflow services that leverage the rich BPEL support for asynchronous services. In this fashion, people and manual tasks become just another asynchronous service from the perspective of the orchestrating process and the BPEL processes stay 100% standard. The other approach has been to standardize the human interactions and go beyond the service invocations. This approach resulted in the workflow specifications emerging around BPEL with the objective to standardize the explicit inclusion of human tasks in BPEL processes. The BPEL4People specification has emerged, which was originally put forth by IBM and SAP in July 2005. Other companies, such as Oracle, Active Endpoints, and Adobe joined later. Finally, this specification is now being advanced within the OASIS BPEL4People Technical Committee. The BPEL4People specification contains two parts: BPEL4People version 1.0, which introduces BPEL extensions to address human interactions in BPEL as a first-class citizen. It defines a new type of basic activity, which uses human tasks as an implementation, and allows specifying tasks local to a process or use tasks defined outside of the process definition. BPEL4People is based on the WS-HumanTask specification that it uses for the actual specification of human tasks. Web Services Human Task (WS-HumanTask) version 1.0 introduces the definition of human tasks, including their properties, behavior, and a set of operations used to manipulate human tasks. It also introduces a coordination protocol in order to control autonomy and lifecycle of service-enabled human tasks in an interoperable manner. The most important extensions introduced in BPEL4People are people activities and people links. People activity is a new BPEL activity used to define user interactions; in other words, tasks that a user has to perform. For each people activity, the BPEL server must create work items and distribute them to users eligible to execute them. People activities can have input and output variables and can specify deadlines. To specify the implementation of people activities, BPEL4People introduced tasks. Tasks specify actions that users must perform. Tasks can have descriptions, priorities, deadlines, and other properties. To represent tasks to users, we need a client application that provides a user interface and interacts with tasks. It can query available tasks, claim and revoke them, and complete or fail them. To associate people activities and the related tasks with users or groups of users, BPEL4People introduced people links. People links are somewhat similar to partner links; they associate users with one or more people activities. People links are usually associated with generic human roles, such as process initiator, process stakeholders, owners, and administrators. The actual users that are associated with people activities can be determined at design time, deployment time, or runtime. BPEL4People anticipates the use of directories such as LDAP to select users. However, it doesn't define the query language used to select users. Rather, it foresees the use of LDAP filters, SQL, XQuery, or other methods. BPEL4People proposes complex extensions to the BPEL specification. However, so far it is still quite high level and doesn't yet specify the exact syntax of the new activities mentioned above. Until the specification becomes more concrete, we don't expect vendors to implement the proposed extensions. But while BPEL4People is early in the standardization process, it shows a great deal of promise. The BPEL4People proposal raises an important question: Is it necessary to introduce such complex extensions to BPEL to cover user interactions? Some vendor solutions model user interactions as just another web service, with well-defined interfaces for both BPEL processes and client applications. This approach does not require any changes to BPEL. To become portable, it would only need an industry-wide agreement on the two interfaces. And, of course, both interfaces can be specified with WSDL, which gives developers great flexibility and lets them use practically any environment, language, or platform that supports Web Services. Clearly, a single standard approach has not yet been adopted for extending BPEL to include Human Tasks and workflow services. However, this does not mean that developers cannot use BPEL to develop business processes with user interactions. Human Task integration with BPEL To interleave user interactions with service invocations in BPEL processes we can use a workflow service, which interacts with BPEL using standard WSDL interfaces. This way, the BPEL process can assign user tasks and wait for responses by invoking the workflow service using the same syntax as for any other service. The BPEL process can also perform more complex operations such as updating, completing, renewing, routing, and escalating tasks. After the BPEL process has assigned tasks to users, users can act on the tasks by using the appropriate applications. The applications communicate with the workflow service by using WSDL interfaces or another API (such as Java) to acquire the list of tasks for selected users, render appropriate user interfaces, and return results to the workflow service, which forwards them to the BPEL process. User applications can also perform other tasks such as reassign, escalate, route, suspend, resume, and withdraw. Finally, the workflow service may allow other communication channels, such as e-mail and SMS, as shown in the following figure: Oracle Human Workflow concepts Oracle SOA Suite 11g provides the Human Workflow component, which enables including human interaction in BPEL processes in a relatively easy way. The Human Workflow component consists of different services that handle various aspects of human interaction with business process and expose their interfaces through WSDL; therefore, BPEL processes invoke them just like any other service. The following figure shows the overall architecture of the Oracle Workflow services: As we can see in the previous figure, the Workflow consists of the following services: Task Service exposes operations for task state management, such as operations to update a task, complete a task, escalate a task, reassign a task, and so on. When we add a human task to the BPEL process, the corresponding partner link for the Task Service is automatically created. Task Assignment Service provides functionality to route, escalate, reassign tasks, and more. Task Query Service enables retrieving the task list for a user based on a search criterion. Task Metadata Service enables retrieving the task metadata. Identity Service provides authentication and authorization of users and lookup of user properties and privileges. Notification Service enables sending of notifications to users using various channels (e-mail, voice message, IM, SMS, and so on). User Metadata Service manages metadata, related to workflow users, such as user work queues, preferences, and so on. Runtime Configuration Service provides functionality for managing metadata used in the task service runtime environment. Evidence Store Service supports management of digitally-signed workflow tasks. BPEL processes use the Task Service to assign tasks to users. More specifically, tasks can be assigned to: Users: Users are defined in an identity store configured with the SOA infrastructure. Groups: Groups contain individual users, which can claim a task and act upon it. Application roles: Used to logically group users and other roles. These roles are application specific and are not stored in the identity store. Assigning tasks to groups or roles is more flexible, as every user in a certain group (role) can review the task to complete it. Oracle SOA Suite 11g provides three methods for assigning users, groups, and application roles to tasks: Static assignment: Static users, groups, or application roles can be assigned at design time. Dynamic assignment: We can define an XPath expression to determine the task participant at runtime. Rule-based assignment: We can create a list of participants with complex expressions. Once the user has completed the task, the BPEL process receives a callback from the Task Service with the result of the user action. The BPEL process continues to execute. The Oracle Workflow component provides several possibilities regarding how users can review the tasks that have been assigned to them, and take the corresponding actions. The most straightforward approach is to use the Oracle BPM Worklist application. This application comes with Oracle SOA Suite 11g and allows users to review the tasks, to see the task details, and to select the decision to complete the task. If the Oracle BPM Worklist application is not appropriate, we can develop our own user interface in Java (using JSP, JSF, Swing, and so on) or almost any other environment that supports Web Services (such as .NET for example). In this respect, the Workflow service is very flexible and we can use a portal, such as Oracle Portal, a web application, or almost any other application to review the tasks. The third possibility is to use e-mail for task reviews. We use e-mails over the Notification service. Workflow patterns To simplify the development of workflows, Oracle SOA Suite 11g provides a library of workflow patterns (participant types). Workflow patterns define typical scenarios of human interactions with BPEL processes. The following participant types are supported: Single approver: Used when a participant maps to a user, group, or role. Parallel: Used if multiple users have to act in parallel (for example, if multiple users have to provide their opinion or vote). The percentage of required user responses can be specified. Serial: Used if multiple users have to act in a sequence. A management chain or a list of users can be specified. FYI (For Your Information): Used if a user only needs to be notified about a task, but a user response is not required. With these, we can realize various workflow patterns, such as: Simple workflow: Used if a single user action is required, such as confirmation, decision, and so on. A timeout can also be specified. Simple workflow has two extension patterns: Escalation: Provides the ability to escalate the task to another user or role if the original user does not complete the task in the specified amount of time. Renewal: Provides the ability to extend the timeout if the user does not complete the task in the specified time. Sequential workflow: Used if multiple users have to act in a sequence. A management chain or a list of users can be specified. Sequential workflow has one extension pattern: Escalation: Same functionality as above. Parallel workflow: Used if multiple users have to act in parallel (for example, if multiple users have to provide their opinion or vote). The percentage of required user responses can be specified. This pattern has an extension pattern: Final reviewer: Is used when the final review has to act after parallel users have provided feedback. Ad-hoc (dynamic) workflow: Used to assign the task to one user, who can then route the task to other user. The task is completed when the user does not route it forward. FYI workflow: Used if a user only needs to be notified about a task, but a user response is not required. Task continuation: Used to build complex workflow patterns as a chain of simple patterns (those described above).
Read more
  • 0
  • 0
  • 3481

article-image-python-testing-coverage-analysis
Packt
01 Jun 2011
13 min read
Save for later

Python Testing: Coverage Analysis

Packt
01 Jun 2011
13 min read
Python Testing Cookbook Over 70 simple but incredibly effective recipes for taking control of automated testing using powerful Python testing tools Introduction A coverage analyzer can be used while running a system in production, but what are the pros and cons, if we used it this way? What about using a coverage analyzer when running test suites? What benefits would this approach provide compared to checking systems in production? Coverage helps us to see if we are adequately testing our system. But it must be performed with a certain amount of skepticism. This is because, even if we achieve 100 percent coverage, meaning every line of our system was exercised, in no way does this guarantee us having no bugs. A quick example involves a code we write and what it processes is the return value from a system call. What if there are three possible values, but we only handle two of them? We may write two test cases covering our handling of it, and this could certainly achieve 100 percent statement coverage. However, it doesn't mean we have handled the third possible return value; thus, leaving us with a potentially undiscovered bug. 100 percent code coverage can also be obtained by condition coverage but may not be achieved with statement coverage. The kind of coverage we are planning to target should be clear. Another key point is that not all testing is aimed at bug fixing. Another key purpose is to make sure that the application meets our customer's needs. This means that, even if we have 100 percent code coverage, we can't guarantee that we are covering all the scenarios expected by our users. This is the difference between 'building it right' and 'building the right thing'. In this article, we will explore various recipes to build a network management application, run coverage tools, and harvest the results. We will discuss how coverage can introduce noise, and show us more than we need to know, as well as introduce performance issues when it instruments our code. We will also see how to trim out information we don't need to get a concise, targeted view of things. This article uses several third-party tools in many recipes. Spring Python (http://springpython.webfactional.com) contains many useful abstractions. The one used in this article is its DatabaseTemplate, which offers easy ways to write SQL queries and updates without having to deal with Python's verbose API. Install it by typing pip install springpython. Install the coverage tool by typing pip install coverage. This may fail because other plugins may install an older version of coverage. If so, uninstall coverage by typing pip uninstall coverage, and then install it again with pip install coverage. Nose is a useful test runner.   Building a network management application For this article, we will build a very simple network management application, and then write different types of tests and check their coverage. This network management application is focused on digesting alarms, also referred to as network events. This is different from certain other network management tools that focus on gathering SNMP alarms from devices. For reasons of simplicity, this correlation engine doesn't contain complex rules, but instead contains simple mapping of network events onto equipment and customer service inventory. We'll explore this in the next few paragraphs as we dig through the code. How to do it... With the following steps, we will build a simple network management application. Create a file called network.py to store the network application. Create a class definition to represent a network event. class Event(object): def __init__(self, hostname, condition, severity, event_time): self.hostname = hostname self.condition = condition self.severity = severity self.id = -1 def __str__(self): return "(ID:%s) %s:%s - %s" % (self.id, self.hostname, self.condition, self.severity) hostname: It is assumed that all network alarms originate from pieces of equipment that have a hostname. condition: Indicates the type of alarm being generated. Two different alarming conditions can come from the same device. severity: 1 indicates a clear, green status; and 5 indicates a faulty, red status. id: The primary key value used when the event is stored in a database. Create a new file called network.sql to contain the SQL code. Create a SQL script that sets up the database and adds the definition for storing network events. CREATE TABLE EVENTS ( ID INTEGER PRIMARY KEY, HOST_NAME TEXT, SEVERITY INTEGER, EVENT_CONDITION TEXT ); Code a high-level algorithm where events are assessed for impact to equipment and customer services and add it to network.py. from springpython.database.core import* class EventCorrelator(object): def __init__(self, factory): self.dt = DatabaseTemplate(factory) def __del__(self): del(self.dt) def process(self, event): stored_event, is_active = self.store_event(event) affected_services, affected_equip = self.impact(event) updated_services = [ self.update_service(service, event) for service in affected_services] updated_equipment = [ self.update_equipment(equip, event) for equip in affected_equip] return (stored_event, is_active, updated_services, updated_equipment) The __init__ method contains some setup code to create a DatabaseTemplate. This is a Spring Python utility class used for database operations. See http://static.springsource.org/spring- python/1.2.x/sphinx/html/dao.html for more details. We are also using sqlite3 as our database engine, since it is a standard part of Python. The process method contains some simple steps to process an incoming event. We first need to store the event in the EVENTS table. This includes evaluating whether or not it is an active event, meaning that it is actively impacting a piece of equipment. Then we determine what equipment and what services the event impacts. Next, we update the affected services by determining whether it causes any service outages or restorations. Then we update the affected equipment by determining whether it fails or clears a device. Finally, we return a tuple containing all the affected assets to support any screen interfaces that could be developed on top of this. Implement the store_event algorithm. def store_event(self, event): try: max_id = self.dt.query_for_int("""select max(ID) from EVENTS""") except DataAccessException, e: max_id = 0 event.id = max_id+1 self.dt.update("""insert into EVENTS (ID, HOST_NAME, SEVERITY, EVENT_CONDITION) values (?,?,?,?)""", (event.id, event.hostname, event.severity, event.condition)) is_active = self.add_or_remove_from_active_events(event) return (event, is_active) This method stores every event that is processed. This supports many things including data mining and post mortem analysis of outages. It is also the authoritative place where other event-related data can point back using a foreign key. The store_event method looks up the maximum primary key value from the EVENTS table. It increments it by one. It assigns it to event.id. It then inserts it into the EVENTS table. Next, it calls a method to evaluate whether or not the event should be add to the list of active events, or if it clears out existing active events. Active events are events that are actively causing a piece of equipment to be unclear. Finally, it returns a tuple containing the event and whether or not it was classified as an active event. For a more sophisticated system, some sort of partitioning solution needs to be implemented. Querying against a table containing millions of rows is very inefficient. However, this is for demonstration purposes only, so we will skip scaling as well as performance and security. Implement the method to evaluate whether to add or remove active events. def add_or_remove_from_active_events(self, event): """Active events are current ones that cause equipment and/or services to be down.""" if event.severity == 1: self.dt.update("""delete from ACTIVE_EVENTS where EVENT_FK in ( select ID from EVENTS where HOST_NAME = ? and EVENT_CONDITION = ?)""", (event.hostname,event.condition)) return False else: self.dt.execute("""insert into ACTIVE_EVENTS (EVENT_FK) values (?)""", (event.id,)) return True When a device fails, it sends a severity 5 event. This is an active event and in this method, a row is inserted into the ACTIVE_EVENTS table, with a foreign key pointing back to the EVENTS table. Then we return back True, indicating this is an active event. Add the table definition for ACTIVE_EVENTS to the SQL script. CREATE TABLE ACTIVE_EVENTS ( ID INTEGER PRIMARY KEY, EVENT_FK, FOREIGN KEY(EVENT_FK) REFERENCES EVENTS(ID) ); This table makes it easy to query what events are currently causing equipment failures. Later, when the failing condition on the device clears, it sends a severity 1 event. This means that severity 1 events are never active, since they aren't contributing to a piece of equipment being down. In our previous method, we search for any active events that have the same hostname and condition, and delete them. Then we return False, indicating this is not an active event. Write the method that evaluates the services and pieces of equipment that are affected by the network event. def impact(self, event): """Look up this event has impact on either equipment or services.""" affected_equipment = self.dt.query( """select * from EQUIPMENT where HOST_NAME = ?""", (event.hostname,), rowhandler=DictionaryRowMapper()) affected_services = self.dt.query( """select SERVICE.* from SERVICE join SERVICE_MAPPING SM on (SERVICE.ID = SM.SERVICE_FK) join EQUIPMENT on (SM.EQUIPMENT_FK = EQUIPMENT.ID where EQUIPMENT.HOST_NAME = ?""", (event.hostname,), rowhandler=DictionaryRowMapper()) return (affected_services, affected_equipment) We first query the EQUIPMENT table to see if event.hostname matches anything. Next, we join the SERVICE table to the EQUIPMENT table through a many-to many relationship tracked by the SERVICE_MAPPING table. Any service that is related to the equipment that the event was reported on is captured. Finally, we return a tuple containing both the list of equipment and list of services that are potentially impacted. Spring Python provides a convenient query operation that returns a list of objects mapped to every row of the query. It also provides an out-of-the-box DictionaryRowMapper that converts each row into a Python dictionary, with the keys matching the column names. Add the table definitions to the SQL script for EQUIPMENT, SERVICE, and SERVICE_MAPPING. CREATE TABLE EQUIPMENT ( ID INTEGER PRIMARY KEY, HOST_NAME TEXT UNIQUE, STATUS INTEGER ); CREATE TABLE SERVICE ( ID INTEGER PRIMARY KEY, NAME TEXT UNIQUE, STATUS TEXT ); CREATE TABLE SERVICE_MAPPING ( ID INTEGER PRIMARY KEY, SERVICE_FK, EQUIPMENT_FK, FOREIGN KEY(SERVICE_FK) REFERENCES SERVICE(ID), FOREIGN KEY(EQUIPMENT_FK) REFERENCES EQUIPMENT(ID) ); Write the update_service method that stores or clears service-related even and then updates the service's status based on the remaining active events. def update_service(self, service, event): if event.severity == 1: self.dt.update("""delete from SERVICE_EVENTS where EVENT_FK in ( select ID from EVENTS where HOST_NAME = ? and EVENT_CONDITION = ?)""", (event.hostname,event.condition)) else: self.dt.execute("""insert into SERVICE_EVENTS (EVENT_FK, SERVICE_FK) values (?,?)""", (event.id,service["ID"])) try: max = self.dt.query_for_int( """select max(EVENTS.SEVERITY) from SERVICE_EVENTS SE join EVENTS on (EVENTS.ID = SE.EVENT_FK) join SERVICE on (SERVICE.ID = SE.SERVICE_FK) where SERVICE.NAME = ?""", (service["NAME"],)) except DataAccessException, e: max = 1 if max > 1 and service["STATUS"] == "Operational": service["STATUS"] = "Outage" self.dt.update("""update SERVICE set STATUS = ? where ID = ?""", (service["STATUS"], service["ID"])) if max == 1 and service["STATUS"] == "Outage": service["STATUS"] = "Operational" self.dt.update("""update SERVICE set STATUS = ? where ID = ?""", (service["STATUS"], service["ID"])) if event.severity == 1: return {"service":service, "is_active":False} else: return {"service":service, "is_active":True} Service-related events are active events related to a service. A single event can be related to many services. For example, what if we were monitoring a wireless router that provided Internet service to a lot of users, and it reported a critical error? This one event would be mapped as an impact to all the end users. When a new active event is processed, it is stored in SERVICE_EVENTS for each related service. Then, when a clearing event is processed, the previous service event must be deleted from the SERVICE_EVENTS table. Add the table defnition for SERVICE_EVENTS to the SQL script. CREATE TABLE SERVICE_EVENTS ( ID INTEGER PRIMARY KEY, SERVICE_FK, EVENT_FK, FOREIGN KEY(SERVICE_FK) REFERENCES SERVICE(ID), FOREIGN KEY(EVENT_FK) REFERENCES EVENTS(ID) ); It is important to recognize that deleting an entry from SERVICE_EVENTS doesn't mean that we delete the original event from the EVENTS table. Instead, we are merely indicating that the original active event is no longer active and it does not impact the related service. Prepend the entire SQL script with drop statements, making it possible to run the script for several recipes DROP TABLE IF EXISTS SERVICE_MAPPING; DROP TABLE IF EXISTS SERVICE_EVENTS; DROP TABLE IF EXISTS ACTIVE_EVENTS; DROP TABLE IF EXISTS EQUIPMENT; DROP TABLE IF EXISTS SERVICE; DROP TABLE IF EXISTS EVENTS; Append the SQL script used for database setup with inserts to preload some equipment and services. INSERT into EQUIPMENT (ID, HOST_NAME, STATUS) values (1, 'pyhost1', 1); INSERT into EQUIPMENT (ID, HOST_NAME, STATUS) values (2, 'pyhost2', 1); INSERT into EQUIPMENT (ID, HOST_NAME, STATUS) values (3, 'pyhost3', 1); INSERT into SERVICE (ID, NAME, STATUS) values (1, 'service-abc', 'Operational'); INSERT into SERVICE (ID, NAME, STATUS) values (2, 'service-xyz', 'Outage'); INSERT into SERVICE_MAPPING (SERVICE_FK, EQUIPMENT_FK) values (1,1); INSERT into SERVICE_MAPPING (SERVICE_FK, EQUIPMENT_FK) values (1,2); INSERT into SERVICE_MAPPING (SERVICE_FK, EQUIPMENT_FK) values (2,1); INSERT into SERVICE_MAPPING (SERVICE_FK, EQUIPMENT_FK) values (2,3); Finally, write the method that updates equipment status based on the current active events. def update_equipment(self, equip, event): try: max = self.dt.query_for_int( """select max(EVENTS.SEVERITY) from ACTIVE_EVENTS AE join EVENTS on (EVENTS.ID = AE.EVENT_FK) where EVENTS.HOST_NAME = ?""", (event.hostname,)) except DataAccessException: max = 1 if max != equip["STATUS"]: equip["STATUS"] = max self.dt.update("""update EQUIPMENT set STATUS = ?""", (equip["STATUS"],)) return equip Here, we need to find the maximum severity from the list of active events for a given host name. If there are no active events, then Spring Python raises a DataAccessException and we translate that to a severity of 1. We check if this is different from the existing device's status. If so, we issue a SQL update. Finally, we return the record for the device, with its status updated appropriately. How it works... This application uses a database-backed mechanism to process incoming network events, and checks them against the inventory of equipment and services to evaluate failures and restorations. Our application doesn't handle specialized devices or unusual types of services. This real-world complexity has been traded in for a relatively simple application, which can be used to write various test recipes. Events typically map to a single piece of equipment and to zero or more services. A service can be thought of as a string of equipment used to provide a type of service to the customer. New failing events are considered active until a clearing event arrives. Active events, when aggregated against a piece of equipment, define its current status. Active events, when aggregated against a service, defines the service's current status.  
Read more
  • 0
  • 0
  • 3472
Modal Close icon
Modal Close icon