How-To Tutorials

Web enabling business data is one of the key devices used to advertise and market products. This can be done with various technologies such as VB, ASP, JSP, ASP.Net and many others. This article shows how you may view data from a table on a MySQL database server on a web page using ASP.NET. The table used in this tutorial was the one described in the first article in this series on Exporting data from MS Access 2003 to MySQL. This article by Dr. Jay Krishnaswamy explains how to populate a GridView on an ASP.NET web page by data retrieved from a MySQL Server. MySQL.Data.MySqlClient is a connector (provider) provided by MySQL which you can use with the .NET Framework applications whose details may be reviewed here. MySQL is well integrated with Visual Studio (MySQL Visual Studio Tools: MySQL.VisualStudio.dll). Overview We first create an ASP.NET 3.5 Website Project (even .NET Framework 2.0 should be OK) in Visual Studio 2008. We then drag and drop a GridView ASP.NET control on to the Default.aspx page. We will then use the smart task on the GridView and follow it up to bring data to the GridView. Then we build the web site project and display the data in the GridView on the Default.aspx page. Create a ASP.NET 3.5 Web Site Project Launch Visual Studio 2008. Click File | New |Web Site... to open up the New Web Site window as shown. Change the default name of the web site to something suitable. Herein it is named WebMySQL as shown. Drag and drop a GridView Control From Toolbox under Data find the GridView Control. Drag and drop this control on to the Default.aspx page as shown. The GridView is 'unbound' when it is dropped and has a few template columns and the smart tasks menu. The menu item is shown in its drop-down state and  displays the menu items under 'Choose Data Source'. Click on the <New Data Source...> item in Choose data source. This will bring up the Data Source Configuration wizard as shown. Herein you need to choose a source of the data you are trying to bring into the application to be bound to the GridView control. You have several options here and for the present article we will be using data from a database. Click on the Database icon as shown in the previous figure. With this you will be specifying an instance of SQLDataSource1 as your source of data. Click OK. This will take you to the next window shown here. Herein you will try to establish a connection to the data source. In the combo-box shown you may see some of the existing connections you have previously established one of which may initially show up. Herein we will be making a new connection. Click the New Connection... button. This brings up the Add Connection window which gets displayed with the default data source, the Microsoft SQL Server Compact 3.5 as shown. Connecting to MySQL Before establishing the connection make sure that your MySQL Server is running. If you have not started you may do so as described in the article mentioned earlier(the first article). You can start the server from the command line as shown in the next figure. Click the Change... button to open the Change Data Source window as shown in the next figure. This window shows a number of Data Sources one of which is the MySQL Database. Scroll down and highlight MySQL Database as shown and click OK. This will bring you back to the Add Connection window with form controls appropriate for making a connection to a MySQL Database. The Server name; user name and Password are appropriate to the MySQL Server on the local computer and you should enter those appropriate for your installation. You may also test the connection as shown. Click OK after the connection is successful. This adds the connection information to the Configure Data Source wizard. You may expand the connection string item to review the connection string created by your entries. Click Next. Here you have an option to save the connection string to the Application Configuration File. This is a recommended practice and hence shown checked. Click Next. Here you will be selecting the set of columns that you want to bring in to your application. It has already chosen the 'employees' table on the MySQL database Testmove. Choose several columns from the list of columns. The SELECT statement is shown at the bottom of the above figure. If you were to click Next you would probably face a page which throws an exception. The square braces [ ] shown for each of the columns is not acceptable to the server.  Click on the first option, "Specify a custom SQL Statement or stored procedure" and then click Next. This opens the "Define Custom Statements or Stored Procedures" page with a Query Builder... button. Here you can not only select columns but also other data modification operations such as Update, Insert and Delete. For now we will be doing just a selection.

This function is only available to administrators, so use this role prudently. During the installation of the Oracle VM Manager, a default admin account is created. And with this admin's account we can go about managing the users and groups. Managing Users Here it is possible to create new users, delete older or unwanted ones, assign different roles to those users, reset user password, and so on. Let's break it up into a few topics and have a look at it. Creating a User Viewing or editing details Changing a role Deleting a User Creating a User To create a User, perform the following: On the Administrator's page click User tab and then click on the Create button: Enter the necessary information such as: Username (avoid using user, manager, and administrator as username) Password Retype Password First Name Last Name Valid Email address We can select the account status, it could either be locked or unlocked and is only accessible when it's unlocked. We can lock an account for security reasons by using the status Locked. We can grant the following roles to this newly created user—User, Manager, or Administrator. Then select the Server Pools for this user and also select the group to which this user should belong to. Click on the Confirm button to confirm the information and we will get this information: As we can see in the preceding screenshot this is a plain user and has no groups or servers assigned to it. However this was unlocked and was granted a User role. Viewing or editing a User Now let's view the User we just created. Click on the User tab on the Administrator page: Click on the Show link to view the Server Pools that the user is allowed to use: We can now edit account details such as change email address, change account status, and so on. Let's change the User's email address: Modifying the account status to either locked or unlocked: Changing the role: Next, add the User to the Server Pool: Removing a User from groups or Server Pools: Changing a User's role Lets change regular Users' role to Administrator: On the Administrator's page, select the newly created User and click on the Edit button. Select the role and click Apply to effectively assign the role to the User: Once applied, we will be presented with the following screen: Deleting User To delete a User, we need to do the following on the Administrator's page. We can carry out a search and then select the User that we want to delete. Click on the Delete button and confirm the User you want to delete:

Why integrate Asterisk with wireless technologies? This is a good question, and is worthy of consideration at the outset of the article. As an open-standards (never mind open source) telephony platform, Asterisk is ideally positioned to connect with all manner of devices, and this is excellent news for those that are involved in designing and deploying Asterisk-based solutions, as we are able to choose the best options rather than being restricted by proprietary compatibility issues. Getting back to the question, or rather the answer to the question, mobility is one of the main reasons people want to hook Asterisk up to the wireless world—both mobility within the office environment and the requirements of a mobile workforce outside the office. Even in this day and age, many international travelers are plagued by heavy roaming costs for their mobile phones. Ironically, a good proportion of them probably carry devices with Wi-Fi and VoIP capabilities, some even staying in hotels with free Wi-Fi, if only they knew! Other reasons that people and corporations may be looking for wireless solutions include rapid deployments, cost reductions (in implementation costs, running costs,or both), or it may be that they need a portable PBX solution that can be moved around with them without the hassle of running cables and so on, every time they arrive at a new location. The following table shows business drivers and technology enablers involved: Business drivers Technology enablers (and issues) Mobility within the office Work anywhere in or around the building Mobile workforce International travel Field staff Multi-site enterprises Rapid implementations New office installations Fast office installation Cost reductions Reduced "hard" infrastructure (cabling, switches, and so on) Reduced real-estate needed Temporary deployments Hot desking Business continuity* Disaster recovery* Wireless access points are already very common, both within offices and in public locations Wi-Fi and WiMAX technology (may suffer with NAT and firewall issues) Wireless routing means phones (and PCs) can now be rolled out very quickly SIP and IAX2 trunks are usually much more cost effective that traditional trunks IP PBXs can be physically much smaller, meaning less rack/floor space is used The power consumption of an IP PBX can be a lot less than a traditional PBX. These reasons in particular are a great sell for VoIP, not just wireless. Wireless technology overview With this weight of reasons to think about wireless, our next job is to look at the options available so that we can deploy the best technology in each given application. To help us consider the options, I have split the next section into two parts—one that considers wireless handsets and one that considers wireless networks. There are many types of wireless handsets on the market today, some have been around for quite some time while others are relatively new. Most are aimed at the consumer, a few are definite enterprise plays, and there is even an "industrial strength" offering. Let's outline each type and home in on their advantages and disadvantages. Wi-Fi (only) phones Wi-Fi (only) phones have probably been around the longest, but in most cases, the vendors do not seem to have refined their offerings following the experience of their first products. Rather, Wi-Fi (only) handsets seem to be disappearing in favor of dual-mode cell phones. These devices started out as a piece of Wi-Fi kit that someone attached to a SIP stack and added on a microphone, speaker, and some basic audio-processing apparatus—and it shows. These phones are usually not excellent. Their issues often include poor battery life and less than perfect audio. Also, something that could be a big issue depending on your specific application, is the ability to roam from one Wi-Fi hotspot to another, which has now been addressed in specification 802.11n. But I have not yet seen many phones which implement this advance. Advantages Disadvantages Truly mobile Inside the office or home Other hotspots worldwide Allows PBX extension to travel. Small and portable. Configuration is difficult. Battery life is generally short. Voice quality can be poor. Not a wide choice of phones. If the phone does not have a web browser, you may not be able to connect to hotspots that require a login. One notable exception to majority of Wi-Fi (only) handsets is the Polycom range. As you would expect, these phones are very well constructed—there is even a rough use version for demanding environments, and they work well too. For more information, look up UT Starcomm, Hitachi, or Linksys Wi-Fi phone. SIP desk phones with a wireless link Imagine all the positive attributes of a good SIP "normal" desk phone—great looks, familiar feel, excellent functionality, high speech quality, and sturdy construction. Now make it wireless! This is exactly the idea that Linksys (and others like Mitel) have implemented by bringing out a wireless Ethernet bridge (a device that connects into a standard Ethernet socket on any device and gives it Wi-Fi connectivity), which is specifically designed to fit into the void in the base of their phones. This brings together the standard-looking office phone (the sort that seems familiar to users and will not scare them) with the convenience of wireless. The intention is not to go wandering around with this phone, as it still needs mains power, but to be able to locate it wherever you want in the office without needing a cabled Ethernet connection, nice! Advantages Disadvantages Very user friendly Great for office use Comprehensive features Good for temporary and flexible deployments Additional work of configuring and connecting the wireless Ethernet bridge The devices still need power, and so are not truly wireless Adds significant extra cost per extension Of course, any wireless Ethernet bridge and any SIP desk phone could be used, as these are standard interfaces that we are talking about. You could just have a wireless Ethernet bridge for a whole room going into a switch and then run cables to a number of phones (if you needed a few), but the elegance and simplicity of the Linksys solution does make it stand out. Dual-mode (GSM and SIP) phones and PDA/smart phones Within the last two to three years, these devices have really taken off. There are offerings from UT Starcomm (who also make Wi-Fi [only] handsets) and Pirelli Communications, but it is the Nokia handsets that have set the standard. Any high end "E" (business) or "N" (multimedia) series Nokia handset will have both Wi-Fi and SIP connectivity, and a SIP client is available for the iPhone, and for Windows mobile phones and PDA devices too. Great news for those wishing to integrate with Asterisk! These handsets (putting call costs to one side for a moment) provide the ultimate mobility, as they will enable calling through Wi-Fi access points or over the regular mobile network Often held up as examples of FMC (Fixed Mobile Convergence), GSM/SIP phones are not really that—they are really two phones in one case, a GSM (or 3G) phone and a SIP phone. The only converged thing about them is that they share a microphone, an earpiece, and the contacts list. These devices cannot currently be configured to intelligently route calls over GSM or IP, depending on, say, cost. You have to choose your preferred method of communication, and all calls will go that way unless you choose the alternative (with an inconvenient two key-press method) on a per-call basis. This is something of an issue in countries like the UK, where it is usually cheapest to call a mobile from a mobile, and most business customers will have their mobile device on a monthly plan that includes minutes—so they will want calls to mobiles routed over the mobile network, and calls to landlines routed over the SIP (Wi-Fi) network. Having said that, two things have emerged to make things a little more user-friendly. Firstly, the UT Starcomm phones have two "place call" buttons—one that directs the call over the GSM (or 3G) network and one that directs the call over the SIP (Wi-Fi) network. Secondly, some third-party companies have developed small applications which run on the Nokia handsets (which use the Symbian S60 operating system) to do Least Cost Routing, which to my mind is an excellent development. Those little gripes aside, can you see the power of a single handset which is both a standard mobile phone AND an extension of your Asterisk PBX, by virtue of its SIP and Wi-Fi capabilities? To give you an idea of the potential benefits, co-author David Duffett describes his experiences: As someone who travels regularly, I continue to be impressed when my Nokia phone rings while I am in, say, Johannesburg, South Africa because someone back in the UK called my office number! It freaks them out when they find out you are not really in the office, and they are also impressed when you tell them that the call between your telephone system (Asterisk) and your mobile phone is FREE! It is heart-warming to find yourself in a hotel that offers free WiFi, able to make calls back to other extensions in the office at zero cost. And, as the internet (in general) and WiFi access points improve, I am finding that there is no noticeable difference in quality between a regular mobile call and a call placed over the internet. I have recently been experimenting with placing SIP calls over the 3G network (as opposed to using WiFi internet access) and, where the 3G network is good (mainly town and city centers at present), call quality has been acceptable. This means that if you are on an unlimited data plan as part of your mobile package, you could be calling your office (and any locations peering with it) free. Advantages Disadvantages Typically better voice quality than Wi-Fi (only) phones User interface is generally better than Wi-Fi (only) alternatives Great mobility Always on-net, just variable costs VoIP is an "application" Respected vendors Battery life is reduced by having Wi-Fi on, in addition to the regular phone Complex hotspot attachment processes in some environments (not a problem with the phone, just the Wi-Fi hook up) Sometimes frustrating steps to choose call route   For more information, look at Nokia, Pirelli, or UT Starcomm dual-mode VoIP phone, or Windows mobile SIP.

Datasource definition A datasource is what JasperReports uses to obtain data for generating a report. Data can be obtained from databases, XML files, arrays of objects, collections of objects, and XML files. In this article, we will focus on using databases as a datasource. Database for our reports We will use a MySQL database to obtain data for our reports. The database is a subset of public domain data that can be downloaded from http://dl.flightstats.us. The original download is 1.3 GB, so we deleted most of the tables and a lot of data to trim the download size considerably. MySQL dump of the modified database can be found as part of code download at http://www.packtpub.com/files/code/8082_Code.zip. The flightstats database contains the following tables: aircraft aircraft_models aircraft_types aircraft_engines aircraft_engine_types The database structure can be seen in the following diagram: The flightstats database uses the default MyISAM storage engine for the MySQL RDBMS, which does not support referential integrity (foreign keys). That is why we don't see any arrows in the diagram indicating dependencies between the tables. Let's create a report that will show the most powerful aircraft in the database. Let's say, those with horsepower of 1000 or above. The report will show the aircraft tail number and serial number, the aircraft model, and the aircraft's engine model. The following query will give us the required results: SELECT a.tail_num, a.aircraft_serial, am.model as aircraft_model, ae.model AS engine_model FROM aircraft a, aircraft_models am, aircraft_engines ae WHERE a.aircraft_engine_code in (select aircraft_engine_code from aircraft_engines where horsepower >= 1000) AND am.aircraft_model_code = a.aircraft_model_code AND ae.aircraft_engine_code = a.aircraft_engine_code The above query retrieves the following data from the database: Generating database reports There are two ways to generate database reports—either by embedding SQL queries into the JRXML report template or by passing data from the database to the compiled report through a datasource. We will discuss both of these techniques. We will first create the report by embedding the query into the JRXML template. Then, we will generate the same report by passing it through a datasource containing the database data. Embedding SQL queries into a report template JasperReports allows us to embed database queries into a report template. This can be achieved by using the <queryString> element of the JRXML file. The following example demonstrates this technique: <?xml version="1.0" encoding="UTF-8" ?> <jasperReport xsi_schemaLocation="http://jasperreports.sourceforge.net/jasperreports http://jasperreports.sourceforge.net/xsd/jasperreport.xsd" name="DbReport"> <queryString> <![CDATA[select a.tail_num, a.aircraft_serial, am.model as aircraft_model, ae.model as engine_model from aircraft a, aircraft_models am, aircraft_engines ae where a.aircraft_engine_code in ( select aircraft_engine_code from aircraft_engines where horsepower >= 1000) and am.aircraft_model_code = a.aircraft_model_code and ae.aircraft_engine_code = a.aircraft_engine_code]]> </queryString> <field name="tail_num" class="java.lang.String" /> <field name="aircraft_serial" class="java.lang.String" /> <field name="aircraft_model" class="java.lang.String" /> <field name="engine_model" class="java.lang.String" /> <pageHeader> <band height="30"> <staticText> <reportElement x="0" y="0" width="69" height="24" /> <textElement verticalAlignment="Bottom" /> <text> <![CDATA[Tail Number: ]]> </text> </staticText> <staticText> <reportElement x="140" y="0" width="79" height="24" /> <text> <![CDATA[Serial Number: ]]> </text> </staticText> <staticText> <reportElement x="280" y="0" width="69" height="24" /> <text> <![CDATA[Model: ]]> </text> </staticText> <staticText> <reportElement x="420" y="0" width="69" height="24" /> <text> <![CDATA[Engine: ]]> </text> </staticText> </band> </pageHeader> <detail> <band height="30"> <textField> <reportElement x="0" y="0" width="69" height="24" /> <textFieldExpression class="java.lang.String"> <![CDATA[$F{tail_num}]]> </textFieldExpression> </textField> <textField> <reportElement x="140" y="0" width="69" height="24" /> <textFieldExpression class="java.lang.String"> <![CDATA[$F{aircraft_serial}]]> </textFieldExpression> </textField> <textField> <reportElement x="280" y="0" width="69" height="24" /> <textFieldExpression class="java.lang.String"> <![CDATA[$F{aircraft_model}]]> </textFieldExpression> </textField> <textField> <reportElement x="420" y="0" width="69" height="24" /> <textFieldExpression class="java.lang.String"> <![CDATA[$F{engine_model}]]> </textFieldExpression> </textField> </band> </detail> </jasperReport> The <queryString> element is used to embed a database query into the report template. In the given code example, the <queryString> element contains the query wrapped in a CDATA block for execution. The <queryString> element has no attributes or subelements other than the CDATA block containing the query. Text wrapped inside an XML CDATA block is ignored by the XML parser. As seen in the given example, our query contains the > character, which would invalidate the XML block if it wasn't inside a CDATA block. A CDATA block is optional if the data inside it does not break the XML structure. However, for consistency and maintainability, we chose to use it wherever it is allowed in the example. The <field> element defines fields that are populated at runtime when the report is filled. Field names must match the column names or alias of the corresponding columns in the SQL query. The class attribute of the <field> element is optional; its default value is java.lang.String. Even though all of our fields are strings, we still added the class attribute for clarity. In the last example, the syntax to obtain the value of a report field is $F{field_name}, where field_name is the name of the field as defined. The next element that we'll discuss is the <textField> element. Text fields are used to display dynamic textual data in reports. In this case, we are using them to display the value of the fields. Like all the subelements of <band>, text fields must contain a <reportElement> subelement indicating the text field's height, width, and x, y coordinates within the band. The data that is displayed in text fields is defined by the <textFieldExpression> subelement of <textField>. The <textFieldExpresson> element has a single subelement, which is the report expression that will be displayed by the text field and wrapped in an XML CDATA block. In this example, each text field is displaying the value of a field. Therefore, the expression inside the <textFieldExpression> element uses the field syntax $F{field_name}, as explained before. Compiling a report containing a query is no different from compiling a report without a query. It can be done programmatically or by using the custom JasperReports jrc ANT task. Generating the report As we have mentioned previously, in JasperReports terminology, the action of generating a report from a binary report template is called filling the report. To fill a report containing an embedded database query, we must pass a database connection object to the report. The following example illustrates this process: package net.ensode.jasperbook; import java.sql.Connection; import java.sql.DriverManager; import java.sql.SQLException; import java.util.HashMap; import net.sf.jasperreports.engine.JRException; import net.sf.jasperreports.engine.JasperFillManager; public class DbReportFill { Connection connection; public void generateReport() { try { Class.forName("com.mysql.jdbc.Driver"); connection = DriverManager.getConnection("jdbc:mysql: //localhost:3306/flightstats?user=user&password=secret"); System.out.println("Filling report..."); JasperFillManager.fillReportToFile("reports/DbReport. jasper", new HashMap(), connection); System.out.println("Done!"); connection.close(); } catch (JRException e) { e.printStackTrace(); } catch (ClassNotFoundException e) { e.printStackTrace(); } catch (SQLException e) { e.printStackTrace(); } } public static void main(String[] args) { new DbReportFill().generateReport(); } } As seen in this example, a database connection is passed to the report in the form of a java.sql.Connection object as the last parameter of the static JasperFillManager.fillReportToFile() method. The first two parameters are as follows: a string (used to indicate the location of the binary report template or jasper file) and an instance of a class implementing the java.util.Map interface (used for passing additional parameters to the report). As we don't need to pass any additional parameters for this report, we used an empty HashMap. There are six overloaded versions of the JasperFillManager.fillReportToFile() method, three of which take a connection object as a parameter. For simplicity, our examples open and close database connections every time they are executed. It is usually a better idea to use a connection pool, as connection pools increase the performance considerably. Most Java EE application servers come with connection pooling functionality, and the commons-dbcp component of Apache Commons includes utility classes for adding connection pooling capabilities to the applications that do not make use of an application server. After executing the above example, a new report, or JRPRINT file is saved to disk. We can view it by using the JasperViewer utility included with JasperReports. In this example, we created the report and immediately saved it to disk. The JasperFillManager class also contains methods to send a report to an output stream or to store it in memory in the form of a JasperPrint object. Storing the compiled report in a JasperPrint object allows us to manipulate the report in our code further. We could, for example, export it to PDF or another format. The method used to store a report into a JasperPrint object is JasperFillManager.fillReport(). The method used for sending the report to an output stream is JasperFillManager.fillReportToStream(). These two methods accept the same parameters as JasperFillManager.fillReportToFile() and are trivial to use once we are familiar with this method. Refer to the JasperReports API for details. In the next example, we will fill our report and immediately export it to PDF by taking advantage of the net.sf.jasperreports.engine.JasperRunManager.runReportToPdfStream() method. package net.ensode.jasperbook; import java.io.IOException; import java.io.InputStream; import java.io.PrintWriter; import java.io.StringWriter; import java.sql.Connection; import java.sql.DriverManager; import java.util.HashMap; import javax.servlet.ServletException; import javax.servlet.ServletOutputStream; import javax.servlet.http.HttpServlet; import javax.servlet.http.HttpServletRequest; import javax.servlet.http.HttpServletResponse; import net.sf.jasperreports.engine.JasperRunManager; public class DbReportServlet extends HttpServlet { protected void doGet(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException { Connection connection; response.setContentType("application/pdf"); ServletOutputStream servletOutputStream = response .getOutputStream(); InputStream reportStream = getServletConfig() .getServletContext().getResourceAsStream( "/reports/DbReport.jasper"); try { Class.forName("com.mysql.jdbc.Driver"); connection = DriverManager.getConnection("jdbc:mysql: //localhost:3306/flightstats?user=dbUser&password=secret"); JasperRunManager.runReportToPdfStream(reportStream, servletOutputStream, new HashMap(), connection); connection.close(); servletOutputStream.flush(); servletOutputStream.close(); } catch (Exception e) { // display stack trace in the browser StringWriter stringWriter = new StringWriter(); PrintWriter printWriter = new PrintWriter(stringWriter); e.printStackTrace(printWriter); response.setContentType("text/plain"); response.getOutputStream().print(stringWriter.toString()); } } } The only difference between static and dynamic reports is that for dynamic reports we pass a connection to the report for generating a database report. After deploying this servlet and pointing the browser to its URL, we should see a screen similar to the following screenshot:  

WebSphere MQ overview WebSphere MQ formerly known as MQ Series is IBM's enterprise messaging solution. In a nutshell, MQ provides the mechanisms for messaging both in point-to-point and publish-subscribe. However, it guarantees to deliver a message only once. This is important for critical business applications which implement messaging. An example of a critical system could be a banking payments system where messages contain messages pertaining to money transfer between banking systems, so guaranteeing delivery of a debit/credit is paramount in this context. Aside from guaranteed delivery, WMQ is often used for messaging between dissimilar systems and the WMQ software provides programming interfaces in most of the common languages, such as Java, C, C++, and so on. If you are using WebSphere, then it is common to find that WMQ is often used with WebSphere when WebSphere is hosting message-enabled applications. It is important that the WebSphere administrator understands how to configure WebSphere resources so that application can be coupled to the MQ queues. Overview of WebSphere MQ example To demonstrate messaging using WebSphere MQ, we are going to re-configure the previously deployed JMS Tester application so that it will use a connection factory which communicates with a queue on a WMQ queue manager as opposed to using the default provider which we demonstrated earlier. Installing WebSphere MQ Before we can install our demo messaging application, we will need to download and install WebSphere MQ 7.0. A free 90-day trial can be found at the following URL: http://www.ibm.com/developerworks/downloads/ws/wmq/. Click the download link as shown below. You will be prompted to register as an IBM website user before you can download the WebSphere MQ Trial. Once you have registered and logged in, the download link above will take you to a page which lists download for different operating systems. Select WebSphere MQ 7.0 90-day trial from the list of available options as shown below. Click continue to go to the download page. You may be asked to fill out a questionnaire detailing why you are evaluating WebSphere MQ (WMQ). Fill out the question as you see fit and submit to move to the download page. As shown above, make sure you use the IBM HTTP Download director as it will ensure that your download will resume, even if your Internet loses a connection. If you do not have a high-speed Internet connection, you can try downloading a free 90-day trial of WebSphere MQ 7.0 overnight while you are asleep. Download the trial to a temp folder, for example c:temp, on your local machine. The screenshot above shows how the IBM HTTP Downloader will prompt for a location where you want to download it to. Once the WMQ install file has been downloaded, you can then upload the file using an appropriate secure copy utility like Winscp to an appropriate folder like /apps/wmq_install on your Linux machine. Once you have the file uploaded to Linux, you can then decompress the file and run the installer to install WebSphere MQ. Running the WMQ installer Now that you have uploaded the WMQv700Trial-x86_linux.tar file on your Linux machine, and follow these steps: You can decompress the file using the following command: gunzip ./WMQv700Trial-x86_linux.tar.gz Then run the un-tar command: tar -xvf ./ WMQv700Trial-x86_linux.tar Before we can run the WMQ installations, we need to accept the license agreement by running the following command: ./mqlicense.sh –accept To run the WebSphere MQ installation, type the following commands: rpm -ivh MQSeriesRuntime-7.0.0-0.i386.rpmrpm -ivh MQSeriesServer-7.0.0-0.i386.rpmrpm -ivh MQSeriesSamples-7.0.0-0.i386.rpm As a result of running the MQSeriesServer installation, a new user called mqm was created. Before running any WMQ command, we need to switch to this user using the following command: su - mqm Then, we can run commands like the dspmqver command which can be run to check that WMQ was installed correctly. To check whether WMQ is installed, run the following command: /opt/mqm/bin/dspmqver The result will be the following message as shown in the screenshot below: Creating a queue manager Before we can complete our WebSphere configuration, we need to create a WMQ queue manager and a queue, then we will use some MQ command line tools to put a test message on an MQ queue and get a message from an MQ queue. To create a new queue manager called TSTDADQ1, use the following command: crtmqm TSTDADQ1 The result will be as shown in the image below. We can now type the following command to list queue managers: dspmq The result of running the dspmq command is shown in the image below. To start the queue manager (QM), type the following command: strmqm The result of starting the QM will be similar to the image below. Now that we have successfully created a QM, we now need to add a queue called LQ.Test where we can put and get messages. To create a local queue on the TSTDADQ1 QM, type the following commands in order: runmqsc TSTDADQ1 You are now running the MQ scripting command line, where you can issue MQ commands to configure the QM. To create the queue, type the following command and hit Enter: define qlocal(LQ.TEST) Then immediately type the following command: end Hit Enter to complete the QM configuration, as shown by the following screenshot. You can use the following command to see if your LQ.TEST queue exists. echo "dis QLOCAL(*)" | runmqsc TSTDADQ1 | grep -i test You have now added a local queue called Q.Test to the TSTDADQ1 queue manager. runmqsc TSTDADQ1DEFINE LISTENER(TSTDADQ1.listener) TRPTYPE (TCP) PORT(1414)START LISTENER(TSTDADQ1.listener)End You can type the following command to ensure that your QM listener is running. ps -ef | grep mqlsr The result will be similar to the image below. To create a default channel, you can run the following command. runmqsc TSTDADQ1DEFINE CHANNEL(SYSTEM.ADMIN.SVRCONN) CHLTYPE(SVRCONN)End We can now use a sample MQ program called amqsput which we can use to put and get a test message from a queue to ensure that our MQ configuration is working before we continue to configure WebSphere. Type the following command to put a test message on the LQ.Test queue: /opt/mqm/samp/bin/amqsput LQ.TEST TSTDADQ1 Then you can type a test message: Test Message and hit Enter; this will put a message on the LQ.Test queue and will exit you from the AMQSPUTQ command tool. Now that we have put a message on the queue, we can read the message by using the MQ Sample command tool called amqsget. Type the following command to get the message you posted earlier: /opt/mqm/samp/bin/amqsget LQ.TEST TSTDADQ1 The result will be that all messages on the LQ.TEST queue will be listed and then the tool will timeout after a few seconds as shown below. We need to do two final steps to complete and that is to add the root user to the mqm group. This is not a standard practice in an enterprise, but we have to do this because our WebSphere installation is running as root. If we did not do this, we would have to reconfigure the user which the WebSphere process is running under and then add the new user to MQ security. To keep things simple, ensure that root is a member of the mqm group, by typing the following command: usermod -a -G mqm root We also need to change WMQ security to ensure that all users of the mqm group have access to all the objects of the TSTDADQ1 queue manager. To change WMQ security to give access to all objects in the QM, type the following command: setmqaut -m TSTDADQ1 -t qmgr -g mqm +all Now, we are ready to re-continue our configuring WebSphere and create the appropriate QCF and queue destinations to access WMQ from WebSphere.

Configuring wireless devices Each device will have its own user interface-either through the display on the device (if it's a phone) or through a web interface that you can browse to (some Wi-Fi phones will have both). The actual entries used for setting up the device to work with Asterisk are standard SIP parameters like username, password, host, and so on, and therefore, the next screenshot has been included (taken from the Siemens S460IP phone) as an example: What is worth looking into in more detail are the Nokia dual-mode phones, as there are a couple of parameters necessary for successful connection with Asterisk, which are not as widely known as they should be. These "lesser known" parameters are responsible for many people giving up on connecting their devices to Asterisk. I will use the Nokia E90 as an example: Most of the parameters are self explanatory, but the two parameters we pay special attention to are the Public user name in the SIP profile itself, and the Realm in the submenus for both the Proxy server and the Registrar server. Further in this section, some screenshots of each of the screens you will need to go through to set up your Nokia phone as an extension on your Asterisk PBX have been included. Start by selecting the Connection in the Settings screen: Then select SIP settings: Select or create the relevant SIP profile from the submenu: Inside the SIP profile, complete the settings—the Public user name must be the SIP profile name (from sip.conf)@: Now move down to the Proxy server and Registrar server sections: It is here that the all-important Realm must be entered—Asterisk sets this as "asterisk" by default, but you can change it in the [general] section of sip.conf. The Registrar server screen needs to be the same as the previous one: Although the IP address used (see preceding screenshot) is internal, it could equally be an Internet IP address such as 80.229.203.112 Note that the Realm is defined in the [general] section of sip.conf by the realm= line. If this line is missing, then the realm will default to "asterisk".

Adding jQuery to your page As we have discussed above you can download the latest version of jQuery from jQuery site (http://jquery.com/) and can be added as a reference to your web pages accordingly. You can reference a local copy of jQuery using <script> tag in the page. Either you can reference your local copy or you can directly reference remote copy from jQuery.com or Google Ajax API (http://ajax.googleapis.com/ajax/libs/jquery/1.3/jquery.min.js) Prerequisite Knowledge In order to understand the code, one should have the basic knowledge of HTML, CSS, JavaScript and basic knowledge of jQuery. Ingredients Used HTML CSS jQuery Photoshop (Used for Designing the Button Background) Preview/Download If you would like to see the working example, click the following link: http://www.developersnippets.com/snippets/jquery/alphabetical_glossary/alphabetical_glossary.html. And if you would like to download the snippet, click the following link: http://www.developersnippets.com/snippets/jquery/alphabetical_glossary.zip. Figure 1: Snapshot of "Simple Alphabetical Glossary using jQuery" Figure 2: Overview of div’s and Listings used Figure 3: Image used for Listings (CSS Sprite) Successfully tested The above application is successfully tested on various browsers like IE 6.0, IE 7, IE 8, Mozilla Firefox (Latest Version), Google Chrome and Safari Browser (4.0.2) respectively. HTML Code Below is the HTML code with comments for you to better understand. <div id="body-container"> <div class="glossary-container"> <ul class="firstUL"> <li id="a" class="selected">A</li> <li id="b">B</li> <li id="c">C</li> <li id="d">D</li> <li id="e">E</li> <li id="f">F</li> <li id="g">G</li> <li id="h">H</li> </ul> </div> <!-- Close of glossary-container --> <div class="content-container"> <!-- A --> <div id="content-for-a" style="background-color:#d2e2fc"> <!-- Content for A --> </div> <!-- B --> <div id="content-for-b"> <!-- Content for B --> </div> <!-- C --> <div id="content-for-c"> <!-- Content for C --> </div> <!-- D --> <div id="content-for-d"> <!-- Content for D --> </div> </div> <!-- Close of content-container --></div> <!-- Close of body-container --> Explanation of HTML Code To get the UI shown in above screenshot, I have written some HTML div tags to hold the Glossary terms in one container and the results in another container. CSS styles are used to assign some nice colors. <div id="body-container">.....</div> I have used "body-container" as the main container to catch hold of other two div containers. <div class="glossary-container">.....</div> "glossary-container" contains a glossary term that is Alphabets, as of now in this example I have used alphabets only from "A" to "H". <div class="content-container">.....</div> In "content-container" I have arranged all the results or definitions of glossary terms, for each and every glossary term I have used separate <div> tags like for Alphabet "A" --- <div id="content-for-a" style="background-color:#d2e2fc">....</div>, and given default background color as "background-color:#d2e2fc" which highlights the definition accordingly. After going through the HTML code, we will have a glance on CSS styling. This is very important to give a good look and feel to the application. Explanation of CSS Code <style>/* Making margin and padding to 0, since by default the body will be allocated some amount of pixels of margin and padding. */body{ margin:0; padding:0;}#body-container{ width:415px; /* Given a constant width of 415px to body-container div */ height:500px; /* Given a constant height of 500px to the div */ margin:0 auto; /* This will align the div to center */ border:1px solid #3285ef; /* Giving the border */}#body-container .glossary-container{ clear:both; /* This will not allow floating elements on either sides */}#body-container .content-container{ height:430px; /* Given a constant height of 430px to the div */ width:415px; /* Given a constant width of 415px to the div */ overflow:auto; /* Scroll bar is shown when content is more than specified height */ font-family:'Arial',Verdana,Tahoma; /* Taken the default font to 'Arial' */ font-size:10pt; /* Making font size to 10 points */ clear:both; /* This will not allow floating elements on either sides */}#body-container .content-container div{ padding-left:10px; /* Left padding given as 10px */ border-bottom:1px #666666 solid; /* In order to separate each terms given bottom border color as #666666 (gray) with 1px */}#body-container .content-container div h2{ margin-top:0px; /* Making the top margin to 0px */}#body-container .content-container p.return-to-top{ color:#0066FF; /* Giving text color to Return to top text */ text-decoration:underline; /* The text will be underlined */ text-align:right; /* Text will be aligned to right */ margin-right:10px; /* Given some margin 10px to right */ cursor:pointer; /* Making the cursor to 'hand' */}.firstUL{ padding:0px 0px 0px 10px; /* Given some padding to left and 0 padding to top, right, bottom */ margin:0px; /* margin to 0px */ background-color:#3285ef; /* Given background color */}.firstUL li { background:transparent url(images/link_sprite_img.jpg) no-repeat scroll 0 0; /* For all li’s(listings) given default background image using CSS Sprite concept */ display:inline; /* Listings will be placed in a line */ font-family:'Arial',Verdana,Tahoma; /* Setting the font to 'Arial' */ font-size:16pt; /* Setting the font size to 16 points */ font-weight:bold; /* Making the text to bold */ padding:10px 15px 22px; /* Given some padding to top, right, bottom and left */ line-height:70px; /* This property specifies the line height */ cursor:pointer; /* Making the cursor to 'hand' */}.firstUL li.selected{ background:transparent url(images/link_sprite_img.jpg) no-repeat scroll 0px -57px; /* When any listing is highlighted, we are given the background to image using CSS Sprite concept */ color:#ffffff; /* Making the font color 'white' */ font-weight:bold; /* Making text bold */}</style> Explanation of jQuery Code In order to work your jQuery code, as mentioned above in "Adding jQuery to your page" section, we need to include jQuery JavaScript file onto your web page using <script> tag. <!-- jQuery --><script language="javascript" type="text/javascript" src="js/jquery.js"></script> I have downloaded the jQuery file from jQuery site and kept it on my local machine. To see the scrolling effect, you need to include the following plugin: <!-- scrollTo Plugin --><script language="javascript" type="text/javascript" src="js/jquery.scrollTo-min.js"></script> You can get the above plugin at this location—http://plugins.jquery.com/project/ScrollTo The above two .js files should be in included in <head> tag in order to utilize those in our code. jQuery Code <script language="javascript" type="text/javascript">$(document).ready(function() { //below code is for high-lighting the link and scroll to particular DOM Element as well $(".firstUL li").each(function() { $(this).click(function() { //On click of any Alphabet $(".firstUL li").removeClass("selected"); //Initially remove "selected" class if any $(this).addClass("selected"); //Add "selected" class for the clicked one elementClick = $(this).attr("id"); //get respective 'Id' for example 'a','b','c'.. etc., $(".content-container").scrollTo($("#content-for-"+elementClick), 800); //scrollTo particular DOM Element $(".content-container div").css({'background-color' : '#ffffff'}); //set the background color to default, that is white $(".content-container #content-for-"+elementClick).css({'background-color' : '#d2e2fc'}); //set the background color to light-blue to that div }); }); //When "Return to Top" is clicked highlight the first Alphabet that 'A' and scroll to top. $('.return-to-top').click(function(){ $(".firstUL li").each(function() { $(".firstUL li").removeClass("selected"); //Remove classname "selected" }); $("#a").addClass("selected"); //Add a class named "selected" to the first Alphabet $(".content-container").scrollTo($("#content-for-a"), 800); //This is for scrolling to particular element that is "A" here... $(".content-container div").css({'background-color' : '#ffffff'}); //set the background color to default, that is white $(".content-container #content-for-a").css({'background-color' : '#d2e2fc'}); //set the background color to light-blue to that div });});</script> Summary In this article, we saw how to create a Simple Alphabetical Glossary using jQuery. We have learnt how we can highlight particular DOM Element, how we can scroll-to particular div element using scroll-to plugin, and learnt how we can add the background-color to a div using CSS properties.

How can a company or organization minimize bandwidth costs when maintaining multiple Ubuntu installations? With bandwidth becoming the currency of the new millennium, being responsible with the bandwidth you have can be a real concern. In this article by Christer Edwards, we will learn how to create, maintain and make available a local Ubuntu repository mirror, allowing you to save bandwidth and improve network efficiency with each machine you add to your network. (For more resources on Ubuntu, see here.) Background Before I begin, let me give you some background into what prompted me to initially come up with these solutions. I used to volunteer at local university user groups whenever they held "installfests". I always enjoyed offering my support and expertise with new Linux users. I have to say, the distribution that we helped deploy the most was Ubuntu. The fact that Ubuntu's parent company, Canonical, provided us with pressed CDs didn't hurt! Despite having more CDs than we could handle we still regularly ran into the same problem. Bandwidth. Fetching updates and security errata was always our bottleneck, consistently slowing down our deployments. Imagine you are in a conference room with dozens of other Linux users, each of them trying to use the internet to fetch updates. Imagine how slow and bogged down that internet connection would be. If you've ever been to a large tech conference you've probably experienced this first hand! After a few of these "installfests" I started looking for a solution to our bandwidth issue. I knew that all of these machines were downloading the same updates and security errata, therefore duplicating work that another user had already done. It just seemed so inefficient. There had to be a better way. Eventually I came across two solutions, which I will outline below. Apt-Mirror The first method makes use of a tool called “apt-mirror”. It is a perl-based utility for downloading and mirroring the entire contents of a public repository. This may likely include packages that you don't use and will not use, but anything stored in a public repository will also be stored in your mirror. To configure apt-mirror you will need the following: apt-mirror package (sudo aptitude install apt-mirror) apache2 package (sudo aptitude install apache2) roughly 15G of storage per release, per architecture Once these requirements are met you can begin configuring the apt-mirror tool. The main items that you'll need to define are: storage location (base_path) number of download threads (nthreads) release(s) and architecture(s) you want The configuration is done in the /etc/apt/mirror.list file. A default config was put in place when you installed the package, but it is generic. You'll need to update the values as mentioned above. Below I have included a complete configuration which will mirror Ubuntu 8.04 LTS for both 32 and 64 bit installations. It will require nearly 30G of storage space, which I will be putting on a removable drive. The drive will be mounted at /media/STORAGE/. # apt-mirror configuration file#### The default configuration options (uncomment and change to override)###set base_path /media/STORAGE/# set mirror_path $base_path/mirror# set skel_path $base_path/skel# set var_path $base_path/var## set defaultarch <running host architecture>set nthreads 20## 8.04 "hardy" i386 mirrordeb-i386 http://us.archive.ubuntu.com/ubuntu hardy main restricted universe multiversedeb-i386 http://us.archive.ubuntu.com/ubuntu hardy-updates main restricted universe multiversedeb-i386 http://us.archive.ubuntu.com/ubuntu hardy-security main restricted universe multiversedeb-i386 http://us.archive.ubuntu.com/ubuntu hardy-backports main restricted universe multiversedeb-i386 http://us.archive.ubuntu.com/ubuntu hardy-proposed main restricted universe multiversedeb-i386 http://us.archive.ubuntu.com/ubuntu hardy main/debian-installer restricted/debian-installer universe/debian-installer multiverse/debian-installerdeb-i386 http://packages.medibuntu.org/ hardy free non-free# 8.04 "hardy" amd64 mirrordeb-amd64 http://us.archive.ubuntu.com/ubuntu hardy main restricted universe multiversedeb-amd64 http://us.archive.ubuntu.com/ubuntu hardy-updates main restricted universe multiversedeb-amd64 http://us.archive.ubuntu.com/ubuntu hardy-security main restricted universe multiversedeb-amd64 http://us.archive.ubuntu.com/ubuntu hardy-backports main restricted universe multiversedeb-amd64 http://us.archive.ubuntu.com/ubuntu hardy-proposed main restricted universe multiversedeb-amd64 http://us.archive.ubuntu.com/ubuntu hardy main/debian-installer restricted/debian-installer universe/debian-installer multiverse/debian-installerdeb-amd64 http://packages.medibuntu.org/ hardy free non-free# Cleaning sectionclean http://us.archive.ubuntu.com/clean http://packages.medibuntu.org/ It should be noted that each of the repository lines within the file should begin with deb-i386 or deb-amd64. Formatting may have changed based on the web formatting. Once your configuration is saved you can begin to populate your mirror by running the command: apt-mirror Be warned that, based on your internet connection speeds, this could take quite a long time. Hours, if not more than a day for the initial 30G to download. Each time you run the command after the initial download will be much faster, as only incremental updates are downloaded. You may be wondering if it is possible to cancel the transfer before it is finished and begin again where it left off. Yes, I have done this countless times and I have not run into any issues. You should now have a copy of the repository stored locally on your machine. In order for this to be available to other clients you'll need to share the contents over http. This is why we listed Apache as a requirement above. In my example configuration above I mirrored the repository on a removable drive, and mounted it at /media/STORAGE. What I need to do now is make that address available over the web. This can be done by way of a symbolic link. cd /var/www/sudo ln -s /media/STORAGE/mirror/us.archive.ubuntu.com/ubuntu/ ubuntu The commands above will tell the filesystem to follow any requests for “ubuntu” back up to the mounted external drive, where it will find your mirrored contents. If you have any problems with this linking double-check your paths (as compared to those suggested here) and make sure your link points to the ubuntu directory, which is a subdirectory of the mirror you pulled from. If you point anywhere below this point your clients will not properly be able to find the contents. The additional task of keeping this newly downloaded mirror updated with the latest updates can be automated by way of a cron job. By activating the cron job your machine will automatically run the apt-mirror command on a regular daily basis, keeping your mirror up to date without any additional effort on your part. To activate the automated cron job, edit the file /etc/cron.d/apt-mirror. There will be a sample entry in the file. Simply uncomment the line and (optionally) change the “4” to an hour more convenient for you. If left with its defaults it will run the apt-mirror command, updating and synchronizing your mirror, every morning at 4:00am. The final step needed, now that you have your repository created, shared over http and configured to automatically update, is to configure your clients to use it. Ubuntu clients define where they should grab errata and security updates in the /etc/apt/sources.list file. This will usually point to the default or regional mirror. To update your clients to use your local mirror instead you'll need to edit the /etc/apt/sources.list and comment the existing entries (you may want to revert to them later!) Once you've commented the entries you can create new entries, this time pointing to your mirror. A sample entry pointing to a local mirror might look something like this: deb http://192.168.0.10/ubuntu hardy main restricted universe multiversedeb http://192.168.0.10/ubuntu hardy-updates main restricted universe multiversedeb http://192.168.0.10/ubuntu hardy-security main restricted universe multiverse Basically what you are doing is recreating your existing entries but replacing the archive.ubuntu.com with the IP of your local mirror. As long as the mirrored contents are made available over http on the mirror-server itself you should have no problems. If you do run into problems check your apache logs for details. By following these simple steps you'll have your own private (even portable!) Ubuntu repository available within your LAN, saving you future bandwidth and avoiding redundant downloads.

(For more resources on Solr, see here.) Lets get started. Communicating with Solr There are a few dimensions to the options available for communicating with Solr: Direct HTTP or a convenient client API Applications interact with Solr over HTTP. This can either be done directly (by hand, but by using an HTTP client of your choice), or it might be facilitated by a Solr integration API such as SolrJ or Solr Flare, which in turn use HTTP. An exception to HTTP is offered by SolrJ, which can optionally be used in an embedded fashion with Solr (so-called Embedded Solr) to avoid network and inter process communication altogether. However, unless you are sure you really want to embed Solr within another application, this option is discouraged in favor of writing a custom Solr updating request handler. Data streamed remotely or from Solr's Filesystem Even though an application will be communicating with Solr over HTTP, it does not have to send Solr data over this channel. Solr supports what it calls remote streaming. Instead of giving Solr the data directly, it is given a URL that it will resolve. It might be an HTTP URL, but more likely it is a filesystem based URL, applicable when the data is already on Solr's machine. Finally, in the case of Solr's DataImportHandler, the data can be fetched from a database. Data formats The following are the different data formats: Solr-XML: Solr has a specific XML schema it uses to specify documents and their fields. It supports instructions to delete documents and to perform optimizes and commits too. Solr-binary: Analogous to Solr-XML, it is an efficient binary representation of the same structure. This is only supported by the SolrJ client API. CSV: CSV is a character separated value format (often a comma). Rich documents like PDF, XLS, DOC, PPT to Solr: The text data extracted from these formats is directed to a particular field in your Solr schema. Finally, Solr's DIH DataImportHandler contrib add-on is a powerful capability that can communicate with both databases and XML sources (for example: web services). It supports configurable relational and schema mapping options and supports custom transformation additions if needed. The DIH uniquely supports delta updates if the source data has modification dates. We'll use the XML, CSV, and DIH options in bringing the MusicBrainz data into Solr from its database to demonstrate Solr's capability. Most likely, an application would use just one format. Before these approaches are described, we'll discuss curl and remote streaming, which are foundational topics. Using curl to interact with Solr Solr receives commands (and possibly the associated data) through HTTP POST. Solr lets you use HTTP GET too (for example, through your web browser). However, this is an inappropriate HTTP verb if it causes something to change on the server, as happens with indexing. For more information on this concept, read about REST at: http://en.wikipedia.org/wiki/Representational_State_Transfer. One way to send an HTTP POST is through the Unix command line program curl (also available on Windows through Cygwin). Even if you don't use curl, it is very important to know how we're going to use it, because the concepts will be applied no matter how you make the HTTP messages. There are several ways to tell Solr to index data, and all of them are through HTTP POST: Send the data as the entire POST payload (only applicable to Solr's XML format). curl does this with data-binary (or some similar options) and an appropriate content-type header reflecting that it's XML. Send some name-value pairs akin to an HTML form submission. With curl, such pairs are proceeded by -F. If you're giving data to Solr to be indexed (as opposed to it looking for it in a database), then there are a few ways to do that: Put the data into the stream.body parameter. If it's small, perhaps less than a megabyte, then this approach is fine. The limit is configured with the multipartUploadLimitInKB setting in solrconfig.xml. Refer to the data through either a local file on the Solr server using the stream.file parameter or a URL that Solr will fetch it from through the stream.url parameter. These choices are a feature that Solr calls remote streaming. Here is an example of the first choice. Let's say we have an XML file named artists.xml in the current directory. We can post it to Solr using the following command line: curl http://localhost:8983/solr/update -H 'Content-type:text/xml; charset=utf-8' --data-binary @artists.xml If it succeeds, then you'll have output that looks like this: <?xml version="1.0" encoding="UTF-8"?> <response> <lst name="responseHeader"> <int name="status">0</int><int name="QTime">128</int> </lst> </response> To use the solr.body feature for the example above, you would do this: curl http://localhost:8983/solr/update -F solr.body=@artists.xml In both cases, the @ character instructs curl to get the data from the file instead of being @artists.xml literally. If the XML is short, then you can just as easily specify it literally on the command line: curl http://localhost:8983/solr/update -F stream.body=' <commit />' Notice the leading space in the value. This was intentional. In this example, curl treats @ and < to mean things we don't want. In this case, it might be more appropriate to use form-string instead of -F. However, it's more typing, and I'm feeling lazy. Remote streaming In the examples above, we've given Solr the data to index in the HTTP message. Alternatively, the POST request can give Solr a pointer to the data in the form of either a file path accessible to Solr or an HTTP URL to it. The file path is accessed by the Solr server on its machine, not the client, and it must also have the necessary operating system file permissions too. However, just as before, the originating request does not return a response until Solr has finished processing it. If you're sending a large CSV file, then it is practical to use remote streaming. Otherwise, if the file is of a decent size or is already at some known URL, then you may find remote streaming faster and/or more convenient, depending on your situation. Here is an example of Solr accessing a local file: curl http://localhost:8983/solr/update -F stream.file=/tmp/artists.xml To use a URL, the parameter would change to stream.url, and we'd specify a URL. We're passing a name-value parameter (stream.file and the path), not the actual data. Remote streaming must be enabled In order to use remote streaming (stream.file or stream.url), you must enable it in solrconfig.xml. It is disabled by default and is configured on a line that looks like this: <requestParsers enableRemoteStreaming="true" multipartUploadLimitInKB="2048" />

(For more resources on Solr, see here.) Direct database and XML import The capability for Solr to get data directly from a database or HTTP GET accessible XML is distributed with Solr as a contrib module, and it is known as the DataImportHandler (DIH in short). The complete reference documentation for this capability is here at http://wiki.apache.org/solr/DataImportHandler, and it's rather thorough. In this article, we'll only walk through an example to see how it can be used with the MusicBrainz data set. In short, the DIH offers the following capabilities: Imports data from databases through JDBC (Java Database Connectivity) Imports XML data from a URL (HTTP GET) or a file Can combine data from different tables or sources in various ways Extraction/Transformation of the data Import of updated (delta) data from a database, assuming a last-updated date A diagnostic/development web page Extensible to support alternative data sources and transformation steps As the MusicBrainz data is in a database, the most direct method to get data into Solr is definitely through the DIH using JDBC. Getting started with DIH DIH is not a direct part of Solr. Hence it might not be included in your Solr setup. It amounts to a JAR file named something like apache-solr-dataimporthandler-1.4.jar, which is probably already embedded within the solr.war file. You can use jar -tf solr.war to see. Alternatively, it may be placed in <solr-home>/lib, which is alongside the conf directory we've been working with. For database connectivity, we need to ensure that the JDBC driver is on the Java classpath. Placing it in <solr-home>/lib is a convenient way to do this. The DIH needs to be registered with Solr in solrconfig.xml. Here is how it is done: <requestHandler name="/dataimport"class="org.apache.solr.handler.dataimport.DataImportHandler"><lst name="defaults"><str name="config">mb-dih-artists-jdbc.xml</str></lst></requestHandler> mb-dih-artists-jdbc.xml (mb being short for MusicBrainz) is a file in <solr-home>/conf, which is used to configure DIH. It is possible to specify some configuration aspects in this request handler configuration instead of the dedicated configuration file. However, I recommend that it all be in the DIHconfig file, as in our example here. Given below is an mb-dih-artists-jdbc.xml file with a rather long SQL query: <dataConfig> <dataSource name="jdbc" driver="org.postgresql.Driver" url="jdbc:postgresql://localhost/musicbrainz_db" user="musicbrainz" readOnly="true" autoCommit="false" /> <document> <entity name="artist" dataSource="jdbc" pk="id" query=" select a.id as id, a.name as a_name, a.sortname as a_name_sort, a.begindate as a_begin_date, a.enddate as a_end_date, a.type as a_type ,array_to_string( array(select aa.name from artistalias aa where aa.ref = a.id ) , '|') as a_alias ,array_to_string( array(select am.name from v_artist_members am where am.band = a.id order by am.id) , '|') as a_member_name ,array_to_string( array(select am.id from v_artist_members am where am.band = a.id order by am.id) , '|') as a_member_id, (select re.releasedate from release re inner join album r on re.album = r.id where r.artist = a.id order by releasedate desc limit 1) as a_release_date_latest from artist a " transformer="RegexTransformer,DateFormatTransformer, TemplateTransformer"> <field column = "id" template="Artist:${artist.id}" /> <field column = "type" template="Artist" /> <field column = "a_begin_date" dateTimeFormat="yyyy-MM-dd" /> <field column = "a_end_date" dateTimeFormat="yyyy-MM-dd" /> <field column = "a_alias" splitBy="|" /> <field column = "a_member_name" splitBy="|"/> <field column = "a_member_id" splitBy="|" /> </entity> </document></dataConfig> The DIH development console Before describing the configuration details, we're going to take a look at the DIH development console. It is accessed by going to this URL (modifications may be needed for your host, port, core, and so on):http://localhost:8983/solr/admin/dataimport.jsp The development console looks like the following screenshot: The screen is divided into two panes: on the left is the DIH control form, which includes an editable version of the DIH configuration file and on the right is the command output as raw XML. The screen works quite simply. The form essentially results in submitting a URL to the right pane. There's no real server-side logic to this interface beyond the standard DIH command invocations being executed on the right. The last section on DIH in this article goes into more detail on submitting a command to the DIH. DIH DataSources of type JdbcDataSource The DIH configuration file starts with the declaration of one or more data sources using the element <dataSource/>, which refers to either a database, a file, or an HTTP URL, depending on the type attribute. It defaults to a value of JdbcDataSource. Those familiar with JDBC should find the driver and url attributes with accompanying user and password straightforward—consult the documentation for your driver/database for further information. readOnly is a boolean that will set a variety of other JDBC options appropriately when set to true. And batchSize is an alias for the JDBC fetchSize and defaults to 500. There are numerous JDBC oriented attributes that can be set as well. I would not normally recommend learning about a feature by reading source code, but this is an exception. For further information, read org.apache.solr.handler.dataimport.JdbcDataSource.java Efficient JDBC configuration Many database drivers in the default configurations (including those for PostgreSQL and MySQL) fetch all of the query results into the memory instead of on-demand or using a batch/fetch size. This may work well for typical database usage like OLTP (Online Transaction Processing systems), but is completely unworkable for ETL (Extract Transform and Load) usage such as this. Configuring the driver to stream the data requires driver-specific configuration options. You may need to consult relevant documentation for the JDBC driver. For PostgreSQL, set autoCommit to false. For MySQL, set batchSize to -1(The DIH detects the -1 and replaces it with Java's Integer.MIN_VALUE, which triggers the special behavior in MySQL's JDBC driver). For Microsoft SQL Server, set responseBuffering to adaptive. Further information about specific databases is at :http://wiki.apache.org/solr/DataImportHandlerFaq.. DIH documents, entities After the declaration of <dataSource/> element(s) is the <document/> element. In turn, this element contains one or more <entity/> elements. In this sample configuration, we're only getting artists. However, if we wanted to have more than one type in the same index, then another could be added. The dataSource attribute references a correspondingly named element earlier. It is only necessary if there are multiple to choose from, but we've put it here explicitly anyway. The main piece of an entity used with a JDBC data source is the query attribute, which is the SQL query to be evaluated. You'll notice that this query involves some sub-queries, which are made into arrays and then transformed into strings joined by spaces. The particular functions used to do these sorts of things are generally database specific. This is done to shoe-horn multi-valued data into a single row in the results. It may create a more complicated query, but it does mean that the database does all of the heavy lifting so that all of the data Solr needs for an artist is in the row. An alternative with DIH is to declare other entities within the entity. If you aren't using a database or if you wish to mix in another data source (even if it's of a different type), then you will be forced to do that. See the Solr DIH Wiki page for examples: http://wiki.apache.org/solr/DataImportHandler. The DIH also supports a delta query, which is a query that selects time-stamped data with dates after the last queried date. This won't be covered here, but you can find more information at the previous URL. DIH fields and transformers Within the <entity/> are some <field/>elements that declare how the columns in the query map to Solr. The field element must have a column attribute that matches the corresponding named column in the SQL query. The name attribute is the Solr schema field name that the column is going into. If it is not specified (and it never is for our example), then it defaults to the column name. Use the SQL as a keyword as we've done to use the same names as the Solr schema instead of the database schema. This reduces the number of explicit mappings needed in <field/> elements and shortens existing ones. When a column in the result can be placed directly into Solr without further processing, there is no need to specify the field declaration, because it is implied. An attribute of the entity declaration that we didn't mention yet is transformer. This declares a comma-separated list of transformers that manipulate the transfer of data from the JDBC resultset into a Solr field. These transformers evaluate a field, if it has an attribute it uses to do its job. More than one might operate on a given field. Therefore, the order in which the transformers are declared in matters. Here are the attributes we've used: template: It is used by TemplateTransformer and declares text, which might include variable name substitutions using ${name} syntax. To access an existing field, use the entityname.columnname syntax. splitBy: It is used by RegexTransformer and splits a single string value into a multi-value by looking for the specified character. dateTimeFormat: It is used by DateFormatTransformer. This is a Java date/time format pattern http://java.sun.com/j2se/1.5.0/docs/api/java/text/SimpleDateFormat.html). If the type of the field in the schema is a date, then it is necessary to ensure Solr can interpret the format. Alternatively, ensure that the string matches the ISO-8601 format, which looks like this: 1976-10-23T23:59:59.000Z. As in all cases in Solr, when specifying dates you can use its so-called "DateMath" syntax such as appending /DAY to tell Solr to round the date to a day.

At the end of this article, we will be able to: Identify the purpose of the several downloads that can be found at the JasperReports web site Set up our development and execution environment to successfully add reporting capabilities to our Java applications Identify the required libraries for adding reporting capabilities to Java applications Identify the optional libraries that can be used to enhance the reporting capabilities of our Java applications Downloading JasperReports JasperReports is distributed as a JAR file that needs to be added to the CLASSPATH of any application we wish to add reporting capabilities to. JasperReports can be downloaded from http://jasperforge.org/plugins/project/project_home.php?group_id=102. Clicking on the Download link around the center of the page will take us to the JasperReports download page on SourceForge.net. The specific version on your screen might be different; at the time of writing, the latest stable version of JasperReports is 3.5.2. It is not always clear what exactly is downloaded by clicking on these links; thus, we will provide a brief explanation of what each link is for. jasperreports-3.5.2-applet.jar: This first download link is for a JAR file containing a subset of the JasperReports functionality. Specifically, it contains classes that can be used to display JasperPrint files, which are reports in JasperReports' native format. This file is offered as a separate download as a convenience for developers; it can be used for applications or applets that don't need full reporting capabilities, yet need to display generated reports. Even though the filename has a suffix of "applet", there is nothing preventing us from using it with standalone applications, without the overhead that the missing JasperReports classes would add to the download. This file is approximately 346 KB in size. jasperreports-3.5.2.jar: This is the second download link, and it is the complete JasperReports class library. It contains all classes necessary to compile, fill, and export reports, but does not include any additional libraries that JasperReports depends on. This is the minimum file requirement for us to add full reporting capabilities to our Java applications. However, if we choose to download this file, we need to download the JasperReports dependencies separately. This file is approximately 2.2 MB in size. jasperreports-3.5.2-javaflow.jar: This is the third download link, and it is the javaflow version of JasperReports. This version of JasperReports should be used in environments where multithreading is discouraged. This file is approximately 2.2 MB in size. jasperreports-3.5.2-project.tar.gz: This is the fourth download link, and it contains the complete JasperReports class library plus all the required and optional libraries. In addition, it contains the entire file as a gzipped TAR file, which is common in Unix and Unix-like systems, such as Linux. This download also includes the JasperReports source code and a lot of source code providing examples of JasperReports' functionality. This gzipped TAR file is approximately 42 MB in size jasperreports-3.5.2-project.zip: The fifth download link, like the fourth link, contains the complete JasperReports class library plus all the required and optional libraries, along with the JasperReports source code. However, it contains the files in ZIP format, which is more commonly used under Microsoft Windows. This file is approximately 51 MB in size. Unless Internet connection speed is an issue, we recommend downloading one of the last two mentioned files, as they include everything we need to create reports with JasperReports. Another good reason to download one of these files is that the included examples are a great way to learn how to implement the different JasperReports features. All of the examples in the file come with an ANT build file containing targets to compile and execute them. We will refer to this file as the JasperReports project file, or more succinctly, as the project file. Once we have downloaded the appropriate file for our purposes, we need to set up our environment to be able to start creating reports. In the next section, we discuss how to do this, assuming that the project file was downloaded. Setting up our environment To set up our environment to get ready for creating reports, we need to extract the JasperReports project ZIP file to a location of our choice. Once we extract the project ZIP file, we should see a jasperreports-3.5.2 directory (The actual name of the directory may vary slightly depending on the version of JasperReports.) containing the following files and directories: build: This directory contains the compiled JasperReports class files. build.xml: This is an ANT build file, which builds the JasperReports source code. If we don't intend to modify JasperReports, we don't need to use this file as JasperReports is distributed in compiled form. changes.txt: This file explains the differences between the current and previous versions of the JasperReports' class library. demo: This directory contains various examples demonstrating several aspects of JasperReports functionality. dist: This directory contains a JAR file containing the standard, javaflow, and applet versions of the JasperReports library. We should add one of these JAR files to our CLASSPATH to take advantage of JasperReports functionality. docs: This directory contains a quick reference guide to most XML tags used in JasperReports templates. lib: This directory contains all the libraries needed to build JasperReports and to use it in our applications. license.txt: This file contains the full text of the LGPL license. pom.xml: This is a Maven 2 POM file used to build JasperReports with Maven, just like build.xml. We don't need this file because JasperReports is distributed in compiled form. readme.txt: This file contains instructions on how to build and execute the supplied examples. src: This directory contains the JasperReports source code. Getting up and running quicklyTo get up and run quickly, the files to add to the CLASSPATH are the JasperReports JAR files, and all the JAR files under the lib directory in the project ZIP file. By adding these files to the CLASSPATH, we don't have to worry about the CLASSPATH when implementing additional functionality, for example, when exporting to PDF or producing charts. JasperReports class library For all JasperReports-related tasks, we need to add the JasperReports library to our CLASSPATH. The JasperReports library can be found under the dist directory in the project file; it is named jasperreports-3.5.2.jar. Depending on the version of JasperReports, the filename will vary. Required libraries for report compilation The project file described earlier contains all of the required supporting libraries. Once that file is downloaded, all the required libraries can be found under the lib subdirectory of the directory created when extracting the ZIP file. JasperReports uses these required files for XML parsing. Therefore, they are needed when compiling JRXML files programmatically, not for filling or displaying reports. JRXML files can be compiled using a custom ANT task provided by JasperReports. If we choose to compile our JRXML files using this custom ANT target, these required libraries need to be added to the CLASSPATH variable of the ANT build file. There are example build files included in the project file and also at http://www.packtpub.com/files/code/8082_Code.zip. The following discussion about libraries highlights the fact that JasperReports makes extensive use of Apache Commons. Apache Commons Digester The Commons Digester library includes utility classes used to initialize Java objects from XML files. JasperReports takes advantage of the Digester component of the Apache Commons repository to implement its XML parsing functionality. Version 3.5.2 of the JasperReports project ZIP file includes version 1.7 of Commons Digester. The filename is commons-digester-1.7.jar, and it must be on your CLASSPATH for your JasperReports application to work correctly. If you downloaded the bare JasperReports class library, you will need to download Commons Digester separately from http://commons.apache.org/digester/. Apache Commons Collections Another component of the Apache Commons suite is the Collections component. This component provides functionality to complement and augment the Java Collections framework. JasperReports takes advantage of the Collections component of Apache Commons to implement some of its functionality. Like all required libraries included in the project ZIP file, the Commons Collections library can be found under the lib subdirectory of the directory created when extracting the project ZIP file. JasperReports project file version 3.5.2 includes version 2.1 of Commons Collections, distributed as a JAR file named commons-collections-2.1.jar. If you have downloaded the bare JasperReports class library, you will need to download Commons Collections separately from http://commons.apache.org/collections/. Apache Commons Logging Apache Commons Logging is a component of the Apache Commons that provides components to aid developers with sending data to log files. JasperReports takes advantage of this component, which can be found on the lib directory of the project ZIP file. The version included with JasperReports 3.5.2 is Commons Logging 1.0.2. The file to be added to your CLASSPATH is commons-logging-1.0.2.jar. If you have downloaded the bare JasperReports class library, you will need to download Commons Logging separately from http://commons.apache.org/logging/. Apache Commons BeanUtils The last library that JasperReports requires for compiling reports is Apache Commons BeanUtils. BeanUtils is a library that provides easy-to-use wrappers around the Java reflection and introspection APIs. Version 3.5.2 of the JasperReports project ZIP file includes BeanUtils 1.8; the file to add to your CLASSPATH is commons-beanutils-1.8.jar. If you have downloaded the bare JasperReports class library, you will need to download Commons BeanUtils separately from http://commons.apache.org/beanutils/. Optional libraries and tools There are a number of libraries that are required only if we wish to take advantage of some of JasperReports' features. These optional libraries and their uses are listed next. Apache ANT JasperReports comes bundled with some custom ANT targets for previewing report designs and for viewing reports serialized in JasperReports' native format. Although not strictly necessary, it is very helpful to have ANT available to take advantage of these custom targets. ANT can be downloaded from http://ant.apache.org/. JDT compiler JDT (Java Development Tools) compiler is the Java compiler included with the Eclipse IDE. The JDT compiler is needed only if the JasperReports application is running under a Java Runtime Environment (JRE) and not under a full JDK. When creating reports, JasperReports creates temporary Java files and compiles them. When using a JDK, JasperReports takes advantage of tools.jar for this functionality. As a JRE does not include tools.jar, the JDT compiler is needed. The JasperReports project file version 3.5.2 includes version 3.1.1 of the JDT compiler. It can be found under the lib directory of the directory created when extracting the project ZIP file. The file to add to your CLASSPATH is jdt-compiler-3.1.1.jar. This file cannot be downloaded separately; therefore, if we need to execute our code under a JRE. We need to download the JasperReports project ZIP file because it includes this file needed for report compilation. JDBC driver When using a JDBC datasource, which is the most common datasource for JasperReports generated reports, the appropriate JDBC driver for our specific RDBMS is needed. The following table lists popular relational database systems and the required JAR files to add to the CLASSPATH. The exact filenames may vary depending on the version, target JDK, and supported JDBC version. The filenames shown here reflect the latest stable versions targeted to the latest available JDK and the latest available version of JDBC at the time of writing. RDBMS Driver JAR Files Firebird jaybird-2.1.6.jar HSQLDB hsqldb.jar JavaDB/Derby (included with JDK 1.6+) derby.jar (embedded) derbyclient.jar (network) MySQL mysql-connector-java-5.1.7-bin.jar Oracle ojdbc6.jar PostgreSQL postgresql-8.3-604.jdbc4.jar SQL Server sqljdbc_1.2.2828.100_enu.exe (for Windows systems) sqljdbc_1.2.2828.100_enu.tar.gz (for Unix systems) Sybase Jconnect60.zip   The JasperReports project file includes the JDBC driver for HSQLDB. Consult your RDBMS documentation for information on where to download the appropriate JDBC driver for your RDBMS. iText iText is an open source library for creating and manipulating PDF files. It is needed in our CLASSPATH only if we want to export our reports to PDF or RTF format. Version 3.5.2 of the JasperReports project file includes iText version 2.1.0; the file to add to your CLASSPATH is iText-2.1.0.jar. The iText library can be downloaded separately from http://www.lowagie.com/iText/. JFreeChart JFreeChart is an open source library for creating professional looking charts, including 2D and 3D pie charts, 2D and 3D bar charts, and line charts. It is needed in our CLASSPATH only if we intend to add charts to our reports. JFreeChart 1.0.12 can be found on the lib directory inside the JasperReports project file version 3.5.2. The file to add to the CLASSPATH is jfreechart-1.0.12.jar. JFreeChart can be downloaded separately from http://www.jfree.org/jfreechart/. JExcelApi JExcelApi is a Java library that allows Java applications to read, write, and modify Microsoft Excel files. We need JExcelApi in our CLASSPATH only if we need to export our reports to XLS format. JasperReports 3.5.2 includes JExcelApi version 2.6. To add XLS exporting capabilities to our reports, the file we need to add to our CLASSPATH is jxl-2.6.jar. JExcelApi can be downloaded separately from http://jexcelapi.sourceforge.net/. Summary This article covered the required and optional libraries needed to add reporting capabilities to Java applications. All the libraries covered in this article are needed at both compile time and runtime. The article provided an explanation of the different files available for downloading on JasperReports' web site and in which conditions it is appropriate to use them. We also saw which libraries are required for report compilation under a JDK and the additional libraries required when compiling JRXML templates under a JRE. Besides, we also learned which libraries are required when using JDBC datasources for our reports, and finally the libraries required when exporting our reports to several formats.

(For more resources on Solr, see here.) Faceting, after searching, is arguably the second-most valuable feature in Solr. It is perhaps even the most fun you'll have, because you will learn more about your data than with any other feature. Faceting enhances search results with aggregated information over all of the documents found in the search to answer questions such as the ones mentioned  below, given a search on MusicBrainz releases: How many are official, bootleg, or promotional? What were the top five most common countries in which the releases occurred? Over the past ten years, how many were released in each year? How many have names in these ranges: A-C, D-F, G-I, and so on? Given a track search, how many are < 2 minutes long, 2-3, 3-4, or more? Moreover, in addition, it can power term-suggest aka auto-complete functionality, which enables your search application to suggest a completed word that the user is typing, which is based on the most commonly occurring words starting with what they have already typed. So if a user started typing siamese dr, then Solr might suggest that dreams is the most likely word, along with other alternatives. Faceting, sometimes referred to as faceted navigation, is usually used to power user interfaces that display this summary information with clickable links that apply Solr filter queries to a subsequent search. If we revisit the comparison of search technology to databases, then faceting is more or less analogous to SQL's group by feature on a column with count(*). However, in Solr, facet processing is performed subsequent to an existing search as part of a single request-response with both the primary search results and the faceting results coming back together. In SQL, you would need to potentially perform a series of separate queries to get the same information. A quick example: Faceting release types Observe the following search results. echoParams is set to explicit (defined in solrconfig.xml) so that the search parameters are seen here. This example is using the standard handler (though perhaps dismax is more typical). The query parameter q is *:*, which matches all documents. In this case, the index I'm using only has releases. If there were non-releases in the index, then I would add a filter fq=type%3ARelease to the URL or put this in the handler configuration, as that is the data set we'll be using for most of this article. I wanted to keep this example brief so I set rows to 2. Sometimes when using faceting, you only want the facet information and not the main search, so you would set rows to 0, if that is the case. It's important to understand that the faceting numbers are computed over the entire search result, which is all of the releases in this example, and not just the two rows being returned. <?xml version="1.0" encoding="UTF-8"?> <response> <lst name="responseHeader"> <int name="status">0</int> <int name="QTime">160</int> <lst name="params"> <str name="wt">standard</str> <str name="rows">2</str> <str name="facet">true</str> <str name="q">*:*</str> <str name="fl">*,score</str> <str name="qt">standard</str> <str name="facet.field">r_official</str> <str name="f.r_official.facet.missing">true</str> <str name="f.r_official.facet.method">enum</str> <str name="indent">on</str> </lst> </lst> <result name="response" numFound="603090" start="0" maxScore="1.0"> <doc> <float name="score">1.0</float> <str name="id">Release:136192</str> <str name="r_a_id">3143</str> <str name="r_a_name">Janis Joplin</str> <arr name="r_attributes"><int>0</int><int>9</int> <int>100</int></arr> <str name="r_name">Texas International Pop Festival 11-30-69</str> <int name="r_tracks">7</int> <str name="type">Release</str> </doc> <doc> <float name="score">1.0</float> <str name="id">Release:133202</str> <str name="r_a_id">6774</str> <str name="r_a_name">The Dubliners</str> <arr name="r_attributes"><int>0</int></arr> <str name="r_lang">English</str> <str name="r_name">40 Jahre</str> <int name="r_tracks">20</int> <str name="type">Release</str> </doc> </result> <lst name="facet_counts"> <lst name="facet_queries"/> <lst name="facet_fields"> <lst name="r_official"> <int name="Official">519168</int> <int name="Bootleg">19559</int> <int name="Promotion">16562</int> <int name="Pseudo-Release">2819</int> <int>44982</int> </lst> </lst> <lst name="facet_dates"/> </lst> </response> The facet related search parameters are highlighted at the top. The facet.missing parameter was set using the field-specific syntax, which will be explained shortly. Notice that the facet results (highlighted) follow the main search result and are given a name facet_counts. In this example, we only faceted on one field, r_official, but you'll learn in a bit that you can facet on as many fields as you desire. The name attribute holds a facet value, which is simply an indexed term, and the integer following it is the number of documents in the search results containing that term, aka a facet count. The next section gives us an explanation of where r_official and r_type came from. MusicBrainz schema changes In order to get better self-explanatory faceting results out of the r_attributes field and to split its dual-meaning, I modified the schema and added some text analysis. r_attributes is an array of numeric constants, which signify various types of releases and it's official-ness, for lack of a better word. As it represents two different things, I created two new fields: r_type and r_official with copyField directives to copy r_attributes into them: <field name="r_attributes" type="integer" multiValued="true" indexed="false" /><!-- ex: 0, 1, 100 --> <field name="r_type" type="rType" multiValued="true" stored="false" /><!-- Album | Single | EP |... etc. --> <field name="r_official" type="rOfficial" multiValued="true" stored="false" /><!-- Official | Bootleg | Promotional --> And: <copyField source="r_attributes" dest="r_type" /> <copyField source="r_attributes" dest="r_official" /> In order to map the constants to human-readable definitions, I created two field types: rType and rOfficial that use a regular expression to pull out the desired numbers and a synonym list to map from the constant to the human readable definition. Conveniently, the constants for r_type are in the range 1-11, whereas r_official are 100-103. I removed the constant 0, as it seemed to be bogus. <fieldType name="rType" class="solr.TextField" sortMissingLast="true" omitNorms="true"> <analyzer> <tokenizer class="solr.KeywordTokenizerFactory"/> <filter class="solr.PatternReplaceFilterFactory" pattern="^(0|1dd)$" replacement="" replace="first" /> <filter class="solr.LengthFilterFactory" min="1" max="100" /> <filter class="solr.SynonymFilterFactory" synonyms="mb_attributes.txt" ignoreCase="false" expand="false"/> </analyzer> </fieldType> The definition of the type rOfficial is the same as rType, except it has this regular expression: ^(0|dd?)$. The presence of LengthFilterFactory is to ensure that no zero-length (empty-string) terms get indexed. Otherwise, this would happen because the previous regular expression reduces text fitting unwanted patterns to empty strings. The content of mb_attributes.txt is as follows: # from: http://bugs.musicbrainz.org/browser/mb_server/trunk/ # cgi-bin/MusicBrainz/Server/Release.pm#L48 #note: non-album track seems bogus; almost everything has it 0=>Non-Album Track 1=>Album 2=>Single 3=>EP 4=>Compilation 5=>Soundtrack 6=>Spokenword 7=>Interview 8=>Audiobook 9=>Live 10=>Remix 11=>Other 100=>Official 101=>Promotion 102=>Bootleg 103=>Pseudo-Release It does not matter if the user interface uses the name (for example: Official) or constant (for example: 100) when applying filter queries when implementing faceted navigation, as the text analysis will let the names through and will map the constants to the names. This is not necessarily true in a general case, but it is for the text analysis as I've configured it above. The approach I took was relatively simple, but it is not the only way to do it. Alternatively, I might have split the attributes and/or mapped them as part of the import process. This would allow me to remove the multiValued setting in r_official. Moreover, it wasn't truly necessary to map the numbers to their names, as a user interface, which is going to present the data, could very well map it on the fly. Field requirements The principal requirement of a field that will be faceted on is that it must be indexed. In addition to all but the prefix faceting use case, you will also want to use text analysis that does not tokenize the text. For example, the value Non-Album Track is indexed the way it is in r_type. We need to be careful to escape the space where this appeared in mb_attributes.txt. Otherwise, faceting on this field would show tallies for Non-Album and Track separately. Depending on the type of faceting you want to do and other needs you have like sorting, you will often find it necessary to have a copy of a field just for faceting. Remember that with faceting, the facet values returned in search results are the actual terms indexed, and not the stored value, which isn't even used.

A couple of days ago, a client asked me for an easy way to develop PHP applications. Naturally, I told him the best way would be to pay me for doing it! Just kidding… "Use the NetBeans IDE", I said to him, almost automatically. "But… isn’t NetBeans something related to Java?" he answered back, with a startled grin on his face. "My dear friend, you can use NetBeans to develop software on Java, PHP, C++, and almost any other programming language I’ve heard of! You could even use it to work on a WordPress live Web site!" I said to him triumphantly. And that’s when a small light bulb lit up inside my mind... I introduced him to VirtualBox and the world of virtual machines, the Turnkey Linux LAMP appliance, and how to make all these software applications collaborate between each other. And that’s how this article was born. So now, my dear readers, let’s get on with the action! Oh, and feel free to skip any section you don’t need, ok? Download and Install VirtualBox Go to the VirtualBox Web site and download the most recent version: http://download.virtualbox.org/virtualbox/3.0.4/VirtualBox-3.0.4-50677-Win.exe After downloading VirtualBox, install it with all the default options (remember to register with Sun!). Download the Turnkey LAMP appliance Go to the Turnkey Linux Web site and download the LAMP appliance: http://www.turnkeylinux.org/download?file=turnkey-lamp-2009.02-hardy-x86.iso . Download the NetBeans PHP IDE Go to the Sun Web site and download the NetBeans PHP IDE here. After downloading NetBeans, install it with the default options. Create a virtual machine Open VirtualBox and click on the New button to create a virtual machine (VM). In the following screenshot I used the VirtualPHPDev name for my VM, but you can use whatever you want; just don’t use strange characters or signs: Choose Linux and Ubuntu as the Operating System and Version, respectively. Click on Next to continue. Leave the default values for RAM and hard disk. When finished, click on the VirtualBox Settings button to open your virtual machine’s settings dialog, select the CD/DVD-ROM category, enable the Mount CD/DVD drive option, select the ISO Image File button and then click on the Invoke Virtual Media Manager button . Click on the Add button in the Virtual Media Manager to open up the Select a CD/DVD-ROM disk image file, go to the directory where you downloaded the Turnkey LAMP appliance and double-click on it to add it to the Virtual Media Manager: Click on Select to close the Virtual Media Manager, and then click on OK to exit the Settings dialog. Now you can click on the Start button to start your LAMP virtual machine: Select the Install to hard disk option in the Turnkey Linux menu screen and press Enter to start installing the LAMP appliance on your virtual machine. Eventually the following screen will show up: Select the Guided option and press Enter to continue. The installer will ask if you want to save changes to disk. Select Yes and press Enter. The installer will start the disk formatting process and then you’ll get to the root password screen. Type the password twice for the root user, and then do the same for the MySQL user. The installer will continue and, after a while, the following installation completion screen will show up: Select No, press Enter and then close the virtual machine typing shutdown -r now at the system prompt. You’ll return to the VirtualBox main screen. With your LAMP virtual machine selected, click on the Settings button to open the settings dialog box. Select the CD/DVD-ROM category and disable the Mount CD/DVD drive option. Then select the Network category, change the Attached to: option to Bridged Adapter and click on OK to continue: You can start your LAMP virtual machine now. When ready, the following screen will appear: The Bridged Adapter mode lets your virtual machine act as if it were another PC on the LAN, and consequently, it will have a different IP address. Write down the IP address shown in the Turnkey Linux Configuration Console screen, because you’ll need it to configure the NetBeans IDE later. You can minimize the LAMP virtual machine window now.

Editing load tests The load can contain one or more scenarios for testing. The scenarios can be edited any time during the design. To edit a scenario, select the scenario you want to edit and right-click to edit the test mix, browser mix, or network mix in the existing scenario or add a new scenario to the load test. The context menu has different options for editing as shown here: The Add Scenario... will open the same wizard, which we used before adding the scenario to the load test. We can keep adding the scenarios as much as we need. The scenario properties window also helps us modify some properties such as the Think Profile and the Think Time Between Test Iteration and the scenario Name. The Add Tests... option is used for adding more tests to the test mix from the tests list in the project. We can add as many tests as required to be part of the test mix. The Edit Test Mix... option is used for editing the test mix in the selected scenario. This option will open a dialog with the selected tests and distribution. Using this Edit Test Mix window we can: Change the test mix model listed in the drop–down. Add new tests to the list and modify the distribution percentage. Select an initial test that executes before other tests for each virtual server. The browse option next to it opens a dialog showing all the tests from the project from which we can select the initial test. Similar to the initial test, we can choose a test which is the final test to run during the test execution. The same option is used here to select the test from the list of available tests. The Edit Browser Mix... option opens the Edit Browser Mix dialog from where you can select the new browser to be included to the browser mix and delete or change the existing browsers selected in the mix. The Edit Network Mix... option opens the Edit Network Mix dialog from where you can add new browsers to the list and modify the distribution percentages. We can change or delete the existing network mix. For changing the existing load pattern, select the Load Pattern under the Scenarios and open the Properties window which shows the current patterns properties. You can change or choose any pattern from the available patterns in the list as shown in the following screenshots: The Run Settings can be multiple for the load tests, but at any time only one can be active. To make the run settings active, select Run Settings, right-click and select Set as Active. The properties of the run settings can be modified directly using the properties window. The properties that can be modified include results storage, SQL tracing, test iterations, timings, and the web test connections. Adding context parameters The web tests can have context parameters added to them. The context parameter is used in place of the common values of multiple requests in the web test. For example, every request has the same web server name, which can be replaced by the context parameters. So whenever the web server changes, we can just change the context parameter value, which replaces all the requests with the new server name. We know that the load test can have web tests and unit tests in the list. If there is a change in the web server for the load test other than what we used for web tests then we will end up modifying the context parameter values in all the web tests used in the load tests. Instead of this, we can include another context parameter in the load test with the same name used in the web tests. The context parameter added to the load test will override the same context parameter used in the web tests. To add new context parameter to the load test, select the Run Settings and right-click to choose the Add Context Parameter option, which adds a new context parameter. For example, the context parameter used in the web test has the web server value as this.Context.Add("WebServer1", "http://localhost:49459"); Now to overwrite this in load tests, add a new context parameter with the same name as shown below: Results store All information collected during the load test run is stored in the central result store. The load test results store contains the data collected by the performance counters and the violation information and errors that occurred during the load test. The result store is the SQL server database created using the script loadtestresultsrepository.sql, which contains all the SQL queries to create the objects required for the result store. If there are no controllers involved in the test and if it is the local test, we can create the results store sql database using SQL Express. Running the script creates the store using SQL Express. Running this script once on the local machine is enough for creating the result store. This is a global central store for all the load tests in the local machine. To create the store, open the visual studio command prompt and run the command with the actual drive where you have installed the visual studio. cd c:Program FilesMicrosoft Visual Studio 9.0Common7IDE In the same folder, run the following command which creates the database store SQLCMD /S localhostsqlexpress /i loadtestresultsrepository.sql If you have any other SQL Server and if you want to use that to have the result store then you can run the script on that server and use that server in connection parameters for the load test. For example, if you have the SQL Server name as SQLServer1 and if the result store has to be created in that store, then run the command as below: SQLCMD /S SQLServer1 -U <user name> -P <password> -iloadtestresultsrepository.sql All these commands create the result store database in the SQL Server. If you look at the tables created in the store, it would look like this: If you are using a controller for the load tests, the installation of the controller itself takes care of creating the results store on the controller machine. The controller can be installed using the Visual Studio 2008 Team Test Load agent Product. To connect to the SQL Server result store database, select the Test option from the Visual Studio IDE and then select the Administer Test Controller window. This option would be available only in the controller machine. If the result store is on a different machine or the controller machine, select the controller from the list or select <Local-No controller>, if it is in the local machine without any controller. Then select the Load Test Results store using the browse button and close the Administer test Controller window. The controllers are used for administering the agent computers and these controller and agents form the rig. Multiple agents are required to simulate a large number of loads from different locations. All the performance data collected from all these agents are saved at the central result store at the controller or any global store configured at the controller. Running the load test Load tests are run like any other test in Visual Studio. Visual Studio also provides multiple options for running the load test. One is through the Test View window where all the tests are listed. We can select the load test, right-click and choose the option Run Selection option, which starts the load tests to run. The second option is to use the Test List Editor. Select the load test from the test list in the test lists editor and choose the option to run the selected tests from the test list editor toolbar. The third option is the built-in run option in the load test editor toolbar. Select the load test from the project and open the load test. This opens the load test in the load test editor. The toolbar for this load test editor has the option to run the currently opened load test. The fourth option is through the command line command. MS Test command line utility is used for running the test. This utility is installed along with the Visual Studio Team System for Test. Open the Visual Studio command prompt. From the folder where the load test resides, run the following command to start the load test mstest /testcontainer:LoadTest1.loadtest In all the above cases, the load test editor will show the progress during the test run. But the other option does not show the progress instead stores the result to the result store repository. It can be loaded later to see the test result and analyze it. You can follow these steps to open the last run tests: Open the menu option Test | Windows | Test Runs. From the Connect drop-down, select the location for the test results store. On selecting this, you can see the trace files of the last run tests getting loaded in the window. Select the test run name from the list and double-click to open the test results for the selected run. Double-click the test result shown in the Results window that connects to the store repository, fetches the data for the selected test result, and presents in the load test window. The end result of the load test editor window will look like the one shown in the following screenshot with all the performance counter values and the violation points. More details about the graph are given under the Graphical View subsection.

Data access applications We have just deployed an application that did not require database connectivity. Often, applications in the business world require access to a RDBMS to fulfill their business objective. If an application requires the ability to retrieve from, or store information in, a database, then you will need to create a data source which will allow the application to connect and use the database (DB). Looking at the figure below, we can see the logical flow of the sample data access application that we are going to install. The basic idea of the application is to display a list of tables that exist in a database schema. Since the application requires a database connection, we need to configure WebSphere before we can deploy the application. We will now cover the preparation work before we install our application. Data sources Each data source is associated with a JDBC provider that is configured for access to a specific database type. The data source provides connectivity which allows an application to communicate with the database. Preparing our sample database Before you create a data source, you need to ensure that the appropriate client database driver software is installed. For our demonstration, we are going to use Oracle Express Edition (Oracle XE) for Linux which is the free version of Oracle. We are using version Oracle XE 10g for Linux and the download size is about 210MB, so it will take time to download. We installed Oracle XE using the default install option for installing an RPM. The administration process is fully documented on Oracle's web site and in the documentation which is installed with the product. We could have chosen to use many open source/free databases, however their explanations and configurations would detract from the point. We have chosen to use Oracle's free RDBMS called Oracle XE, and JDBC with Oracle XE is quite easy to configure. By following these steps, you will be able to apply the same logic to any of the major vendors' full RDMS products, that is, DB/2, Oracle, SQL Server, and so on. Another reason why we chose Oracle XE is that it is an enterprise-ready DB and is administered by a simple web interface and comes with sample databases. We need to test that we can connect to our database without WebSphere so that we can evaluate the DB design. To do this, we will need to install Oracle XE. We will now cover the following steps one by one. Download Oracle XE from Oracle's web site using the following URL:http://www.oracle.com/technology/products/database/xe/index.html. Transfer the oracle-xe-10.2.0.1-1.0.i386.rpm file to an appropriate directory on your Linux server using WinSCP (Secure Copy) or your chosen Secure FTP client. Since the XE installer uses X Windows, ensure that you have Xming running. Then install Oracle XE by using the rpm command, as shown here: rpm -ivh oracle-xe-10.2.0.1-1.0.i386.rpm Follow the installer steps as prompted: HTTP port = 8080 Listener port = 1521 SYS & SYSTEM / password = oracle Autostart = y Oracle XE requires 1024 minimum swap space and requires 1.5 GB of disk space to install. Ensure that Oracle XE is running. You can now access the web interface via a browser from the local machine; by default, XE will only accept a connection locally. As shown in the following figure, we have a screenshot of using Firefox to connect to OracleXE using the URL http://localhost:8080/apex. The reason we use Firefox on Linux is that this is the most commonly installed default browser on the newer Linux distributions. When the administration application loads, you will be presented with a login screen as seen in the following screenshot. You can log in using the username SYSTEM and password oracle as set by your installation process. Oracle XE comes with a pre-created user called HR which is granted ownership to the HR Schema. However, the account is locked by default for security reasons and so we need to unlock the HR user account. To unlock an account, we need to navigate to the Database Users | Manage Users screen, as demonstrated in the following screenshot: You will notice that the icon for the HR user is locked. You will see a small padlock on the HR icon, as seen in this figure: Click on the HR user icon and unlock the account as shown in the following figure. You need to reset the password and change Account Status to Unlocked, and then click Alter User to set the new password. The following figure shows that the HR account is unlocked: The HR account is now unlocked as seen above. Log out and log back into the administration interface using the HR user to ensure that the account is now unlocked. Another good test to perform to ensure connectivity to Oracle is to use an Oracle admin tool called sqlplus. Sqlplus is a command line tool which database administrators can use to administer Oracle. We are going to use sqlplus to do a simple query to list the tables in the HR schema. To run sqlplus, we need to set up an environment variable called $ORACLE_HOME which is required to run sqlplus. To set $ORACLE_HOME, type the following command in a Linux shell: export ORACLE_HOME=/usr/lib/oracle/xe/app/oracle/product/10.2.0/server If you have installed Oracle XE in a non-default location, then you may have to use a different path. To run sqlplus, type the following command: <oracle_home>/bin/sqlplus The result will be a login screen as shown below: You will be prompted for a username. Type the following command: hr@xe<enter> For the password, type the following command: hr<enter> When you have successfully logged in, you can type the following commands in the SQL prompt: SELECT TABLE_NAME FROM user_tables<enter> /<enter> The / command means execute the command buffer. The result will be a list of tables in the HR schema, as shown in the following screenshot: We have now successfully verified that Oracle works from a command line, and thus it is very likely that WebSphere will also be able to communicate with Oracle. Next, we will cover how to configure WebSphere to communicate with Oracle. JDBC providers Deployed applications use JDBC providers to communicate with RDBMS. The JDBC provider object provides the actual JDBC driver implementation class for access to a specific database type, that is, Oracle, SQL Server, DB/2, and so on. You associate a data source with a JDBC provider. A data source provides the connection to the RDBMS. The JDBC provider and the data source provide connectivity to a database. Creating a JDBC provider Before creating a JDBC provider, you will need to understand the application's resource requirements, that is, the data sources that the application references. You should know the answer to the following questions: Does your application require a data source? Not all applications use a database. The security credentials required to connect to the database. Often databases are secured and you will need a username and password to access a secure database. Are there any web components (Servlets, JSP, and so on) or EJBs which need to access a database. Answering these questions will determine the amount of configuration required for your database connectivity configurations. To create a JDBC provider, log into the administration console and click on the JDBC Provider link in the JDBC category of the Resources section located in the left-hand panel of the administration console as shown below. We need to choose an appropriate scope from the Scope drop-down pick list. Scope determines how the provider will be seen by applications. We will talk more about scope in the JNDI section. For now, please choose the Cell scope as seen below. Click New and the new JDBC provider wizard is displayed. Select the Database type as Oracle, Provider type as Oracle JDBC Driver, Implementation type as Connection pool data source, and Name for the new JDBC provider. We are going to enter MyJDBCDriver as the provider name as seen in the previous screenshot. We also have to choose an Implementation type. There are two implementation types for Oracle JDBC Drivers. The table below explains the two different types. Implementation Type Description Connection pool data source Use Connection Pool datasource if your application does not require connection that supports two-phase commit transactions... XA Datasource Use XA Datasource if your application requires two-phase commit transactions. Click Next to go to the database classpath screen. As shown in the following screenshot, enter the database class path information for the JDBC provider. As long as you have installed Oracle XE using the default paths, you will be able to use the following path in the Directory location field: /usr/lib/oracle/xe/oracle/product/10.2.0/server/jdbc/lib. Click Next to proceed to the next step, where you will be presented with a summary as shown in the following screenshot. Review the JDBC provider information that you have entered and click Finish. You will now be prompted to save the JDBC provider configuration. Click Save, as shown in the following screenshot. Saving this will persist the configuration to disk the resources to resources.xml. Before we finish, we need to update the JDBC Provider with the correct JAR file as the default one is not the one that we wish to use as it was assuming a later Oracle driver which we are not using. To change the driver, we must first select the driver that we created earlier called MyJDBCDriver as shown in the following screenshot: In the screen presented, we are going to change the Classpath field from: ${ORACLE_JDBC_DRIVER_PATH}/ojdbc6.jar to ${ORACLE_JDBC_DRIVER_PATH}/ojdbc14.jar Since WAS 7.0 is the latest version of WebSphere, the wizard already knows about the new version of the oracle 11g JDBC Driver. We are connecting to Oracle XE 10g and the driver for this is ojdbc14.jar.The classpath file can contain a list of paths or JAR file names which together form the location for the resource provider classes. Class path entries are separated by using the ENTER key and must not contain path separator characters (such as ; or :). Class paths can contain variable (symbolic) names that can be substituted using a variable map. Check your driver installation notes for specific JAR file names that are required. Click Apply and save the configuration.