Search icon CANCEL
Subscription
0
Cart icon
Your Cart (0 item)
Close icon
You have no products in your basket yet
Save more on your purchases! discount-offer-chevron-icon
Savings automatically calculated. No voucher code required.
Arrow left icon
Explore Products
Best Sellers
New Releases
Books
Events
Videos
Audiobooks
Packt Hub
Free Learning
Arrow right icon
timer SALE ENDS IN
0 Days
:
00 Hours
:
00 Minutes
:
00 Seconds

How-To Tutorials

7018 Articles
Packt
31 Dec 2013
5 min read
Save for later

Geolocation – using PhoneGap features to improve an app's functionality, write once use everywhere

Packt
31 Dec 2013
5 min read
(For more resources related to this topic, see here.) Step 1 – define global variables We include the following variables and functions in our file: var map; var latitud; var longitud; var xmlDoc = loadXml("puntos.xml"); var marker; var markersArray = []; function loadXml(xmlUrl) { var xmlhttp; if (window.XMLHttpRequest) { xmlhttp = new XMLHttpRequest(); } else { xmlhttp = new ActiveXObject("Microsoft.XMLHTTP"); } xmlhttp.open("GET", xmlUrl, false); xmlhttp.send(); xmlDoc = xmlhttp.responseXML; return xmlDoc; } With this code, we build our first function called loadXmlthat just loads the information into the XML file and then in to the smart phone memory. Step 2 – get current position We will build the following functions that we need to show our current position in Google Maps: getCurrentPosition: This function receives an object with the latitude and longitude. These values are set to two variables defined as global variables with the same name. This function receives three parameters, which are as follows: A function to manage the current latitude and longitude A function to manage errors if they occur when the device is trying to get the position Options to configure maximumAge: This is the time required to keep our position on the cache in milliseconds timeout: This is the maximum time required to wait for an answer from the getCurrentPosition function in milliseconds enableHighAccuracy: This provides a more accurate location The values of these functions can be set to either True or False. The following is the code for the preceding functions: getCurrentPosition(coordinates, errors, { maximumAge : 3000, timeout : 15000, enableHighAccuracy : true } The following is the code for the coordinates and errors functions: function coordinates(position) { latitud = position.coords.latitude; /* saving latitude */ longitud = position.coords.longitude; /* saving longitude*/ loadMap(); }// end function function errors(err) { /* Managing errors */ if (err.code == 0) { alert("Oops! Something is wrong"); } if (err.code == 1) { alert("Oops! Please accept share your position with us."); } if (err.code == 2) { alert("Oops! We can't get your current position."); } if (err.code == 3) { alert("Oops! Timeout!"); } }// end errors LocateMe: This function checks whether the device supports the PhoneGap geolocalization API or not. If the device supports the API, then this function calls a method to obtain the current position. If the device doesn't support geolocalization API, then we will use the current position's function, getCurrrentPosition, explained previously. The complete code for the LocateMe function is as follows: function locateMe() { if (navigator.geolocation) { /* The browser have geolocalization */ navigator.geolocation.getCurrentPosition(coordinates, errors, { maximumAge : 3000, timeout : 15000, enableHighAccuracy : true }); } else { alert('Oops! Your browser doesn't support geolocalization'); } } Step 3 – showing the current position using Google Maps To do this, we use a function that we called loadMap. This function is responsible for loading a map using Google Maps API with the current position. Also, we use a JavaScript object called myOptions with three properties, zoom to define the zoom, center to define where the maps will be centered, and mapTypeId to indicate the map style (that is Satellite). Following the code, we find two lines: the first one initializes the actualHeight variable according to the value that is returned by the getActualContentHeight function. This function returns the height that the map needs to have according to the screen size. In the second line, we change the height of the div "map canvas" through the jQuery Mobile method's CSS. In the following code file, we set the variable map with the google.maps.Map object that receives the parameters, the HTML element, and the myOptions object. We use an additional event to resize the map when the screen size changes. Now, we create a new marker object google.maps.Marker with four properties: position for the place of the marker, map that is the global variable, icon to define the image that we want to use as a marker, and the tooltip with the property title. We have two functions: the first one is createMarkers that allows us to read the XML file with the points uploaded in memory and, after that, puts the markers in the map with our second function addMarker that receives four parameters (the global map variable, point name, address and phone, and the google.maps.LatLng object with the point coordinates) to build the marker. We add a click event to show all the information about the place. Finally, we use the removePoints function that clears the maps, thereby deleting the points loaded. This is the code for the JS file: function loadMap() { var latlon = new google.maps.LatLng(latitud, longitud); var myOptions = { zoom : 17, center : latlon, mapTypeId : google.maps.MapTypeId.ROADMAP }; map = new google.maps.Map(document.getElementById('map_canvas'), myOptions); var mapDiv = document.getElementById('map_canvas'); google.maps.event.addDomListener(mapDiv, 'resize', function(){ google.maps.event.trigger(map,'resize'); }); var coorMarker = new google.maps.LatLng(latitud, longitud); marker = new google.maps.Marker({/* Create the marker */ position : coorMarker, map : map, icon : 'images/green-dot.png', title : "Where am I?" }); } Step 4 – edit the HTML file To implement the preceding functions, we need to add some code lines in our HTML file. We need to add the script line to include the new JS file, and add a complete JavaScript in the HTML head tag to execute the locateMe function. Now that the device is ready, another PhoneGap function calls watchPosition to update the position when the user is moving. Summary This article has shown us how to implement Geolocation in our app. It also shows how to use the API provided by PhoneGap to use in our app. The code for pointing out our current location was also discussed. Resources for Article: Further resources on this subject: Using Location Data with PhoneGap [Article] Configuring the ChildBrowser plugin [Article] Creating and configuring a basic mobile application [Article]
Read more
  • 0
  • 0
  • 3494

article-image-using-redis-hostile-environment-advanced
Packt
27 Dec 2013
12 min read
Save for later

Using Redis in a hostile environment (Advanced)

Packt
27 Dec 2013
12 min read
(For more resources related to this topic, see here.) How to do it... Anyone who can read the files that Redis uses to persist your dataset has a full copy of all your data. Worse, anyone who can write to those files can, with a minimal amount of effort and some patience, change the data that your Redis server contains. Both of these things are probably not what you want, and thankfully it isn't particularly difficult to prevent. All you have to do is prevent anyone but the user running your Redis server from accessing the data directory your Redis instance is using. The simplest way to achieve this is by changing the owner of the directory to the user who runs your Redis server, and then disallow all privileges to everyone else, like this: Determine the user under whom you are running your Redis instance. You can typically find this out by running ps caux |grep redis-server. The name in the first column is the user under which Redis is running. Determine the directory in which Redis is storing its files. If you don't already know this, you can ask Redis by running CONFIG GET dir from within redis-cli. Ensure that the user running your Redis instance owns its data directory: chown <redisuser> /path/to/redis/datadir Restrict permissions on the data directory so that only the owner can access it at all: chmod 0700 /path/to/redis/datadir It is important that you protect the Redis data directory, and not individual data files, because Redis is regularly rewriting those data files, and the permissions you choose won't necessarily be preserved on the next rewrite. It is also a good practice to restrict access to your redis.conf file, because in some cases it can contain sensitive data. This is simply achieved: chmod 0600 /path/to/redis.conf If you run your Redis using applications on a server which is shared with other people, your Redis instance is at pretty serious risk of abuse. The most common way of connecting to Redis is via TCP, which can only limit access based on the address connecting to it. On a shared server, that address is shared amongst everyone using it, so anyone else on the same server as you can connect to your Redis. Not cool! If, however, the programs that need to access your Redis server are on the same machine as the Redis server, there is another, more secure, method of connection called Unix sockets. A Unix socket looks like a file on disk, and you can control its permissions just like a file, but Redis can listen on it (and clients can connect to it), in a very similar way to a TCP socket. Enabling Redis to listen on a Unix socket is fairly straightforward: Set the port parameter to 0 in your Redis configuration file. This will tell Redis to not listen on a TCP socket. This is very important to prevent miscreants from still being able to connect to your Redis server while you're happily using a Unix socket. Set the unixsocket parameter in your Redis configuration file to a fully-qualified filename where you want the socket to exist. If your Redis server runs as the same user as your client programs (which is common in shared-hosting situations), I recommend making the name of the file redis.sock, in the same directory as your Redis dataset. So, if you keep your Redis data in /home/joe/redis, set unixsocket to /home/joe/redis/redis.sock. Set the unixsocketperm parameter in your Redis configuration file to 600, or a more relaxed permission set if you know what you're doing. Again, this assumes that your Redis server and Redis-using programs are running as the same user. If they're not, you'll probably need a dedicated group and things get a lot more complicated—and beyond the scope of what can be covered in this guide. Once you've changed those configuration parameters and restarted Redis, you should find that the file you specified for unixsocket has magically appeared, and you can no longer connect to Redis using TCP. All that remains to do now is to configure your Redis-using programs to connect using the Unix socket, which is something you should find how to do in the manual for your particular Redis client library or application. Configuring Redis to use Unix sockets is all well and good when it's practical, but what about if you need to connect to Redis over a network? In that case, you'll need to let Redis listen on a TCP socket, but you should at least limit the computers that can connect to it with a suitable firewall configuration. While the properly paranoid systems administrator runs their systems with a default deny firewalling policy, not everyone shares this philosophy. However, given that by default, anyone who can connect to your Redis server can do anything they want with it, you should definitely configure a firewall on your Redis servers to limit incoming TCP connections to those which are coming from machines that have a legitimate need to talk to your Redis server. While it won't protect you from all attacks, it will cut down significantly on the attack surface, which is an important part of a defense-in-depth security strategy. Unfortunately, it is hard to give precise commands to configure a firewall ruleset, because there are so many firewall management tools in common use on systems today. In the interest of addressing the greatest common factor, though, I'll provide a set of Linux iptables rules, which should be translatable to whatever means of managing your firewall (whether it be an iptables wrapper of some sort on Linux, or a pf-based system on a BSD). In all of the following commands, replace the word <port> with the TCP port that your Redis server listens on. Also, note that these commands will temporarily stop all traffic to your Redis instance, so you'll want to avoid doing this on a live server. Setting up your firewall in an init script is the best course of action. Insert a rule that will drop all traffic to your Redis server port by default: iptables -I INPUT -p tcp --dport <port> -j DROP For each IP address you want to allow to connect, run these two commands to let the traffic in: iptables -I INPUT -p tcp --dport <port> -s <clientIP> -j ACCEPT iptables -I OUTPUT -p tcp --sport <port> -d <clientIP> -j ACCEPT A firewall is great, but sometimes you can't trust everyone with access to a machine that needs to talk to your Redis instance. In that case, you can use authentication to provide a limited amount of protection against miscreants: Select a very strong password. Redis is not hardened against repeated password guessing, so you want to make this very long and very random. If you make the password too short, an attacker can just write a program that tries every possible password very quickly and guess your password that way, not cool! Thankfully, since humans should rarely be typing this password, it can be a complete jumble, and very long. I like the command pwgen -sy 32 1 for all my "generating very strong password" needs. Configure all your clients to authenticate against the server, by sending the following command when they first connect to the server: AUTH <password> Edit your Redis configuration file to include a line like this: requirepass "\:d!&!:Y<p'TXBI0"ys96rfH]lxaA7|E" If your selected password contains any double-quotes, you'll need to escape them with a backslash (so " would become "), as I've done in the preceding example. You'll also need to double any actual backslashes (so becomes \), again as I've done in the password of the preceding example. Let the configuration changes take effect by restarting Redis. The authentication password cannot be changed at runtime. If you don't need certain commands, or want to limit the use of certain commands to a subset of clients, you can use the rename-command configuration parameter. Like firewalling, restricting, or disabling commands reduces your attack surface, but is not a panacea. The simplest solution to the risk of a dangerous command is to disable it. For example, if you want to stop anyone from accidentally (or deliberately) nuking all the data in your Redis server with a single command, you might decide to disable the FLUSHDB and FLUSHALL commands, by putting the following in your Redis config file: rename-command FLUSHDB ""rename-command FLUSHALL "" This doesn't stop someone from enumerating all the keys in your dataset with KEYS * and then deleting them all one-by-one, but it does raise the bar somewhat. If you never wanted to delete keys (but, say, only let them expire) you could disable the DEL command; although all that would probably do is encourage the wily cracker to enumerate all your keys and run PEXPIRE 1 over them. Arms races are a terrible thing... While disabling commands entirely is great when it can be done, you sometimes need a particular command, but you'd prefer not to give access to it to absolutely everyone—commands that can cause serious problems if misused, such as CONFIG. For those cases, you can rename the command to something hard-to-guess, as shown in the following command: rename-command CONFIG somegiantstringnobodywouldguess It's important to not make the new name of the command something easy-to-guess. Like the AUTH command, which we discussed previously, someone who wanted to do bad things could easily write a program to repeatedly guess what you've renamed your commands to. For any environment in which you can't trust the network (which these days is pretty much everywhere, thanks to the NSA and the Cloud), it is important to consider the possibility of someone watching all your data as it goes over the wire. There's little point configuring authentication, or renaming commands, if an attacker can watch all your data and commands flow back and forth. The least-worst option we have for generically securing network traffic from eavesdropping is still the Secure Sockets Layer (SSL). Redis doesn't support SSL natively; however, through the magic of the stunnel program, we can create a secure tunnel between Redis clients and servers. The setup we will build will look like the following diagram: In order to set this up, you'll need to do the following: In your redis.conf, ensure that Redis is only listening on 127.0.0.1, by setting the bind parameter: bind 127.0.0.1 Create a private key and certificate, which stunnel will use to secure the network communications. First, create a private key and a certificate request, by running: openssl req -out /etc/ssl/redis.csr -keyout /etc/ssl/redis.key -nodes -newkey rsa:2048 This will ask you all sorts of questions which you can answer with whatever you like. Create the self-signed certificate itself, by running: openssl x509 -req -days 3650 -signkey /etc/ssl/redis.key -in /etc/ssl/redis.csr -out /etc/ssl/redis.crt Finally, stunnel expects to find the private key and the certificate in the same file, so we'll concatenate the two together into one file: cat /etc/ssl/redis.key /etc/ssl/redis.crt >/etc/ssl/redis.pem Now, we've got our SSL keys, we can start stunnel on the server side, configuring it to listen out for SSL connections, and forward them to our local Redis server: stunnel -d 46379 -r 6379 -p /etc/ssl/redis.pem If your local Redis instance isn't listening on port 6379, or if you'd like to change the public port that stunnel listens on, you can, of course, adjust the preceding command line to suit. Also, don't forget to open up your firewall for the port you're listening on! Once you run the preceding command, you should be returned to a command line pretty quickly, because stunnel runs in the background. Although you examine your listening ports with netstat -ltn, you will still find that port 46379 is listening. If that's the case, you're done configuring the server. On the client(s), the process is somewhat simpler, because you don't have to create a whole new key pair. However, you do need the certificate from the server, because you want to be able to verify that you're connecting to the right SSL-enabled service. There's little point in using SSL if an attacker can just set up a fake SSL service and trick you into connecting to it. To set up the client, do the following: Copy /etc/ssl/redis.crt from the server to the same location on the client. Start stunnel on the client, as shown in the following code snippet: stunnel -c -v 3 -A /etc/ssl/redis.crt -d 127.0.0.1:56379 -r 192.0.2.42:46379 Replace 192.0.2.42 with the IP address of your Redis server. Verify that stunnel is listening correctly by running netstat -ltn, and look for something listening on port 56379. Reconfigure your client to connect to 127.0.0.1:56379, rather than directly to the remote Redis server. Summary This article contains an assortment of quick enhancements that you can deploy to your systems to protect them from various threats, which are frequently encountered on the Internet today. Resources for Article: Further resources on this subject: Implementing persistence in Redis (Intermediate) [Article] Python Text Processing with NLTK: Storing Frequency Distributions in Redis [Article] Coding for the Real-time Web [Article]
Read more
  • 0
  • 0
  • 5755

Packt
26 Dec 2013
21 min read
Save for later

Implementing the Naïve Bayes classifier in Mahout

Packt
26 Dec 2013
21 min read
(for more resources related to this topic, see here.) Bayes was a Presbyterian priest who died giving his "Tractatus Logicus" to the prints in 1795. The interesting fact is that we had to wait a whole century for the Boolean calculus before Bayes' work came to light in the scientific community. The corpus of Bayes' study was conditional probability. Without entering too much into mathematical theory, we define conditional probability as the probability of an event that depends on the outcome of another event. In this article, we are dealing with a particular type of algorithm, a classifier algorithm. Given a dataset, that is, a set of observations of many variables, a classifier is able to assign a new observation to a particular category. So, for example, consider the following table: Outlook Temperature Temperature Humidity Humidity Windy Play Numeric Nominal Numeric Nominal Overcast 83 Hot 86 High FALSE Yes Overcast 64 Cool 65 Normal TRUE Yes Overcast 72 Mild 90 High TRUE Yes Overcast 81 Hot 75 Normal FALSE Yes Rainy 70 Mild 96 High FALSE Yes Rainy 68 Cool 80 Normal FALSE Yes Rainy 65 Cool 70 Normal TRUE No Rainy 75 Mild 80 Normal FALSE Yes Rainy 71 Mild 91 High TRUE No Sunny 85 Hot 85 High FALSE No Sunny 80 Hot 90 High TRUE No Sunny 72 Mild 95 High FALSE No Sunny 69 Cool 70 Normal FALSE Yes Sunny 75 Mild 70 Normal TRUE Yes The table itself is composed of a set of 14 observations consisting of 7 different categories: temperature (numeric), temperature (nominal), humidity (numeric), and so on. The classifier takes some of the observations to train the algorithm and some as testing it, to create a decision for a new observation that is not contained in the original dataset. There are many types of classifiers that can do this kind of job. The classifier algorithms are part of the supervised learning data-mining tasks that use training data to infer an outcome. The Naïve Bayes classifier uses the assumption that the fact, on observation, belongs to a particular category and is independent from belonging to any other category. Other types of classifiers present in Mahout are the logistic regression, random forests, and boosting. Refer to the page https://cwiki.apache.org/confluence/display/MAHOUT/Algorithms for more information. This page is updated with the algorithm type, actual integration in Mahout, and other useful information. Moving out of this context, we could describe the Naïve Bayes algorithm as a classification algorithm that uses the conditional probability to transform an initial set of weights into a weight matrix, whose entries (row by column) detail the probability that one weight is associated to the other weight. In this article's recipes, we will use the same algorithm provided by the Mahout example source code that uses the Naïve Bayes classifier to find the relation between works of a set of documents. Our recipe can be easily extended to any kind of document or set of documents. We will only use the command line so that once the environment is set up, it will be easy for you to reproduce our recipe. Our dataset is divided into two parts: the training set and the testing set. The training set is used to instruct the algorithm on the relation it needs to find. The testing set is used to test the algorithm using some unrelated input. Let us now get a first-hand taste of how to use the Naïve Bayes classifier. Using the Mahout text classifier to demonstrate the basic use case The Mahout binaries contain ready-to-use scripts for using and understanding the classical Mahout dataset. We will use this dataset for testing or coding. Basically, the code is nothing more than following the Mahout ready-to-use script with the corrected parameter and the path settings done. This recipe will describe how to transform the raw text files into weight vectors that are needed by the Naïve Bayes algorithm to create the model. The steps involved are the following: Converting the raw text file into a sequence file Creating vector files from the sequence files Creating our working vectors Getting ready The first step is to download the datasets. The dataset is freely available at the following link: http://people.csail.mit.edu/jrennie/20Newsgroups/20news-bydate.tar.gz. For classification purposes, other datasets can be found at the following URL: http://sci2s.ugr.es/keel/category.php?cat=clas#sub2. The dataset contains a post of 20 newsgroups dumped in a text file for the purpose of machine learning. Anyway, we could have also used other documents for testing purposes, but we will suggest how to do this later in the recipe. Before proceeding, in the command line, we need to set up the working folder where we decompress the original archive to have shorter commands when we need to insert the full path of the folder. In our case, the working folder is /mnt/new; so, our working folder's command-line variables will be set using the following command: export WORK_DIR=/mnt/new/ You can create a new folder and change the WORK_DIR bash variable accordingly. Do not forget that to have these examples running, you need to run the various commands with a user that has the HADOOP_HOME and MAHOUT_HOME variables in its path. To download the dataset, we only need to open up a terminal console and give the following command: wget http://people.csail.mit.edu/jrennie/20Newsgroups/20news-bydate.tar.gz Once your working dataset is downloaded, decompress it using the following command: tar –xvzf 20news-bydate.tar.gz You should see the folder structure as shown in the following screenshot: The second step is to sequence the whole input file to transform them into Hadoop sequence files. To do this, you need to transform the two folders into a single one. However, this is only a pedagogical passage, but if you have multiple files containing the input texts, you could parse them separately by invoking the command multiple times. Using the console command, we can group them together as a whole by giving the following command in sequence: rm -rf ${WORK_DIR}/20news-all mkdir ${WORK_DIR}/20news-all cp -R ${WORK_DIR}/20news-bydate*/*/* ${WORK_DIR}/20news-all Now, we should have our input folder, which is the 20news-all folder, ready to be used: The following screenshot shows a bunch of files, all in the same folder: By looking at one single file, we should see the underlying structure that we will transform. The structure is as follows: From: xxx Subject: yyyyy Organization: zzzz X-Newsreader: rusnews v1.02 Lines: 50 jaeger@xxx (xxx) writes: >In article xxx writes: >>zzzz "How BCCI adapted the Koran rules of banking". The >>Times. August 13, 1991. > > So, let's see. If some guy writes a piece with a title that implies > something is the case then it must be so, is that it? We obviously removed the e-mail address, but you can open this file to see its content. For any newsgroup of 20 news items that are present on the dataset, we have a number of files, each of them containing a single post to a newsgroup without categorization. Following our initial tasks, we need to now transform all these files into Hadoop sequence files. To do this, you need to just type the following command: ./mahout seqdirectory -i ${WORK_DIR}/20news-all -o ${WORK_DIR}/20news-seq This command brings every file contained in the 20news-all folder and transforms them into a sequence file. As you can see, the number of corresponding sequence files is not one to one with the number of input files. In our case, the generated sequence files from the original 15417 text files are just one chunck-0 file. It is also possible to declare the number of output files and the mappers involved in this data transformation. We invite the reader to test the different parameters and their uses by invoking the following command: ./mahout seqdirectory --help The following table describes the various options that can be used with the seqdirectory command: Parameter Description --input (-i) input his gives the path to the job input directory. --output (-o) output The directory pathname for the output. --overwrite (-ow) If present, overwrite the output directory before running the job. --method (-xm) method The execution method to use: sequential or mapreduce. The default is mapreduce. --chunkSize (-chunk) chunkSize The chunkSize values in megabyte. The default is 64 Mb. --fileFilterClass (-filter) fileFilterClass The name of the class to use for file parsing.The default is org.apache.mahout.text.PrefixAdditionFilter. --keyPrefix (-prefix) keyPrefix The prefix to be prepended to the key of the sequence file. --charset (-c) charset The name of the character encoding of the input files.The default is UTF-8. --overwrite (-ow) If present, overwrite the output directory before running the job. --help (-h) Prints the help menu to the command console. --tempDir tempDir If specified, tells Mahout to use this as a temporary folder. --startPhase startPhase Defines the first phase that needs to be run. --endPhase endPhase Defines the last phase that needs to be run To examine the outcome, you can use the Hadoop command-line option fs. So, for example, if you would like to see what is in the chunck-0 file, you could type in the following command: hadoop fs -text $WORK_DIR/20news-seq/chunck-0 | more In our case, the result is as follows: /67399 From:xxx Subject: Re: Imake-TeX: looking for beta testers Organization: CS Department, Dortmund University, Germany Lines: 59 Distribution: world NNTP-Posting-Host: tommy.informatik.uni-dortmund.de In article <xxxxx>, yyy writes: |> As I announced at the X Technical Conference in January, I would like |> to |> make Imake-TeX, the Imake support for using the TeX typesetting system, |> publically available. Currently Imake-TeX is in beta test here at the |> computer science department of Dortmund University, and I am looking |> for |> some more beta testers, preferably with different TeX and Imake |> installations. The Hadoop command is pretty simple, and the syntax is as follows: hadoop fs –text <input file> In the preceding syntax, <input file> is the sequence file whose content you will see. Our sequence files have been created, and until now, there has been no analysis of the words and the text itself. The Naïve Bayes algorithm does not work directly with the words and the raw text, but with the weighted vector associated to the original document. So now, we need to transform the raw text into vectors of weights and frequency. To do this, we type in the following command: ./mahout seq2sparse -i ${WORK_DIR}/20news-seq -o ${WORK_DIR}/20news- vectors -lnorm -nv -wt tfidf The following command parameters are described briefly: The -lnorm parameter instructs the vector to use the L_2 norm as a distance The -nv parameter is an optional parameter that outputs the vector as namedVector The -wt parameter instructs which weight function needs to be used We end the data-preparation process with this step. Now, we have the weight vector files that are created and ready to be used by the Naïve Bayes algorithm. We will clear a little while this last step algorithm. This part is about tuning the algorithm for better performance of the Naïve Bayes classifier. How to do it… Now that we have generated the weight vectors, we need to give them to the training algorithm. But if we train the classifier against the whole set of data, we will not be able to test the accuracy of the classifier. To avoid this, you need to divide the vector files into two sets called the 80-20 split. This is a good data-mining approach because if you have any algorithm that should be instructed on a dataset, you should divide the whole bunch of data into two sets: one for training and one for testing your algorithm. A good dividing percentage is shown to be 80 percent and 20 percent, meaning that the training data should be 80 percent of the total while the testing ones should be the remaining 20 percent. To split data, we use the following command: ./mahout split -i ${WORK_DIR}/20news-vectors/tfidf-vectors --trainingOutput ${WORK_DIR}/20news-train-vectors --testOutput ${WORK_DIR}/20news-test-vectors --randomSelectionPct 40 --overwrite --sequenceFiles -xm sequential As result of this command, we will have two new folders containing the training and testing vectors. Now, it is time to train our Naïves Bayes algorithm on the training set of vectors, and the command that is used is pretty easy: ./mahout trainnb -i ${WORK_DIR}/20news-train-vectors -el -o ${WORK_DIR}/model -li ${WORK_DIR}/labelindex -ow Once finished, we have our training model ready to be tested against the remaining 20 percent of the initial input vectors. The final console command is as follows: ./mahout testnb -i ${WORK_DIR}/20news-test-vectors -m ${WORK_DIR}/model -l ${WORK_DIR}/labelindex\ -ow -o ${WORK_DIR}/20news-testing The following screenshot shows the output of the preceding command: How it works... We have given certain commands and we have seen the outcome, but you've done this without an understanding of why we did it and above all, why we chose certain parameters. The whole sequence could be meaningless, even for an experienced coder. Let us now go a little deeper in each step of our algorithm. Apart from downloading the data, we can divide our Naïve Bayes algorithm into three main steps: Data preparation Data training Data testing In general, these are the three procedures for mining data that should be followed. The data preparation steps involve all the operations that are needed to create the dataset in the format that is required for the data mining procedure. In this case, we know that the original format was a bunch of files containing text, and we transformed them into a sequence file format. The main purpose of this is to have a format that can be handled by the map reducing algorithm. This phase is a general one as the input format is not ready to be used as it is in most cases. Sometimes, we also need to merge some data if they are divided into different sources. Sometimes, we also need to use Sqoop for extracting data from different datasources. Data training is the crucial part; from the original dataset, we extract the information that is relevant to our data mining tasks, and we bring some of them to train our model. In our case, we are trying to classify if a document can be inserted in a certain category based on the frequency of some terms in it. This will lead to a classifier that using another document can state if this document is under a previously found category. The output is a function that is able to determinate this association. Next, we need to evaluate this function because it is possible that one good classification in the learning phase is not so good when using a different document. This three-phased approach is essential in all classification tasks. The main difference relies on the type of classifier to be used in the training and testing phase. In this case, we use Naïve Bayes, but other classifiers can be used as well. In the Mahout framework, the available classifiers are Naïve Bayes, Decision Forest, and Logistic Regression. As we have seen, the data preparation consists basically of creating two series of files that will be used for training and testing purposes. The step to transform the raw text file into a Hadoop sequence format is pretty easy; so, we won't spend too long on it. But the next step is the most important one during data preparation. Let us recall it: mahout seq2sparse -i ${WORK_DIR}/20news-seq -o ${WORK_DIR}/20news- vectors -lnorm -nv -wt tfidf This computational step basically grabs the whole text from the chunck-0 sequence file and starts parsing it to extract information from the words contained in it. The input parameters tell the utility to work in the following ways: The -i parameter is used to declare the input folder where all the sequence files are stored The -o parameter is used to create the output folder where the vector containing the weights is stored The -nv parameter tells Mahout that the output format should be in the namedVector format The -wt parameter tells which frequency function to use for evaluating the weight of every term to a category The -lnorm parameter is a function used to normalize the weights using the L_2 distance The -ow: parameter overwrites the previously generated output results The -m: parameter gives the minimum log-likelihood ratio The whole purpose of this computation step is to transform the sequence files that contain the documents' raw text in the sequence files containing vectors that count the frequency of the term. Obviously, there are some different functions that count the frequency of a term within the whole set of documents. So, in Mahout, the possible values for the wt parameter are tf and tfidf. The Tf value is the simpler one and counts the frequency of the term. This means that the frequency of the Wi term inside the set of documents is the ratio between the total occurrence of the word over the total number of words. The second one considers the sum of every term frequency using a logarithmic function like this one: In the preceding formula, Wi is the TF-IDF weight of the word indexed by i. N is the total number of documents. DFi is the frequency of the i word in all the documents. In this preprocessing phase, we notice that we index the whole corpus of documents so that we are sure that even if we divide or split in the next phase, the documents are not affected. We compute a word frequency; this means that the word was contained in the training or testing set. So, the reader should grasp the fact that changing this parameter can affect the final weight vectors; so, based on the same text, we could have very different outcomes. The lnorm value basically means that while the weight can be a number ranging from 0 to an upper positive integer, they are normalized to 1 as the maximum possible weight for a word inside the frequency range. The following screenshot shows the output of the output folder: Various folders are created for storing the word count, frequency, and so on. Basically, this is because the Naïve Bayes classifier works by removing all periods and punctuation marks from the text. Then, from every text, it extracts the categories and the words. The final vector file can be seen in the tfidf-vectors folder, and for dumping vector files to normal text ones, you can use the vectordump command as follows: mahout vectordump -i ${WORK_DIR}/20news-vectors/tfidf-vectors/ part-r-00000 –o ${WORK_DIR}/20news-vectors/tfidf-vectors/part-r-00000dump The dictionary files and word files are sequence files containing the association within the unique key/word created by the MapReduce algorithm using the command: hadoop fs -text $WORK_DIR/20news-vectors/dictionary.file-0 | more one can see for example adrenal_gland 12912 adrenaline 12913 adrenaline.com 12914| The splitting of the dataset into training and testing is done by using the split command-line option of Mahout. The interesting parameter in this case is that randomSelectionPct equals 40. It uses a random selection to evaluate which point belongs to the training or the testing dataset. Now comes the interesting part. We are ready to train using the Naïve Bayes algorithm. The output of this algorithm is the model folder that contains the model in the form of a binary file. This file represents the Naïve Bayes model that holds the weight Matrix, the feature and label sums, and the weight normalizer vectors generated so far. Now that we have the model, we test it on the training set. The outcome is directly shown on the command line in terms of a confusion matrix. The following screenshot shows the format in which we can see our result. Finally, we test our classifier on the test vector generated by the split instruction. The output in this case is a confusion matrix. Its format is as shown in the following screenshot: We are now going to provide details on how this matrix should be interpreted. As you can see, we have the total classified instances that tell us how many sentences have been analyzed. Above this, we have the correctly/incorrectly classified instances. In our case, this means that on a test set of weighted vectors, we have nearly 90 percent of the corrected classified sentences against an error of 9 percent. But if we go through the matrix row by row, we can see at the end that we have different newsgroups. So, a is equal to alt.atheism and b is equal to comp.graphics. So, a first look at the detailed confusion matrix tells us that we did the best in classification against the rec.sport.hockey newsgroup, with a value of 418 that is the highest we have. If we take a look at the corresponding row, we understand that of these 418 classified sentences, we have 403/412; so, 97 percent of all of the sentences were found in the rec.sport.hockey newsgroup. But if we take a look at the comp.os.ms-windows.miscwe newsgroup, we can see overall performance is low. The sentences are not so centered around the same new newsgroup; so, it means that we find and classify the sentences in ms-windows in another newsgroup, and so we do not have a good classification. This is reasonable as sports terms like "hockey" are really limited to the hockey world, while sentences about Microsoft could be found both on Microsoft specific newsgroups and in other newsgroups. We encourage you to give another run to the testing phase on the training phase to see the output of the confusion matrix by giving the following command: ./bin/mahout testnb -i ${WORK_DIR}/20news-train-vectors -m ${WORK_DIR}/model -l ${WORK_DIR}/labelindex -ow -o ${WORK_DIR}/20news-testing As you can see, the input folder is the same for the training phase, and in this case, we have the following confusion matrix: In this case, we can see it using the same set both as the training and testing phase. The first consequence is that we have a rise in the correctly classified sentences by an order of 10 percent, which is even bigger if you remember that in terms of weighted vectors with respect to the testing phase, we have a size that is four times greater. But probably the most important thing is that the best classification has now moved from the hockey newsgroup to the sci.electronics newsgroup. There's more We use exactly the same procedure used by the Mahout examples contained in the binaries folder that we downloaded. But you should now be aware that starting all process need only to change the input files from the initial folder. So, for the willing reader, we suggest you download another raw text file and perform all the steps in another type of file to see the changes that we have compared to the initial input text. We would suggest that non-native English readers also look at the differences that we have by changing the initial input set with one not written in English. Since the whole text is transformed using only weight vectors, the outcome does not depend on the difference between languages but only on the probability of finding certain word couples. As a final step, using the same input texts, you could try to change the way the algorithm normalizes and counts the words to create the vector sparse weights. This could be easily done by changing, for example, the -wt tfidf parameter into the command line Mahout seq2sparce. So, for example, an alternative run of the seq2sparce Mahout could be the following one: mahout seq2sparse -i ${WORK_DIR}/20news-seq -o ${WORK_DIR}/20news- vectors -lnorm -nv -wt tfidf Finally, we not only choose to run the Naïve Bayes classifier for classifying words in a text document but also the algorithm that uses vectors of weights so that, for example, it would be easy to create your own vector weights.
Read more
  • 0
  • 0
  • 3207

article-image-acting-proxy-httpproxymodule
Packt
24 Dec 2013
9 min read
Save for later

Acting as a proxy (HttpProxyModule)

Packt
24 Dec 2013
9 min read
(For more resources related to this topic, see here.) The HttpProxyModule allows Nginx to act as a proxy and pass requests to another server. location / {   proxy_pass        http://app.localhost:8000; } Note when using the HttpProxyModule (or even when using FastCGI), the entire client request will be buffered in Nginx before being passed on to the proxy server. Explaining directives Some of the important directives of the HttpProxyModule are as follows. proxy_pass The proxy_pass directive sets the address of the proxy server and the URI to which the location will be mapped. The address may be given as a hostname or an address and port, for example: proxy_pass http://localhost:8000/uri/; Or, the address may be given as an UNIX socket path: proxy_pass http://unix:/path/to/backend.socket:/uri/; path is given after the word unix between two colons. You can use the proxy_pass directive to forward headers from the client request to the proxied server. proxy_set_header Host $host; While passing requests, Nginx replaces the location in the URI with the location specified by the proxy_pass directive. If inside the proxied location, URI is changed by the rewrite directive and this configuration will be used to process the request. For example: location  /name/ {   rewrite      /name/([^/] +)  /users?name=$1  break;   proxy_pass   http://127.0.0.1; } A request URI is passed to the proxy server after normalization as follows: Double slashes are replaced by a single slash Any references to current directory like "./" are removed Any references to the previous directory like "../" are removed. If proxy_pass is specified without a URI (for example in "http://example.com/request",/request is the URI part), the request URI is passed to the server in the same form as sent by a client location /some/path/ {   proxy_pass http://127.0.0.1; } If you need the proxy connection to an upstream server group to use SSL, your proxy_pass rule should use https:// and you will also have to set your SSL port explicitly in the upstream definition. For example: upstream https-backend {   server 10.220.129.20:443; }   server {   listen 10.220.129.1:443;   location / {     proxy_pass https://backend-secure;   } } proxy_pass_header The proxy_pass_header directive allows transferring header lines forbidden for response. For example: location / {   proxy_pass_header X-Accel-Redirect; } proxy_connect_timeout The proxy_connect_timeout directive sets a connection timeout to the upstream server. You can't set this timeout value to be more than 75 seconds. Please remember that this is not the response timeout, but only a connection timeout. This is not the time until the server returns the pages which is configured through proxy_read_timeout directive. If your upstream server is up but hanging, this statement will not help as the connection to the server has been made. proxy_next_upstream The proxy_next_upstream directive determines in which cases the request will be transmitted to the next server: error: An error occurred while connecting to the server, sending a request to it, or reading its response timeout: The timeout occurred during the connection with the server, transferring the request, or while reading the response from the server invalid_header: The server returned an empty or incorrect response http_500: The server responded with code 500 http_502: The server responded with code 502 http_503: The server responded with code 503 http_504: The server responded with code 504 http_404: The server responded with code 404 off: Disables request forwarding Transferring the request to the next server is only possible if there is an error sending the request to one of the servers. If the request sending was interrupted due to an error or some other reason, the transfer of request will not take place. proxy_redirect The proxy_redirect directive allows you to manipulate the HTTP redirection by replacing the text in the response from the upstream server. Specifically, it replaces text in the Location and Refresh headers. The HTTP Location header field is returned in response from a proxied server for the following reasons: To indicate that a resource has moved temporarily or permanently. To provide information about the location of a newly created resource. This could be the result of an HTTP PUT. Let us suppose that the proxied server returned the following: Location: http://localhost:8080/images/new_folder If you have the proxy_redirect directive set to the following: proxy_redirect http://localhost:8080/images/ http://xyz/; The Location text will be rewritten to be similar to the following: Location: http://xyz/new_folder/. It is possible to use some variables in the redirected address: proxy_redirect http://localhost:8000/ http://$location:8000; You can also use regular expressions in this directive: proxy_redirect ~^(http://[^:]+):d+(/.+)$ $1$2; The value off disables all the proxy_redirect directives at its level. proxy_redirect off; proxy_set_header The proxy_set_header directive allows you to redefine and add new HTTP headers to the request sent to the proxied server. You can use a combination of static text and variables as the value of the proxy_set_header directive. By default, the following two headers will be redefined: proxy_set_header Host $proxy_host; proxy_set_header Connection Close; You can forward the original Host header value to the server as follows: proxy_set_header Host $http_host; However, if this header is absent in the client request, nothing will be transferred. It is better to use the variable $host; its value is equal to the request header Host or to the basic name of the server in case the header is absent from the client request. proxy_set_header Host $host; You can transmit the name of the server together with the port of the proxied server: proxy_set_header Host $host:$proxy_port; If you set the value to an empty string, the header is not passed to the upstream proxied server. For example, if you want to disable the gzip compression on upstream, you can do the following: proxy_set_header  Accept-Encoding  ""; proxy_store The proxy_store directive sets the path in which upstream files are stored, with paths corresponding to the directives alias or root. The off directive value disables local file storage. Please note that proxy_store is different from proxy_cache. It is just a method to store proxied files on disk. It may be used to construct cache-like setups (usually involving error_page-based fallback). This proxy_store directive parameter is off by default. The value can contain a mix of static strings and variables. proxy_store   /data/www$uri; The modification date of the file will be set to the value of the Last-Modified header in the response. A response is first written to a temporary file in the path specified by proxy_temp_path and then renamed. It is recommended to keep this location path and the path to store files the same to make sure it is a simple renaming instead of creating two copies of the file. Example: location /images/ {   root                 /data/www;   error_page           404 = @fetch; }   location /fetch {   internal;   proxy_pass           http://backend;   proxy_store          on;   proxy_store_access   user:rw  group:rw  all:r;   proxy_temp_path      /data/temp;   alias                /data/www; } In this example, proxy_store_access defines the access rights of the created file. In the case of an error 404, the fetch internal location proxies to a remote server and stores the local copies in the /data/temp folder. proxy_cache The proxy_cache directive either turns off caching when you use the value off or sets the name of the cache. This name can then be used subsequently in other places as well. Let's look at the following example to enable caching on the Nginx server: http {   proxy_cache_path  /var/www/cache levels=1:2 keys_zone=my-     cache:8m max_size=1000m inactive=600m;   proxy_temp_path /var/www/cache/tmp;   server {     location / {       proxy_pass http://example.net;       proxy_cache my-cache;       proxy_cache_valid  200 302  60m;       proxy_cache_valid  404      1m;     }   } } The previous example creates a named cache called my-cache. It sets up the validity of the cache for response codes 200 and 302 to 60m, and for 404 to 1m, respectively. The cached data is stored in the /var/www/cache folder. The levels parameter sets the number of subdirectory levels in the cache. You can define up to three levels. The name of key_zone is followed by an inactive interval. All the inactive items in my-cache will be purged after 600m. The default value for inactive intervals is 10 minutes.   Chapter 5 of the book, Creating Your Own Module, is inspired by the work from Mr. Evan Miller which can be found at http://www.evanmiller.org/nginx-modules-guide.html.   Summary In this article we looked at several standard HTTP modules. These modules provide a very rich set of functionalities by default. You can disable these modules if you please at the time of configuration. However, they will be installed by default if you don't. The list of modules and their directives in this chapter is by no means exhaustive. Nginx's online documentation can provide you with more details. Resources for Article: Introduction to nginx [Article] Nginx Web Services: Configuration and Implementation [Article] Using Nginx as a Reverse Proxy [Article]
Read more
  • 0
  • 2
  • 18563

article-image-microsoft-dac-2012
Packt
24 Dec 2013
5 min read
Save for later

Microsoft DAC 2012

Packt
24 Dec 2013
5 min read
(For more resources related to this topic, see here.) Microsoft DAC 2012 Architecture Dynamic Access Control is not just a single feature, but an end-to-end file server solution based on the following features in Windows Server 2012: Windows authorization and audit engine supporting expression-based access control Kerberos Version 5 support for user and device claims File classification infrastructure that support claims RMS support that can be extended for further file types from third-party vendors API to extend the solution with custom classification and audit tools Building blocks The Dynamic Access Control solution can be logically divided into the following main components to get a better, granular overview: Infrastructure requirements User and device claims Expression-based ACEs Classification enhancements Central access and audit policies Access-denied assistance You need to get a quick overview of the most important facts of Dynamic Access Control. We will start the overview with the infrastructure requirements. Infrastructure requirements For basic deployment of Dynamic Access Control, you do not need to put in a big effort. To use claims for authorization and auditing, there is only a need for the following components: A Claim is something that Active Directory states about a specific object (user or computer). A Claim may include the user, a unique Security Identifier (SID), department classification of a file or other attributes of a file, user, or computer. At least one Windows 2012, or a newer, domain controller Configure DAC objects, which are: Claim Types Central Access Rules Central Access Policies Administering with Active Directory Administrative Center (ADAC) or Remote Server Administration Tools (RSAT) for installed on Windows 8/ Windows Server 2012 or newer Group policy to deploy Central Access Policies to your file servers Group policy to enable the KDC support for claims Group policy to enable the Kerberos client support for claims All the file servers that use DAC must be 2012 or newer Windows 8 or newer client computers must be part of that domain (only required when using device claims) AD RMS role enabled and configured if you want to use automatic encryption The following figure shows the basic deployment and configuration that needs to be done You need to enable claims support on domain controllers and clients (disabled by default) DAC stores all configurations in the Active Directory configuration partition Group policies are used to configure DAC on file servers and clients The File Server Resource Manager (FSRM) brings up many features such as File Server Classification Infrastructure (FCI) Dynamic Access Control also works over organization boundaries with Claims Transformation Policies (CTP) However, what's happens if you don't use Windows 8 clients? For non-Windows 8 / Windows Server 2012, such as XP, Vista or Windows 7, the user doesn't need to worry about claims. In that case, the 2012-based file server will query the Active Directory services and forward the claims request to get information about the claims the user or the machine provides. As you can see in the previous figure, DAC works between different Active Directory Forests (Active Directory instance of an organization), and Claims Transformation Policies will provide the functionality to translate the claims definitions between two or more organizations. To prepare for this scenario, you need to establish a Forest Trust between the Active Directory Forests and the Domain Function Level (DFL), which in both the Forest Root domains must be Windows 2012 or higher. Right now, this is a challenge but also a necessary requirement. There is no need for Claim Transformation Rules inside a Forest., This works fine out of the box because Dynamic Access Control objects are stored in the configuration part of the Active Directory and the whole Forest knows the relevant information. User and device claims Traditionally, you may have secured access to files by using NTFS file permissions and security groups. With this configuration, we were restricted to making policy decisions based on the user's group membership and the number of groups will explode. Therefore, if we wanted to include the device to control access, there was no chance to do this in an earlier version of the Windows Server. Another limitation was the requirement for folder or file access based on a certificate. Before Windows 2012 Dynamic Access Control, there was no way for the built-in functionality to include devices or certificates. DAC now integrates claims into Windows Authentication so that we can use Active Directory attributes from users and computers to control access to our information stored on file servers such as a location, department, or project. DAC will only be used as complementary technology and is not a replacement for security groups. The following figure shows the new combinations you can use for authorization: This opens new ways of giving permissions on files and folders, such as: Allow | Read, Write | If (@User.Department == @File.Department) AND (@Device.Managed == True) There is no development knowledge required to implement a Dynamic Access Control solution. Summary In this article we saw the benefits and the main components of DAC. We saw the Microsoft DAC 2012 as an end-to-end file server solution, which provides a better granular overview of the file servers. Resources for Article: Further resources on this subject: Article Marketing on CMS, Blogs, and Online Magazines to Improve Sales [Article] User Interface in Production [Article] Setting up the most Popular Journal Articles in your Personalized Community in Liferay Portal [Article]
Read more
  • 0
  • 0
  • 2458

article-image-creating-identity-and-resource-pools
Packt
24 Dec 2013
7 min read
Save for later

Creating Identity and Resource Pools in Cisco Unified Computing System

Packt
24 Dec 2013
7 min read
Computers and their various peripherals have some unique identities such as Universally Unique Identifiers (UUIDs), Media Access Control (MAC) addresses of Network Interface Cards (NICs), World Wide Node Numbers (WWNNs) for Host Bus Adapters (HBAs), and others. These identities are used to uniquely identify a computer system in a network. For traditional computers and peripherals, these identities were burned into the hardware and, hence, couldn't be altered easily. Operating systems and some applications rely on these identities and may fail if these identities are changed. In case of a full computer system failure or failure of a computer peripheral with unique identity, administrators have to follow cumbersome firmware upgrade procedures to replicate the identities of the failed components on the replacement components. The Unified Computing System (UCS) platform introduced the idea of creating identity and resource pools to abstract the compute node identities from the UCS Manager (UCSM) instead of using the hardware burned-in identities. In this article, we'll discuss the different pools you can create during UCS deployments and server provisioning. We'll start by looking at what pools are and then discuss the different types of pools and show how to configure each of them. Understanding identity and resource pools The salient feature of the Cisco UCS platform is stateless computing . In the Cisco UCS platform, none of the computer peripherals consume the hardware burned-in identities. Rather, all the unique characteristics are extracted from identity and resource pools, which reside on the Fabric Interconnects (FIs) and are managed using UCSM. These resource and identity pools are defined in an XML format, which makes them extremely portable and easily modifiable. UCS computers and peripherals extract these identities from UCSM in the form of a service profile. A service profile has all the server identities including UUIDs, MACs, WWNNs, firmware versions, BIOS settings, and other server settings. A service profile is associated with the physical server using customized Linux OS that assigns all the settings in a service profile to the physical server. In case of server failure, the failed server needs to be removed and the replacement server has to be associated with the existing service profile of the failed server. In this service profile association process, the new server will automatically pick up all the identities of the failed server, and the operating system or applications dependent upon these identities will not observe any change in the hardware. In case of peripheral failure, the replacement peripheral will automatically acquire the identities of the failed component. This greatly improves the time required to recover a system in case of a failure. Using service profiles with the identity and resource pools also greatly improves the server provisioning effort. A service profile with all the settings can be prepared in advance while an administrator is waiting for the delivery of the physical server. The administrator can create service profile templates that can be used to create hundreds of service profiles; these profiles can be associated with the physical servers with the same hardware specifications. Creating a server template is highly recommended as this greatly reduces the time for server provisioning. This is because a template can be created once and used for any number of physical servers with the same hardware. Server identity and resource pools are created using the UCSM. In order to better organize, it is possible to define as many pools as are needed in each category. Keep in mind that each defined resource will consume space in the UCSM database. It is, therefore, a best practice to create identity and resource pool ranges based on the current and near-future assessments. For larger deployments, it is best practice to define a hierarchy of resources in the UCSM based on geographical, departmental, or other criteria; for example, a hierarchy can be defined based on different departments. This hierarchy is defined as an organization, and the resource pools can be created for each organizational unit. In the UCSM, the main organization unit is root, and further suborganizations can be defined under this organization. The only consideration to be kept in mind is that pools defined under one organizational unit can't be migrated to other organizational units unless they are deleted first and then created again where required. The following diagram shows how identity and resource pools provide unique features to a stateless blade server and components such as the mezzanine card: Learning to create a UUID pool UUID is a 128-bit number assigned to every compute node on a network to identify the compute node globally. UUID is denoted as 32 hexadecimal numbers. In the Cisco UCSM, a server UUID can be generated using the UUID suffix pool. The UCSM software generates a unique prefix to ensure that the generated compute node UUID is unique. Operating systems including hypervisors and some applications may leverage UUID number binding. The UUIDs generated with a resource pool are portable. In case of a catastrophic failure of the compute node, the pooled UUID assigned through a service profile can be easily transferred to a replacement compute node without going through complex firmware upgrades. Following are the steps to create UUIDs for the blade servers: Log in to the UCSM screen. Click on the Servers tab in the navigation pane. Click on the Pools tab and expand root. Right-click on UUID Suffix Pools and click on Create UUID Suffix Pool as shown in the following screenshot: In the pop-up window, assign the Name and Description values to the UUID pool. Leave the Prefix value as Derived to make sure that UCSM makes the prefix unique. The selection of Assignment Order as Default is random. Select Sequential to assign the UUID sequentially. Click on Next as shown in the following screenshot: Click on Add in the next screen. In the pop-up window, change the value for Size to create a desired number of UUIDs. Click on OK and then on Finish in the previous screen as shown in the following screenshot: In order to verify the UUID suffix pool, click on the UUID Suffix Pools tab in the navigation pane and then on the UUID Suffixes tab in the work pane as shown in the following screenshot: Learning to create a MAC pool MAC is a 48-bit address assigned to the network interface for communication in the physical network. MAC address pools make server provisioning easier by providing scalable NIC configurations before the actual deployment. Following are the steps to create MAC pools: Log in to the UCSM screen. Click on the LAN tab in the navigation pane. Click on the Pools tab and expand root. Right-click on MAC Pools and click on Create MAC Pool as shown in the following screenshot: In the pop-up window, assign the Name and Description values to the MAC pool. The selection of Default as the Assignment Order value is random. Select Sequential to assign the MAC addresses sequentially. Click on Next as shown in the following screenshot: Click on Add in the next screen. In the pop-up window, change Size to create the desired number of MAC addresses. Click on OK and then on Finish in the previous screen as shown in the following screenshot: In order to verify the MAC pool, click on the MAC Pools tab in the navigation pane and then on the MAC Addresses tab in the work pane as shown in the following screenshot:
Read more
  • 0
  • 0
  • 13381
Unlock access to the largest independent learning library in Tech for FREE!
Get unlimited access to 7500+ expert-authored eBooks and video courses covering every tech area you can think of.
Renews at €18.99/month. Cancel anytime
article-image-using-faceted-search-searching-finding
Packt
24 Dec 2013
11 min read
Save for later

Using Faceted Search, from Searching to Finding

Packt
24 Dec 2013
11 min read
(For more resources related to this topic, see here.) Looking at Solr's standard query parameters The basic engine of Solr is Lucene, so Solr accepts a query syntax based on the Lucene one, even if there are some minor differences, they should not affect our experiments, as they involve more advanced behavior. You can find an explanation on the Solr Query syntax on wiki at: http://wiki.apache.org/solr/SolrQuerySyntax. Let's see some example of a query using the basic parameters. Before starting our tests, we need to configure a new core again, in the usual way. Sending Solr's query parameters over HTTP It is important to take care of the fact that our queries to Solr are sent over the HTTP protocol (unless we are using Solr in embedded mode, as we will see later). With cURL we can handle the HTTP encoding of parameters, for example: >> curl -X POST 'http://localhost:8983/solr/paintings/select?start=3&rows=2&fq=painting&wt=json&indent=true' --data-urlencode 'q=leonardo da vinci&fl=artist title' This command can be instead of the following command: >> curl -X GET "http://localhost:8983/solr/paintings/select?q=leonardo%20da%20vinci&fq=painting&start=3&row=2&fl=artist%20title&wt=json&indent=true" Please note how using the --data-urlencode parameter in the example we can write the parameters values including characters which needs to be encoded over HTTP. Testing HTTP parameters on browsers On modern browsers such as Firefox or Chrome you can look at the parameters directly into the provided console. For example using Chrome you can open the console (with F12): In the previous image you can see under Query String Parameters section on the right that the parameters are showed on a list, and we can easily switch between the encoded and the more readable un-encoded value's version. If don't like using Chrome or Firefox and want a similar tool, you can try the Firebug lite (http://getfirebug.com/firebuglite). This is a JavaScript library conceived to port firebug plugin functionality ideally to every browser, by adding this library to your HTML page during the test process. Choosing a format for the output When sending a query to Solr directly (by the browser or cURL) we can ask for results in multiple formats, including for example JSON: >> curl -X GET 'http://localhost:8983/solr/paintings/select?q=*:*&wt=json&indent=true' Time for action – searching all documents with pagination When performing a query we need to remember we are potentially asking for a huge number of documents. Let's observe how to manage partial results using pagination: For example think about the q=*:* query as seen in previous examples which was used for asking all the documents, without a specific criteria. In a case like this, in order to avoid problems with resources, Solr will send us actually only the first ones, as defined by a parameter in the configuration. The default number of returned results will be 10, so we need to be able to ask for a second group of results, and a third, and so on and on until there are. This is what is generally called a pagination of results, similarly as for scenarios involving SQL. Executing the command: >> curl -X GET "http://localhost:8983/solr/paintings/select?q=*:*&start=0&rows=0&wt=json&indent=true" We should obtain a result similar to this (the number of documents numFound and the time spent for processing query QTime could vary, depending on your data and your system): In the previous image we see the same results in two different ways: on the right side you'll recognize the output from cURL and on the left side of the browser you see how the results directly in the browser window. In the second example we had the Json View plugin installed in the browser, which gives a very helpful visualization of JSON, with indentation and colors. You can install it if you want for Chrome at: https://chrome.google.com/webstore/detail/jsonview/chklaanhfefbnpoihckbnefhakgolnmc For Firefox the plugin can be installed from: https://addons.mozilla.org/it/firefox/addon/jsonview/ Note how even if we have found 12484 documents, we are currently seeing none of them in the results! What just happened? In this very simple example, we already use two very useful parameters: start and rows, which we should always think as a couple, even if we may be using only one of them explicitly. We could change the default values for these parameters from the solrconfig.xml file, but this is generally not needed: The start value defines the original index of the first document returned in the response, from the ones matching our search criteria, and starting from value 0. The default value will again start at 0. The rows parameter is used to define how many documents we want in the results. The default value will be 10 for rows. So if for example we only want the second and third document from the results, we can obtain them by the query: >> curl -X GET "http://localhost:8983/solr/paintings/select?q=*:*&start=1 &rows=2&wt=json&indent=true' In order to obtain the second document in the results we need to remember that the enumeration starts from 0 (so the second will be at 1), while to see the next group of documents (if present), we will send a new query with values such as, start=10, rows=10, and so on. We are still using the wt and indent parameters only to have results formatted in a clear way. The start/rows parameters play roles in this context which are quite similar to the OFFSET/LIMIT clause in SQL. This process of segmenting the output to be able to read it in group or pages of results is usually called pagination, and it is generally handled by some programming code. You should know this mechanism, so you could play with your test even on a small segment of data without a loss of generalization. I strongly suggest you to always add these two parameters explicitly in your examples. Time for action – projecting fields with fl Another important parameter to consider is fl, that can be used for fields projection, obtaining only certain fields in the results: Suppose now that we are interested on obtaining the titles and artist reference for all the documents: >>curl -X GET 'http://localhost:8983/solr/paintings/select?q=artist:*&wt=json&indent=true&omitHeader=true&fl=title,artist' We will obtain an output similar to the one shown in the following image: Note that the results will be indented as requested, and will not contain any header to be more readable. Moreover the parameters list does not need to be written in a specific order. The previous query could be rewritten also: >>curl -X GET 'http://localhost:8983/solr/paintings/select?q=artist:*&wt=json&indent=true&omitHeader=true&fl=title&fl=artist' Here we ask for field projection one by one, if needed (for example when using HTML and JavaScript widget to compose the query following user's choices). What just happened? The fl parameter stands for fields list. By using this parameter we can define a comma-separated list of fields names that explicitly define what fields are projected in the results. We can also use a space to separate fields, but in this case we should use the URL encoding for the space, writing fl=title+artist or fl=title%20artist. If you are familiar with relational databases and SQL, you should consider the fl parameter. It is similar to the SELECT clause in SQL statements, used to project the selected fields in the results. In a similar way writing fl=author:artist,title corresponds to the usage of aliases for example, SELECT artist AS author, title. Let's see the full list of parameters in details: The parameter q=artist:* is used in this case in place of a more generic q=*:*, to select only the fields which have a value for the field artist. The special character * is used again for indicating all the values. The wt=json, indent=true parameters are used for asking for an indented JSON format. The omitHeader=true parameter is used for omit the header from the response. The fl=title,artist parameter represents the list of the fields to be projected for the results. Note how the fields are projected in the results without using the order asked in fl, as this has no particular sense for JSON output. This order will be used for the CSV response writer that we will see later, however, where changing the columns order could be mandatory. In addition to the existing field, which can be added by using the * special character, we could also ask for the projection of the implicit score field. A composition of these two options could be seen in the following query: >>curl -X GET 'http://localhost:8983/solr/paintings/select?q=artist:*&wt=json&indent=true&omitHeader=true&fl=*,score' This will return every field for every document, including the score field explicitly, which is sometimes called a pseudo-field, to distinguish it from the field defined by a schema. Time for action – selecting documents with filter query Sometimes it's useful to be able to narrow the collection of documents on which we are currently performing our search. It is useful to add some kind of explicit linked condition on the logical side for navigation on data, and will also have good impact on performances too. It is shown in the following example: It shows how the default search is restricted by the introduction of a fq=annunciation condition. What just happened? The first result in this simple example shows that we obtain results similar to what we could have obtained by a simple q=annunciation search. Filtered query can be cached (as well as facets, that we will see later), improving performance by reducing the overhead of performing the same query many times, and accessing documents of large datasets to the same group many times. In this case the analogy with SQL seems less convincing, but q=dali and fq=abstract:painting can be seen corresponding to WHERE conditions in SQL. The fq parameters will then be a fixed condition. In our scenario, we could define for example specific endpoints with pre-defined filter query by author, to create specific channels. In this case instead of passing the parameters every time we could set them on solrconfig.xml. Time for action – searching for similar terms with Fuzzy search Even if the wildcard queries are very flexible, sometimes they simply cannot give us a good results. There could be some weird typo in the term, and we still want to obtain some good results wherever it is possible under certain confidence conditions: If I want to write painting and I actually search for plainthing, for example: >> curl – X GET 'http://localhost:8983/solr/paintings/select?q=abstract:plainthing~0.5&wt=json' Suppose we have a person using a different language, who searched for leonardo by misspelling the name: >> curl -X GET 'http://localhost:8983/solr/paintings/select?q=abstract:lionardo~0.5&wt=json' In both cases the examples use misspelled words to be more recognizable, but the same syntax can be used for intercept existing similar words. What just happened? Both the preceding examples work as expected. The first gives us documents containing the term painting, the second gives us documents containing leonardo instead. Note that the syntax plainthing^0.5 represents a query that matches with a certain confidence, so for example we will also obtain occurrences of documents with the term paintings, which is good, but on a more general case we could receive weird results. In order to properly set up the confidence value there are not many options, apart from doing tests. Using fuzzy search is a simple way to obtain a suggested result for alternate forms of search query, just like when we trust some search engine's similar suggestions in the did you mean approaches.
Read more
  • 0
  • 0
  • 1945

article-image-handling-dom-dart
Packt
24 Dec 2013
15 min read
Save for later

Handling the DOM in Dart

Packt
24 Dec 2013
15 min read
(For more resources related to this topic, see here.) A Dart web application runs inside the browser (HTML) page that hosts the app; a single-page web app is more and more common. This page may already contain some HTML elements or nodes, such as <div> and <input>, and your Dart code will manipulate and change them, but it can also create new elements. The user interface may even be entirely built up through code. Besides that, Dart is responsible for implementing interactivity with the user (the handling of events, such as button-clicks) and the dynamic behavior of the program, for example, fetching data from a server and showing it on the screen. We explored some simple examples of these techniques. Compared to JavaScript, Dart has simplified the way in which code interacts with the collection of elements on a web page (called the DOM tree). This article teaches you this new method using a number of simple examples, culminating with a Ping Pong game. The following are the topics: Finding elements and changing their attributes Creating and removing elements Handling events Manipulating the style of page elements Animating a game Ping Pong using style(s) How to draw on a canvas – Ping Pong revisited Finding elements and changing their attributes All web apps import the Dart library dart:html; this is a huge collection of functions and classes needed to program the DOM (look it up at api.dartlang.org). Let's discuss the base classes, which are as follows: The Navigator class contains info about the browser running the app, such as the product (the name of the browser), its vendor, the MIME types supported by the installed plugins, and also the geolocation object. Every browser window corresponds to an object of the Window class, which contains, amongst many others, a navigator object, the close, print, scroll and moveTo methods, and a whole bunch of event handlers, such as onLoad, onClick, onKeyUp, onMouseOver, onTouchStart, and onSubmit. Use an alert to get a pop-up message in the web page, such as in todo_v2.dart: window.onLoad.listen( (e) => window.alert("I am at your disposal") ); If your browser has tabs, each tab opens in a separate window. From the Window class, you can access local storage or IndexedDB to store app data on the client The Window object also contains an object document of the Document class, which corresponds to the HTML document. It is used to query for, create, and manipulate elements within the document. The document also has a list of stylesheets (objects of the StyleSheet class)—we will use this in our first version of the Ping Pong game. Everything that appears on a web page can be represented by an object of the Node class; so, not only are tags and their attributes nodes, but also text, comments, and so on. The Document object in a Window class contains a List<Node> element of the nodes in the document tree (DOM) called childNodes. The Element class, being a subclass of Node, represents web page elements (tags, such as <p>, <div>, and so on); it has subclasses, such as ButtonElement, InputElement, TableElement, and so on, each corresponding to a specific HTML tag, such as <button>, <input>, <table>, and so on. Every element can have embedded tags, so it contains a List<Element> element called children. Let us make this more concrete by looking at todo_v2.dart, solely for didactic purposes—the HTML file contains an <input> tag with the id value task, and a <ul> tag with the id value list: <div><input id="task" type="text" placeholder="What do you want to do?"/> <p id="para">Initial paragraph text</p> </div> <div id="btns"> <button class="backgr">Toggle background color of header</button> <button class="backgr">Change text of paragraph</button> <button class="backgr">Change text of placeholder in input field and the background color of the buttons</button> </div> <div><ul id="list"/> </div> In our Dart code, we declare the following objects representing them: InputElement task; UListElement list; The following list object contains objects of the LIElement class, which are made in addItem(): var newTask = new LIElement(); You can see the different elements and their layout in the following screenshot: The screen of todo_v2 Finding elements Now we must bind these objects to the corresponding HTML elements. For that, we use the top-level functions querySelector and querySelectorAll; for example, the InputElement task is bound to the <input> tag with the id value task using: task = querySelector('#task'); . Both functions take a string (a CSS selector) that identifies the element, where the id value task will be preceded by #. CSS selectors are patterns that are used in .css files to select elements that you want to style. There are a number of them, but, generally, we only need a few basic selectors (for an overview visit http://www.w3schools.com/cssref/css_selectors.asp). If the element has an id attribute with the value abc, use querySelector('#abc') If the element has a class attribute with value abc, use querySelector('.abc') To get a list of all elements with the tag <button>, use querySelectorAll('button') To get a list of all text elements, use querySelectorAll('input[type="text"]') and all sorts of combinations of selectors; for example, querySelectorAll('#btns .backgr') will get a list of all elements with the backgr class that are inside a tag with the id value btns These functions are defined on the document object of the web page, so in code you will also see document.querySelector() and document.querySelectorAll(). Changing the attributes of elements All objects of the Element class have properties in common, such as classes, hidden, id, innerHtml, style, text, and title; specialized subclasses have additional properties, such as value for a ProgressElement method. Changing the value of a property in an element makes the browser re-render the page to show the changed user interface. Experiment with todo_v2.dart: import 'dart:html'; InputElement task; UListElement list; Element header; List<ButtonElement> btns; main() { task = querySelector('#task'); list = querySelector('#list'); task.onChange.listen( (e) => addItem() ); // find the h2 header element: header = querySelector('.header'); (1) // find the buttons: btns = querySelectorAll('button'); (2) // attach event handler to 1st and 2nd buttons: btns[0].onClick.listen( (e) => changeColorHeader() ); (3) btns[1].onDoubleClick.listen( (e) => changeTextPara() ); (4) // another way to get the same buttons with class backgr: var btns2 = querySelectorAll('#btns .backgr'); (5) btns2[2].onMouseOver.listen( (e) => changePlaceHolder() );(6) btns2[2].onClick.listen((e) => changeBtnsBackColor() ); (7) addElements(); } changeColorHeader() => header.classes.toggle('header2'); (8) changeTextPara() => querySelector('#para').text = "You changed my text!"; (9) changePlaceHolder() => task.placeholder = 'Come on, type something in!'; (10) changeBtnsBackColor() => btns.forEach( (b) => b.classes.add('btns_backgr')); (11) void addItem() { var newTask = new LIElement(); (12) newTask.text = task.value; (13) newTask.onClick.listen( (e) => newTask.remove()); task.value = ''; list.children.add(newTask); (14) } addElements() { var ch1 = new CheckboxInputElement(); (15) ch1.checked = true; document.body.children.add(ch1); (16) var par = new Element.tag('p'); (17) par.text = 'I am a newly created paragraph!'; document.body.children.add(par); var el = new Element.html('<div><h4><b>A small divsection</b></h4></div>'); (18) document.body.children.add(el); var btn = new ButtonElement(); btn.text = 'Replace'; btn.onClick.listen(replacePar); document.body.children.add(btn); var btn2 = new ButtonElement(); btn2.text = 'Delete all list items'; btn2.onClick.listen( (e) => list.children.clear() ); (19) document.body.children.add(btn2); } replacePar(Event e) { var el2 = new Element.html('<div><h4><b>I replaced this div!</b></h4></div>'); el.replaceWith(el2); (20) } Comments for the numbered lines are as follows: We find the <h2> element via its class. We get a list of all the buttons via their tags. We attach an event handler to the Click event of the first button, which toggles the class of the <h2> element, changing its background color at each click (line (8)). We attach an event handler to the DoubleClick event of the second button, which changes the text in the <p> element (line (9)). We get the same list of buttons via a combination of CSS selectors. We attach an event handler to the MouseOver event of the third button, which changes the placeholder in the input field (line (10)). We attach a second event handler to the third button; clicking on it changes the background color of all buttons by adding a new CSS class to their classes collection (line (11)). Every HTML element also has an attribute Map where the keys are the attribute names; you can use this Map to change an attribute, for example: btn.attributes['disabled'] = 'true'; Please refer to the following document to see which attributes apply to which element: https://developer.mozilla.org/en-US/docs/HTML/Attributes Creating and removing elements The structure of a web page is represented as a tree of nodes in the Document Object Model (DOM). A web page can start its life with an initial DOM tree, marked up in its HTML file, and then the tree can be changed using code; or, it can start off with an empty tree, which is then entirely created using code in the app, that is every element is created through a constructor and its properties are set in code. Elements are subclasses of Node; they take up a rectangular space on the web page (with a width and height). They have, at most, one parent Element in which they are enclosed and can contain a list of Elements—their children (you can check this with the function hasChildNodes() that returns a bool function). Furthermore, they can receive events. Elements must first be created before they can be added to the list of a parent element. Elements can also be removed from a node. When elements are added or removed, the DOM tree is changed and the browser has to re-render the web page. An Element object is either bound to an existing node with the querySelector method of the document object or it can be created with its specific constructor, such as that in line (12) (where newTask belongs to the class LIElement—List Item element) or line (15). If useful, we could also specify the id in the code, such as in newTask.id = 'newTask'; If you need a DOM element in different places in your code, you can improve the performance of your app by querying it only once, binding it to a variable, and then working with that variable. After being created, the element properties can be given a value such as that in line (13). Then, the object (let's name it elem) is added to an existing node, for example, to the body node with document.body.children.add(elem), as in line (16), or to an existing node, as list in line (14). Elements can also be created with two named constructors from the Element class: Like Element.tag('tagName') in line (17), where tagName is any valid HTML tag, such as <p>, <div>, <input>, <select>, and so on. Like Element.html('htmlSnippet') in line (18), where htmlSnippet is any valid combination of HTML tags. Use the first constructor if you want to create everything dynamically at runtime; use the second constructor when you know what the HTML for that element will be like and you won't need to reference its child elements in your code (but by using the querySelector method, you can always find them if needed). You can leave the type of the created object open using var, or use the type Element, or use the specific class name (such as InputElement)—use the latter if you want your IDE to give you more specific code completion and warnings/errors against the possible misuse of types. When hovering over a list item, the item changes color and the cursor becomes a hand icon; this could be done in code (try it), but it is easier to do in the CSS file: #list li:hover { color: aqua; font-size:20 px; font-weight: bold; cursor: pointer; } To delete an Element elem from the DOM tree, use elem.remove(). We can delete list items by clicking on them, which is coded with only one line: newTask.onClick.listen( (e) => newTask.remove() ); To remove all the list items, use the List function clear(), such as in line (19). Replace elem with another element elem2 using elem.replaceWith(elem2), such as in line (20). Handling events When the user interacts with the web form, such as when clicking on a button or filling in a text field, an event fires; any element on the page can have events. The DOM contains hooks for these events and the developer can write code (an event handler) that the browser must execute when the event fires. How do we add an event handler to an element (which is also called registering an event handler)?. The general format is: element.onEvent.listen( event_handler ) (The spaces are not needed, but can be used to make the code more readable). Examples of events are Click, Change, Focus, Drag, MouseDown, Load, KeyUp, and so on. View this as the browser listening to events on elements and, when they occur, executing the indicated event handler. The argument that is passed to the listen() method is a callback function and has to be of the type EventListener; it has the signature: void EventListener(Event e) The event handler gets passed an Event parameter, succinctly called e or ev, that contains more specific info on the event, such as which mouse button should be pressed in case of a mouse event, on which object the event took place using e.target, and so on. If an event is not handled on the target object itself, you can still write the event handler in its parent, or its parent's parent, and so on up the DOM tree, where it will also get executed; in such a situation, the target property can be useful in determining the original event object. In todo_v2.dart, we examine the various event-coding styles. Using the general format, the Click event on btns2[2] can be handled using the following code: btns2[2].onClick.listen( changeBtnsBackColor ); where changeBtnsBackColor is either the event handler or callback function. This function is written as: changeBtnsBackColor(Event e) => btns.forEach( (b) => b.classes.add('btns_backgr')); Another, shorter way to write this (such as in line (7)) is: btns2[2].onClick.listen( (e) => changeBtnsBackColor() ); changeBtnsBackColor() => btns.forEach( (b) => b.classes.add('btns_backgr')); When a Click event occurs on btns2[2], the handler changeBtnsBackColor is called. In case the event handler needs more code lines, use the brace syntax as follows: changeBtnsBackColor(Event e) { btns.forEach( (b) => b.classes.add('btns_backgr')); // possibly other code } Familiarize yourself with these different ways of writing event handlers. Of course, when the handler needs only one line of code, there is no need for a separate method, as in the following code: newTask.onClick.listen( (e) => newTask.remove() ); For clarity, we use the function expression syntax => whenever possible, but you can inline the event handler and use the brace syntax along with an anonymous function, thus avoiding a separate method. So instead of executing the following code: task.onChange.listen( (e) => addItem() ); we could have executed: task.onChange.listen( (e) { var newTask = new LIElement(); newTask.text = task.value; newTask.onClick.listen( (e) => newTask.remove()); task.value = ''; list.children.add(newTask); } ); JavaScript developers will find the preceding code very familiar, but it is also used frequently in Dart code, so make yourself acquainted with the code pattern ( (e) {...} );. The following is an example of how you can respond to key events (in this case, on the window object) using the keyCode and ctrlKey properties of the event e: window.onKeyPress .listen( (e) { if (e.keyCode == KeyCode.ENTER) { window.alert("You pressed ENTER"); } if (e.ctrlKey && e.keyCode == CTRL_ENTER) { window.alert("You pressed CTRL + ENTER"); } }); In this code, the constant const int CTRL_ENTER = 10; is used. (The list of keyCodes can be found at http://www.cambiaresearch.com/articles/15/javascript-char-codes-key-codes). Manipulating the style of page elements CSS style properties can be changed in the code as well: every element elem has a classes property, which is a set of CSS classes. You can add a CSS class as follows: elem.classes.add ('cssclass'); as we did in changeBtnsBackColor (line (11)); by adding this class, the new style is immediately applied to the element. Or, we can remove it to take away the style: elem.classes.remove ('cssclass'); The toggle method (line (8)) elem.classes.toggle('cssclass'); is a combination of both: first the cssclass is applied (added), the next time it is removed, and, the time after that, it is applied again, and so on. Working with CSS classes is the best way to change the style, because the CSS definition is separated from the HTML markup. If you do want to change the style of an element directly, use its style property elem.style, where the cascade style of coding is very appropriate, for example: newTask.style ..fontWeight = 'bold' ..fontSize = '3em' ..color = 'red';
Read more
  • 0
  • 0
  • 5445

article-image-adding-raster-layers
Packt
24 Dec 2013
9 min read
Save for later

Adding Raster Layers

Packt
24 Dec 2013
9 min read
(For more resources related to this topic, see here.) This article will cover about working with raster layers. The collection of sections is composed of most common use cases regarding the handling of raster layers in the Google Maps JavaScript API. Raster is one of the prime data types in the GIS world and Google Maps JavaScript API presents an extensive set of tools to integrate external sources of imagery. Also, the API enables the application developers to change the styling of its own base maps with a palette of practically unlimited array of choices. This article will introduce you to change the styling of base maps and then continue with the display of raster data, focusing on external Tiled Map Services (TMS) where the raster layer is composed of organized tiles in the map display.(ex: Openstreetmap). Lastly, there a number of raster layers (traffic, transit, weather, bicycle and Panoramio) that are to be presented within their integration with Google Maps JavaScript API. Styling of Google base maps Google base maps show a variety of details, such as water bodies (oceans, seas, rivers, lakes etc.), roads, parks, and built-up areas (residential, industrial and so on.). As you have observed in the first article, all these are shown in predefined cartographic parameters. With the styling capability of base maps, you have a virtually unlimited set of choices in terms of cartographic representation of base maps. In your web or mobile applications, it is very beneficial to have diversity of representations (in all different colour schemes with different emphasis) in terms of keeping your audience more involved and having maps blend in nicely into your website design. The following steps will guide you through the process of changing the styling of base maps. Getting ready… We can continue from the 1st part of Article 1 – Google Maps JavaScript API Basics – as we do not need to call back the basics of getting the map and so on. How to do it… Your result will look like bluish Google Maps, if you follow the steps below: Create an array of styles as follows: var bluishStyle = [ { stylers: [ { hue: "#009999" }, { saturation: -5 }, { lightness: -40 } ] },{ featureType: "road", elementType: "geometry", stylers: [ { lightness: 100 }, { visibility: "simplified" } ] }, { featureType: "water", elementType: "geometry", stylers: [ { hue: "#0000FF" }, {saturation:-40} ] }, { featureType: "administrative.neighborhood", elementType: "labels.text.stroke", stylers: [ { color: "#E80000" }, {weight: 1} ] },{ featureType: "road", elementType: "labels.text", stylers: [ { visibility: "off" } ] }, { featureType: "road.highway", elementType: "geometry.fill", stylers: [ { color: "#FF00FF" }, {weight: 2} ] } ] Add your styles array to the initMap() function. Within the initMap() function, create a styledMapType object with its name and referencing the your styles array: var bluishStyledMap = new google.maps.StyledMapType(bluishStyle, {name: "Bluish Google Base Maps with Pink Highways"}); Add mapTypeControlOptions object having mapTypeIds property to your original mapOptions object: var mapOptions = { center: new google.maps.LatLng(39.9078, 32.8252), zoom: 10, mapTypeControlOptions: {mapTypeIds: [google.maps.MapTypeId.ROADMAP, 'new_bluish_style']} }; Relate the new mapTypeId to your styledMapType object: map.mapTypes.set('new_bluish_style', bluishStyledMap); And lastly, set this new mapTypeId to be displayed: map.setMapTypeId('new_bluish_style'); You can observe the bluish styled Google base maps, as seen above. How it works... Firstly, let's observe the bluishStyle array consisting of one or more google.maps.MapTypeStyle objects arranged like this: var bluishStyle = [ { featureType: '', elementType: '', stylers: [ {hue: ''}, {saturation: ''}, {lightness: ''}, // etc... ] }, { featureType: '', // etc... } ] In this array, you can include several styles (specified in google.maps.MapTypeStyleElementType) for different map features (specified in google.maps.MapTypeStyleFeatureType) and their respective elements – their geometries, labels, and so on (specified in google.maps.MapTypeStyleElementType). Map features embrace the types of geographic representations that are found in the base maps. Administrative areas, landscape features, points of interest, roads and water bodies are examples of map features. In addition to these general definitions of map features, Google Maps JavaScript API enables you to specify subtypes of these features. For example, you may wish to be specific on which poi types to change the default style by giving the featureType property as follows: featureType: 'poi.school' Or, you can be more specific on landscape map features: featureType: 'landscape.man_made' More about google.maps.MapTypeStyleFeatureType object Complete listing on MapTypeStyleFeatureType object can be found at (https://developers.google.com/maps/documentation/javascript/reference#MapTypeStyleFeatureType) in details. Please note that, the first element of our styles array does not include any featureType property, making the styler options be valid for the entire base map: { stylers: [ { hue: "#009999" }, { saturation: -5 }, { lightness: -40 } ] } In addition to google.maps.MapTypeStyleFeatureType and their constants, you can also detail google.maps.MapTypeStyleElementType for each map feature: the geometries, geometry strokes and fills, labels, label texts (also, text fill and stroke) and label icons. Having this opportunity, you can style the geometries of roads in different settings than their related icons. In our article, you have disabled the visibility of all road label texts, not touching their geometry or label icons: { featureType: "road", elementType: "labels.text", stylers: [ { visibility: "off" } ] } More about google.maps.MapTypeStyleElementType object Complete listing on MapTypeStyleElementType object can be found at (https://developers.google.com/maps/documentation/javascript/reference#MapTypeStyleElementType) in details. For every map feature type and its element type, you can specify a google.maps.MapTypeStyler that covers the options of hue, lightness, saturation, gamma, inverse_lightness, visibility and weight options as an array. In our article, the styler options that make the highway road appear as pink are: { featureType: "road.highway", elementType: "geometry.fill", stylers: [ { color: "#FF00FF" }, {weight: 2} ] } where color option in the stylers array is a RGB Hex string of a pink tone and weight defines the weight of the feature in pixels. More about google.maps.MapTypeStyler object Complete listing on MapTypeStyler object can be found at (https://developers.google.com/maps/documentation/javascript/reference#MapTypeStyler) in details. After defining the styles array in our initMap() function, we have created a StyledMapType object: var bluishStyledMap = new google.maps.StyledMapType(bluishStyle, {name: "Bluish Google Base Maps with Pink Highways"}); This object takes two arguments, first one is the styles array, the second one is a google.maps.StyledMapTypeOptions object. . Here, we have included only the name property, however you can additionally include maxZoom and minZoom properties between which this StyledMapType object will be displayed. You can note that, the value that we have assigned for the name property is displayed in the interface, as you can see in the screen snapshot of this article. Once we have created the StyledMapType object, we have added an additional object called mapTypeControlOptions that takes mapTypeIds array in the mapOptions object replacing the mapTypeId property: var mapOptions = { center: new google.maps.LatLng(39.9078, 32.8252), zoom: 10, mapTypeControlOptions: {mapTypeIds: [google.maps.MapTypeId.ROADMAP, 'new_bluish_style']} }; This enables us to add multiple styles in addition to the standard ROADMAP map type. Next comes the step of linking the mapTypeId ('new_bluish_style') that we have specified in mapTypeIds array with the StyledMapType object (bluishStyledMap): map.mapTypes.set('new_bluish_style', bluishStyledMap); After linking the mapTypeId with the StyledMapType, we can finish with: map.setMapTypeId('new_bluish_style'); so that, the map interface opens with a base map styled within our intentions. In our article, we have covered how to style the base maps according to our taste. We have made use of google.maps.MapTypeStyle object to select the features types (google.maps.MapTypeStyleFeatureType) and related elements (google.maps.MapTypeStyleElementType) and styled them using the google.maps.MapTypeStyler object. Then, we have added our StyledMapType object to the map, showing our own styling of the base maps of Google Maps. There's more... Using StyledMapType object is only one of way handling user defined styled base maps in Google Maps JavaScript API. Another, yet simpler usage is through specifying the styles array in mapOptions object's styles property: var mapOptions = { center: new google.maps.LatLng(39.9078, 32.8252), zoom: 10, mapTypeId: google.maps.MapTypeId.ROADMAP, styles: bluishStyle }; Or; after defining our mapOptions object, we can add the styles property later by: map.setOptions({styles: bluishStyle }); There is an important difference between using StyledMapType object and mapOptions object's style property. Using StyledMapType object enables us to define a number of (virtually infinite) styles as map types. In addition, these map types can be seen in the map type control in the map interface, so that it is very easy to change back and forth between map types for the user. However, if the styles are attached to the map by mapOptions object's style property, there is no way for the user to change between multiple styles. In fact, in the map type control, there will be option for your new selecting the styles, because styles are not attached to a StyledMapType object, therefore cannot be identified as map types. Styled Map Wizard http://gmaps-samples-v3.googlecode.com/svn/trunk/styledmaps/wizard/index.html Preparing style arrays is a detailed job, with many cartographic details. Finding the correct combination in stylers for each feature and element type would take too much time, especially if you have the only way of editing in a text editor. Google has done a great job on preparing "The Styled Map Wizard" for easing this time consuming task that enables you to perform all your styling in an interface so, you can overview what you are changing in real time. After you finish your work, you can export your changes as JSON to be used as styles array in your application. Summary In this article, we presented the addition of external raster data through a series of steps alongside Google layers such as the Tile, Traffic, Transit, and Weather layers. Resources for Article: Further resources on this subject: Including Google Maps in your Posts Using Apache Roller 4.0 [Article] QR Codes, Geolocation, Google Maps API, and HTML5 Video [Article] Google Earth, Google Maps and Your Photos: a Tutorial Part II [Article]
Read more
  • 0
  • 0
  • 1639

article-image-creating-ice-and-snow-materials
Packt
24 Dec 2013
8 min read
Save for later

Creating the ice and snow materials

Packt
24 Dec 2013
8 min read
(For more resources related to this topic, see here.) Getting ready We will create the ice and the snow using a single material, and mix them using a new technique. Select the iceberg mesh and add a new material to it. How to do it... We will now see the steps required to create the ice as well as the snow material. Creating ice The following are the steps to create ice: Add Glossy BSDF and Glass BSDF and mix them using a Mix Shader node. Let's put Glossy in the first socket and Glass in the second one. As input for the color of both the BSDFs, we will use a color Mix node with Color1 as white and Color2 as RGB 0.600, 1.00, 0.760. As input for the Fac value of the color Mix node, we will use a Voronoi Texture node with the Generated coordinates, Intensity mode, and Scale 100. Invert the color output using an Invert node and plug it into the Fac value of the color Mix node. As the input for Roughness of the Glossy BSDF, we will use the Layer Weight node's Facing output with a Blend value of 0.800. Then we will plug this into a ColorRamp node and set the color stops as shown in the following screenshot. The first color stop is HSV 0.000, 0.000, 0.090 and the second one is HSV 0.000, 0.000, 0.530. Remember to plug the ColorRamp node into the Glossy BSDF roughness socket. Finally, set Glass BSDF node's IOR to 1.310 and Roughness to 0.080. Now we will create the Fac input for the Shader Mix node of the two BSDFs. Now we will add Noise Texture to the Generated coordinates with a Scale of 130, Detail of 1, and Distortion of 0.500. Plug this into a ColorRamp node and set the color stops as shown in the the following screenshot: Let's now add a Subsurface Scattering node. Set the mode to Compatible, the Scale to 10.000 and the Radius to 0.070, 0.070, 0.10. As a color input, let's add another color Mix node with Color1 as RGB 0.780, 0.960, 1.000 and Color2 as RGB 0.320, 0.450, 0.480. The Fac input for the color Mix node will be the same as for the color Mix node of the Glass and Glossy BSDFs. Now mix the SSS node with the mix of the other two BSDFs, using an Add Shader node. Now, we will create the normals for the shader. Add three Image Texture nodes. In the first one, let's load the IceScratches.jpg file. We will use the Generated coordinates with a Scale of XYZ 20.000, 20.000, 20.000. Set the projection mode to Box and the Blend to 0.500. In the second Image Texture node, load the ice_snow_DISP.pngfile, using the UV coordinates. Finally, load the ice_snow_NRM.png file in the third Image Texture node, using again the UV textures. Now let's mix the IceScratches.jpg with the ice_snow_DISP.png textures, using a color Mix node with the Displacement Texture into the Color1 socket and the scratches texture into the Color2 socket. Set the Fac value to 0.100. Plug the mix of the textures into the Height socket of a Bump node and then plug the ice_snow_NRM.png texture into the Color socket of a Normal Map node. Finally, plug this one into the Normal socket of the Bump node. Set the Normal Map node's Strength to 0.050, the Bump node's Strength to 0.500 and the Distance to 1.000. Plug the Bump node into all of the BSDFs we added so far. Frame every node we created and label the frame ICE. Creating snow The nodes we will add now will still be within the same material, but outside the ICE group we just created. Add a Glossy and a Subsurface Scatter BSDF nodes. Mix them using a Mix Shader node with 20 percent of influence from the Glossy BSDF node. Set both the colors to white. Also, set the SSS Scale to 3.00 and the Radius to 0.400, 0.400, 0.450. Set the Glossy mode to GGX and the Roughness to 0.600. Add a Noise Textures node and set Scale to 2000.000, Detail to 2.000 and Distortion to 0.000. We will use Generated coordinates for this texture. Connect the Fac output of the Noise Texture node to the Height socket of the Bump node and set the Strength to 0.200 and the Distance to 1.000. Connect the Normal output of the Noise Texture node to the Normal input of the Subsurface Scatter BSDF and Glossy BSDF nodes. Now let's mix the Mix Shader node of the Subsurface Scatter BSDF and Glossy BSDF nodes with an Emission shader using another Mix Shader node. Add new Noise Textures, this time with Scale as 2500.000, Detail as 2.000, and Distortion as 0.000. Connect the Fac output of the Noise Texture node to the Color input of the Gamma node, with a Gamma value of 8.000, and then add the Color output of the Gamma node to the Fac input of the ColorRamp node. We will set up the color stops as seen in the next screenshot. Connect the ColorRamp node's Color socket to the Fac socket of the previous Mix Shader node. Remember to use the Color output of the ColorRamp node. Frame all these nodes and label the frame SNOW. Mixing ice and snow Add a Geometry(Add | Input) and a Normal node (Add | Vectors). Connect the Normal output from the Geometry node to the Normal input of the Normal node. Now connect the Dot output of the Normal node to the first socket of a Multiply node and set the mode to Multiply and the second value to 2.000. Add a Mix shader node and connect the ICE frame into the first Shader socket and the SNOW frame into the second one. Finally, connect the output of the Multiply node into the Fac value of the Mix Shader node. How it works... Let's see the most interesting points of this material in detail. For the ice material, we used a Voronoi Texture node to create a pattern for the surface color. Then we mixed the Glass and Glossy BSDF nodes using a Noise Texture node to simulate both, the more and less transparent areas: for example, the ice may be less transparent due to some part of it being covered with snow, difference in purity of the water, or the thickness of the ice. Then we mixed the two BSDFs with a Subsurface Scatter node to simulate the dispersion of the lighting inside the ice. Note that we used the ColorRamp node quite often in order to fine tune the various mixing and inputs. The snow material is quite similar for the main concept, but is missing the refractive part of the ice. Here we did something else. We used a Noise Texture node with some tweaking (Gamma and ColorRamp) to make it really contrasted to mix an emission shader to the rest of the material. This will create a small emission dot over the snow surface that we will use in compositing to create the flakes. It is really interesting how we mixed the two materials. We wanted the snow to be placed only on the flat surfaces of the iceberg, while we wanted the slopes to be just ice. To obtain this effect, we extracted the normal information about the mesh and used it to understand which part of the mesh is facing upward. We must imagine the normals to be working like the sunlight falling on the surface of the earth. Half of the sphere is in darkness, and half is hit by the light. We can decide from which direction the light hits the surface. Now imagine the same principle applied to the shape of our mesh. In this way we can create a white mask on the areas that are hit by the normal sphere direction. With the Normal node, we can orient this effect wherever we want. The default position of the sphere is exactly what we needed: the parts of the mesh that are faced upward are made white, while the rest of the mesh is black. Turning the sphere around will make the direction of the ramp that has been created, change accordingly. The sphere, maybe, is not the best way to set these kind of things as it lacks a bit of precision (as for the setting of the sky), but this will probably change in the future with something that allows more precise settings. Finally we used a Multiply node to multiply the value coming from the Normal node and increased the contrast of the mask. There's more... The normal method we just saw in this article is not the only way of mixing materials. Just some time ago, two new plugins have been developed. The first one allows us to obtain the same results we got in this article by creating a weight map or a vertex paint based on the slope of the mesh, while the second creates the same based on the altitude. This opens up many possibilities not only in terms of material creation, but also for the distribution of particle systems! The plugins can be downloaded from the following links, where we can find some instructions about them: http://blenderthings.blogspot.nl/2013/09/height-to-vertex-weights-blender-addon.html See also In the following link, Andrew Price teaches us how to create a different kind of ice material; for example, material that is more suitable for ice cubes. Surely worth a watch! http://www.blenderguru.com/videos/how-to-create-realistic-ice/
Read more
  • 0
  • 0
  • 27160
article-image-working-tooltips
Packt
23 Dec 2013
6 min read
Save for later

Working with Tooltips

Packt
23 Dec 2013
6 min read
(For more resources related to this topic, see here.) The jQuery team introduced their version of the tooltip as part of changes to Version 1.9 of the library; it was designed to act as a direct replacement for the standard tooltip used in all browsers. The difference here, though, was that whilst you can't style the standard tooltip, jQuery UI's replacement is intended to be accessible, themeable, and completely customizable. It has been set to display not only when a control receives focus, but also when you hover over that control, which makes it easier to use for keyboard users. Implementing a default tooltip Tooltips were built to act as direct replacements for the browser's native tooltips. They will recognize the default markup of the title attribute in a tag, and use it to automatically add the additional markup required for the widget. The target selector can be customized though using tooltip's items and content options. Let's first have a look at the basic structure required for implementing tooltips. In a new file in your text editor, create the following page: <!DOCTYPE HTML> <html> <head> <meta charset="utf-8"> <title>Tooltip</title> <link rel="stylesheet" href="development- bundle/themes/redmond/jquery.ui.all.css"> <style> p { font-family: Verdana, sans-serif; } </style> <script src = "js/jquery-2.0.3.js"></script> <script src = "development- bundle/ui/jquery.ui.core.js"></script> <script src = "development-bundle/ui/jquery.ui.widget.js"> </script> <script src = "development-bundle/ui/jquery.ui.position.js"> </script> <script src = "development-bundle/ui/jquery.ui.tooltip.js"> </script> <script> $(document).ready(function($){ $(document).tooltip(); }); </script> </head> <body> <p>Lorem ipsum dolor sit amet, consectetur adipiscing elit. Nulla blandit mi quis imperdiet semper. Fusce vulputate venenatis fringilla. Donec vitae facilisis tortor. Mauris dignissim nibh ac justo ultricies, nec vehicula ipsum ultricies. Mauris molestie felis ligula, id tincidunt urna consectetur at. Praesent <a href="http://www. ipsum.com" title="This was generated from www.ipsum.com">blandit</a> faucibus ante ut semper. Pellentesque non tristique nisi. Ut hendrerit tempus nulla, sit amet venenatis felis lobortis feugiat. Nam ac facilisis magna. Praesent consequat, risus in semper imperdiet, nulla lorem aliquet nisi, a laoreet nisl leo rutrum mauris.</p> </body> </html> Save the code as tooltip1.html in your jqueryui working folder. Let's review what was used. The following script and CSS resources are needed for the default tooltip widget configuration: jquery.ui.all.css jquery-2.0.3.js jquery.ui.core.js jquery.ui.widget.js jquery.ui.tooltip.js The script required to create a tooltip, when using the title element in the underlying HTML can be as simple as this, which should be added after the last <script> element in your code, as shown in the previous example: <script> $(document).ready(function($){ $(document).tooltip(); }); </script> In this example, when hovering over the link, the library adds in the requisite aria described by the code for screen readers into the HTML link. The widget then dynamically generates the markup for the tooltip, and appends it to the document, just before the closing </body> tag. This is automatically removed as soon as the target element loses focus. ARIA, or Accessible Rich Internet Applications, provides a way to make content more accessible to people with disabilities. You can learn more about this initiative at https://developer.mozilla.org/en-US/docs/Accessibility/ARIA. It is not necessary to only use the $(document) element when adding tooltips. Tooltips will work equally well with classes or selector IDs; using a selector ID, will give a finer degree of control. Overriding the default styles When styling the Tooltip widget, we are not limited to merely using the prebuilt themes on offer, we can always elect to override existing styles with our own. In our next example, we’ll see how easy this is to accomplish, by making some minor changes to the example from tooltip1.html. In a new document, add the following styles, and save it as tooltipOverride.css, within the css folder: p { font-family: Verdana, sans-serif; } .ui-tooltip { background: #637887; color: #fff; } Don't forget to link to the new style sheet from the <head> of your document: <link rel="stylesheet" href="css/tooltipOverride.css"> Before we continue, it is worth explaining a great trick for styling tooltips before committing the results to code. If you are using Firefox, you can download and install the Toggle JS add-on for Firefox, which is available from https://addons.mozilla.org/en-US/firefox/addon/toggle-js/. This allows us to switch off JavaScript on a per-page basis; we can then hover over the link to create the tooltip, before expanding the markup in Firebug and styling it at our leisure. Save your HTML document as tooltip2.html. When we run the page in a browser, you should see the modified tooltip appear when hovering over the link in the text: Using prebuilt themes If creating completely new styles by hand is overkill for your needs, you can always elect to use one of the prebuilt themes that are available for download from the jQuery UI site. This is a really easy change to make. We first need to download a copy of the replacement theme; in our example, we’re going to use one called Excite Bike. Let’s start by browsing to http://jqueryui.com/download/, then deselecting the Toggle All option. We don’t need to download the whole library, just the theme at the bottom, change the theme option to display Excite Bike then select Download. Next, open a copy of tooltip2.html then look for this line: <link rel="stylesheet" href="development-bundle/themes/redmond /jquery.ui.all.css"> You will notice the highlighted word in the above line. This is the name of the existing theme. Change this to excite-bike then save the document as tooltip3.html, then remove the tooltipOverride.css link, and you’re all set. The following is our replacement theme in action: With a single change of word, we can switch between any of the prebuilt themes available for use with jQuery UI (or indeed even any of the custom ones that others have made available online), as long as you have downloaded and copied the theme into the appropriate folder. There may be occasions though, were we need to tweak the settings. This gives us the best of both worlds, where we only need to concentrate on making the required changes. Let’s take a look at how we can alter an existing theme, using ThemeRoller.
Read more
  • 0
  • 0
  • 12279

article-image-creating-direct2d-game-window-class
Packt
23 Dec 2013
12 min read
Save for later

Creating a Direct2D game window class

Packt
23 Dec 2013
12 min read
(For more resources related to this topic, see here.) To put some graphics on the screen; the first step for us would be creating a new game window class that will use Direct2D. This new game window class will derive from our original game window class, while adding the Direct2D functionality. Open Visual Studio. Add a new class to the project called GameWindow2D. We need to change its declaration to: public class GameWindow2D : GameWindow, IDispoable As you can see, it inherits from the GameWindow class meaning that it has all of the public and protected members of the GameWindow class, as though we had implemented them again in this class. It also implements the IDisposable interface, just as the GameWindow class does. Also, don't forget to add a reference to SlimDX to this project if you haven't already. We need to add some using statements to the top of this class file as well. They are all the same using statements that the GameWindow class has, plus one more. The new one is SlimDX.Direct2D. They are as follows: using System.Windows.Forms; using System.Diagnostics; using System.Drawing; using System; using SlimDX; using SlimDX.Direct2D; using SlimDX.Windows; Next, we need to create a handful of member variables: WindowRenderTarget m_RenderTarget; Factory m_Factory; PathGeometry m_Geometry; SolidColorBrush m_BrushRed; SolidColorBrush m_BrushGreen; SolidColorBrush m_BrushBlue; The first variable is a WindowRenderTarget object. The term render target is used to refer to the surface we are going to draw on. In this case, it is our game window. However, this is not always the case. Games can render to other places as well. For example, rendering into a texture object is used to create various effects. One example would be a simple security camera effect. Say, we have a security camera in one room and a monitor in another room. We want the monitor to display what our security camera sees. To do this, we can render the camera's view into a texture, which can then be used to texture the screen of the monitor. Of course, this has to be re-done in every frame so that the monitor screen shows what the camera is currently seeing. This idea is useful in 2D too. Back to our member variables, the second one is a Factory object that we will be using to set up our Direct2D stuff. It is used to create Direct2D resources such as RenderTargets. The third variable is a PathGeometry object that will hold the geometry for the first thing we will draw, which will be a rectangle. The last three variables are all SolidColorBrush objects. We use these to specify the color we want to draw something with. There is a little more to them than that, but that's all we need right now. The constructor Let's turn our attention now to the constructor of our Direct2D game window class. It will do two things. Firstly, it will call the base class constructor (remember the base class is the original GameWindow class), and it will then get our Direct2D stuff initialized. The following is the initial code for our constructor: public GameWindow2D(string title, int width, int height,   bool fullscreen)     : base(title, width, height, fullscreen) {     m_Factory = new Factory();     WindowRenderTargetProperties properties = new       WindowRenderTargetProperties();     properties.Handle = FormObject.Handle;     properties.PixelSize = new Size(width, height);     m_RenderTarget = new WindowRenderTarget(m_Factory,       properties); } In the preceding code, the line starting with a colon is calling the constructor of the base class for us. This ensures that everything inherited from the base class is initialized. In the body of the constructor, the first line creates a new Factory object and stores it in our m_Factory member variable. Next, we create a WindowRenderTargetProperties object and store the handle of our RenderForm object in it. Note that FormObject is one of the properties defined in our GameWindow base class. Remember that the RenderForm object is a SlimDX object that represents a window for us to draw on. The next line saves the size of our game window in the PixelSize property. The WindowRenderTargetProperties object is basically how we specify the initial configuration for a WindowRenderTarget object when we create it. The last line in our constructor creates our WindowRenderTarget object, storing it in our m_RenderTarget member variable. The two parameters we pass in are our Factory object and the WindowRenderTargetProperties object we just created. A WindowRenderTarget object is a render target that refers to the client area of a window. We use the WindowRenderTarget object to draw in a window. Creating our rectangle Now that our render target is set up, we are ready to draw stuff, but first we need to create something to draw! So, we will add a bit more code at the bottom of our constructor. First, we need to initialize our three SolidColorBrush objects. Add these three lines of code at the bottom of the constructor: m_BrushRed = new SolidColorBrush(m_RenderTarget, new Color4(1.0f,   1.0f, 0.0f, 0.0f)); m_BrushGreen = new SolidColorBrush(m_RenderTarget, new   Color4(1.0f, 0.0f, 1.0f, 0.0f)); m_BrushBlue = new SolidColorBrush(m_RenderTarget, new Color4(1.0f,   0.0f, 0.0f, 1.0f)); This code is fairly simple. For each brush, we pass in two parameters. The first parameter is the render target we will use this brush on. The second parameter is the color of the brush, which is an ARGB (Alpha Red Green Blue) value. The first parameter we give for the color is 1.0f. The f character on the end indicates that this number is of the float data type. We set alpha to 1.0 because we want the brush to be completely opaque. A value of 0.0 will make it completely transparent, and a value of 0.5 will be 50 percent transparent. Next, we have the red, green, and blue parameters. These are all float values in the range 0.0 to 1.0 as well. As you can see for the red brush, we set the red channel to 1.0f and the green and blue channels are both set to 0.0f. This means we have maximum red, but no green or blue in our color. With our SolidColorBrush objects set up, we now have three brushes we can draw with, but we still lack something to draw! So, let's fix that by adding some code to make our rectangle. Add this code to the end of the constructor: m_Geometry = new PathGeometry(m_RenderTarget.Factory); using (GeometrySink sink = m_Geometry.Open()) {     int top = (int) (0.25f * FormObject.Height);     int left = (int) (0.25f * FormObject.Width);     int right = (int) (0.75f * FormObject.Width);     int bottom = (int) (0.75f * FormObject.Height);     PointF p0 = new Point(left, top);     PointF p1 = new Point(right, top);     PointF p2 = new Point(right, bottom);     PointF p3 = new Point(left, bottom);     sink.BeginFigure(p0, FigureBegin.Filled);     sink.AddLine(p1);     sink.AddLine(p2);     sink.AddLine(p3);     sink.EndFigure(FigureEnd.Closed);     sink.Close(); } This code is a bit longer, but it's still fairly simple. The first line creates a new PathGeometry object and stores it in our m_Geometry member variable. The next line starts the using block and creates a new GeometrySink object that we will use to build the geometry of our rectangle. The using block will automatically dispose of the GeometrySink object for us when program execution reaches the end of the using block. The using blocks only work with objects that implement the IDisposable interface. The next four lines calculate where each edge of our rectangle will be. For example, the first line calculates the vertical position of the top edge of the rectangle. In this case, we are making the rectangle's top edge be 25 percent of the way down from the top of the screen. Then, we do the same thing for the other three sides of our rectangle. The second group of four lines of code creates four Point objects and initializes them using the values we just calculated. These four Point objects represent the corners of our rectangle. A point is also often referred to as a vertex. When we have more than one vertex, we call them vertices (pronounced as vert-is-ces). The final group of code has six lines. They use the GeometrySink and the Point objects we just created to set up the geometry of our rectangle inside the PathGeometry object. The first line uses the BeginFigure() method to begin the creation of a new geometric figure. The next three lines each add one more line segment to the figure by adding another point or vertex to it. With all four vertices added, we then call the EndFigure() method to specify that we are done adding vertices. The last line calls the Close() method to specify that we are finished adding geometric figures, since we can have more than one if we want. In this case, we are only adding one geometric figure, our rectangle. Drawing our rectangle Since our rectangle never changes, we don't need to add any code to our UpdateScene() method. We will override the base class's UpdateScene() method anyway, in case we need to add some code in here later, which is given as follows: public override void UpdateScene(double frameTime) {     base.UpdateScene(frameTime); } As you can see, we only have one line of code in this override modifier of the base class's UpdateScene() method. It simply calls the base class's version of this method. This is important because the base class's UpdateScene() method contains our code that gets the latest user input data each frame. Now, we are finally ready to write the code that will draw our rectangle on the screen! We will override the RenderScene() method so we can add our custom code. The following is the code: public override void RenderScene() {     if ((!this.IsInitialized) || this.IsDisposed)     {         return;     }     m_RenderTarget.BeginDraw();     m_RenderTarget.Clear(ClearColor);     m_RenderTarget.FillGeometry(m_Geometry, m_BrushBlue);     m_RenderTarget.DrawGeometry(m_Geometry, m_BrushRed, 1.0f);     m_RenderTarget.EndDraw(); } First, we have an if statement, which happens to be identical to the one we put in the base class's RenderScene() method. This is because we are not calling the base class's RenderScene() method, since the only code in it is this if statement. Not calling the base class version of this method will give us a slight performance boost, since we don't have the overhead of that function call. We could do the same thing with the UpdateScene() method as well. In this case we didn't though, because the base class version of that method has a lot more code in it. In your own projects you may want to copy and paste that code into your override of the UpdateScene() method. The next line of code calls the render target's BeginDraw() method to tell it that we are ready to begin drawing. Then, we clear the screen on the next line by filling it with the color stored in the ClearColor property that is defined by our GameWindow base class. The last three lines draw our geometry twice. First, we draw it using the FillGeometry() method of our render target. This will draw our rectangle filled in with the specified brush (in this case, solid blue). Then, we draw the rectangle a second time, but this time with the DrawGeometry() method. This draws only the lines of our shape but doesn't fill it in, so this draws a border on our rectangle. The extra parameter on the DrawGeometry() method is optional and specifies the width of the lines we are drawing. We set it to 1.0f, which means the lines will be one-pixel wide. And the last line calls the EndDraw() method to tell the render target that we are finished drawing. Cleanup As usual, we need to clean things up after ourselves when the program closes. So, we need to add override of the base class's Dispose(bool) method. We've already done this a few times, so it should be somewhat familiar and is not shown here. Our blue rectangle with a red border As you might guess, there is a lot more you can do with drawing geometry. You can draw curved line segments and draw shapes with gradient brushes too for example. You can also draw text on the screen using the render target's DrawText() method. But since we have limited space on these pages, we're going to look at how to draw bitmap images on the screen. These images are something that make up the graphics of most 2D games. Summary In this article, we first made a simple demo application that drew a rectangle on the screen. Then, we got a bit more ambitious and built a 2D tile-based game world. Resources for Article: Further resources on this subject: HTML5 Games Development: Using Local Storage to Store Game Data [Article] Flash Game Development: Creation of a Complete Tetris Game [Article] Interface Designing for Games in iOS [Article]
Read more
  • 0
  • 0
  • 13536

article-image-getting-started-fusion-applications
Packt
23 Dec 2013
10 min read
Save for later

Getting Started with Fusion Applications

Packt
23 Dec 2013
10 min read
(For more resources related to this topic, see here.) Oracle Fusion Applications (OFA) is an exciting entrant in the Enterprise Resource Planning (ERP) application arena. It is based on leading practices leveraged from Oracle's existing ERP suites, namely Oracle E-Business Suite, Oracle PeopleSoft, and Siebel. However, it fundamentally changes the architecture and approach for end-to-end ERP functionality. To state that Oracle Fusion Applications is unique is an understatement. The application has been architected differently from any prior ERP application. It has been developed using industry-leading platforms that seamlessly work together. It can be deployed in multiple models, from on-premise to cloud, and various hybrid options in between. This book will progressively explain each new feature of Oracle Fusion Applications. It will provide a detailed explanation of what each feature does, and how to configure it. Best practices are specified, where there are multiple implementation options to help the reader make an informed decision. In this introductory section, we will put the uniqueness of Oracle Fusion Applications in context by looking at the evolution of ERP applications. We will look at the architecture and deployment models for Oracle Fusion Applications. We will then review the technology platforms on which Oracle Fusion Applications is built. Changing the application's ecosystem Oracle Fusion Applications and the business functionalities that it provides have evolved. New business processes have usually been the trigger for driving change in systems and applications. However, in recent years, technology has opened up novel approaches to perform business processes, and in some cases, opened up entirely new business domains (for example, Cloud or Elastic computings). There is a healthy history of business needs and technology advances complementing each other. However, within the age of computers, technology has led to business innovation. Business users have benefited from the computing capability and flexibility that technical developments have brought to businesses. The PC era put meaningful computing capability on the desktop. This drove a change from mainframe-based monolithic data centers to business processes that empowered the end user. An inventory report was no longer something that arrived once a month. Business users now had the capability to get business information they needed, when they needed it. This capability profoundly changed the speed of business, and more importantly, the speed in which business decisions were made. Departments such as sales, engineering, and finance now had visibility to each other's vital data. This led to efficiencies in business and ultimately higher profits and customer satisfaction. This flexibility in the business domain during the client-server period soon extended to the consumer with the advent of the Internet. Consumers could now interact directly with businesses, which enabled a completely new dimension of customer centricity. Business users saw a delayed benefit of the Internet. The need to be interconnected outside the organization evolved with the advent of concepts, such as Service Oriented Architecture (SOA). This introduced business-process flexibility as well as business-process delegation. A need for business-process orchestration was realized. The advent of elastic computing based on cloud-based computing models has created entirely new possibilities for businesses. Now, entire applications and their intrinsic functionalities are made available on demand, and can scale to business needs. This has opened up new avenues of doing business as well as business models that have never been seen before. Applications have started to evolve to embrace this new approach, and Oracle Fusion Applications is one such example. The following diagram visualizes the evolution of computing paradigms and their impact on businesses. It shows how applications evolved along with changing business models. It provides an overlay with the functionality that was inherent within those applications. The diagram shows a gradual migration of functionality away from the core of the organization to a highly distributed model. This evolution can be explained by the technology drivers that were available during each computing era. In the mainframe era where computing resources were at a premium, applications were focused on computations. Interactions with the mainframes required advanced skills and training. There was marginal attention paid to the user interface and distributed computing. With the advent of the client-server model, computing resources became distributed, and this led to the first era of true business applications. The client layer focused on presentation and simplistic data validation. The server layer provided data persistence and complex processing. This approach allowed the first wave of true ERP applications to be developed. A whole host of applications from multiple vendors surfaced during this period. This included applications such as Oracle E-Business Suite, PeopleSoft, Siebel, SAP, and Great Plains, apart from several industry-specific applications. These server-oriented ERP applications had complex data structures, which resulted in large monolithic databases. Also, the business-processing logic was tightly coupled with the database structures resulting in applications that provided good business functionality, but they had two major drawbacks, which are as follows: They were hard to modify or customize Integrations outside of the applications were complex Middleware and SOA With the advent of the Internet, there was a move towards moving business applications to a web-based frontend. This resulted in the evolution of middleware that could insulate the data layer from the presentation layer. However, functionality provided by ERP applications did not fundamentally change. The Internet simply provided a powerful new mechanism for users to interact with the business applications. Internet-based connectivity did provide new approaches for business-to-business (B2B) communication. This also led to progress in the areas of business-to-customer (B2C) capabilities. However, core business processes in the areas of Finance, Order Management, Supply Chain, and Human Resource Management remained mostly unchanged. No more monolithic applications The introduction of middleware provided a unique ability for business applications to communicate with each other. This communication could now be real time and could be orchestrated for multiple back-and-forth messaging patterns. This led to the development of standards that would allow the exchange of information between applications. It also resulted in a specialized approach in business processing, wherein niche applications for specialized functionality could be integrated with larger ERP applications. Typical examples of this were specialized customer service systems integrated with ERP systems, or asset management systems integrated with core financial systems. The following diagram shows the separation of the presentation and data layers. Middleware orchestrates the business processes, and in many cases, also contains the business logic that is the core process for the organization: While the middleware products were evolving, the SOA-based approach for service integration had also been maturing. The confluence of these two accelerated the ability to weave together functionality across applications. The following diagram illustrates the realization of the SOA-based approach in the middleware layer. Middleware products such as Oracle Fusion Middleware provide comprehensive message transformation and routing capabilities. However, the ability to provide process orchestration is vital to aligning business processes with siloed applications. As seen in the following diagram, when applications are connected using middleware, they appear as providers of functionality. This functionality is then orchestrated into an end-to-end business process by the middleware connecting these applications. The technology stack is now much more closely aligned with the business process than it has ever been in the past. Middleware and SOA approaches allowed the orchestration of processes that spanned multiple applications. The large ERP applications that had appeared "monolithic" up to this point now resembled the silos of functionality. ERP systems started to appear restrictive as they were limited in their flexibility, but continued to be the primary storage of institutional data. The immense flexibility provided by SOA (enabled by middleware) stood in stark contrast to the restrictions imposed by "big-box" ERP software. The impact of Cloud Cloud-based deployments provided the ability to provide elastic resource capabilities. This has several technical and cost advantages. However, from a business perspective, it opened up the possibility of a "plug-in" functionality that did not have to be developed in-house. This was a tremendously liberating aspect for the business, and its impact is similar to the dawn of the Internet age. You could now browse for functionality that was required, and pay-per-use for the same. Using middleware, this functionality could be integrated with back-end systems. It is this empowering of the business that is now fundamentally changing the ERP space. The uniqueness of Oracle Fusion Applications The Oracle Fusion Middleware product suite has integration and process orchestration features required for this new generation of ERP applications. It provides the platform for integration and process orchestration that is vital to a flexible ERP. Oracle has leveraged its Fusion Middleware capability to build Oracle Fusion Applications. However, Oracle Fusion Applications are different in the following unique ways: Leading practices: Oracle Fusion Applications is built on leading practices from Oracle's stable of existing ERP products. It has those features that were considered the most meaningful and commonly used from these existing ERP product lines. Oracle Fusion Middleware: Oracle Fusion Middleware does provide comprehensive SOA-based integration. It is an industry leading middleware platform. Oracle Fusion Applications is built on this platform, and hence, inherits the value that technology stacks brings. Oracle Fusion Middleware brings some additional benefits to Oracle Fusion Applications, which are as follows: Standards based on the J2EE platform: This is important as using Java (as opposed to a proprietary language) opens up a vast community of people who are familiar with the technology platform. Web-based frontend: The UI is completely web-based. There is no thick client that needs to be deployed on client desktops. BPEL-based workflow: Prebuilt business process flows are built using BPEL workflows using the BPMN notification. This replaces the implicit workflow technology that used to be highly proprietary to the application vendor. Deployment: Oracle Fusion Applications has multiple deployment options. These include the following: On-premise: This provides a bare metal install on in-house hardware. This involves a full install of the technology's platform and the entire application. Cloud model: This is similar to the on-premise model except that the application is installed on the infrastructure that is on the cloud. SaaS model: SaaS stands for Software as a Service model. The application is hosted by Oracle and the subscription to the application is on a pay-per-use model. The users are not responsible for maintaining the application. The cloud models have the additional option of being single tenant (single business users for the entire application), or multi-tenant (multiple businesses running within the same application). Thus, Oracle Fusion Applications has a wide range of deployment options that can scale based on the needs of an organization. The uniqueness of Oracle Fusion Applications is summarized in the following diagram: The combination of a technology platform along with leading business practices, delivered across multiple channels, makes Oracle Fusion Applications a unique ERP. It is best suited to provide the value that business users expect from their business applications. The architecture of Oracle Fusion Applications The architecture of Oracle Fusion Applications is shown in the following diagram at a high-level to set a baseline understanding. Components of this architecture will be discussed in detail throughout this book. Administration of key components and areas that need special attention will also be addressed. The following diagram shows the foundational building blocks on which Oracle Fusion Applications is built. It also lists the guiding design principles that have driven the design and architecture of the overall application. Summary In this article we have discussed the evolution of Enterprise Resource Applications and the impact that technology has had on this evolution. We have also seen how functionality can be externalized to create a unique approach to business applications. The architecture of Oracle Fusion Applications was reviewed, and the uniqueness of this suite of applications was also discussed. Resources for Article: Further resources on this subject: Remote Job Agent in Oracle 11g Database with Oracle Scheduler [Article] Introduction to Oracle Service Bus & Oracle Service Registry [Article] Oracle Integration and Consolidation Products [Article]
Read more
  • 0
  • 0
  • 1630
Packt
23 Dec 2013
5 min read
Save for later

GLSL – How to Set up the Shaders from the Host Application Side

Packt
23 Dec 2013
5 min read
(For more resources related to this topic, see here.) Setting up geometry Let's say that our mesh (a quad formed by two triangles) has the following information: vertex positions and texture coordinates. Also, we will arrange the data interleaved in the array. struct MyVertex { float x, y, z; float s, t; }; MyVertex geometry[] =  {{0,1,0,0,1}, {0,0,0,0,0}, {1,1,0,1,1},{1,0,0,1,0}}; // Let's create the objects that will encapsulate our geometry data GLuint vaoID, vboID; glGenVertexArray(1, &vaoID); glBindVertexArray(vaoID); glGenBuffers(1, &vboID); glBindBuffer(GL_ARRAY_BUFFER, vboID); // Attach our data to the OpenGL objects glBufferData(GL_ARRAY_BUFFER, 4 * sizeof(MyVertex), &geometry[0].x,GL_DYNAMIC_DRAW); // Specify the format of each vertex attribute glEnableVertexAttribArray(0); glVertexAttribPointer(0, 3, GL_FLOAT, GL_FALSE, sizeof(MyVertex), NULL); glEnableVertexAttribArray(1); glVertexAttribPointer(1, 2, GL_FLOAT, GL_FALSE, sizeof(MyVertex),(void*)(sizeof(float)*3)); At this point, we created the OpenGL objects, set up correctly each vertex attribute format and uploaded the data to GPU. Setting up textures Setting up textures follows the same pattern. First create the OpenGL objects, then fill the data and the format in which it is provided. const int width = 512; const int height = 512; const int bpp = 32; struct RGBColor { unsigned char R,G,B,A; }; RGBColor textureData[width * height]; for(size_t y = 0; y < height; ++y) for(size_t x = 0; x < width; ++x)                textureData[y*height+x] = …; //fill your texture here // Create GL object GLuint texID; glGenTextures(1, &texID); glBindTexture(GL_TEXTURE_2D, texID); // Fill up the data and set the texel format glTexImage2D(GL_TEXTURE_2D, 0, GL_RGBA, width, height, 0, GL_RGBA,GL_UNSIGNED_BYTE, &textureData[0].R); // Set texture format data: interpolation and clamping modes glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR); glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_LINEAR); glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_S, GL_REPEAT); glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_T, GL_REPEAT); And that's all about textures. Later we will use the texture ID to place the texture into a slot, and that slot number is the information that we will pass to the shader to tell it where the texture is placed in order to locate it. Setting up shaders In order to setup the shaders, we have to carry out some steps:  load up the source code, compile it and associate to a shader object, and link all shaders together into a program object. char* vs[1]; vs[0] = "#version 430\nlayout (location = 0) in vec3 PosIn;layout (location = 1) in vec2 TexCoordIn;smooth out vec2 TexCoordOut;uniform mat4 MVP; void main() { TexCoordOut = TexCoordIn;gl_Position = MVP * vec4(PosIn, 1.0);}"; char fs[1]; fs = "#version 430\n uniform sampler2D Image; smooth in vec2 TexCoord;out vec4 FBColor;void main() {FBColor = texture(Image, TexCoord);}"; // Upload source code and compile it GLuint pId, vsId, fsId; vsId = glCreateShader(GL_VERTEX_SHADER); glShaderSource(vsId, 1, (const char**)&vs, NULL); glCompileShader(vsId); // Check for compilation errors GLint status = 0, bufferLength = 0; glGetShaderiv(vsId, GL_COMPILE_STATUS, &status); if(!status) { char* infolog = new char[bufferLength + 1]; glGetShaderiv(vsId, GL_INFO_LOG_LENGTH, &bufferLength); glGetShaderInfoLog(vsId, bufferLength, NULL, infolog); infolog[bufferLength] = 0; printf("Shader compile errors / warnings: %s\n", infolog); delete [] infolog; } The process for the fragment shader is exactly the same. The only change is that the shader object must be created as fsId = glCreateShader(GL_FRAGMENT_SHADER); // Now let's proceed to link the shaders into the program object pId = glCreateProgram(); glAttachShader(pId, vsId); glAttachShader(pId, fsId); glLinkProgram(pId); glGetProgramiv(pId, GL_LINK_STATUS, &status); if(!status) { char* infolog = new char[bufferLength + 1]; glGetProgramiv(pId, GL_INFO_LOG_LENGTH, &bufferLength); infolog[bufferLength] = 0; printf("Shader linking errors / warnings: %s\n", infolog); delete [] infolog; } // We do not need the vs and fs anymore, so it is same mark them for deletion. glDeleteShader(vsId); glDeleteShader(fsId); The last things to upload to the shader are two uniform variables: the one that corresponds with the view-projection matrix and the one that represents the texture. Those are uniform variables and are set in the following way: // First bind the program object where we want to upload the variables glUseProgram(pId); // Obtain the "slot number" where the uniform is located in GLint location = glGetUniformLocation(pId, "MVP"); float mvp[16] = {1, 0, 0, 0, 0, 1, 0, 0, 0, 0, 1, 0, 0, 0, 0, 1}; // Set the data into the uniform's location glUniformMatrix4fv(location, 1, GL_FALSE, mvp); // Active the texture slot 0 glActiveTexture(GL_TEXTURE0); // Bind the texture to the active slot glBindTexture(GL_TEXTURE_2D, texID); location = glGetUniformLocation(pId, "Image"); // Upload the texture's slot number to the uniform variable int imageSlot = 0; glUniform1i(location, imageSlot); And that's all. For the other types of shaders, process is all the same: Create shader object, upload source code, compile and link, but using the proper OpenGL types such as GL_GEOMETRY_SHADER or GL_COMPUTE_SHADER. A last step, to draw all these things, is to establish them as active and issue the draw call: glBindVertexArray(vaoID); glBindTexture(GL_TEXTURE_2D, texID); glUseProgram(pId); glDrawArrays(GL_TRIANGLE_STRIP, 0, 4); Resources for Article: Further resources on this subject: The Basics of GLSL 4.0 Shaders [Article] GLSL 4.0: Using Subroutines to Select Shader Functionality [Article] Getting Started with GLSL [Article]
Read more
  • 0
  • 0
  • 19166

article-image-social-engineering-attacks
Packt
23 Dec 2013
6 min read
Save for later

Social Engineering Attacks

Packt
23 Dec 2013
6 min read
(For more resources related to this topic, see here.) Advanced Persistent Threats Advanced Persistent Threats came into existence during cyber espionage between nations. The main motive for these attacks was monetary gain and lucrative information. APTs normally target specific organization. A targeted attack is the prerequisite for APT. The targeted attack commonly exploits vulnerability of the application on the target. The attacker normally crafts a mail with malicious attachment, such as any PDF document or an executable, which is e-mailed to the individual or group of individuals .Generally, these e-mails are prepared with some social engineering element to make it lucrative to the target. The typical characteristics of APTs are as follows: Reconnaissance: The attacker motivates to get to know the target and their environment, specific users, system configurations, and so on. This type of information can be obtained from the metadata tool collector from the targeted organization document, which can be easily collected from the target organization website by using tools such as FOCA, metagoofil, libextractor, and CeWL. Time-to-live: In APTs, attackers utilize the techniques to bypass security and deploy a backdoor to make the access for longer period of time and placing a backdoor so they can always come back in case their actual presence was detected. Advance malware: Attacker utilizes the polymorphic malware in the APT. The polymorphic generally changes throughout its cycle to fool AV detection mechanism. Phishing: Most APT exploiting the target machine application is started by social engineering and spear phishing. Once the target machine is compromised or network credentials are given up, the attackers actively take steps to deploy their own tools to monitor and spread through the network as required, from machine-to-machine, and network-to-network, until they find the information they are looking for. Active Attack: In APT, there is a human element involvement. Everything is not an automatic malicious code. Famous attacks classified under APTs are as follows: Operation Aurora McRAT Hydraq Stuxnet RSA SecureID attacks The APT life cycle covers five phases, which are as follows: Phase 1: Reconnaissance Phase 2: Initial exploitation Phase 3: Establish presence and control Phase 4: Privilege escalation Phase 5: Data extraction Phase 1: Reconnaissance It's a well-known saying, "A war is half won by not only on our strength but also how much we know our enemy". The attacker generally gathers information from variety of sources as initial preparation so definitely it applies to the defendant. Information on specific people mostly higher management people who posses important information, information about specific events, setting up initial attack point and application vulnerability. So there are multiple places such as Facebook, LinkedIn, Google, and many more where the attacker tries to find the information. There are tools that generally assist Social Engineering Framework (SEF) that we have included in the book, Kali Linux Social Engineering, and another one that I suggest is Foca Meta data collector. An attack would be planned based on the information gathering. So employee awareness program must be continuously run to make employees aware that they should not to highlight themselves on the Internet and should be better prepared to defend against these attack. Phase 2: Initial exploitation A spear-phishing attack is considered one of the most advanced targeting attacks, and they are also called advance persistent threat (APT) attacks. Today, many cyber criminals use spear phishing attack to initial exploit the machine. The objective of performing spear-phishing is to gain long term access to different resources of the target for ex-government, military network, or satellite usage. The main motivation of performing such attacks is to gain access to IT environment and utilize zero day exploit found in initial information gathering phase. Why this attack is considered most dangerous because the attacker can spoof its e-mail ID by sending a malicious e-mail. There is a complete example in graphical format implementation of this has been included in the book Kali Linux Social Engineering. Phase 3: Establishing presence and control The main objective of this stage is to deploy full range of attack tools such as backdoor and rootkits to start controlling the environment and stay undetected. The organization need to take care of the outbound connection to deter such attacks because the attack tools make the outbound connection to attacker. Phase 4: Privilege escalation This is one of the key phase in the APT. Once the attacker has breached the network, the next step is to take over the privilege accounts to move the around the targeted network. So the common objective of the attacker is to obtain an administrator level credentials and stay undetected. The best approach to defend against these attacks "assume that the attackers are inside our networks right now and proceed accordingly by blocking the pathways they're travelling to access and steal our sensitive data. Phase 5: Data extraction This is the stage where the attacker has control over one or two machine in the targeted network and have obtained access credentials to supervise it's reach and identified the lucrative data. The only objective left for the attacker to start sending the data from targeted network to one of its own server or on his own machine The attacker has number of option what he can do with this data. The attacker can ask for ransom if the target does not agree to pay the amount, he can threaten to disclose the information in the public, share the zero day exploits, sell the information, or public disclosure. APT defense The defense against the APT attacks mostly based on its characteristics. The APT attacks normally bypass the Network Firewall Defense by attaching exploits within the content carried over the allowed protocol .So deep content filtering is required. In most of the APT attacks custom-developed code or targeting zero day vulnerability is used so no single IPS or antivirus signature will be able to identify the threat so must reply on less definitive indicators. The organization must ask himself what they are trying to protect and perhaps they can apply layer of data loss prevention (DEP) technology. The organization needs to monitor both inbound or outbound network preferably for both web and e-mail communication. Summary In this article, we learned what are APTs and the types of APTs. Resources for Article: Further resources on this subject: Web app penetration testing in Kali [Article] Debugging Sikuli scripts [Article] Customizing a Linux kernel [Article]
Read more
  • 0
  • 0
  • 4326
Modal Close icon
Modal Close icon