How-To Tutorials

(For more resources related to this topic, see here.) Establishing the architecture The recent componentization within Hadoop allows any distributed system to use it for resource management. In Hadoop 1.0, resource management was embedded into the MapReduce framework as shown in the following diagram: Hadoop 2.0 separates out resource management into YARN, allowing other distributed processing frameworks to run on the resources managed under the Hadoop umbrella. In our case, this allows us to run Storm on YARN as shown in the following diagram: As shown in the preceding diagram, Storm fulfills the same function as MapReduce. It provides a framework for the distributed computation. In this specific use case, we use Pig scripts to articulate the ETL/analysis that we want to perform on the data. We will convert that script into a Storm topology that performs the same function, and then we will examine some of the intricacies involved in doing that transformation. To understand this better, it is worth examining the nodes in a Hadoop cluster and the purpose of the processes running on those nodes. Assume that we have a cluster as depicted in the following diagram: There are two different components/subsystems shown in the diagram. The first is YARN, which is the new resource management layer introduced in Hadoop 2.0. The second is HDFS. Let's first delve into HDFS since that has not changed much since Hadoop 1.0. Examining HDFS HDFS is a distributed filesystem. It distributes blocks of data across a set of slave nodes. The NameNode is the catalog. It maintains the directory structure and the metadata indicating which nodes have what information. The NameNode does not store any data itself, it only coordinates create, read, update, and delete (CRUD) operations across the distributed filesystem. Storage takes place on each of the slave nodes that run DataNode processes. The DataNode processes are the workhorses in the system. They communicate with each other to rebalance, replicate, move, and copy data. They react and respond to the CRUD operations of clients. Examining YARN YARN is the resource management system. It monitors the load on each of the nodes and coordinates the distribution of new jobs to the slaves in the cluster. The ResourceManager collects status information from the NodeManagers. The ResourceManager also services job submissions from clients. One additional abstraction within YARN is the concept of an ApplicationMaster. An ApplicationMaster manages resource and container allocation for a specific application. The ApplicationMaster negotiates with the ResourceManager for the assignment of resources. Once the resources are assigned, the ApplicationMaster coordinates with the NodeManagers to instantiate containers. The containers are logical holders for the processes that actually perform the work. The ApplicationMaster is a processing-framework-specific library. Storm-YARN provides the ApplicationMaster for running Storm processes on YARN. HDFS distributes the ApplicationMaster as well as the Storm framework itself. Presently, Storm-YARN expects an external ZooKeeper. Nimbus starts up and connects to the ZooKeeper when the application is deployed. The following diagram depicts the Hadoop infrastructure running Storm via Storm-YARN: As shown in the preceding diagram, YARN is used to deploy the Storm application framework. At launch, Storm Application Master is started within a YARN container. That, in turn, creates an instance of Storm Nimbus and the Storm UI. After that, Storm-YARN launches supervisors in separate YARN containers. Each of these supervisor processes can spawn workers within its container. Both Application Master and the Storm framework are distributed via HDFS. Storm-YARN provides command-line utilities to start the Storm cluster, launch supervisors, and configure Storm for topology deployment. We will see these facilities later in this article. To complete the architectural picture, we need to layer in the batch and real-time processing mechanisms: Pig and Storm topologies, respectively. We also need to depict the actual data. Often a queuing mechanism such as Kafka is used to queue work for a Storm cluster. To simplify things, we will use data stored in HDFS. The following depicts our use of Pig, Storm, YARN, and HDFS for our use case, omitting elements of the infrastructure for clarity. To fully realize the value of converting from Pig to Storm, we would convert the topology to consume from Kafka instead of HDFS as shown in the following diagram: As the preceding diagram depicts, our data will be stored in HDFS. The dashed lines depict the batch process for analysis, while the solid lines depict the real-time system. In each of the systems, the following steps take place: Step Purpose Pig Equivalent Storm-Yarn Equivalent 1 The processing frameworks are deployed The MapReduce Application Master is deployed and started Storm-YARN launches Application Master and distributes Storm framework 2 The specific analytics are launched The Pig script is compiled to MapReduce jobs and submitted as a job Topologies are deployed to the cluster 3 The resources are reserved Map and reduce tasks are created in YARN containers Supervisors are instantiated with workers 4 The analyses reads the data from storage and performs the analyses Pig reads the data out of HDFS Storm reads the work, typically from Kafka; but in this case, the topology reads it from a flat file Another analogy can be drawn between Pig and Trident. Pig scripts compile down into MapReduce jobs, while Trident topologies compile down into Storm topologies. For more information on the Storm-YARN project, visit the following URL: https://github.com/yahoo/storm-yarn

(For more resources related to this topic, see here.) Installing matplotlib Before experimenting with matplotlib, you need to install it. Here we introduce some tips to get matplotlib up and running without too much trouble. How to do it... We have three likely scenarios: you might be using Linux, OS X, or Windows. Linux Most Linux distributions have Python installed by default, and provide matplotlib in their standard package list. So all you have to do is use the package manager of your distribution to install matplotlib automatically. In addition to matplotlib, we highly recommend that you install NumPy, SciPy, and SymPy, as they are supposed to work together. The following list consists of commands to enable the default packages available in different versions of Linux: Ubuntu: The default Python packages are compiled for Python 2.7. In a command terminal, enter the following command: sudo apt-get install python-matplotlib python-numpy python-scipy python-sympy ArchLinux: The default Python packages are compiled for Python 3. In a command terminal, enter the following command: sudo pacman -S python-matplotlib python-numpy python-scipy python-sympy If you prefer using Python 2.7, replace python by python2 in the package names Fedora: The default Python packages are compiled for Python 2.7. In a command terminal, enter the following command: sudo yum install python-matplotlib numpy scipy sympy There are other ways to install these packages; in this article, we propose the most simple and seamless ways to do it. Windows and OS X Windows and OS X do not have a standard package system for software installation. We have two options—using a ready-made self-installing package or compiling matplotlib from the code source. The second option involves much more work; it is worth the effort to have the latest, bleeding edge version of matplotlib installed. Therefore, in most cases, using a ready-made package is a more pragmatic choice. You have several choices for ready-made packages: Anaconda, Enthought Canopy, Algorete Loopy, and more! All these packages provide Python, SciPy, NumPy, matplotlib, and more (a text editor and fancy interactive shells) in one go. Indeed, all these systems install their own package manager and from there you install/uninstall additional packages as you would do on a typical Linux distribution. For the sake of brevity, we will provide instructions only for Enthought Canopy. All the other systems have extensive documentation online, so installing them should not be too much of a problem. So, let's install Enthought Canopy by performing the following steps: Download the Enthought Canopy installer from https://www.enthought.com/products/canopy. You can choose the free Express edition. The website can guess your operating system and propose the right installer for you. Run the Enthought Canopy installer. You do not need to be an administrator to install the package if you do not want to share the installed software with other users. When installing, just click on Next to keep the defaults. You can find additional information about the installation process at http://docs.enthought.com/canopy/quick-start.html. That's it! You will have Python 2.7, NumPy, SciPy, and matplotlib installed and ready to run. Plotting one curve The initial example of Hello World! for a plotting software is often about showing a simple curve. We will keep up with that tradition. It will also give you a rough idea about how matplotlib works. Getting ready You need to have Python (either v2.7 or v3) and matplotlib installed. You also need to have a text editor (any text editor will do) and a command terminal to type and run commands. How to do it... Let's get started with one of the most common and basic graph that any plotting software offers—curves. In a text file saved as plot.py, we have the following code: import matplotlib.pyplot as plt X = range(100) Y = [value ** 2 for value in X] plt.plot(X, Y) plt.show() Assuming that you installed Python and matplotlib, you can now use Python to interpret this script. If you are not familiar with Python, this is indeed a Python script we have there! In a command terminal, run the script in the directory where you saved plot.py with the following command: python plot.py Doing so will open a window as shown in the following screenshot: The window shows the curve Y = X ** 2 with X in the [0, 99] range. As you might have noticed, the window has several icons, some of which are as follows: : This icon opens a dialog, allowing you to save the graph as a picture file. You can save it as a bitmap picture or a vector picture. : This icon allows you to translate and scale the graphics. Click on it and then move the mouse over the graph. Clicking on the left button of the mouse will translate the graph according to the mouse movements. Clicking on the right button of the mouse will modify the scale of the graphics. : This icon will restore the graph to its initial state, canceling any translation or scaling you might have applied before. How it works... Assuming that you are not very familiar with Python yet, let's analyze the script demonstrated earlier. The first line tells Python that we are using the matplotlib.pyplot module. To save on a bit of typing, we make the name plt equivalent to matplotlib.pyplot. This is a very common practice that you will see in matplotlib code. The second line creates a list named X, with all the integer values from 0 to 99. The range function is used to generate consecutive numbers. You can run the interactive Python interpreter and type the command range(100) if you use Python 2, or the command list(range(100)) if you use Python 3. This will display the list of all the integer values from 0 to 99. In both versions, sum(range(100)) will compute the sum of the integers from 0 to 99. The third line creates a list named Y, with all the values from the list X squared. Building a new list by applying a function to each member of another list is a Python idiom, named list comprehension. The list Y will contain the squared values of the list X in the same order. So Y will contain 0, 1, 4, 9, 16, 25, and so on. The fourth line plots a curve, where the x coordinates of the curve's points are given in the list X, and the y coordinates of the curve's points are given in the list Y. Note that the names of the lists can be anything you like. The last line shows a result, which you will see on the window while running the script. There's more... So what we have learned so far? Unlike plotting packages like gnuplot, matplotlib is not a command interpreter specialized for the purpose of plotting. Unlike Matlab, matplotlib is not an integrated environment for plotting either. matplotlib is a Python module for plotting. Figures are described with Python scripts, relying on a (fairly large) set of functions provided by matplotlib. Thus, the philosophy behind matplotlib is to take advantage of an existing language, Python. The rationale is that Python is a complete, well-designed, general purpose programming language. Combining matplotlib with other packages does not involve tricks and hacks, just Python code. This is because there are numerous packages for Python for pretty much any task. For instance, to plot data stored in a database, you would use a database package to read the data and feed it to matplotlib. To generate a large batch of statistical graphics, you would use a scientific computing package such as SciPy and Python's I/O modules. Thus, unlike many plotting packages, matplotlib is very orthogonal—it does plotting and only plotting. If you want to read inputs from a file or do some simple intermediary calculations, you will have to use Python modules and some glue code to make it happen. Fortunately, Python is a very popular language, easy to master and with a large user base. Little by little, we will demonstrate the power of this approach.

(For more resources related to this topic, see here.) Element scenes in After Effects Element 3D scenes, unfortunately, are currently limited to five object groups. An object group is a collection of geometrical objects that is animated together. Under some circumstances, you may have multiple objects in a group, but not in this example. As a result, much of this article is devoted to tricks to overcome the limitations in Element 3D. We're going to cover the following topics: Saving your objects to your library so that they may be used in other instances of the Element plugin Setting up your objects in the After Effects interface and generating null objects to animate them easily Setting up your camera and lights in the scene and creating a new null object to create a focal point for the depth of field Let's get started! Saving your objects First things first: Let's create a folder in your Model Browser window. Navigate to your default objects directory for Element 3D (Documents/VideoCopilot/Models). Create a folder called BookModels inside that directory. Now, open your AEX project and go into the Scene Setup window in Element 3D. Your folder won't show up in the Model Browser (because there are no objects in that folder just yet). Let's save the lampshade glass to the presets now. The following screenshot shows the interface for saving your objects: Right-click on the lampshade glass object and select Save Model Preset. Now, you will see the directory we created. Unfortunately, there is no way to create a new folder from within this window yet. (Hence the need to navigate to the directory using your Windows Explorer (or Finder on Mac) to create the directory first.) Save all of your objects to this directory. Save them as shown in the following list: lampshade(glass) Lamp(base) Table WineBottle PepperShaker SaltShaker Note that we created a pepper shaker (in addition to our salt shaker). This was done by saving the salt shaker, then changing the texture do something darker, and saving it again as a pepper shaker. This can be great help when creating slight variants in your objects. The following screenshot shows how your Model Browser window should look: Preparing our scene Now that we have all our objects saved as presets, we can start populating our scene. Start off by deleting all your objects from the scene (click on the small X on the right side of the object bar in the Scene window). Don't worry. This is why we set them up as presets. Now we can bring them back in the right order. Setting up the lamp Let's start with the lamp. We'll be putting the lamp back in its lampshade and adding bulbs to the sockets. To do this, let's execute the following steps: In your Model Browser window, click on your lamp, then your lampshade, navigate to your starter pack, and click on bulb_on. Then in your Scene window, put the lamp in group 1, the shade in group 2, and the bulbs in group 3, as shown in the following screenshot: But wait. There's more! We have one bulb for four light sockets (it's a pretty huge bulb). That's okay! Element 3D wasn't really designed as a full 3D animation package. This is probably why it still has some of the limitations it does. Instead, it was designed as a way to use 3D objects as particles and replicate them in interesting ways. We'll get more into these features in the advanced sections, but it's time to get your feet wet by replicating the bulbs. Replicating the bulbs The shade and lamp should appear perfectly together (as if they were modeled together). The bulb is a stock asset of Element 3D, so it must be adjusted using the following steps: First, let's create a camera (so that we can accurately look around). Create a camera layer called Camera 1 using the stock's 50 mm preset. Remember, you can move your camera around using the camera tool (press C on your keyboard, then the mouse buttons will let you move your view around). Position your camera over the lamp so that we can see through the lampshades. Now, open the effect controls again for the ElementWineAndLamp layer. We don't need to touch anything in group 1 or 2 (that is, our lampshade and lamp). Since we created a symmetrical lamp, all we need to do is position one bulb and replicate it (like an array). Positioning and replicating an object Open Group 3 and then the Particle Replicator option. If we set the replicator shape to a plane, we will be able to replicate them all symmetrically on a plane. Pretty self explanatory, right? Let's set that first, as setting it later will change the coordinate system for our object, and as a result, it will look like it rests. Down the list, a little away from the Replicator Shape parameter are our Position XY, Position Z, and Scale Shape parameters. Mess with these to get your bulb directly in one of the bulb sockets. Then, you can turn the Particle Count parameter up to 4, and you'll end up with four bulbs positioned directly in the sockets. If they're not lined up, you're not out of luck! Directly under the Scale Shape parameter is Scale XYZ. It's important to note that Scale XYZ does not scale the object. Instead, it scales the replicator shape. There is no way in Element 3D to scale objects on their individual axis as of yet. Just remember, Scale Shape scales the geometry and Scale XYZ scales the replicator. You can also use the Particle Size parameter inside Particle Look to adjust object sizes. If you used your own lamp for this exercise, play around; you'll get it. If you downloaded the sample lamp, this screenshot shows the correct settings to get the bulbs into place. Don't worry if they're not exact. They aren't the hero objects in this animation. They're just there in case we see them either through the lampshades, or catch a glimpse. The result and setting should look similar to what is shown in the following screenshot: Lighting the lamp Luckily, lighting our scene is relatively easy, there's a lamp! Let's create an ambient light first. If our scene is going to be believable, we're going to need a blue light for ambient lighting. Create a new light with the type as Ambient, a light blue color, and an intensity of 10 %. Why so low? We want dramatic lighting in this scene. We'll keep the details without losing contrast with a very low ambient light. Call this light Ambient Light. The settings should look similar to what is shown in the following screenshot: So, why did we make it blue? Well, for a lamp to be on, it's probably night. If you look at any film footage, night is really denoted by shades of blue. Therefore, the light of night is blue. The same reason our lamp bulbs will have a bit of orangey-yellow to them. They're incandescent bulbs, which have a very warm look (also called color temperature). If you're going to be an animation god, you have to pay attention to everything around you: timings, lighting, and colors (actual colors and not just perceived colors). With all this in mind, let's create our first (of four) bulb lights by performing the following steps: Create a point light that is slightly orangey-yellow (be subtle) with an intensity of 100 percent. Don't worry just yet about any shadows or falloffs. These will actually be controlled within Element 3D. Name this light Light 1. The settings should look similar to what is shown in the following screenshot: Now, position this light directly on your first light bulb. Make sure you rotate your camera to ensure that it's placed properly. Once that's done, you can simply duplicate the light (Ctrl + D) and position the next on another bulb, either on the x or the z axis, and eventually have four lights (one on each bulb) in addition to your ambient light. The result should look like the following screenshot. Remember, lighting is the key to stunning looks. Always take the time to light your scenes well. The following screenshot shows the location of the lights directly on the bulbs: We'll finalize our lighting's falloff and the illusion of shadows in a bit. For now, let's move on. Adding the table and wine bottle Obviously, we're not going to be able to add the wine bottle, salt and pepper shakers, and the table to one layer of Element in AEX. Hence, we have the naming convention calling out the lamp and the bottle. We're going to break up our other objects across other layers within AEX. We'll get to that, but don't worry, there's a trick to do this. First, let's set up this layer. Place the table and move it (and scale it) using the techniques you already learned for positioning the first lamp. Position this table under the lamp. Do not move the lamp. Then, do the same with the wine bottle, placing it on top of the table. Use group 4 for the table and group 5 for the bottle. So far, your scene should look similar to what is shown in the following screenshot: Finishing the initial setup Now we have all of the five groups used up, but before we can add more objects (in other layers), we want to finalize our render settings (so we don't have to remember them and keep adjusting more Element layers). This will make all our layers look the same for rendering and give the illusion they were all on the same layer. Faking shadows Again, there is no ray tracing in Element 3D, and because the lights are native to AEX and they only project shadows on objects, AEX can see for itself that using AEX's shadow engine isn't an option. So we have to fake them. This can be done with something called ambient occlusion (AO). When the light goes into a corner, it has a hard time bouncing out again. This means that by nature, corners are darker than non-corners. This principal is called ambient occlusion. Element 3D does have this feature. So, by turning the AO way up, we can fake shadows. Some surfaces receive shadows better than others, so we'll have to play around a bit. Start by going into your E3D effect controls. Under Render Settings, you'll see Ambient Occlusion. Tick the Enable AO checkbox. Also, turn the intensity up to 10 and crank up the samples to 50. The sample amount is simple: the larger the number, the smoother the AO. However, the smoother the AO, the longer the render time. Crank up the AO Radius option to 3.2 and the AO Gamma option to 3.2. Watch what happens on your Preview window while you do this. Your settings should look similar to what is shown in the following screenshot: Metals don't receive shadows very well, so let's go turn it down on the gold. Go back into your scene settings, and on the gold shader (down in the advanced section), turn down your AO Amount option to 0.42. The final result should look similar to what is shown in the following screenshot: Light falloff When a candle is very bright at its flame but against a far wall, the wall is lit dimly. This effect is true with light bulbs as well. This principle is called Light Falloff. To add some realism and dramatic lighting to our scene, let's adjust this setting. Inside Render Settings in the Element effect controls, you'll see Lighting. Turn the Light Falloff setting up to 2.49. The larger this number, the more severe the falloff. The result should look similar to what is shown in the following screenshot:

(For more resources related to this topic, see here.) Sending a message from a website Sending messages from a website has many uses; sending notifications to users is one good example. In this example, we're going to present you with a form where you can enter a phone number and message and send it to your user. This can be quickly adapted for other uses. Getting ready The complete source code for this recipe can be found in the Chapter6/Recipe1/ folder. How to do it... Ok, let's learn how to send an SMS message from a website. The user will be prompted to fill out a form that will send the SMS message to the phone number entered in the form. Download the Twilio Helper Library from https://github.com/twilio/twilio-php/zipball/master and unzip it. Upload the Services/ folder to your website. Upload config.php to your website and make sure the following variables are set: <?php $accountsid = ''; // YOUR TWILIO ACCOUNT SID $authtoken = ''; // YOUR TWILIO AUTH TOKEN $fromNumber = ''; // PHONE NUMBER CALLS WILL COME FROM ?> Upload a file called sms.php and add the following code to it: <!DOCTYPE html> <html> <head> <title>Recipe 1 – Chapter 6</title> </head> <body> <?php include('Services/Twilio.php'); include("config.php"); include("functions.php"); $client = new Services_Twilio($accountsid, $authtoken); if( isset($_POST['number']) && isset($_POST['message']) ){ $sid = send_sms($_POST['number'],$_POST['message']); echo "Message sent to {$_POST['number']}"; } ?> <form method="post"> <input type="text" name="number" placeholder="Phone Number...." /><br /> <input type="text" name="message" placeholder="Message...." /><br /> <button type="submit">Send Message</button> </form> </body> </html> Create a file called functions.php and add the following code to it: <?php function send_sms($number,$message){ global $client,$fromNumber; $sms = $client->account->sms_messages->create( $fromNumber, $number, $message ); return $sms->sid; } How it works... In steps 1 and 2, we downloaded and installed the Twilio Helper Library for PHP. This library is the heart of your Twilio-powered apps. In step 3, we uploaded config.php that contains our authentication information to talk to Twilio's API. In steps 4 and 5, we created sms.php and functions.php, which will send a message to the phone number we enter. The send_sms function is handy for initiating SMS conversations; we'll be building on this function heavily in the rest of the article. Allowing users to make calls from their call logs We're going to give your user a place to view their call log. We will display a list of incoming calls and give them the option to call back on these numbers. Getting ready The complete source code for this recipe can be found in the Chapter9/Recipe4 folder. How to do it... Now, let's build a section for our users to log in to using the following steps: Update a file called index.php with the following content: <?php session_start(); include 'Services/Twilio.php'; require("system/jolt.php"); require("system/pdo.class.php"); require("system/functions.php"); $_GET['route'] = isset($_GET['route']) ? '/'.$_GET['route'] : '/'; $app = new Jolt('site',false); $app->option('source', 'config.ini'); #$pdo = Db::singleton(); $mysiteURL = $app->option('site.url'); $app->condition('signed_in', function () use ($app) { $app->redirect( $app->getBaseUri().'/login',!$app->store('user')); }); $app->get('/login', function() use ($app){ $app->render( 'login', array(),'layout' ); }); $app->post('/login', function() use ($app){ $sql = "SELECT * FROM `user` WHERE `email`='{$_POST['user']}' AND `password`='{$_POST['pass']}'"; $pdo = Db::singleton(); $res = $pdo->query( $sql ); $user = $res->fetch(); if( isset($user['ID']) ){ $_SESSION['uid'] = $user['ID']; $app->store('user',$user['ID']); $app->redirect( $app->getBaseUri().'/home'); }else{ $app->redirect( $app->getBaseUri().'/login'); } }); $app->get('/signup', function() use ($app){ $app->render( 'register', array(),'layout' ); }); $app->post('/signup', function() use ($app){ $client = new Services_Twilio($app->store('twilio.accountsid'), $app->store('twilio.authtoken') ); extract($_POST); $timestamp = strtotime( $timestamp ); $subaccount = $client->accounts->create(array( "FriendlyName" => $email )); $sid = $subaccount->sid; $token = $subaccount->auth_token; $sql = "INSERT INTO 'user' SET `name`='{$name}',`email`='{$email }',`password`='{$password}',`phone_number`='{$phone_number}',`sid` ='{$sid}',`token`='{$token}',`status`=1"; $pdo = Db::singleton(); $pdo->exec($sql); $uid = $pdo->lastInsertId(); $app->store('user',$uid ); // log user in $app->redirect( $app->getBaseUri().'/phone-number'); }); $app->get('/phone-number', function() use ($app){ $app->condition('signed_in'); $user = $app->store('user'); $client = new Services_Twilio($user['sid'], $user['token']); $app->render('phone-number'); }); $app->post("search", function() use ($app){ $app->condition('signed_in'); $user = get_user( $app->store('user') ); $client = new Services_Twilio($user['sid'], $user['token']); $SearchParams = array(); $SearchParams['InPostalCode'] = !empty($_POST['postal_code']) ? trim($_POST['postal_code']) : ''; $SearchParams['NearNumber'] = !empty($_POST['near_number']) ? trim($_POST['near_number']) : ''; $SearchParams['Contains'] = !empty($_POST['contains'])? trim($_ POST['contains']) : '' ; try { $numbers = $client->account->available_phone_numbers->getList('US', 'Local', $SearchParams); if(empty($numbers)) { $err = urlencode("We didn't find any phone numbers by that search"); $app->redirect( $app->getBaseUri().'/phone-number?msg='.$err); exit(0); } } catch (Exception $e) { $err = urlencode("Error processing search: {$e->getMessage()}"); $app->redirect( $app->getBaseUri().'/phone-number?msg='.$err); exit(0); } $app->render('search',array('numbers'=>$numbers)); }); $app->post("buy", function() use ($app){ $app->condition('signed_in'); $user = get_user( $app->store('user') ); $client = new Services_Twilio($user['sid'], $user['token']); $PhoneNumber = $_POST['PhoneNumber']; try { $number = $client->account->incoming_phone_numbers->create(array( 'PhoneNumber' => $PhoneNumber )); $phsid = $number->sid; if ( !empty($phsid) ){ $sql = "INSERT INTO numbers (user_id,number,sid) VALUES('{$u ser['ID']}','{$PhoneNumber}','{$phsid}');"; $pdo = Db::singleton(); $pdo->exec($sql); $fid = $pdo->lastInsertId(); $ret = editNumber($phsid,array( "FriendlyName"=>$PhoneNumber, "VoiceUrl" => $mysiteURL."/voice?id=".$fid, "VoiceMethod" => "POST", ),$user['sid'], $user['token']); } } catch (Exception $e) { $err = urlencode("Error purchasing number: {$e->getMessage()}"); $app->redirect( $app->getBaseUri().'/phone-number?msg='.$err); exit(0); } $msg = urlencode("Thank you for purchasing $PhoneNumber"); header("Location: index.php?msg=$msg"); $app->redirect( $app->getBaseUri().'/home?msg='.$msg); exit(0); }); $app->route('/voice', function() use ($app){ }); $app->get('/transcribe', function() use ($app){ }); $app->get('/logout', function() use ($app){ $app->store('user',0); $app->redirect( $app->getBaseUri().'/login'); }); $app->get('/home', function() use ($app){ $app->condition('signed_in'); $uid = $app->store('user'); $user = get_user( $uid ); $client = new Services_Twilio($user['sid'], $user['token']); $app->render('dashboard',array( 'user'=>$user, 'client'=>$client )); }); $app->get('/delete', function() use ($app){ $app->condition('signed_in'); }); $app->get('/', function() use ($app){ $app->render( 'home' ); }); $app->listen(); Upload a file called dashboard.php with the following content to your views folder: <h2>My Number</h2> <?php $pdo = Db::singleton(); $sql = "SELECT * FROM `numbers` WHERE `user_ id`='{$user['ID']}'"; $res = $pdo->query( $sql ); while( $row = $res->fetch() ){ echo preg_replace("/[^0-9]/", "", $row['number']); } try { ?> <h2>My Call History</h2> <p>Here are a list of recent calls, you can click any number to call them back, we will call your registered phone number and then the caller</p> <table width=100% class="table table-hover tabled-striped"> <thead> <tr> <th>From</th> <th>To</th> <th>Start Date</th> <th>End Date</th> <th>Duration</th> </tr> </thead> <tbody> <?php foreach ($client->account->calls as $call) { # echo "<p>Call from $call->from to $call->to at $call->start_time of length $call->duration</p>"; if( !stristr($call->direction,'inbound') ) continue; $type = find_in_list($call->from); ?> <tr> <td><a href="<?=$uri?>/call?number=<?=urlencode($call->from)?>"><?=$call->from?></a></td> <td><?=$call->to?></td> <td><?=$call->start_time?></td> <td><?=$call->end_time?></td> <td><?=$call->duration?></td> </tr> <?php } ?> </tbody> </table> <?php } catch (Exception $e) { echo 'Error: ' . $e->getMessage(); } ?> <hr /> <a href="<?=$uri?>/delete" onclick="return confirm('Are you sure you wish to close your account?');">Delete My Account</a> How it works... In step 1, we updated the index.php file. In step 2, we uploaded dashboard.php to the views folder. This file checks if we're logged in using the $app->condition('signed_in') method, which we discussed earlier, and if we are, it displays all incoming calls we've had to our account. We can then push a button to call one of those numbers and whitelist or blacklist them. Summary Thus in this article we have learned how to send messages and make phone calls from your website using Twilio. Resources for Article: Further resources on this subject: Make phone calls, send SMS from your website using Twilio [article] Trunks in FreePBX 2.5 [article] Trunks using 3CX: Part 1 [article]

(For more resources related to this topic, see here.) Installing the PostgreSQL database server Most operating systems that come with a package management solution for open source software make PostgreSQL packages available for installation. In this recipe, we will install PostgreSQL from a package and set it up on your system. Installing the server package automatically installs the PostgreSQL command-line client package, as well. How to do it... Perform the following steps to install the PostgreSQL database server: Install the PostgreSQL database server package. In most package repositories, the PostgreSQL server package is simply named postgresql-server. If your distribution allows you to select among different versions of PostgreSQL, the package names will contain version numbers such as postgresql-9.1 or postgresql93-server. Pick the package with the latest version unless you have reasons to stick with an older one. Click the Refresh Modules link at the bottom of Webmin's main menu and reload the browser to update the menu. Navigate to Servers | PostgreSQL Database Server. You should see a screen that lists installed databases. It should include the default databases such as postgresql and template1. If you do not see the list of databases, but instead a message which indicates that the database system has not yet been initialized, click the Initialize Database button. At the bottom of the screen, if you see the message, Warning: The Perl modules DBI and DBD::Pg are not installed on your system, click the link and follow Webmin's instructions to install the missing Perl modules. Navigate to System | Bootup and Shutdown and verify that the init script, postgresql, is set to start at boot. If it isn't, select its checkbox and click the Start Now and On Boot button. How it works... Webmin helps you find and install the postgresql-server package from your distribution's repositories. The package installs the database server, client, and an init script that starts the server during system boot. Before Postgres can be used to manage databases, a new cluster must be created. A PostgreSQL cluster is a collection of databases managed by a single server. Creating the cluster involves the creation of a directory in which the database files will be stored, and filling role, because all new databases in the cluster will be made by copying this template. If your package installation script does not initialize a database cluster for you, you can ask Webmin to do it by clicking the Initialize Database button. This runs the following subcommand of the init script: /etc/rc.d/init.d/postgresql initdb Locating the PostgreSQL server configuration files The main configuration file of the PostgreSQL server is usually named postgresql.conf, and is stored in the database cluster data directory by default. Various system distributions move this configuration outside of the data directory and place it in a different location, for example, in the /etc/ directory. In this recipe, we will demonstrate how to find the postgresql.conf and change it to modify the server's configuration. Webmin does not assist you in the modification of the basic settings of PostgreSQL, so you will need to edit the configuration file manually. Getting ready Make sure that the PostgreSQL server is installed and running, and that you are able to connect to it via Webmin before starting. The recipe, Installing the PostgreSQL database server, provides more information. How to do it... Follow these steps to locate PostgreSQL's main configuration file on your system: Navigate to Servers | PostgreSQL Database Server. Click the icon of the default database, postgres. Click the Execute SQL button. Enter the following SQL command in the provided text area: SHOW config_file; Click the Execute button and you will see the output of the SQL command, which provides the full path to the main server configuration file, as shown in the following screenshot: How it works... When an init script starts the PostgreSQL server, it may specify the location of the database cluster's data directory or the location of the server's main configuration file (customarily called postgresql.conf). By default, the main configuration file is stored inside of the data directory, but package maintainers often move it to a different location (such as /etc/) to keep system configuration files in order. The SQL command, SHOW config_file;, can be used to check where the main configuration file is located. There's more... The location of other configuration files and the values of other settings can also be displayed using the SQL SHOW command. Determining location of other configuration files and data files Use the following commands to check where other configuration files are located: Setting Command Main configuration file (postgresql.conf) SHOW config_file; Data directory SHOW data_directory; Host-based access configuration file (pg_hba.conf) SHOW hba_file; Identity mapping file (pg_ident.conf) SHOW ident_file; Directory where the Unix-domain socket will be created SHOW unix_socket_directory; Checking values of other settings You can also reveal the values of all settings by issuing the following command: SHOW all; Allowing access to PostgreSQL over the network Programs that access PostgreSQL databases, which are called clients, may be running on the same machine as the server. In this case, the client and server will communicate most efficiently using a Unix-domain socket, a channel of inter-process communication accessed through the filesystem such as a file or directory. Access to a socket is controlled by filesystem permissions. Other client programs may be able to communicate only over TCP network sockets. These clients may connect to the local server using the loopback interface and IP address of 127.0.0.1. However, if a client program is located on a machine other than the server, then communication between them must take place over the network using the TCP protocol. There are a number of ways to set up network connections for PostgreSQL. The most efficient but least secure method is to use a direct unencrypted connection between the client and server. This method has the drawback that unencrypted information could potentially be eavesdropped upon or even modified in transit over the network. Because database systems are usually designed to be as efficient as possible, this type of communication is used often, but should only be deployed inside of a secure network. We will describe how to enable this type of communication in this recipe. In order to make network access to your PostgreSQL server more secure, you can choose to encrypt the transferred information using SSL. This prevents eavesdropping and man-in-the-middle attacks, but leaves the PostgreSQL server's network port exposed and potentially vulnerable to brute-force password guessing and other attacks. If you really need security, for instance, to access your database server over the Internet, you should probably choose a third option: send the PostgreSQL traffic over an encrypted SSH tunnel. This is the least efficient of the described transmission methods, but it generates the fewest security concerns. For more information, take a look at the recipe, Accessing the PostgreSQL server over an SSH tunnel. Getting ready In this recipe, we will prepare your PostgreSQL server to accept incoming network connections. In order to test the connection, we will need access to two computers attached to the same network: the server and a client machine. Make note of the server and client's IP or domain name before starting. How to do it... The steps in this recipe will be divided into five sections: First, we'll instruct PostgreSQL to listen for incoming network connections on the standard port (5432). Next, we'll create a database user named dbuser. Then, we will create a database named testtdb. We will allow remote access to the database. And finally, we will test the setup by connecting to our server from a secondary client machine. Perform the following steps to instruct the PostgreSQL server to listen for network connections: Allow incoming TCP traffic to port 5432 through your firewall. Find the location of the PostgreSQL main server configuration file (postgresql.conf). Refer to the recipe, Locating the PostgreSQL server configuration files, for detailed instructions. Within the postgresql.conf file, find the line with the listen_addresses directive. This line may be commented out (start with the # character). Change the line to the following: listen_addresses = '*' The most effective way to edit files on your server is to use an editor such as Vim or Nano in a terminal session (for example, over SSH). But to make a small change in a configuration file, you do not need to leave Webmin. We must restart the server after making configuration changes. Navigate to Servers | PostgreSQL Database Server, click the Stop PostgreSQL Server button, and then click the Start PostgreSQL Server button. Your PostgreSQL server will now listen for incoming network connections on port 5432. Perform the following steps to create a new user: Navigate to Servers | PostgreSQL Database Server | PostgreSQL Users. Click the Create new user link. Set Username to dbuser and assign a strong password in the Password field. Answer No to the Can create databases? and Can create users? questions. Set Valid until to Forever: Click the Create button. Perform the following steps to create a database: Navigate to Servers | PostgreSQL Database Server Click the Create a new database link. Set Database name to testdb. Set Owned by user to dbuser. Set Template database to template1: Click the Create button. Perform the following steps to grant a user remote access to the database: Navigate to Servers | PostgreSQL Database Server. Click the Allowed Hosts icon. Click the Create a new allowed host link. Set Host address to Single host and enter the IP address of the client computer (for example, 10.10.10.100). If the client can connect from more then one IP, you can specify a subnet by providing a network and netmask or CIDR length. For instance, to grant access to all computers in the 10.10.10.* subnet, you could specify the network as 10.10.10.0 and either the netmask as 255.255.255.0 or the CIDR length as 24. Set SSL connection required? to Yes. You can shave off a little performance overhead by not using SSL, but you should only do that on entirely trusted networks. Set Database to testdb. Set Users to Listed users and enter dbuser. Set Authentication mode to MD5 encrypted password: Click the Create button. We'll need to restart the server one more time to load the new access configuration. Navigate to Servers | PostgreSQL Database Server, click the Stop PostgreSQL Server button, and then click the Start PostgreSQL Server button. On a busy production system it would be a bad idea to restart the database server unnecessarily, although that is the sure way of reloading all settings. After changing access settings, you don't really need to restart the server. You could send it a SIGHUP signal instead. This signal instructs Postgres to reload its configuration. On systems equipped with the pg_ctl program, this can be achieved by issuing the following command: $ sudo pg_ctl reload On systems with the pg_ctlcluster command, you will need to specify the server version and cluster name, for example: $ sudo pg_ctlcluster 9.1 main reload Testing the connection Try to connect to your database server from the client machine that uses the IP we specified. If your other machine has the PostgreSQL command-line client installed, you can test the connection by typing in this command at the terminal. However, substitute postgresql-host with the IP or domain name of your Postgres server as follows: $ psql -h postgresql-host -U dbuser testdb testdb=# q If the connection is successful, you should arrive at the PostgreSQL prompt (testdb=#). Type q and press Enter to exit. How it works... In order to enable network access to the PostgreSQL database server, we needed to modify two configuration files. We edited the main configuration file (postgresql.conf) manually to instruct the server to listen for incoming network connections on all network interfaces. The second file, which was edited through Webmin's interface, is the host-based authentication configuration (pg_hba.conf). This file instructs the server which users should be allowed to connect from which network hosts and how they should be required to authenticate. Webmin added the following line to pg_hba.conf: hostssl testdb dbuser 10.10.10.100 255.255.255.255 md5 The preceding line instructs the server to accept SSL connections to the testdb database by the dbuser user if the connection originated from the IP address 10.10.10.100. The user should be asked to provide an MD5-encrypted password for authentication. Another line in pg_hba.conf can look like the following: local all postgres peer This line instructs the server to accept connections made locally over the Unix socket. These connections use the peer authentication method, which checks the username of the system account running the connecting client program. If the system username matches a Postgres account name, then the connection is considered authenticated. Password checking is not performed in peer authentication. The preceding line of code will allow the system account postgres to access all databases.

(For more resources related to this topic, see here.) Installing Visual Studio tools Part of the coding that is needed to consume an external service is performed in Visual Studio. This is why we must install both Visual Studio 2010 and Visual Studio Tools for Microsoft Dynamics AX 2012 before we can create Visual Studio projects and add them to the AOT. Although you can develop applications that integrate with Microsoft Dynamics AX 2012 using other versions of Visual Studio, such as Visual Studio 2012, Visual Studio Tools are only available for Visual Studio 2010. To install Visual Studio tools, perform the following steps: Run the Microsoft Dynamics AX 2012 setup. Go to the Install section and select Microsoft Dynamics AX Components. Click on the Next button to move to the next screen and select Add or modify existing components. Look under the Developer Tools node and select Visual Studio Tools. Go through the rest of the setup wizard to complete the installation process. Installing Visual Studio Tools will add the following extensions to Visual Studio: The Application Explorer option that is available in Visual Studio by navigating to View | Application Explorer. Enabling it will display the AOT in Visual Studio. Two new templates that are available when you create a new project in Visual Studio — Report Model and EP Web Applications. An option to add Visual Studio projects to the AOT. This is the option we're interested in when consuming web services.

(For more resources related to this topic, see here.) Creating the game controller In order to design a controller, we first need to know what sort of game is going to be played. I am going to explain how to make a game where the player is told a letter, and he/she has to press the button of that letter as quickly as possible. They then are told about another letter. The player has to hit as many buttons correctly as they can in a 30 second time limit. There are many ways in which this game can be varied; instead of ordering the player to press a particular button, the game could ask the player a multiple-choice question, and instead of colors, the buttons could be labeled with Yes, No, Maybe or different colors. You could give the player multiple commands at once, and make sure that he/she presses all the buttons in the right order. It would even be possible to make a huge controller and treat it as more of a board game. I will leave the game design up to you, but I recommend that you follow the instructions in this article until the end, and then change things to your liking once you know how everything works. The controller base So now that we know how the game is going to be played, it's time to design the controller. This is what my design looks like with four different letters: Make sure each button area is at least a little bigger than a paper clip, as these are what the buttons will be made of. I recommend a maximum of eight buttons. Draw your design on to the card, decorate it however you like, and then cut it out. Adding buttons Now for each button, we need to perform the following steps: Poke two small holes in the card, roughly 3 cm apart (or however long your paper clips are), as shown in the following figure. Use a sharp pencil or a pair of scissors to do this. Push a paper fastener through each hole and open them out, as shown in the following figures: Wrap a paper clip around the head of one of the fasteners, and (if necessary) bend it so that it grips the fastener tightly, as shown in the following figure: Bend the other end of the paper clip up very slightly, so it doesn't touch the second fastener unless you press down on it, as shown in the following figure: Turn the card over and tape one leg of each fastener in place, making sure that they don't touch, as shown in the following figure: Tape a length of wire to each of the two remaining legs of the fasteners. The ends of the wires should be exposed metal so that electricity can flow through the wire, paper fastener, and paper clip (as shown in the following figure). You may like to delay this step until later, when you have a better idea of how long the wire should be. Connecting to the Raspberry Pi Now that the controller is ready, it's time to connect it to the Raspberry Pi. One of the things that distinguishes the Raspberry Pi from a normal computer is its set of general purpose input/output (GPIO) pins. These are the 26 pins at the top-left corner of the Raspberry Pi, just above the logo. As the name suggests, they can be used for any purpose, and are capable of both sending and receiving signals. The preceding figure shows what each of the pins does. In order to create a (useful) circuit, we need to connect one of the power pins to one of the ground pins, with some sort of electrical component in between. The GPIO pins are particularly useful because we can make them behave like either power or ground pins, and they can also detect what they're connected to. Note that there are two versions of the pin numbering system. You will almost certainly have a revision 2 Raspberry Pi. The revision 2 board has two mounting holes, while the revision 1 board has none. (These holes are surrounded by metal and are large enough to put a screw through. It's easy to spot them if they're there.) It is safest to simply not use any of the pins that have different numbers in different revisions. To connect your controller to the Raspberry Pi, connect one wire from each button to a 3V3 Power pin, and each of the remaining wires to a different GPIO pin (one with GPIO in its name as in the previous figure). In my example, I will use pins 22, 23, 24, and 25. Everything is now connected as shown in the following figure: Summary In this article, we used the Python programming language to create a game. We created an electronic circuit to act as the game controller, and used code to detect when the buttons were being pressed. We learned the basics of the Python language, and saw how separating the code into multiple functions makes it more flexible and easier to manage. Resources for Article: Further resources on this subject: Installing MAME4All (Intermediate) [Article] Creating a file server (Samba) [Article] Clusters, Parallel Computing, and Raspberry Pi – A Brief Background [Article]

(For more resources related to this topic, see here.) The commonly used input components and their features The PrimeFaces Extensions team created some basic form components that are frequently used in registration forms. These frequently used components are the InputNumber component that formats the input fields with numeric strings, the KeyFilter component for filtering the keyboard input whereas select components such as TriStateCheckbox and TriStateCheckboxMany are used for adding a new state to the select Boolean checkbox and Many checkbox components in an order. Understanding the InputNumber component The InputNumber component can be used to format the input form fields with custom number strings. The main features of this component include support for currency symbols, min and max values, negative numbers, and many more rounding methods. The component development is based on the autoNumeric jQuery plugin. The InputNumber component features are basically categorized into two main sections: Common usage Validations, conversions, and rounding methods Common usage The InputNumber use case is used for basic common operations such as appending currency symbols on either side of the number (that is, prefix and suffix notations), custom decimal and thousand separators, minimum and maximum values, and custom decimal places. The following XHTML code is used to create InputNumber with all possible custom options in the form of attributes: <pe:inputNumber id="customInput" value="#{inputNumberController.value}" symbol=" $" symbolPosition="p" decimalSeparator="," thousandSeparator="." minValue="-99.99" maxValue="99.99" decimalPlaces="4" > Validations, conversions, and rounding methods The purpose of this use case is just like any other standard JSF and PrimeFaces input components where we can also apply different types of converters and validators to the InputNumber component. Apart from these regular features, you can also control the empty input display with different types of options such as empty, sign, and zero values. The InputNumber component is specific to Numeric types; rounded methods are a commonly used feature for InputNumber in web applications. You can use the roundMethod attribute of InputNumber; its default value is Round-Half-Up Symmetric. Exploring the KeyFilter component to restrict input data On a form-based screen, you need to restrict the input on specific input components based on the component's nature and functionality. Instead of approaching plain JavaScript with regularExpressions, the Extensions team provided the KeyFilter component to filter the keyboard input. It is not a standalone component and always depends on the input components by referring through the for attribute. TriStateCheckbox and TriStateManyCheckbox Both the TriStateCheckbox and TriStateManyCheckbox components provide a new state to the standard SelectBooleanCheckbox and SelectManyBooleanCheckbox components respectively. Each state is mapped to the 0, 1, and 2 string values. TriStateCheckbox can be customized using the title, custom icons for three states, and item label features. Just as with any other standard input component, you can apply the Ajax behavior listeners to this component as well. Managing events using the TimeLine component TimeLine is an interactive visualization chart for scheduling and manipulating the events in a certain period of time. The time axis scale can be auto-adjusted and ranges from milliseconds to years. The events can take place on a single date or a particular date range. The TimeLine component supports many features such as read only events, editable events, grouping events, client-side and server-side API, and drag-and-drop. Understanding the MasterDetail component and its various features The MasterDetail component allows us to group data contents into multiple levels and save the web page space for the remaining important areas of the application. The grouped data is maintained in a hierarchical manner and can be navigated through flexible built-in breadcrumbs or command components to go forward and backward in the web interface. Each level in the content flow is represented by a MasterDetailLevel component. This component will hold the PrimeFaces/JSF data iterative or form components inside the grouping components. You can also switch between levels with the help of the SelectDetailLevel handler, which is based on Ajax, and dynamically load the levels through Ajax behavior. The SelectDetailLevel handler can be attached to the ajaxified PrimeFaces components and standard JSF components. These components also support the header and footer facets. Introducing exporter components and its features The PrimeFaces Core dataExporter component works very well on the plain dataTable components along with providing some custom features. But the PrimeFaces Extensions exporter component is introduced to work on all the major features of the dataTable component, provides full control of customization, and extends its features to dataList components. The exporter component is used to extract and report the tabular form of data in different formats. This component is targeted to work with all major features of dataTable, subTable, and other data iteration components such as dataList as well. Currently, the supported file formats are PDF and Excel. Both headerText and footerText columns are supported along with the value holders. CKEditor The CKEditor is a WYSIWYG text editor that is to be used for the web pages that bring the desktop editing application features from Microsoft Word and OpenOffice to web applications. The text being edited in this editor makes the results similar to what you can see after the web page is published. The CKEditor component is available as a separate JAR file from the Extensions library; this library or dependency needs to be included on demand. The CKEditor component provides more custom controls with the custom toolbar template and skinning in user interfaces using the theme and interfaceColor properties as compared to the PrimeFaces editor component. The editor component is, by default, displayed with all the controls to make the content customizable. You can supply a few more customizations through interfaceColor to change the interface dynamically and checkDirtyInterval for repeated time interval checks after the content has been changed. To make asynchronous communication calls on the server-side code, many Ajax events are supported by this component. The following XHTML code creates a CKEditor component with custom interface colors: <pe:ckEditor id="editor" value="#{ckeditorController.content}" interfa ceColor="#{ckeditorController.color}" checkDirtyInterval="0"> <p:ajax event="save" listener="#{ckeditorController.saveListener}" update="growl"/> </pe:ckEditor> You can write any number of instances on the same page. There are two ways in which we can customize the CKEditor toolbar component: Toolbar defined with default custom controls: You can customize the editor toolbar by declaring the control names in the form of a string name or an array of strings. CustomConfig JavaScript file for user-defined controls: You can also customize the toolbar by defining the custom config JavaScript file through the customConfig attribute and register the control configuration names on the toolbar attribute. Summary In this article, we discussed some commonly used input components and their features, and introduced how to manage events using the TimeLine component. Then we covered the MasterDetail component and introduced exporter component and its features. Resources for Article: Further resources on this subject: Getting Started with PrimeFaces [Article] JSF2 composite component with PrimeFaces [Article] JSF 2.0 Features: An Extension [Article]

(For more resources related to this topic, see here.) Executing a remote shell command using telnet If you need to connect an old network switch or router via telnet, you can do so from a Python script instead of using a bash script or an interactive shell. This recipe will create a simple telnet session. It will show you how to execute shell commands to the remote host. Getting ready You need to install the telnet server on your machine and ensure that it's up and running. You can use a package manager that is specific to your operating system to install the telnet server package. For example, on Debian/Ubuntu, you can use apt-get or aptitude to install the telnetd package, as shown in the following command: $ sudo apt-get install telnetd $ telnet localhost How to do it... Let us define a function that will take a user's login credentials from the command prompt and connect to a telnet server. Upon successful connection, it will send the Unix 'ls' command. Then, it will display the output of the command, for example, listing the contents of a directory. Listing 7.1 shows the code for a telnet session that executes a Unix command remotely as follows: #!/usr/bin/env python # Python Network Programming Cookbook -- Chapter - 7 # This program is optimized for Python 2.7. # It may run on any other version with/without modifications. import getpass import sys import telnetlib     def run_telnet_session():   host = raw_input("Enter remote hostname e.g. localhost:")   user = raw_input("Enter your remote account: ")   password = getpass.getpass()     session = telnetlib.Telnet(host)     session.read_until("login: ")   session.write(user + "n")   if password:     session.read_until("Password: ")     session.write(password + "n")       session.write("lsn")   session.write("exitn")     print session.read_all()   if __name__ == '__main__':   run_telnet_session() If you run a telnet server on your local machine and run this code, it will ask you for your remote user account and password. The following output shows a telnet session executed on a Debian machine: $ python 7_1_execute_remote_telnet_cmd.py Enter remote hostname e.g. localhost: localhost Enter your remote account: faruq Password:   ls exit Last login: Mon Aug 12 10:37:10 BST 2013 from localhost on pts/9 Linux debian6 2.6.32-5-686 #1 SMP Mon Feb 25 01:04:36 UTC 2013 i686   The programs included with the Debian GNU/Linux system are free software; the exact distribution terms for each program are described in the individual files in /usr/share/doc/*/copyright.   Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent permitted by applicable law. You have new mail. faruq@debian6:~$ ls       down              Pictures               Videos Downloads         projects               yEd Dropbox           Public env               readme.txt faruq@debian6:~$ exit logout How it works... This recipe relies on Python's built-in telnetlib networking library to create a telnet session. The run_telnet_session() function takes the username and password from the command prompt. The getpass module's getpass() function is used to get the password as this function won't let you see what is typed on the screen. In order to create a telnet session, you need to instantiate a Telnet() class, which takes a hostname parameter to initialize. In this case, localhost is used as the hostname. You can use the argparse module to pass a hostname to this script. The telnet session's remote output can be captured with the read_until() method. In the first case, the login prompt is detected using this method. Then, the username with a new line feed is sent to the remote machine by the write() method (in this case, the same machine accessed as if it's remote). Similarly, the password was supplied to the remote host. Then, the ls command is sent to be executed. Finally, to disconnect from the remote host, the exit command is sent, and all session data received from the remote host is printed on screen using the read_all() method. Copying a file to a remote machine by SFTP If you want to upload or copy a file from your local machine to a remote machine securely, you can do so via Secure File Transfer Protocol (SFTP). Getting ready This recipe uses a powerful third-party networking library, Paramiko, to show you an example of file copying by SFTP, as shown in the following command. You can grab the latest code of Paramiko from GitHub (https://github.com/paramiko/paramiko) or PyPI: $ pip install paramiko How to do it... This recipe takes a few command-line inputs: the remote hostname, server port, source filename, and destination filename. For the sake of simplicity, we can use default or hard-coded values for these input parameters. In order to connect to the remote host, we need the username and password, which can be obtained from the user from the command line. Listing 7.2 explains how to copy a file remotely by SFTP, as shown in the following code: #!/usr/bin/env python # Python Network Programming Cookbook -- Chapter - 7 # This program is optimized for Python 2.7. # It may run on any other version with/without modifications.   import argparse import paramiko import getpass     SOURCE = '7_2_copy_remote_file_over_sftp.py' DESTINATION ='/tmp/7_2_copy_remote_file_over_sftp.py '     def copy_file(hostname, port, username, password, src, dst):   client = paramiko.SSHClient()   client.load_system_host_keys()   print " Connecting to %s n with username=%s... n" %(hostname,username)   t = paramiko.Transport((hostname, port))   t.connect(username=username,password=password)   sftp = paramiko.SFTPClient.from_transport(t)   print "Copying file: %s to path: %s" %(SOURCE, DESTINATION)   sftp.put(src, dst)   sftp.close()   t.close()     if __name__ == '__main__':   parser = argparse.ArgumentParser(description='Remote file copy')   parser.add_argument('--host', action="store", dest="host", default='localhost')   parser.add_argument('--port', action="store", dest="port", default=22, type=int)   parser.add_argument('--src', action="store", dest="src", default=SOURCE)   parser.add_argument('--dst', action="store", dest="dst", default=DESTINATION)     given_args = parser.parse_args()   hostname, port =  given_args.host, given_args.port   src, dst = given_args.src, given_args.dst     username = raw_input("Enter the username:")   password = getpass.getpass("Enter password for %s: " %username)     copy_file(hostname, port, username, password, src, dst) If you run this script, you will see an output similar to the following: $ python 7_2_copy_remote_file_over_sftp.py Enter the username:faruq Enter password for faruq:  Connecting to localhost  with username=faruq... Copying file: 7_2_copy_remote_file_over_sftp.py to path: /tmp/7_2_copy_remote_file_over_sftp.py How it works... This recipe can take the various inputs for connecting to a remote machine and copying a file over SFTP. This recipe passes the command-line input to the copy_file() function. It then creates a SSH client calling the SSHClient class of paramiko. The client needs to load the system host keys. It then connects to the remote system, thus creating an instance of the transport class. The actual SFTP connection object, sftp, is created by calling the SFTPClient.from_transport() function of paramiko. This takes the transport instance as an input. After the SFTP connection is ready, the local file is copied over this connection to the remote host using the put() method. Finally, it's a good idea to clean up the SFTP connection and underlying objects by calling the close() method separately on each object.

(For more resources related to this topic, see here.) Android application teardown An Android application is an archive file of the data and resource files created while developing the application. The extension of an Android application is .apk, meaning application package, which includes the following files and folders in most cases: Classes.dex (file) AndroidManifest.xml (file) META-INF (folder) resources.arsc (file) res (folder) assets (folder) lib (folder) In order to verify this, we could simply unzip the application using any archive manager application, such as 7zip, WinRAR, or any preferred application. On Linux or Mac, we could simply use the unzip command in order to show the contents of the archive package, as shown in the following screenshot: Here, we have used the -l (list) flag in order to simply show the contents of the archive package instead of extracting it. We could also use the file command in order to see whether it is a valid archive package. An Android application consists of various components, which together create the working application. These components are Activities, Services, Broadcast Receivers, Content providers, and Shared Preferences. Before proceeding, let's have a quick walkthrough of what these different components are all about: Activities: These are the visual screens which a user could interact with. These may include buttons, images, TextView, or any other visual component. Services: These are the Android components which run in the background and carry out specific tasks specified by the developer. These tasks may include anything from downloading a file over HTTP to playing music in the background. Broadcast Receivers: These are the receivers in the Android application that listen to the incoming broadcast messages by the Android system, or by other applications present in the device. Once they receive a broadcast message, a particular action could be triggered depending on the predefined conditions. The conditions could range from receiving an SMS, an incoming phone call, a change in the power supply, and so on. Shared Preferences: These are used by an application in order to save small sets of data for the application. This data is stored inside a folder named shared_prefs. These small datasets may include name value pairs such as the user's score in a game and login credentials. Storing sensitive information in shared preferences is not recommended, as they may fall vulnerable to data stealing and leakage. Intents: These are the components which are used to bind two or more different Android components together. Intents could be used to perform a variety of tasks, such as starting an action, switching activities, and starting services. Content Providers: These are used to provide access to a structured set of data to be used by the application. An application can access and query its own data or the data stored in the phone using the Content Providers. Now that we know of the Android application internals and what an application is composed of, we can move on to reversing an Android application. That is getting the readable source code and other data sources when we just have the .apk file with us. Reversing an Android application As we discussed earlier, that Android applications are simply an archive file of data and resources. Even then, we can't simply unzip the archive package (.apk) and get the readable sources. For these scenarios, we have to rely on tools that will convert the byte code (as in classes.dex) into readable source code. One of the approaches to convert byte codes to readable files is using a tool called dex2jar. The .dex file is the converted Java bytecode to Dalvik bytecode, making it optimized and efficient for mobile platforms. This free tool simply converts the .dex file present in the Android application to a corresponding .jar file. Please follow the ensuing steps: Download the dex2jar tool from https://code.google.com/p/dex2jar/. Now we can use it to run against our application's .dex file and convert to .jar format. Now, all we need to do is go to the command prompt and navigate to the folder where dex2jar is located. Next, we need to run the d2j-dex2jar.bat file (on Windows) or the d2j-dex2jar.sh file (on Linux/Mac) and provide the application name and path as the argument. Here in the argument, we could simply use the .apk file, or we could even unzip the .apk file and then pass the classes.dex file instead, as shown in the following screenshot: As we can see in the preceding screenshot, dex2jar has successfully converted the .dex file of the application to a .jar file named helloworld-dex2jar.jar. Now, we can simply open this .jar file in any Java graphical viewer such as JD-GUI, which can be downloaded from its official website at http://jd.benow.ca/. Once we download and install JD-GUI, we could now go ahead and open it. It will look like the one shown in the following screenshot: Here, we could now open up the converted .jar file from the earlier step and see all the Java source code in JD-GUI. To open a .jar file, we could simply navigate to File | Open. In the right-hand side pane, we can see the Java sources and all the methods of the Android application. Note that the recompilation process will give you an approximate version of the original Java source code. This won't matter in most cases; however, in some cases, you might see that some of the code is missing from the converted .jar file. Also, if the application developer is using some protections against decompilations such as proguard and dex2jar, when we decompile the application using dex2jar or Apktool, we won't be seeing the exact source code; instead, we will see a bunch of different source files, which won't be the exact representation of the original source code. Using Apktool to reverse an Android application Another way of reversing an Android application is converting the .dex file to smali files. A smali is a file format whose syntax is similar to a language known as Jasmine. We won't be going in depth into the smali file format as of now. For more information, take a look at the online wiki at https://code.google.com/p/smali/wiki/ in order to get an in-depth understanding of smali. Once we have downloaded Apktool and configured it, we are all set to go further. The main advantage of Apktool over JD-GUI is that it is bidirectional. This means if you decompile an application and modify it, and then recompile it back using Apktool, it will recompile perfectly and will generate a new .apk file. However, dex2jar and JD-GUI won't be able to do this similar functionality, as it gives an approximate code and not the exact code. So, in order to decompile an application using Apktool, all we need to do is to pass in the .apk filename along with the Apktool binary. Once decompiled, Apktool will create a new folder with the application name in which all of the files will be stored. To decompile, we will simply go ahead and use apktool d [app-name].apk. Here, the -d flag stands for decompilation. In the following screenshot, we can see an app being decompiled using Apktool: Now, if we go inside the smali folder, we will see a bunch of different smali files, which will contain the code of the Java classes that were written while developing the application. Here, we can also open up a file, change the values, and use Apktool to build it back again. To build a modified application from smali, we will use the b (build) flag in Apktool. apktool d [decompiled folder name] [target-app-name].apk However, in order to decompile, modify, and recompile applications, I would personally recommend using another tool called Virtuous Ten Studio (VTS). This tool offers similar functionalities as Apktool, with the only difference that VTS presents it in a nice graphical interface, which is relatively easy to use. The only limitation for this tool is it runs natively only on the Windows environment. We could go ahead and download VTS from the official download link, http://www.virtuous-ten-studio.com/. The following is a screenshot of the application decompiling the same project: Summary In this article, we covered some of the methods and techniques that are used to reverse the Android applications. Resources for Article: Further resources on this subject: Android Native Application API [Article] Introducing an Android platform [Article] Animating Properties and Tweening Pages in Android 3-0 [Article]

(For more resources related to this topic, see here.) Working with vSphere Distributed Switches A vSphere Distributed Switch (vDS) is similar to a standard switch, but vDS spans across multiple hosts instead of creating an individual switch on each host. The vDS is created at the vCenter level, and the configuration is stored in the vCenter database. A cached copy of the vDS configuration is also stored on each host in case of a vCenter outage. Getting ready Log in to the vCenter Server using the vSphere Web Client. How to do it… In this section, you will learn how to create a vDS, dvportgroup, and manage the ESXi host using the vDS. First, we will create a vSphere Distributed Switch. The steps involved in creating a vDS are as follows: Select the datacenter on which the vDS has to be created. Navigate to Actions | New Distributed Switch...., as shown in the following screenshot Enter the Name and location for the vDS and click on Next. Select the version for the vDS, as shown in the following screenshot, and click on Next: In the Edit settings page, provide the following details: Number of uplinks: This specifies the number of physical NIC of the host which would be part of the vDS. Network I/O Control: This option controls the input/output to the network and can be set to either Enabled or Disabled. Default port group: This option lets you create a default port group. To create one, enable the checkbox and provide the Port group name. Click on Next when finished. In the Ready to complete screen, review the settings and click on Finish. The following steps will create a new distributed port group: The next step after creating a vDS is to create a new port group if it is not been created as part of the vDS. Select the vDS and click on Actions | New Distributed Port Group. Provide the name and select the location for the port group and click on Next. In the Configure settings screen, set the following general properties for the port group: Port binding: This provides us with three options, namely, Static, Dynamic, and Ephemeral (no binding). Static binding: This is selected when a VM is connected to the port group where a port is assigned and reserved for the VM. Only when the VM is deleted, the port is freed up. Ephemeral binding: This port is created and assigned to the VM by the host when a VM is powered on and the port is deleted when the VM is powered off. Dynamic binding: This is depreciated in ESXi 5.x version and is no longer in use, but the option is still available in the vSphere Client. Port allocation: This can be set to either Elastic or Fixed. Elastic: The default port is 8, and when all ports are used, a new set of ports is created automatically Fixed: The ports are fixed to 8, and no additional ports are created when all ports are used up Number of ports: This option is set to 8 by default. Network resource pool: This option is enabled only if a user-defined network pool is created; it can be set even after creating the port group. VLAN type: The available options are None, VLAN, VLAN trunking, and Private VLAN. None: This means that no VLAN is used VLAN: This implies that VLAN is used and the ID has to be specified VLAN trunking: This implies that a group of VLANs is being trunked and their respective ID have to be used Private VLAN: This menu is empty if a private VLAN does not exist In the Ready to complete screen, review the settings and click on Finish. The next step after creating a distributed port group is to add the ESXi host to the vDS. While the host is being added, it is possible to migrate the VMkernel and VM port group from the vSS to vDS, or it can be done later. Now, let's see the steps involved: Select the Distributed Switch in the vSphere Web Client. Navigate to Actions | Add and Manage Hosts. In the Select task screen, select Add hosts, as shown in the following screenshot, and click on Next: Click on the + icon to select hosts to be added and click on OK. Click on Next in the Select new hosts screen. Select the physical network adapters, which will be used as an uplink for the vDS, and click on Next. In the Select virtual network adapters screen, you will have the option to migrate the VMkernel interface to the vDS group; select the appropriate option and click on Next. Review any dependencies on the validation page and click on Next. Optionally, you can migrate the VM Network to the vDS port group in the Select VM network adapters screen by selecting the appropriate option and clicking on Next. In the Ready to complete screen, review the settings and click on Finish. An ESXi host can be removed from the vDS only if there is no VM still connected to the vDS. Make sure the VMs are either migrated to the standard switch or to another vDS. The following steps will remove an ESXi host from the Distributed Switch: Browse to the Distributed Switch in the vSphere Web Client. Navigate to Actions | Add and Manage Hosts. In the Select task screen, select Remove hosts, as shown in the following screenshot, and click on Next: Click on the + icon to select new hosts to be removed and click on OK. Click on Next in the Select hosts screen. In the Ready to complete screen, review the settings and click on Finish. When the entire host is being added to the vDS, you can start to migrate the resources from vSS to vDS. The following steps will help you migrate from a Standard to a Distributed Switch: Select the Distributed Switch in the vSphere Web Client. Navigate to Actions | Migrate VM to Another Network. In the Select source and destination networks screen, you have the option to browse to a specific network or no network for source network migration. These options are described as follows: Specific network: This option allows you to select the VMs residing on a particular port group No network: This option implies that VMs that are not connected to any network will be selected for migration In the Destination network option, browse and select the distributed port group for the VM network and click on Next. Select the VM to migrate and click on Next. In the Ready to complete screen, review the settings and click on Finish. How it works... vSphere Distributed Switches extend the capabilities of virtual networking. vDS can be broken into the following two logical sections; one is the data plane and the other is management plane: Data plane: This is also called the I/O plane and it takes cares of the actual packet switching, filtering, tagging, and all networking-related activities. Management plane: This is also known as the control plane. It is a centralized control to manage and configure the data plane functionality. There's more... It is possible to preserve the vSphere Distributed Switch configuration information to a file. You can use these configurations for other deployments and also as a backup. You can restore the port group configuration in case of any misconfiguration. The following steps will export the vSphere Distributed Switch configuration: Select the vSphere Distributed Switch from the vSphere Web Client. Navigate to Actions | All vCenter Actions | Export Configurations. In Configuration to export, you will have the following two options. Select the appropriate one. Distributed Switch and all port groups Distributed Switch only Click on OK. Exporting would begin, and once done, it would ask for saving the configuration. Click on Yes and provide the path to store the file. The import configuration function can be used to create a copy of the exported vDS from the existing configuration file. The following steps will import the vSphere Distributed Switch configuration file: Select the Distributed Switch from the vSphere Web Client. Navigate to Actions | All vCenter Actions | Import distributed port group. In the Import Port Group Configuration option, browse to the backup file and click on Next. Review the import settings and click on Finish. The following steps will restore the vSphere distributed port group configuration: Select the distributed port group from the vSphere Web Client. Navigate to Actions | All vCenter Actions | Restore Configuration. Select one of the following options and click on OK: Restore to a previous configuration: This allows you to restore the configuration of the port group to your previous snapshot Restore configuration from a file: This allows you to restore to the configuration from the file saved on your local system In the Ready to complete screen, review the settings and click on Finish. Summary In this article, we understood the vSphere networking concepts and how to work with vSphere distributed switches. We also discussed some of the more advanced networking configurations available in the distributed switch. Resources for Article: Further resources on this subject: Windows 8 with VMware View [Article] VMware View 5 Desktop Virtualization [Article] Cloning and Snapshots in VMware Workstation [Article]

(For more resources related to this topic, see here.) In this article, you will learn how to present a single record, multiple records, and master-details records on your page using different components and methodologies. You will also learn how to enable the internationalizing and localizing processes in your application by using a resource bundle and the different options of bundle you can have. Starting from this article onward, we will not use the HR schema. We will rather use the FacerHR schema in the Git repository under the BookDatabaseSchema folder and read the README.txt file for information on how to create the database schema. This schema will be used for the whole book, so you need to do this only once. Make sure you validate your database connection information for your recipes to work without problem. Presenting single records on your page In this recipe, we will address the need for presenting a single record in a page, which is useful specifically when you want to focus on a specific record in the table of your database; for example, a user's profile can be represented by a single record in an employee's table. The application and its model have been created for you; you can see it by cloning the PresentingSingleRecord application from the Git repository. How to do it... In order to present a single record in pages, follow the ensuing steps: Open the PresentingSingleRecord application. Create a bounded task flow by right-clicking on ViewController and navigating to New | ADF Task Flow. Name the task flow single-employee-info and uncheck the Create with Page Fragments option. You can create a task flow with a page fragment, but you will need a page to host it at the end; alternatively, you can create a whole page if the task flow holds only one activity and is not reusable. However, in this case, I prefer to create a page-based task flow for fast deployment cycles and train you to always start from task flow. Add a View activity inside of the task flow and name it singleEmployee. Double-click on the newly created activity to create the page; this page will be based on the Oracle Three Column layout. Close the dialog by pressing the OK button. Navigate to Data Controls pane | HrAppModuleDataControl, drag-and-drop EmployeesView1 into the white area of the page template, and select ADF Form from the drop-down list that appears as you drop the view object. Check the Row Navigation option so that it has the first, previous, next, and last buttons for navigating through the task. Group attributes based on their category, so the Personal Information group should include the EmployeeId, FirstName, LastName, Email, and Phone Number attributes; the Job Information group should include HireDate, Job, Salary, and CommissionPct; and the last group will be Department Information that includes both ManagerId and DepartmentId attributes. Select multiple components by holding the Ctrl key and click on the Group button at the top-right corner, as shown in the following screenshot: Change the Display Label values of the three groups to eInfo, jInfo, and dInfo respectively. The Display Label option is a little misleading when it comes to groups in a form as groups don't have titles. Due to this, Display Label will be assigned to the Id attribute of the af:group component that will wrap the components, which can't have space and should be reasonably small; however, Input Text w/Label or Output Text w/Label will end up in the Label attribute in the panelLabelAndMessage component. Change the Component to Use option of all attributes from ADF Input Text w/Label to ADF Output Text w/Label. You might think that if you check the Read-Only Form option, it will have the same effect, but it won't. What will happen is that the readOnly attribute of the input text will change to true, which will make the input text non-updateable; however, it won't change the component type. Change the Display Label option for the attributes to have more human-readable labels to the end user; you should end up with the following screen: Finish by pressing the OK button. You can save yourself the trouble of editing the Display Label option every time you create a component that is based on a view object by changing the Label attribute in UI Hints from the entity object or view object. More information can be found in the documentation at http://docs.oracle.com/middleware/1212/adf/ADFFD/bcentities.htm#sm0140. Examine the page structure from the Structure pane in the bottom-left corner as shown in the following screenshot. A panel form layout can be found inside the center facet of the page template. This panel form layout represents an ADF form, and inside of it, there are three group components; each group has a panel label and message for each field of the view object. At the bottom of the panel form layout, you can locate a footer facet; expand it to see a panel group layout that has all the navigation buttons. The footer facet identifies the locations of the buttons, which will be at the bottom of this panel form layout even if some components appear inside the page markup after this facet. Examine the panel form layout properties by clicking on the Properties pane, which is usually located in the bottom-right corner. It allows you to change attributes such as Max Columns, Rows, Field Width, or Label Width. Change these attributes to change the form and to have more than one column. If you can't see the Structure or Properties pane, you can see them again by navigating to Window menu | Structure or Window menu | Properties. Save everything and run the page, placing it inside the adf-config task flow; to see this in action, refer to the following screenshot: How it works... The best component to represent a single record is a panel form layout, which presents the user with an organized form layout for different input/output components. If you examine the page source code, you can see an expression like #{bindings.FirstName.inputValue}, which is related to the FirstName binding inside the Bindings section of the page definition where it points to EmployeesView1Iterator. However, iterator means multiple records, then why FirstName is only presenting a single record? It's because the iterator is aware of the current row that represents the row in focus, and this row will always point to the first row of the view object's select statement when you render the page. By pressing different buttons on the form, the Current Row value changes and thus the point of focus changes to reflect a different row based on the button you pressed. When you are dealing with a single record, you can show it as the input text or any of the user input's components; alternatively, you can change it as the output text if you are just viewing it. In this recipe, you can see that the Group component is represented as a line in the user interface when you run the page. If you were to change the panel form layout's attributes, such as Max Columns or Rows, you would see a different view. Max Columns represents the maximum number of columns to show in a form, which defaults to 3 in case of desktops and 2 in case of PDAs; however, if this panel form layout is inside another panel form layout, the Max Columns value will always be 1. The Rows attribute represents the numbers of rows after which we should start a new column; it has a default value of 231-1. You can know more about each attribute by clicking on the gear icon that appears when you hover over an attribute and reading the information on the property's Help page. The benefit of having a panel form layout is that all labels are aligned properly; this organizes everything for you similar to the HTML table component. See also Check the following reference for more information about arranging content in forms: http://docs.oracle.com/middleware/1212/adf/ADFUI/af_orgpage.htm#CDEHDJEA

(For more resources related to this topic, see here.) Exploring 3D hierarchies The ability to parent objects among one another is a versatile feature in Unity3D. So far, in this article, we have seen how hierarchies can be used as a means of associating objects with one another and organizing them into hierarchies. One example of this is the character Prefabs with their child hats we developed for the player Prefab and the racers. In this example, by dragging-and-dropping the hat object onto the player's body, we associated the hat with the object body by putting the child into the parent's coordinate system. After doing this, we saw that the rotations, translations, and scales of the body's transform component were propagated to the hat. In practice, this is how we attach objects to one another in 3D games, that is, by parenting them to different parents and thereby changing their coordinate systems or frames of reference. Another example of using hierarchies as a data structure was for collections. Examples of this are the splineData collections we developed for the NPCs. These objects had a single game object at the head and a collection of data as child objects. We could then perform operations on the head, which lets us process all of the child objects in the collection (in our case, installing the way points). A third use of hierarchies was for animation. Since rotations, translations, and scales that are applied to a parent transform are propagated to all of the children, we have a way of developing fine-tuned motion for a hierarchy of objects. It turns out that characters in games and animated objects alike use this hierarchy of transforms technique to implement their motion. Skinned meshes in Unity3D A skinned mesh is a set of polygons whose positions and rotations are computed based on the positions of the various transforms in a hierarchy. Instead of each GameObject in the hierarchy have its own mesh, a single set of polygons is shared across a number of transforms. This results in a single mesh that we call a skin because it envelops the transforms in a skin-like structure. It turns out that this type of mesh is great for in-game characters because it moves like skin. We will now use www.mixamo.com to download some free models and animations for our game. Acquiring and importing models Let's download a character model for the hero of our game. Open your favorite Internet browser and go to www.mixamo.com. Click on Characters and scroll through the list of skinned models. From the models that are listed free, choose the one that you want to use. At the time of writing this article, we chose Justin from the free models available, as shown in the following screenshot: Click on Justin and you will be presented with a preview window of the character on the next page. Click on Download to prepare the model for download. Note, you may have to sign up for an account on the website to continue. Once you have clicked on Download, the small downloads pop-up window at the bottom-right corner of the screen will contain your model. Open the tab and select the model. Make sure to set the download format to FBX for Unity (.fbx) and then click on Download again. FBX (Filmbox)—originally created by a company of the same name and now owned by AutoDesk—is an industry standard format for models and animations. Congratulations! You have downloaded the model we will use for the main player character. While this model will be downloaded and saved to the Downloads folder of your browser, go back to the character select page and choose two more models to use for other NPCs in the game. At the time of writing this article, we chose Alexis and Zombie from the selection of free models, as shown in the following screenshot: Go to Unity, create a folder in your Project tab named Chapter8, and inside this folder, create a folder named models. Right-click on this folder and select Open In Explorer. Once you have the folder opened, drag-and-drop the two character models from your Download folder into the models folder. This will trigger the process of importing a model into Unity, and you should then see your models in your Project view as shown in the following screenshot: Click on each model in the models folder, and note that the Preview tab shows a t-pose of the model as well as some import options. In the Inspector pane, under the Rig tab, ensure that Animation Type is set to Humanoid instead of Generic. The various rig options tell Unity how to animate the skinned mesh on the model. While Generic would work, choosing Humanoid will give us more options when animating under Mechanim. Let Unity create the avatar definition file for you and you can simply click on Apply to change the Rig settings, as shown in the following screenshot: We can now drag-and-drop your characters to the game from the Project tab into the Scene view of the level, as shown in the following screenshot: Congratulations! You have successfully downloaded, imported, and instantiated skinned mesh models. Now, let's learn how to animate them, and we will do this starting with the main player's character. Exploring the Mechanim animation system The Mechanim animation system allows the Unity3D user to integrate animations into complex state machines in an intuitive and visual way! It also has advanced features that allow the user to fine-tune the motion of their characters, right from the use of blend trees to mix, combine, and switch between animations as well as Inverse Kinematics (IK) to adjust the hands and feet of the character after animating. To develop an FSM for our main player, we need to consider the gameplay, moves, and features that the player represents in the game. Choosing appropriate animations In level one, our character needs to be able to walk around the world collecting flags and returning them to the flag monument. Let's go back to www.mixamo.com, click on Animations, and download the free Idle, Gangnam Style, and Zombie Running animations, as shown in the following screenshot: Building a simple character animation FSM Let's build an FSM that the main character will use. To start, let's develop the locomotion system. Import the downloaded animations to a new folder named anims, in Chapter8. If you downloaded the animations attached to a skin, don't worry. We can remove it from the model, when it is imported, and apply it to the animation FSM that you will build. Open the scene file from the first gameplay level TESTBED1. Drag-and-drop the Justin model from the Projects tab into the Scene view and rename it Justin. Click on Justin and add an Animator component. This is the component that drives a character with the Mechanim system. Once you have added this component, you will be able to animate the character with the Mechanim system. Create a new animator controller and call it JustinController. Drag-and-drop JustinController into the controller reference on the Animator component of the Justin instance. The animator controller is the file that will store the specific Mechanim FSM that the Animator component will use to drive the character. Think of it as a container for the FSM. Click on the Justin@t-pose model from the Project tab, and drag-and-drop the avatar definition file from Model to the Avatar reference on the Animator component on the Justin instance. Go to the Window drop-down menu and select Animator. You will see a new tab open up beside the Scene view. With your Justin model selected, you should see an empty Animator panel inside the tab, as shown in the following screenshot: Right now our Justin model has no animations in his FSM. Let's add the idle animation (named Idle_1) from the Adam model we downloaded. You can drag-and-drop it from the Project view to any location inside this panel. That's all there is to it! Now, we have a single anim FSM attached to our character. When you play the game now, it should show Justin playing the Idle animation. You may notice that the loop doesn't repeat or cycle repeatedly. To fix this, you need to duplicate the animation and then select the Loop Pose checkbox. Highlight the animation child object idle_1 and press the Ctrl + D shortcut to duplicate it. The duplicate will appear outside of the hierarchy. You can then rename it to a name of your choice. Let's choose Idle as shown in the following screenshot: Now, click on Idle, and in the Inspector window, make sure that Loop Pose is selected. Congratulations! Using this Idle animation now results in a character who idles in a loop. Let's take a look at adding the walk animation. Click on the Zombie Running animation, which is a child asset of the Zombie model, and duplicate it such that a new copy appears in the Project window. Rename this copy Run. Click on this animation and make sure to check the Loop Pose checkbox so that the animation runs in cycles. Drag-and-drop the Run animation into the Animator tab. You should now have two animations in your FSM, with the default animation still as Idle; if you run the game, Justin should still just be idle. To make him switch animations, we need to do the following: Add some transitions to the Run animation from the Idle animation and vice versa. Trigger the transitions from a script. You will want to switch from the Idle to the Run animation when the player's speed (as determined from the script) is greater than a small number (let's say 0.1 f). Since the variable for speed only lives in the script, we will need a way for the script to communicate with the animation, and we will do this with parameters. In your Animator tab, note that the FSM we are developing lives in the Base Layer screen. While it is possible to add multiple animation layers by clicking on the + sign under Base Layer, this would allow the programmer to design multiple concurrent animation FSMs that could be used to develop varying degrees/levels of complex animation. Add a new parameter by clicking on the + sign beside the Parameters panel. Select float from the list of datatypes. You should now see a new parameter. Name this speed as shown in the following screenshot: Right-click on the Idle Animation and select Make Transition. You will now see that a white arrow is visible, which extends from Idle, that tracks the position of your mouse pointer. If you now click on the Run animation, the transition from Idle to Run will lock into place. Left-click on the little white triangle of the transition and observe the inspector for the transition itself. Scroll down in the Inspector window to the very bottom and set the condition for the transition to speed greater than 0.1, as shown in the following screenshot: Make the transition from the Run animation cycle back to the Idle cycle by following the same procedure. Right-click on the Run animation to start the transition. Then, left-click on Idle. After this, left-click on the transition once it is locked into place. Then, when the transition is activated in Conditions, set its speed to less than 0.09. Congratulations! Our character will now transition from Idle to Run when the speed crosses the 0.1 threshold. The transition is a nice blend from the first animation to the second over a brief period of time, and this is indicated in the transition graph.

(For more resources related to this topic, see here.) What is CMIS? The goal of CMIS is to provide a standard method for accessing content from different content repositories. Using CMIS service calls, it is possible to navigate through and create content in a repository. CMIS also includes a query language for searching both the metadata and full-text content stored that is stored in a repository. The CMIS standard defines the protocols and formats for the requests and responses of API service calls made to a repository. CMIS acts as a standard interface and protocol for accessing content repositories, something similar to how ANSI-SQL acts as a common-denominator language for interacting with different databases. The use of the CMIS API for accessing repositories brings with it a number of benefits. Perhaps chief among these is the fact that access to CMIS is language neutral. Any language that supports HTTP services can be used to access a CMIS-enabled repository. Client software can be written to use a single API and be deployed to run against multiple CMIS-compliant repositories. Alfresco and CMIS The original draft for CMIS 0.5 was written by EMC, IBM and Microsoft. Shortly after that draft, Alfresco and other vendors joined the CMIS standards group. Alfresco was an early CMIS adopter and offered an implementation of CMIS version 0.5 in 2008. In 2009, Alfresco began hosting an on-line preview of the CMIS standard. The server, accessible via the http://cmis.alfresco.com URL, still exists and implements the latest CMIS standard. As of this writing, that URL hosts a preview of CMIS 1.1 features. In mid-2010, just after the CMIS 1.0 standard was approved, Alfresco released CMIS in both the Alfresco Community and Enterprise editions. In 2012, with Alfresco version 4.0, Alfresco moved from a home grown CMIS runtime implementation to one that uses the Apache Chemistry OpenCMIS Server Framework. From that release, developers have been able to customize Alfresco using the OpenCMIS Java API. Overview of the CMIS Standard Next we discuss the details of the CMIS specification, particularly the domain model, the different services that it provides, and the supported protocol bindings. Domain model (Object model) Every content repository vendor has their own definition of a content or object model. Alfresco, for example, has rich content modeling capabilities, such as types and aspects that can inherit from other types and aspects, and properties that can be assigned attributes like data-type, multi-valued and required. But there are wide differences in the ways in which different vendors have implemented content modeling. In the Documentum ECM system, for example, the generic content type is called dm_document, while in Alfresco it is called cm:content. Another example is the concept of an aspect as used in Alfresco – many repositories do not support that idea. The CMIS Domain Model is an attempt by the CMIS Standardization group to define a framework generic enough that can describe content models and map to concepts used by many different repository vendors. The CMIS Domain Model defines a Repository as a container and an entry point to all content items, from now on called objects. All objects are classified by an Object Type , which describes a common set of Properties (like Type ID, Parent, and Display Name). There are five base types of objects: Document, Folder, Relationship, Policy , Item(available from CMIS 1.1), and these all inherit from Object Type. In addition to the five base object types there are also a number of property types that can be used when defining new properties for an Object type. These are shown in the figure: String, Boolean, Decimal, Integer, and DateTime. Besides these property types there are also the URI, Id, and HTML property types, not shown in the figure. Taking a closer look at each one of the base types, we can see that: Document almost always corresponds to a file, although it need not have any content (when you upload a file via, for example, the AtomPub binding the metadata is created with the first request and the content for the file is posted with the second request). Folder is a container for file-able objects such as folders and documents. Immediately after filing a folder or document into a folder, an implicit parent-child relationship is automatically created. The fileable property of the object type definition specifies whether an object is file-able or not. Relationship object defines a relationship between a target and source object. Objects can have multiple relationships with other objects. The support for relationship objects is optional. Policy is a way of defining administrative policies to manage objects. An object to which a policy may be applied is called a controllable object (controllablePolicy property has to be set to true). For example, a CMIS policy could be used to define a retention policy. A policy is opaque and has no meaning to the repository. It must be implemented and enforced in a repository-specific way. For example, rules might be used in Alfresco to enforce a policy. The support for policy objects is optional. Item (CMIS 1.1) object represents a generic type of a CMIS information asset. For example, this could be a user or group object. Item objects are not versionable and do not have content streams like documents, but they do have properties like all other CMIS objects. The support for item objects is optional. Additional object types can be defined in a repository as custom subtypes of the base types. The Legal Case type shown in the figure above is an example. CMIS services are provided for the discovery of object types that are defined in a repository. However, object type management services, such as the creation, modification, and deletion of an object type, are not covered by the CMIS standard. An object has one primary base object type, such as Document or Folder, which cannot be changed. An object can also have secondary object types applied to it (CMIS 1.1). A secondary type is a named class that may add extra properties to an object in addition to the properties defined by the object's primary base object-type (This is similar to the concept of aspects in Alfresco). Every CMIS object has an opaque and immutable Object Identity (ID), which is assigned by the repository when the object is created. In the case of Alfresco, a Node Reference is created which becomes the Object ID. The ID uniquely identifies an object within a repository regardless of the type of the object. All CMIS objects have a set of named, but not explicitly ordered, properties. Within an object, each property is uniquely identified by its Property ID. In addition, a document object can have a Content Stream, which is then used to hold the actual byte content from a file. A document can also have one or more Renditions, like a thumbnail, a different sized image, or an alternate representation of the content stream. Document or folder objects can have one Access Control List (ACL), which controls access to the document or folder. An ACL is made up of a list of Access Control Entries (ACEs). An ACE in turn represents one or more permissions being granted to a principal, such as a user, group, role, or something similar. All objects and properties are defined in the cmis name-space. From now on we will refer to the different objects and properties via their fully qualified name, for example cmis:document or cmis:name. Services The following CMIS services can access and manage CMIS objects in the repository: Repository Services: These are used to discover information about the repository, including repository ids (more than one repository could be managed by the endpoint). Since many features are optional, this provides a way to find out which are supported. CMIS 1.1 compliant repositories also support the creation of new types dynamically. Methods: getRepositories, getRepositoryInfo , getTypeChildren, getTypeDescendants , getTypeDefinition , createType (CMIS 1.1), updateType (CMIS 1.1), deleteType (CMIS 1.1) Navigation Services: These are used to navigate the folder hierarchy. Methods: getChildren, getDescendants, getFolderTree, getFolderParent, getObjectParents, getCheckedOutDocs. Object Services: These services provide ID-based CRUD (Create, Read, Update, Delete) operations. Methods: createDocument, createDocumentFromSource, createFolder, createRelationship, createPolicy, createItem (CMIS 1.1), getAllowableActions, getObject, getProperties, getObjectByPath, getContentStream, getRenditions, updateProperties, bulkUpdateProperties (CMIS 1.1), moveObject, deleteObject, deleteTree, setContentStream, appendContentStream (CMIS 1.1), deleteContentStream. Multi-filing Services: These services (optional) makes it possible to put an object to several folders (multi-filing) or outside the folder hierarchy (un-filing). This service is not used to create or delete objects. Methods: addObjectToFolder, removeObjectFromFolder. Discovery Services: These are used to search for query-able objects within the Repository (objects with property queryable set to true ). Methods: query , getContentChanges. Versioning Services: These are used to manage versioning of document objects, other objects are not versionable. Whether or not a document can be versioned is controlled by the versionable property in the Object type. Methods: checkOut, cancelCheckOut, checkIn, getObjectOfLatestVersion, getPropertiesOfLatestVersion, getAllVersions. Relationship Services: These (optional) are used to retrieve the relationships in which an object is participating. Methods: getObjectRelationships. Policy Services: These (optional) are used to apply or remove a policy object to an object which has the property controllablePolicy set to true. Methods: applyPolicy, removePolicy, getAppliedPolicies. ACL Services: This service is used to discover and manage the Access Control List (ACL) for an object, if the object has one. Methods: applyACL, and getACL Summary In this article, we introduced the CMIS standard and how it came about. Then we covered the CMIS domain model with its five base object types: document, folder, relationship, policy, and item (CMIS 1.1.). We also learned that the CMIS standard defines a number of services, such as navigation and discovery, which makes it possible to manipulate objects in a content management system repository. Resources for Article: Further resources on this subject: Content Delivery in Alfresco 3 [Article] Getting Started with the Alfresco Records Management Module [Article] Managing Content in Alfresco [Article]

(For more resources related to this topic, see here.) Social media mining using sentiment analysis People are highly opinionated. We hold opinions about everything from international politics to pizza delivery. Sentiment analysis, synonymously referred to as opinion mining, is the field of study that analyzes people's opinions, sentiments, evaluations, attitudes, and emotions through written language. Practically speaking, this field allows us to measure, and thus harness, opinions. Up until the last 40 years or so, opinion mining hardly existed. This is because opinions were elicited in surveys rather than in text documents, computers were not powerful enough to store or sort a large amount of information, and algorithms did not exist to extract opinion information from written language. The explosion of sentiment-laden content on the Internet, the increase in computing power, and advances in data mining techniques have turned social data mining into a thriving academic field and crucial commercial domain. Professor Richard Hamming famously pushes researchers to ask themselves, "What are the important problems in my field?" Researchers in the broad area of natural language processing (NLP) cannot help but list sentiment analysis as one such pressing problem. Sentiment analysis is not only a prominent and challenging research area, but also a powerful tool currently being employed in almost every business and social domain. This prominence is due, at least in part, to the centrality of opinions as both measures and causes of human behavior. This article is an introduction to social data mining. For us, social data refers to data generated by people or by their interactions. More specifically, social data for the purposes of this article will usually refer to data in text form produced by people for other people's consumption. Data mining is a set of tools and techniques used to describe and make inferences about data. We approach social data mining with a potent mix of applied statistics and social science theory. As for tools, we utilize and provide an introduction to the statistical programming language R. The article covers important topics and latest developments in the field of social data mining with many references and resources for continued learning. We hope it will be of interest to an audience with a wide array of substantive interests from fields such as marketing, sociology, politics, and sales. We have striven to make it accessible enough to be useful for beginners while simultaneously directing researchers and practitioners already active in the field towards resources for further learning. Code and additional material will be available online at http://socialmediaminingr.com as well as on the authors' GitHub account, https://github.com/SocialMediaMininginR. The state of communication The state of communication section describes the fundamentally altered modes of social communication fostered by the Internet. The interconnected, social, rapid, and public exchange of information detailed here underlies the power of social data mining. Now more than ever before, information can go viral, a phrase first cited as early as 2004. By changing the manner in which we connect with each other, the Internet changed the way we interact—communication is now bi-directional and many-to-many. Networks are now self-organized, and information travels along every dimension, varying systematically depending on direction and purpose. This new economy with ideas as currency has impacted nearly every person. More than ever, people rely on context and information before making decisions or purchases, and by extension, more and more on peer effects and interactions rather than centralized sources. The traditional modes of communication are represented mainly by radio and television, which are isotropic and one-to-many. It took 38 years for radio broadcasters and 13 years for television to reach an audience of 50 million, but the Internet did it in just four years (Gallup). Not only has the nature of communication changed, but also its scale. There were 50 pages on the World Wide Web (WWW) in 1993. Today, the full impact and scope of the WWW is difficult to measure, but we can get a rough sense of its size: the Indexed Web contains at least 1.7 billion pages as of February 2014 (World Wide Web size). The WWW is the largest, most widely used source of information, with nearly 2.4 billion users (Wikipedia). 70 percent of these users use it daily to both contribute and receive information in order to learn about the world around them and to influence that same world—constantly organizing information around pieces that reflect their desires. In today's connected world, many of us are members of at least one, if not more, social networking service. The influence and reach of social media enterprises such as Facebook is staggering. Facebook has 1.11 billion monthly active users and 751 million monthly active users of their mobile products (Facebook key facts). Twitter has more than 200 million (Twitter blog) active users. As communication tools, they offer a global reach to huge multinational audiences, delivering messages almost instantaneously. Connectedness and social media have altered the way we organize our communications. Today we have dramatically more friends and more friends of friends, and we can communicate with these higher order connections faster and more frequently than ever before. It is difficult to ignore the abundance of mimicry (that is, copying or reposting) and repeated social interactions in our social networks. This mimicry is a result of virtual social interactions organized into reaffirming or oppositional feedback loops. We self-organize these interactions via (often preferential) attachments that form organic, shifting networks. There is little question of whether or not social media has already impacted your life and changed the manner in which you communicate. Our beliefs and perceptions of reality, as well as the choices we make, are largely conditioned by our neighbors in virtual and physical networks. When we need to make a decision, we seek out for opinions of others—more and more of those opinions are provided by virtual networks. Information bounce is the resonance of content within and between social networks often powered by social media such as customer reviews, forums, blogs, microblogs, and other user-generated content. This notion represents a significant change when compared to how information has traveled throughout history; individuals no longer need to exclusively rely on close ties within their physical social networks. Social media has both made our close ties closer and the number of weak ties exponentially greater. Beyond our denser and larger social networks is a general eagerness to incorporate information from other networks with similar interests and desires. The increased access to networks of various types has, in fact, conditioned us to seek even more information; after all, ignoring available information would constitute irrational behavior. These fundamental changes to the nature and scope of communication are crucial due to the importance of ideas in today's economic and social interactions. Today, and in the future, ideas will be of central importance, especially those ideas that bounce and go viral. Ideas that go viral are those that resonate and spur on social movements, which may have political and social purposes or reshape businesses and allow companies such as Nike and Apple to produce outsized returns on capital. This article introduces readers to the tools necessary to measure ideas and opinions derived from social data at scale. Along the way, we'll describe strategies for dealing with Big Data. What is Big Data? People create 2.5 quintillion bytes (2.5 * 1018) of data, or nearly 2.3 million Terabytes of data every day, so much that 90 percent of the data in the world today has been created in the last two years alone. Furthermore, rather than being a large collection of disparate data, much of this data flow consists of data on similar things, generating huge data-sets with billions upon billions of observations. Big Data refers not only to the deluge of data being generated, but also to the astronomical size of data-sets themselves. Both factors create challenges and opportunities for data scientists. This data comes from everywhere: physical sensors used to gather information, human sensors such as the social web, transaction records, and cell phone GPS signals to name a few. This data is not only big but is growing at an increasing rate. The data used in this article, namely, Twitter data, is no exception. Twitter was launched in March 21, 2006, and it took 3 years, 2 months, and 1 day to reach 1 billion tweets. Twitter users now send 1 billion tweets every 2.5 days. The size and scope of Big Data helps us overcome some of the hurdles caused by its low density. For instance, even though each unique piece of social data may have little applicability to our particular task, these small bits of information quickly become useful as we aggregate them across thousands or millions of people. Like the proverbial bundle of sticks—none of which could support inferences alone—when tied together, these small bits of information can be a powerful tool for understanding the opinions of the online populace. The sheer scope of Big Data has other benefits as well. The size and coverage of many social data-sets creates coverage overlaps in time, space, and topic. This allows analysts to cross-refer socially generated sets against one another or against small-scale data-sets designed to examine niche questions. This type of cross-coverage can generate consilience (Osborne)—the principle that states evidence from independent, unrelated sources can converge to form strong conclusions. That is, when multiple sources of evidence are in agreement, the conclusion can be very strong even when none of the individual sources of evidence are very strong on their own. A crucial characteristic of socially generated data is that it is opinionated. This point underpins the usefulness of big social data for sentiment analysis, and is novel. For the first time in history, interested parties can put their fingers to the pulse of the masses because the masses are frequently opining about what is important to them. They opine with and for each other and anyone else who cares to listen. In sum, opinionated data is the great enabler of opinion-based research. Human sensors and honest signals Opinion data generated by humans in real time presents tremendous opportunities. However, big social data will only prove useful to the extent that it is valid. This section tackles the extent to which socially generated data can be used to accurately measure individual and/or group-level opinions head-on. One potential indicator of the validity of socially generated data is the extent of its consumption for factual content. Online media has expanded significantly over the past 20 years. For example, online news is displacing print and broadcast. More and more Americans distrust mainstream media, with a majority (60 percent) now having little to no faith in traditional media to report news fully, accurately, and fairly. Instead, people are increasingly turning to the Internet to research, connect, and share opinions and views. This was especially evident during the 2012 election where social media played a large role in information transmission (Gallup). Politics is not the only realm affected by social Big Data. People are increasingly relying on the opinions of others to inform about their consumption preferences. Let's have a look at this: 91 percent of people report having gone into a store because of an online experience 89 percent of consumers conduct research using search engines 62 percent of consumers end up making a purchase in a store after researching it online 72 percent of consumers trust online reviews as much as personal recommendations 78 percent of consumers say that posts made by companies on social media influence their purchases If individuals are willing to use social data as a touchstone for decision making in their own lives, perhaps this is prima facie evidence of its validity. Other Big Data thinkers point out that much of what people do online constitutes their genuine actions and intentions. The breadcrumbs left from when people execute online transactions, send messages, or spend time on web pages constitute what Alex Petland of MIT calls honest signals. These signals are honest insofar as they are actions taken by people with no subtext or secondary intent. Specifically, he writes the following: "Those breadcrumbs tell the story of your life. It tells what you've chosen to do. That's very different than what you put on Facebook. What you put on Facebook is what you would like to tell people, edited according to the standards of the day. Who you actually are is determined by where you spend time, and which things you buy." To paraphrase, Petland finds some web-based data to be valid measures of people's attitudes when that data is without subtext or secondary intent; what he calls data exhaust. In other words, actions are harder to fake than words. He cautions against taking people's online statements at face value, because they may be nothing more than cheap talk. Anthony Stefanidis of George Mason University also advocates for the use of social data mining. He favorably speaks about its reliability, noting that its size inherently creates a preponderance of evidence. This article takes neither the strong position of Pentland and honest signals nor Stefanidis and preponderance of evidence. Instead, we advocate a blended approach of curiosity and creativity as well as some healthy skepticism. Generally, we follow the attitude of Charles Handy (The Empty Raincoat, 1994), who described the steps to measurement during the Vietnam War as follows: "The first step is to measure whatever can be easily measured. This is OK as far as it goes. The second step is to disregard that which can't be easily measured or to give it an arbitrary quantitative value. This is artificial and misleading. The third step is to presume that what can't be measured easily really isn't important. This is blindness. The fourth step is to say that what can't be easily measured really doesn't exist. This is suicide." The social web may not consist of perfect data, but its value is tremendous if used properly and analyzed with care. 40 years ago, a social science study containing millions of observations was unheard of due to the time and cost associated with collecting that much information. The most successful efforts in social data mining will be by those who "measure (all) what is measurable, and make measurable (all) what is not so" (Rasinkinski, 2008). Ultimately, we feel that the size and scope of big social data, the fact that some of it is comprised of honest signals, and the fact that some of it can be validated with other data, lends it validity. In another sense, the "proof is in the pudding". Businesses, governments, and organizations are already using social media mining to good effect; thus, the data being mined must be at least moderately useful. Another defining characteristic of big social data is the speed with which it is generated, especially when considered against traditional media channels. Social media platforms such as Twitter, but also the web generally, spread news in near-instant bursts. From the perspective of social media mining, this speed may be a blessing or a curse. On the one hand, analysts can keep up with the very fast-moving trends and patterns, if necessary. On the other hand, fast-moving information is subject to mistakes or even abuse. Following the tragic bombings in Boston, Massachusetts (April 15, 2013), Twitter was instrumental in citizen reporting and provided insight into the events as they unfolded. Law enforcement asked for and received help from general public, facilitated by social media. For example, Reddit saw an overall peak in traffic when reports came in that the second suspect was captured. Google Analytics reports that there were about 272,000 users on the site with 85,000 in the news update thread alone. This was the only time in Reddit's history other than Obama AMA that a thread beat the front page in the ratings (Reddit). The downside of this fast-paced, highly visible public search is that masses can be incorrect. This is exactly what happened; users began to look at the details and photos posted and pieced together their own investigation—as it turned out, the information was incorrect. This was a charged event and created an atmosphere that ultimately undermined the good intentions of many. Other efforts such as those by governments (Wikipedia) and companies (Forbes) to post messages favorable to their position is less than well intentioned. Overall, we should be skeptical of tactical (that is, very real time) uses of social media. Summary In this article, we introduced readers to the concepts of social media, sentiment analysis, and Big Data. We described how social media has changed the nature of interpersonal communication and the opportunities it presents for analysts of social data. This article also made a case for the use of quantitative approaches to measure all that is measurable, and make the one which is not so measurable. Resources for Article: Further resources on this subject: Bitcoins – Pools and Mining [Article] Managing your social channels [Article] Social Media for Wordpress: VIP Memberships [Article]