Search icon CANCEL
Subscription
0
Cart icon
Your Cart (0 item)
Close icon
You have no products in your basket yet
Save more on your purchases! discount-offer-chevron-icon
Savings automatically calculated. No voucher code required.
Arrow left icon
Explore Products
Best Sellers
New Releases
Books
Events
Videos
Audiobooks
Packt Hub
Free Learning
Arrow right icon
timer SALE ENDS IN
0 Days
:
00 Hours
:
00 Minutes
:
00 Seconds

How-To Tutorials

7018 Articles
article-image-hive-security
Packt
17 Feb 2016
13 min read
Save for later

Hive Security

Packt
17 Feb 2016
13 min read
In this article by Hanish Bansal, Saurabh Chauhan, and Shrey Mehrotra, the authors of the book, Apache Hive Cookbook, we will cover the following recipes: Securing Hadoop Authorizing Hive Security is a major concern in all the big data frameworks. It is little complex to implement security in distributed systems because the components of different machines need to communicate with each other. It is very important to enable security on the data. (For more resources related to this topic, see here.) Securing Hadoop In today's era of big data, most organizations are concentrating to use Hadoop as a centralized data store. Data size is growing day by day, and organizations want to derive some insights and make decisions using the important information. While everyone is focusing on collecting the data, but having all the data at a centralized place increases the risk of data security. Securing the data access of Hadoop Distributed File System (HDFS) is very important. Hadoop security means restricting the access of data to only authorized users and groups. Further, when we talk about security, there are two major things—Authentication and Authorization. The HDFS supports a permission model for files and directories that is much equivalent to standard POSIX model. Similar to UNIX permissions, each file and directory in HDFS are associated with an owner, a group, and other users. There are three types of permissions in HDFS: read, write, and execute. In contrast to the UNIX permission model, there is no concept of executable files. So in case of files, read (r) permission is required to read a file, and write (w) permission is required to write or append to a file. In case of directories, read (r) permission is required to list the contents of directory, write (w) permission is required to create or delete the files or subdirectories, and execute (x) permission is required to access the child objects (files/subdirectories) of that directory. The following screenshot shows the level of access to each individual entity, namely OWNER, GROUP, and OTHER: The Default HDFS Permission Model As illustrated in the previous screenshot, by default, the permission set for the owner of files or directories is rwx (7), which means the owner of the file or directory is having full permission to read, write, and execute. For the members of group, the permission set is r-x, which means group members can only read and execute the files or directories and they cannot write or update anything in the files or directories. For other members, a permission set is same as a group, that is, other members can only read and execute the files or directories and they cannot write or update anything in files or directories. Although this basic permission model is sufficient to handle a large number of security requirements at the block level, but using this model, you cannot define finer level security to specifically named users or groups. HDFS also has a feature to configure Access Control List (ACL), which can be used to define fine-grained permissions at file level as well as directory level for specifically named users or groups. For example, you want to give read access to users John, Mike, and Kate; then, HDFS ACLs can be used to define such kind of permissions. HDFS ACLs are designed on the base concept of POSIX ACLs of UNIX systems. How to do it… First of all, you will need to enable ACLs in Hadoop. To enable ACL permissions, configure the following property in Hadoop-configure file named hdfs-site.xml located at <HADOOP_HOME>/etc/hadoop/hdfs-site.xml: <property> <name>dfs.namenode.acls.enabled</name> <value>true</value> </property> There are two main commands that are used to configure ACLs: setfacl and getfacl. The command setfacl is used to set Finer Access Control Lists (FACL) for files or directories, and getfacl is used to retrieve Finer Access Control Lists (FACL) for files or directories. Let's see how to use these commands: hdfs dfs -setfacl [-R] [-b |-k -m |-x <acl_specification> <path>] |[--set <acl_specification> <path>] The same command can be run using hadoop fs also, as follows: hadoop fs -setfacl [-R] [-b |-k -m |-x <acl_specification> <path>] |[--set <acl_specification> <path>] This command contains the following elements: -R is used to apply operation recursively for all files and subdirectories under a directory. -b is used to remove all ACLs except the base ACLs. -k is used to remove the default ACLs. -m is used to modify ACLs. Using this option, new entries are added to the existing set of ACLs. -x is used to remove specific ACLs. acl_specification is a comma-separated list of ACLs. path is the path of a file or directory for which ACL has to be applied. --set is used to set new ACLs. It removes all existing ACLs and set the new ACLs only. Now, let's see another command that is used to retrieve the ACLs: hdfs dfs -getfacl [-R] <path> This command can also be run using hadoop fs as follows: hadoop fs -getfacl [-R] <path> This command contains the following elements: -R is used to retrieve ACLs recursively for all files and subdirectories under a directory path is a path of a file or directory of which ACL is to be retrieve The command getfacl will list all default ACLs as well as new ACLs defined for specified files or directories. How it works… If ACLs are defined for a file or directory, then while accessing that file/directory, access is validated as given in the following algorithm: If the username is the same as the owner name of the file, then owner permissions are enforced If username matches with one of named user ACL entry, then those permissions are enforced If a user's group name matches with one of the named group ACL entry, then those permissions are enforced In case multiple ACLs entries found for a user, then the union of all those permissions is enforced If no ACL entry found for a user, then other permissions are enforced Let's assume that we have a file named stock-data containing stock market data. To retrieve all ACLs of this file, run the following command after which the output is shown in the screenshot given later: $ hadoop fs -getfacl /stock-data Because we have not defined any custom ACL for this file, as shown in the previous screenshot, command will return default ACL for this file. You can check the permissions of a file or directory using the ls command also. As shown in the previous screenshot, the permission set for stock-data file is -rw-r--r--, which means read and write access for owner as well as read access for group members and others. In the following command, we give read and write access to user named mike, and the result is shown in the following screenshot: $ hadoop fs -setfacl -m user:mike:rw- /stock-data As shown in the previous screenshot, first, we defined the ACLs for the user mike using setfacl command; then, we retrieved the ACLs using the getfacl command. The output of the getfacl command will list out all default permissions as well as all ACLs. We defined ACLs for the user mike, so in the output, there is an extra row user:mike:rw-. There is an extra row in the output mask::rw-, which defines special mask ACLs entry. Mask is a special type of ACLs that filters out the access for all named users, named groups, and unnamed groups. If you have not defined mask ACL, then its value is calculated using the union of all permissions. In addition to this, the output of the ls command is also changed after defining ACLs. There is an extra plus (+) sign in permissions list that indicates that there are additional ACLs defined for this file or directory. Revoking access of user mike. To remove a specific ACL -x option is used with the setfacl command: $ hadoop fs -setfacl -x user:mike /stock-data In the previous screenshot, after revoking access of user mike, ACLs are updated, and there is no entry for the user mike now. See also You can read more about the permission model in Hadoop at https://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-hdfs/HdfsPermissionsGuide.html. Authorizing Hive Hive authorization is to verifying that a user is authorized to perform particular action. Authentication is about verifying the identity of a user, which is different from the authorization concept. Hive can be used in the following different ways: Using HCatalog API: Hive's Hcatalog API is used to access Hive by many other frameworks such as Apache Pig, MapReduce, Facebook Presto, Spark SQL, and Cloudera Impala. Using the HCatalog API, users have direct access to HDFS data and hive metadata. Hive metadata is directly accessible using metastore server API. Using Hive CLI: Using Hive CLI also, users have direct access to HDFS data and Hive metadata. Hive CLI directly interacts with a Hive metastore server. Currently Hive CLI don't support rich authorization. In next versions of hive, Hive CLI's implementation will be changed to provide better security, and also Hive CLI will interact with HiveServer2 rather than directly interacting with the metastore server. Using ODBC/JDBC and other HiveServer2 clients such as Beeline: These clients don't have direct access to HDFS data and metadata but through HiveServer2. For security purpose, this is the best way to access Hive. How to do it… The following are the various ways of authorization in Hive: Default authorization: the legacy mode: The legacy authorization mode was available in earlier versions of Hive. This authorization scheme prevents the users from doing some unwanted actions. This scheme doesn't prevent malicious users from doing activities. It manages the access control using grant and revoke statements. This mode supports Hive Command Line Interface (Hive CLI). In case of Hive CLI, users have direct access to HDFS files and directories, so they can easily break the security checks. Also in this model for granting privileges, the permissions needed for a user are not defined, which means that any user can grant the access to themselves, so it is not secure to use this model. Storage-based Authorization: As storage perspective, both HDFS data as well as Hive metadata must be accessed only to authorized users. If users use the HCatalog API or Hive CLI, then they have direct access to data. To protect the data, HDFS ACLs are being enabled. In this mode, HDFS permissions work as a single source of truth for protecting data.Generally in Hive metastore, database credentials are configured in Hive configuration file hive-site.xml. Malicious users can easily read the metastore credentials and then cause serious damage to data as well as metadata, so Hive metastore server can also be secured. In this authorization, you can also enable security at metastore level. After enabling metastore security, it will restrict the access on metadata objects by verifying that users have respective system permissions corresponding to different files and directories of metadata objects. To configure the storage-based authorization, set the following properties in the hive-site.xml file: Property Value hive.metastore.pre.event.listeners org.apache.hadoop.hive.ql.security.authorization.AuthorizationPreEventListener hive.security.metastore.authorization.manager org.apache.hadoop.hive.ql.security.authorization.StorageBasedAuthorizationProvider hive.security.metastore.authenticator.manager org.apache.hadoop.hive.ql.security.HadoopDefaultMetastoreAuthenticator hive.security.metastore.authorization.auth.reads true After setting all these configurations, Hive configuration file hive-site.xml will look as follows: <configuration> <property> <name>hive.metastore.pre.event.listeners</name> <value>org.apache.hadoop.hive.ql.security.authorization.AuthorizationPreEventListener</value> </property> <property> <name>hive.security.metastore.authorization.manager</name> <value>org.apache.hadoop.hive.ql.security.authorization.StorageBasedAuthorizationProvider</value> </property> <property> <name>hive.security.metastore.authenticator.manager</name> <value>org.apache.hadoop.hive.ql.security.HadoopDefaultMetastoreAuthenticator</value> </property> <property> <name>hive.security.metastore.authorization.auth.reads</name> <value>true</value> </property> </configuration> hive.metastore.pre.event.listeners: This property is used to define pre-event listener class, which is loaded on the metastore side. APIs of this class are executed before occurring of any event such as creating a database, table, or partition; altering a database, table, or partition; or dropping a database, table, or partition. Configuring this property turns on security at a metastore level. Set value of this property to org.apache.hadoop.hive.ql.security.authorization.AuthorizationPreEventListener. hive.security.metastore.authorization.manager: This property is used to define the authorization provider class for metastore security. The default value of this property is DefaultHiveMetastoreAuthorizationProvider, which provides default legacy authorization described in the previous bullet. To enable storage-based authorization based on Hadoop ACLs, set value of this property to org.apache.hadoop.hive.ql.security.authorization.StorageBasedAuthorizationProvider. You can also write your own custom class to manage authorization and configure this property to enable custom authorization manager. The custom authorization manager class must implement an interface org.apache.hadoop.hive.ql.security.authorization.HiveMetastoreAuthorizationProvider. hive.security.metastore.authenticator.manager: This property is used to define an authentication manager class. Set value of this property to org.apache.hadoop.hive.ql.security.HadoopDefaultMetastoreAuthenticator. You can also write your custom class to manage authentication and configure to this property. The custom authentication manager class must implement an interface org.apache.hadoop.hive.ql.security.HiveAuthenticationProvider. hive.security.metastore.authorization.auth.reads: This property is used to define whether metastore authorization should check for read access or not. The default value of this property is true. SQL Standard-based Authorization: SQL standard-based authorization is the third way of authorizing Hive. Although the previous methodology storage-based authorization also provides access control at level of partitions, tables, and databases, that methodology does not provide access control at more granular level such as columns and rows. This is because storage-based authorization depends on access control provided by HDFS using ACL that controls the access on the level of files and directories. SQL Standard-based authorization can be used to enforce fine-grained security. It is recommended to use as it is fully SQL compliant in its authorization model. There's more Many things can be done with SQL standard-based authorization. Use SQL standard-based authorization for more details. Summary In this article, we learned two different recipes Securing Hadoop and Authorizing Hive. You also learned different terminology of access permissions and their types. You went through various steps to secure Hadoop and learned different ways to perform authorization in Hive. Resources for Article: Further resources on this subject: Hive in Hadoop[article] Processing Tweets with Apache Hive[article] Using Hive non-interactively (Simple)[article]
Read more
  • 0
  • 0
  • 3495

article-image-chrome-custom-tabs
Packt
17 Feb 2016
16 min read
Save for later

Chrome Custom Tabs

Packt
17 Feb 2016
16 min read
Well, most of us know tabs from every day Internet browsing. It doesn't really matter which browser you use; all browsers support tabs and multiple tabs' browsing. This allows us to have more than one website open at the same time and navigate between the opened instances. In Android, things are much the same, but when using WebView, you don't have tabs. This article will give highlights about WebView and the new feature of Android 6, Chrome custom tabs. (For more resources related to this topic, see here.) What is WebView? WebView is the part in the Android OS that's responsible for rendering web pages in most Android apps. If you see web content in an Android app, chances are you're looking at WebView. The major exceptions to this rule are some of the Android browsers, such as Chrome, Firefox, and so on. In Android 4.3 and lower, WebView uses code based on Apple's Webkit. In Android 4.4 and higher, WebView is based on the Chromium project, which is the open source base of Google Chrome. In Android 5.0, WebView was decoupled into a separate app that allowed timely updates through Google Play without requiring firmware updates to be issued, and the same technique was used with Google Play services. Now, let's talk again about a simple scenario: we want to display web content (URL-related) in our application. We have two options: either launch a browser or build our own in-app browser using WebView. Both options have trade-offs or disadvantages if we write them down. A browser is an external application and you can't really change its UI; while using it, you push the users to other apps and you may lose them in the wild. On the other hand, using WebView will keep the users tightly inside. However, actually dealing with all possible actions in WebView is quite an overhead. Google heard our rant and came to the rescue with Chrome custom tabs. Now we have better control over the web content in our application, and we can stitch web content into our app in a cleaner, prettier manner. Customization options Chrome custom tabs allow several modifications and tweaks: The toolbar color Enter and exit animations Custom actions for the toolbar and overflow menu Prestarted and prefetched content for faster loading When to use Chrome custom tabs Ever since WebView came out, applications have been using it in multiple ways, embedding content—local static content inside the APK and dynamic content as loading web pages that were not designed for mobile devices at the beginning. Later on we saw the rise of the mobile web era complete with hybrid applications). Chrome custom tabs are a bit more than just loading local content or mobile-compatible web content. They should be used when you load web data and want to allow simple implementation and easier code maintenance and, furthermore, make the web content part of your application—as if it's always there within your app. Among the reasons why you should use custom tabs are the following: Easy implementation: you use the support library when required or just add extras to your View intent. It's that simple. In app UI modifications, you can do the following: Set the toolbar color Add/change the action button Add custom menu items to the overflow menu Set and create custom in/out animations when entering the tab or exiting to the previous location Easier navigation and navigation logic: you can get a callback notifying you about an external navigation, if required. You know when the user navigates to web content and where they should return when done. Chrome custom tabs allow added performance optimizations that you can use: You can keep the engine running, so to speak, and actually give the custom tab a head start to start itself and do some warm up prior to using it. This is done without interfering or taking away precious application resources. You can provide a URL to load in advance in the background while waiting for other user interactions. This speeds up the user-visible page loading time and gives the user a sense of blazing fast application where all the content is just a click away. While using the custom tab, the application won't be evicted as the application level will still be in the foreground even though the tab is on top of it. So, we remain at the top level for the entire usage time (unless a phone call or some other user interaction leads to a change). Using the same Chrome container means that users are already signed in to sites they connected to in the past; specific permissions that were granted previously apply here as well; even fill data, autocomplete, and sync work here. Chrome custom tabs allow us give the users the latest browser implementation on pre-Lollipop devices where WebView is not the latest version. The implementation guide As discussed earlier, we have a couple of features integrated into Chrome custom tabs. The first customizes the UI and interaction with the custom tabs. The second allows pages to be loaded faster and keeps the application alive. Can we use Chrome custom tabs? Before we start using custom tabs, we want to make sure they're supported. Chrome custom tabs expose a service, so the best check for support is to try and bind to the service. Success means that custom tabs are supported and can be used. You can check out this gist, which shows a helper how to to check it, or check the project source code later on at https://gist.github.com/MaTriXy/5775cb0ff98216b2a99d. After checking and learning that support exists, we will start with the UI and interaction part. Custom UI and tab interaction Here, we will use the well-known ACTION_VIEW intent action, and by appending extras to the intent sent to Chrome, we will trigger changes in the UI. Remember that the ACTION_VIEW intent is compatible with all browsers, including Chrome. There are some phones without Chrome out there, or there are instances where the device's default browser isn't Chrome. In these cases, the user will navigate to the specific browser application. Intent is a convenient way to pass that extra data we want Chrome to get. Don't use any of these flags when calling to the Chrome custom tabs: FLAG_ACTIVITY_NEW_TASK FLAG_ACTIVITY_NEW_DOCUMENT Before using the API, we need to add it to our gradle file: compile 'com.android.support:customtabs:23.1.0' This will allow us to use the custom tab support library in our application: CustomTabsIntent.EXTRA_SESSION The preceding code is an extra from the custom tabs support library; it's used to match the session. It must be included in the intent when opening a custom tab. It can be null if there is no need to match any service-side sessions with the intent. We have a sample project to show the options for the UI called ChubbyTabby at https://github.com/MaTriXy/ChubbyTabby. We will go over the important parts here as well. Our main interaction comes from a special builder from the support library called CustomTabsIntent.Builder; this class will help us build the intent we need for the custom tab: CustomTabsIntent.Builder intentBuilder = new CustomTabsIntent.Builder(); //init our Builder //Setting Toolbar Color int color = getResources().getColor(R.color.primary); //we use primary color for our toolbar as well - you can define any color you want and use it. intentBuilder.setToolbarColor(color); //Enabling Title showing intentBuilder.setShowTitle(true); //this will show the title in the custom tab along the url showing at the bottom part of the tab toolbar. //This part is adding custom actions to the over flow menu String menuItemTitle = getString(R.string.menu_title_share); PendingIntent menuItemPendingIntent = createPendingShareIntent(); intentBuilder.addMenuItem(menuItemTitle, menuItemPendingIntent); String menuItemEmailTitle = getString(R.string.menu_title_email); PendingIntent menuItemPendingIntentTwo = createPendingEmailIntent(); intentBuilder.addMenuItem(menuItemEmailTitle, menuItemPendingIntentTwo); //Setting custom Close Icon. intentBuilder.setCloseButtonIcon(mCloseButtonBitmap); //Adding custom icon with custom action for the share action. intentBuilder.setActionButton(mActionButtonBitmap, getString(R.string.menu_title_share), createPendingShareIntent()); //Setting start and exit animation for the custom tab. intentBuilder.setStartAnimations(this, R.anim.slide_in_right, R.anim.slide_out_left); intentBuilder.setExitAnimations(this, android.R.anim.slide_in_left, android.R.anim.slide_out_right); CustomTabActivityHelper.openCustomTab(this, intentBuilder.build(), Uri.parse(URL), new WebviewFallback(), useCustom);  A few things to notice here are as follows: Every menu item uses a pending intent; if you don't know what a pending intent is, head to http://developer.android.com/reference/android/app/PendingIntent.html When we set custom icons, such as close buttons or an action button, for that matter, we use bitmaps and we must decode the bitmap prior to passing it to the builder Setting animations is easy and you can use animations' XML files that you created previously; just make sure that you test the result before releasing the app The following screenshot is an example of a Chrome custom UI and tab: The custom action button As developers, we have full control over the action buttons presented in our custom tab. For most use cases, we can think of a share action or maybe a more common option that your users will perform. The action button is basically a bundle with an icon of the action button and a pending intent that will be called by Chrome when your user hits the action button. The icon should be 24 dp in height and 24-48 dp in width according to specifications: //Adding custom icon with custom action for the share action intentBuilder.setActionButton(mActionButtonBitmap, getString(R.string.menu_title_share), createPendingShareIntent()); Configuring a custom menu By default, Chrome custom tabs usually have a three-icon row with Forward, Page Info, and Refresh on top at all times and Find in page and Open in Browser (Open in Chrome can appear as well) at the footer of the menu. We, developers, have the ability to add and customize up to three menu items that will appear between the icon row and foot items as shown in the following screenshot: The menu we see is actually represented by an array of bundles, each with menu text and a pending intent that Chrome will call on your behalf when the user taps the item: //This part is adding custom buttons to the over flow menu String menuItemTitle = getString(R.string.menu_title_share); PendingIntent menuItemPendingIntent = createPendingShareIntent(); intentBuilder.addMenuItem(menuItemTitle, menuItemPendingIntent); String menuItemEmailTitle = getString(R.string.menu_title_email); PendingIntent menuItemPendingIntentTwo = createPendingEmailIntent(); intentBuilder.addMenuItem(menuItemEmailTitle, menuItemPendingIntentTwo); Configuring custom enter and exit animations Nothing is complete without a few animations to tag along. This is no different, as we have two transitions to make: one for the custom tab to enter and another for its exit; we have the option to set a specific animation for each start and exit animation: //Setting start and exit animation for the custom tab. intentBuilder.setStartAnimations(this,R.anim.slide_in_right, R.anim.slide_out_left); intentBuilder.setExitAnimations(this, android.R.anim.slide_in_left, android.R.anim.slide_out_right); Chrome warm-up Normally, after we finish setting up the intent with the intent builder, we should call CustomTabsIntent.launchUrl (Activity context, Uri url), which is a nonstatic method that will trigger a new custom tab activity to load the URL and show it in the custom tab. This can take up quite some time and impact the impression of smoothness the app provides. We all know that users demand a near-instantaneous experience, so Chrome has a service that we can connect to and ask it to warm up the browser and its native components. Calling this will ask Chrome to perform the following: The DNS preresolution of the URL's main domain The DNS preresolution of the most likely subresources Preconnection to the destination, including HTTPS/TLS negotiation The process to warm up Chrome is as follows: Connect to the service. Attach a navigation callback to get notified upon finishing the page load. On the service, call warmup to start Chrome behind the scenes. Create newSession; this session is used for all requests to the API. Tell Chrome which pages the user is likely to load with mayLaunchUrl. Launch the intent with the session ID generated in step 4. Connecting to the Chrome service Connecting to the Chrome service involves dealing with Android Interface Definition Language (AIDL). If you don't know about AIDL, read http://developer.android.com/guide/components/aidl.html. The interface is created with AIDL, and it automatically creates a proxy service class for you: CustomTabsClient.bindCustomTabsService() So, we check for the Chrome package name; in our sample project, we have a special method to check whether Chrome is present in all variations. After we set the package, we bind to the service and get a CustomTabsClient object that we can use until we're disconnected from the service: pkgName - This is one of several options checking to see if we have a version of Chrome installed can be one of the following static final String STABLE_PACKAGE = "com.android.chrome"; static final String BETA_PACKAGE = "com.chrome.beta"; static final String DEV_PACKAGE = "com.chrome.dev"; static final String LOCAL_PACKAGE = "com.google.android.apps.chrome"; private CustomTabsClient mClient; // Binds to the service. CustomTabsClient.bindCustomTabsService(myContext, pkgName, new CustomTabsServiceConnection() { @Override public void onCustomTabsServiceConnected(ComponentName name, CustomTabsClient client) { // CustomTabsClient should now be valid to use mClient = client; } @Override public void onServiceDisconnected(ComponentName name) { // CustomTabsClient is no longer valid which also invalidates sessions. mClient = null; } });  After we bind to the service, we can call the proper methods we need. Warming up the browser process The method for this is as follows: boolean CustomTabsClient.warmup(long flags) //With our valid client earlier we call the warmup method. mClient.warmup(0); Flags are currently not being used, so we pass 0 for now. The warm-up procedure loads native libraries and the browser process required to support custom tab browsing later on. This is asynchronous, and the return value indicates whether the request has been accepted or not. It returns true to indicate success. Creating a new tab session The method for this is as follows: boolean CustomTabsClient.newSession(ICustomTabsCallback callback) The new tab session is used as the grouping object tying the mayLaunchUrl call, the VIEW intent that we build, and the tab generated altogether. We can get a callback associated with the created session that would be passed for any consecutive mayLaunchUrl calls. This method returns CustomTabsSession when a session is created successfully; otherwise, it returns Null. Setting the prefetching URL The method for this is as follows: boolean CustomTabsSession.mayLaunchUrl (Uri url, Bundle extras, List<Bundle> otherLikelyBundles) This method will notify the browser that a navigation to this URL will happen soon. Make sure that you call warmup() prior to calling this method—this is a must. The most likely URL has to be specified first, and you can send an optional list of other likely URLs (otherLikelyBundles). Lists have to be sorted in a descending order and the optional list may be ignored. A new call to this method will lower the priority of previous calls and can result in URLs not being prefetched. Boolean values inform us whether the operation has been completed successfully. Custom tabs connection callback The method for this is as follows: void CustomTabsCallback.onNavigationEvent (int navigationEvent, Bundle extras) We have a callback triggered upon each navigation event in the custom tab. The int navigationEvent element is one of the six that defines the state the page is in. Refer to the following code for more information: //Sent when the tab has started loading a page. public static final int NAVIGATION_STARTED = 1; //Sent when the tab has finished loading a page. public static final int NAVIGATION_FINISHED = 2; //Sent when the tab couldn't finish loading due to a failure. public static final int NAVIGATION_FAILED = 3; //Sent when loading was aborted by a user action. public static final int NAVIGATION_ABORTED = 4; //Sent when the tab becomes visible. public static final int TAB_SHOWN = 5; //Sent when the tab becomes hidden. public static final int TAB_HIDDEN = 6; private static class NavigationCallback extends CustomTabsCallback { @Override public void onNavigationEvent(int navigationEvent, Bundle extras) { Log.i(TAG, "onNavigationEvent: Code = " + navigationEvent); } } Summary In this article, we learned about a newly added feature, Chrome custom tabs, which allows us to embed web content into our application and modify the UI. Chrome custom tabs allow us to provide a fuller, faster in-app web experience for our users. We use the Chrome engine under the hood, which allows faster loading than regular WebViews or loading the entire Chrome (or another browser) application. We saw that we can preload pages in the background, making it appear as if our data is blazing fast. We can customize the look and feel of our Chrome tab so that it matches our app. Among the changes we saw were the toolbar color, transition animations, and even the addition of custom actions to the toolbar. Custom tabs also benefit from Chrome features such as saved passwords, autofill, tap to search, and sync; these are all available within a custom tab. For developers, integration is quite easy and requires only a few extra lines of code in the basic level. The support library helps with more complex integration, if required. This is a Chrome feature, which means you get it on any Android device where the latest versions of Chrome are installed. Remember that the Chrome custom tab support library changes with new features and fixes, which is the same as other support libraries, so please update your version and make sure that you use the latest API to avoid any issues. To learn more about Chrome custom tabs and Android 6, refer to the following books: Android 6 Essentials (https://www.packtpub.com/application-development/android-6-essentials) Augmented Reality for Android Application Development (https://www.packtpub.com/application-development/augmented-reality-android-application-development) Resources for Article: Further resources on this subject: Android and iOS Apps Testing at a Glance [Article] Working with Xamarin.Android [Article] Mobile Phone Forensics – A First Step into Android Forensics [Article]
Read more
  • 0
  • 0
  • 5817

article-image-hands-docker-swarm
Packt
17 Feb 2016
16 min read
Save for later

Hands On with Docker Swarm

Packt
17 Feb 2016
16 min read
In this article, we will be taking a look at Docker Swarm. With Docker Swarm, you can create and manage Docker clusters. Swarm can be used to disperse containers across multiple hosts. It also has the ability to know how to scale containers as well. In this article, we will be covering the following topics: Installing Docker Swarm The Docker Swarm components Docker Swarm usage The Docker Swarm commands The Docker Swarm topics (For more resources related to this topic, see here.) Docker Swarm install Let's get things started by the typical way of installing Docker Swarm. Docker Swarm is only available for Linux and Mac OS X. The installation process for both is the same. Let's take a look at how we install Docker Swarm. Installation Ensure that you already have Docker installed, either through the curl command on Linux or through Docker Toolbox on Mac OS X. Once you have the Docker daemon installed, installing Docker Swarm will be simple: $ docker pull swarm One command and you are up and running. That's it! Docker Swarm components What components are involved with Docker Swarm? Let's take a look at the three components of Docker Swarm: Swarm Swarm manager Swarm host Swarm Docker Swarm is the container that runs on each Swarm host. Swarm uses a unique token for each cluster to be able to join the cluster. The Swarm container itself is the one that communicates on behalf of that Docker host to the other Docker hosts that are running Docker Swarm as well as the Docker Swarm manager. Swarm manager The Swarm manager is the host that is the central management point for all the Swarm hosts. The Swarm manager is where you issue all your commands to control nodes. You can switch between the nodes, join nodes, remove nodes, and manipulate the hosts. Swarm host Swarm hosts, which we saw earlier as the Docker hosts, are those that run the Docker containers. The Swarm host is managed from the Swarm manager. The preceding figure is an illustration of all the Docker Swarm components. We see that the Docker Swarm manager talks to each Swarm host that is running the Swarm container. Docker Swarm usage Let's now take look at Swarm usage and how we can do the following tasks: Creating a cluster Joining nodes Removing nodes Managing nodes Creating a cluster Let's start by creating the cluster, which starts with a Swarm manager. We first need a token that can be used to join all the nodes to the cluster: $ docker run --rm swarm create 85b335f95e9a37b679e2ea9e6ad8d6361 We can now use that token to create our Swarm manager: $ docker-machine create -d virtualbox --swarm --swarm-master --swarm-discovery token://85b335f95e9a37b679e2ea9e6ad8d6361 swarm-master Creating VirtualBox VM... Creating SSH key... Starting VirtualBox VM... Starting VM... To see how to connect Docker to this machine, run docker-machine env swarm-master. The swarm-master node is now in VirtualBox. We can see this machine by doing as follows: $ docker-machine ls NAME ACTIVE DRIVER STATE URL SWARM swarm-master virtualbox Running tcp://192.168.99.101:2376 swarm-master (master) Now, let's point Docker Machine at the new Swarm master. The earlier output we saw when we created the Swarm master tells us how to point to the node: $ docker-machine env swarm-master export DOCKER_TLS_VERIFY="1" export DOCKER_HOST="tcp://192.168.99.102:2376" export DOCKER_CERT_PATH="/Users/spg14/.docker/machine/machines/swarm-master" export DOCKER_MACHINE_NAME="swarm-master" # Run this command to configure your shell: # eval "$(docker-machine env swarm-master)" Upon running the previous command, we are told to run the following command to point to the Swarm master: $ eval "$(docker-machine env swarm-master)" Now, if we look at what machines are on our host, we can see that we have the swarm-master host as well. It is set to ACTIVE, which means that we can now run commands against this host: $ docker-machine ls NAME ACTIVE DRIVER STATE URL SWARM swarm-master * virtualbox Running tcp://192.168.99.101:2376 swarm-master (master Joining nodes Again using the token, which we got from the earlier commands, used to create the Swarm manager, we need that same token to join nodes to that cluster: $ docker-machine create -d virtualbox --swarm --swarm-discovery token://85b335f95e9a37b679e2ea9e6ad8d6361 swarm-node1 Now, if we look at the machines on our system, we can see that they are both part of the same Swarm: $ docker-machine ls NAME ACTIVE DRIVER STATE URL SWARM swarm-master * virtualbox Running tcp://192.168.99.102:2376 swarm-master(master) swarm-node1 virtualbox Running tcp://192.168.99.103:2376 swarm-master Listing nodes First, ensure you are pointing at the Swarm master: $ docker-machine ls NAME ACTIVE DRIVER STATE URL SWARM swarm-master * virtualbox Running tcp://192.168.99.102:2376 swarm-master(master) swarm-node1 virtualbox Running tcp://192.168.99.103:2376 swarm-master Now, we can see what machines are joined to this cluster based off the token used to join them all together: $ docker run --rm swarm list token://85b335f95e9a37b679e2ea9e6ad8d6361 192.168.99.102:2376 192.168.99.103:2376 Managing a cluster Let's see how we can do some management of all of the cluster nodes we are creating. So, there are two ways you can go about managing these Swarm hosts and the containers on each host that you are creating. But first, you need to know some information about them, so we will turn to our Docker Machine command again: $ docker-machine ls NAME ACTIVE DRIVER STATE URL SWARM swarm-master * virtualbox Running tcp://192.168.99.102:2376 swarm-master(master) swarm-node1 virtualbox Running tcp://192.168.99.103:2376 swarm-master You can switch to each Swarm host like we have seen earlier by doing something similar to the following—changing the values—and by following the instructions from the output of the command: $ docker-machine env <Node_Name> But this is a lot of tedious work. There is another way we can manage these hosts and see what is going on inside them. Let's take a look at how we can do it. From the previous docker-machine ls command, we see that we are currently pointing at the swarm-master node. So, any Docker commands we issue would go against this host. But, if we run the following, we can get information on the swarm-node1 node: $ docker -H tcp://192.168.99.103:2376 info Containers: 1 Images: 8 Storage Driver: aufs Root Dir: /mnt/sda1/var/lib/docker/aufs Backing Filesystem: tmpfs Dirs: 10 Dirperm1 Supported: true Execution Driver: native-0.2 Logging Driver: json-file Kernel Version: 4.0.9-boot2docker Operating System: Boot2Docker 1.8.2 (TCL 6.4); master : aba6192 - Thu Sep 10 20:58:17 UTC 2015 CPUs: 1 Total Memory: 996.2 MiB Name: swarm-node1 ID: SDEC:4RXZ:O3VL:PEPC:FYWM:IGIK:CFM5:UXPS:U4S5:PNQD:5ULK:TSCE Debug mode (server): true File Descriptors: 18 Goroutines: 29 System Time: 2015-09-16T09:32:27.67035212Z EventsListeners: 1 Init SHA1: Init Path: /usr/local/bin/docker Docker Root Dir: /mnt/sda1/var/lib/docker Labels: provider=virtualbox So, we can see the information on this host such as the number of containers, the numbers of images on the host, as well as information about the CPU, memory, and so on. We can see from the earlier information that one container is running. Let's take a look at what is running on the swarm-node1 host: $ docker -H tcp://192.168.99.103:2376 ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 12a400424c87 swarm:latest "/swarm join --advert" 17 hours ago Up 17 hours 2375/tcp swarm-agent Now, you can use any of the Docker commands using this method against any Swarm host that is listed in the output of your docker-machine ls output. The Docker Swarm commands Now, let's take a look at some Docker Swarm-specific commands that we can use. Let's revert to the ever-so-helpful—the help switch on the Docker Swarm command: $ docker run --rm swarm --help Usage: swarm [OPTIONS] COMMAND [arg...] A Docker-native clustering system Version: 0.4.0 (d647d82) Options: --debug debug mode [$DEBUG] --log-level, -l "info" Log level (options: debug, info, warn, error, fatal, panic) --help, -h show help --version, -v print the version Commands: create, c Create a cluster list, l List nodes in a cluster manage, m Manage a docker cluster join, j join a docker cluster help, h Shows a list of commands or help for one command Using TLS Let's take a look at the options you can use for Docker Swarm as well as the commands that are associated with it. Options Looking over the options from the preceding output, we can see the --debug and --log level switches. The other two are straightforward, as one will just print out the help information and the other one will print out the version number that we can see in the previous output. The options are used after each of the following subcommands of Docker Swarm. For example: $ docker run --rm swarm list --debug $ docker run --rm swarm manage --debug $ docker run --rm swarm create --debug list We looked at the Swarm list command before: $ docker run --rm swarm list token://85b335f95e9a37b679e2ea9e6ad8d6361 192.168.99.102:2376 192.168.99.103:2376 But there is also a switch that we can tack onto the list command and that is the --timeout switch: $ docker run --rm swarm list --timeout 20s token://85b335f95e9a37b679e2ea9e6ad8d6361 This will allow more time to find the nodes that are a part of Swarm. It could take time for the hosts to check, depending upon things such as network latency or if they are running in different parts of the globe. create We have seen how we can create a Swarm cluster as well. What this command actually does is it gives us the token that we need to create the cluster and join all the nodes to it. There are no other switches that can be used with this command as we have seen with other commands: $ docker run --rm swarm create 85b335f95e9a37b679e2ea9e6ad8d6361 manage We can manage a cluster with the manage subcommand in Docker Swarm. An example of this command would look like the following, replacing the information to align with your IP address and Swarm token: $ docker run --rm swarm manage -H tcp://192.168.99.104:2376 token://85b335f95e9a37b679e2ea9e6ad8d6361 The Docker Swarm topics There are three advanced topics we will take a look at in this section: Discovery services Advanced scheduling The Docker Swarm API Discovery services You can also use services such as etcd, ZooKeeper, consul, and many others to automatically add nodes to your Swarm cluster as well as do other things such as list the nodes or manage them. Let's take a look at consul and how you can use it. This will be the same for each discovery service that you might use. It just involves switching out the word consul with the discovery service you are using. On each node, you will need to do something different in how you join the machines. Earlier, we did something like this: $ docker-machine create -d virtualbox --swarm --swarm-discovery token://85b335f95e9a37b679e2ea9e6ad8d6361 swarm-node1 Now, we would do something similar to the following (based upon the discovery service you are using): $ docker-machine create -d virtualbox --swarm join --advertise=<swarm-node1_ip:2376> consul://<consul_ip> swarm-node1 You can now start manage on your laptop or the system that you will be using as the Swarm manager. Before, we would run something like this: $ docker run --rm swarm manage -H tcp://192.168.99.104:2376 token://85b335f95e9a37b679e2ea9e6ad8d6361 Now, we run this with regards to discovery services: $ docker run --rm swarm manage -H tcp://192.168.99.104:2376 consul://<consul_ip> We can also list the nodes in this cluster as well as the discovery service: $ docker run --rm swarm list -H tcp://192.168.99.104:2376 consul://<consul_ip> You can easily switch out consul for another discovery service such as etcd or ZooKeeper; the format will still be the same: $ docker-machine create -d virtualbox --swarm join --advertise=<swarm-node1_ip:2376> etcd://<etcd_ip> swarm-node1 $ docker-machine create -d virtualbox --swarm join --advertise=<swarm-node1_ip:2376> zk://<zookeeper_ip> swarm-node1 Advanced scheduling What is advanced scheduling with regards to Docker Swarm? Docker Swarm allows you to rank nodes within your cluster. It provides three different strategies to do this. These can be used by specifying them with the --strategy switch with the swarm manage command: spread binpack random spread and binpack use the same strategy to rank your nodes. They are ranked based off of the node's available RAM and CPU as well as the number of containers that it has running on it. spread will rank the host with less containers higher than a container with more containers (assuming that the memory and CPU values are the same). spread does what the name implies; it will spread the nodes across multiple hosts. By default, spread is used with regards to scheduling. binpack will try to pack as many containers on as few hosts as possible to keep the number of Swarm hosts to a minimal. random will do just that—it will randomly pick a Swarm host to place a node on. The Swarm scheduler comes with a few filters that can be used as well. These can be assigned with the --filter switch with the swarm manage command. These filters can be used to assign nodes to hosts. There are five filters that are associated with it: constraint: There are three types of constraints that can be assigned to nodes: storage=: This is used if you want to specify a node that is put on a host and has SSD drives in it region=: This is used if you want to set a region; mostly used for cloud computing such as AWS or Microsoft Azure environment=: This can set a node to be put into production, development, or other created environments affinity: This filter is used to create attractions between containers. This means that you can specify a filter name and then have all those containers run on the same node. port: The port filter finds a host that has the open port needed for the node to run; it then assigns the node to that host. So, if you have a MySQL instance and need port 3306 open, it will find a host that has port 3306 open and assign the node to that host for operation. dependency: The dependency filter schedules nodes to run on the same host based off of three dependencies: --volumes-from=dependency --link=dependency:<alias> --net=container:dependency health: The health filter is pretty straightforward. It will prevent the scheduling of nodes to run on unhealthy hosts. The Swarm API Before we dive into the Swarm API, let's first make sure you understand what an API is. An API is defined as an application programming interface. An API consists of routines, protocols, and tools to build applications. Think of an API as the bricks used to build a wall. This allows you to put the wall together using those bricks. What APIs allow you to do is code in the environment you are comfortable in and reach into other environments to do the work you need. So, if you are used to coding in Python, you can still use Python to do all your work while using the Swarm API to do the work in Swarm that you would like done. For example, if you wanted to create a container, you would use the following in your code: POST /containers/create HTTP/1.1 Content-Type: application/json { "Hostname": "", "Domainname": "", "User": "", "AttachStdin": false, "AttachStdout": true, "AttachStderr": true, "Tty": false, "OpenStdin": false, "StdinOnce": false, "Env": null, "Cmd": [ "date" ], "Entrypoint": "", "Image": "ubuntu", "Labels": { "com.example.vendor": "Acme", "com.example.license": "GPL", "com.example.version": "1.0" }, "Mounts": [ { "Source": "/data", "Destination": "/data", "Mode": "ro,Z", "RW": false } ], "WorkingDir": "", "NetworkDisabled": false, "MacAddress": "12:34:56:78:9a:bc", "ExposedPorts": { "22/tcp": {} }, "HostConfig": { "Binds": ["/tmp:/tmp"], "Links": ["redis3:redis"], "LxcConf": {"lxc.utsname":"docker"}, "Memory": 0, "MemorySwap": 0, "CpuShares": 512, "CpuPeriod": 100000, "CpusetCpus": "0,1", "CpusetMems": "0,1", "BlkioWeight": 300, "MemorySwappiness": 60, "OomKillDisable": false, "PortBindings": { "22/tcp": [{ "HostPort": "11022" }] }, "PublishAllPorts": false, "Privileged": false, "ReadonlyRootfs": false, "Dns": ["8.8.8.8"], "DnsSearch": [""], "ExtraHosts": null, "VolumesFrom": ["parent", "other:ro"], "CapAdd": ["NET_ADMIN"], "CapDrop": ["MKNOD"], "RestartPolicy": { "Name": "", "MaximumRetryCount": 0 }, "NetworkMode": "bridge", "Devices": [], "Ulimits": [{}], "LogConfig": { "Type": "json-file", "Config": {} }, "SecurityOpt": [""], "CgroupParent": "" } } You would use the preceding example to create a container; but there are also other things you can do such as inspect containers, get the logs from a container, attach to a container, and much more. Simply put, if you can do it through the command line, there is more than likely something in the API that can be used to tie into to do it through the programming language you are using. The Docker documentation states that the Swarm API is mostly compatible with the Docker Remote API. Now we could list them out in this section. But seeing that the list could change as things could be added into the Docker Swarm API or removed, I believe, it's best to refer to the link to the Swarm API documentation here instead of listing them out, so the information is not outdated: https://docs.docker.com/swarm/api/swarm-api/ The Swarm cluster example We will now go through an example of how to set up a Docker Swarm cluster: # Create a new Docker host with Docker Machine $ docker-machine create --driver virtualbox swarm # Point to the new Docker host $ eval "$(docker-machine env swarm)" # Generate a Docker Swarm Discovery Token $ docker run swarm create # Launch the Swarm Manager $ docker-machine create --driver virtualbox --swarm --swarm-master --swarm-discovery token://<DISCOVERY_TOKEN> swarm-master # Launch a Swarm node $ docker-machine create --driver virtualbox --swarm --swarm-discovery token://<DISCOVERY_TOKEN> swarm_node-01 # Launch another Swarm node $ docker-machine create --driver virtualbox --swarm --swarm-discovery token://<DISCOVERY_TOKEN> swarm_node-02 # Point to our Swarm Manager $ eval "$(docker-machine env swarm-master)" # Execute 'docker info' command to view information about your envionment $ docker info # Execute 'docker ps -a'; will show you all the containers running as well as how they are joined to the same Swarm cluster $ docker ps -a # Run simple test $ docker run hello-world # You can then execute the 'docker ps -a' command again to see what node it ran on $ docker ps -a # You will want to look at the column labeled 'NAMES'. If you continue to re-run the 'docker run hello-world' command/container you will see it will run on a different Swarm node Summary In this article, we took a dive into Docker Swarm. We took a look at how to install Docker Swarm and the Docker Swarm components; these are what make up Docker Swarm. We took a look at how to use Docker Swarm; joining, listing, and managing Swarm nodes. We reviewed the Swarm commands and how to use them. We also covered some advanced Docker Swarm topics such as advanced scheduling for your jobs, discovery services to discover new containers to add to Docker Swarm, and the Docker Swarm API that you can use to tie your own code to perform the Swarm commands. Resources for Article: Further resources on this subject: Introduction to Docker[article] Docker in Production[article] Speeding Vagrant Development With Docker[article]
Read more
  • 0
  • 0
  • 4292

article-image-asking-permission-getting-your-head-around-marshmallows-runtime-permissions
Packt
17 Feb 2016
8 min read
Save for later

Asking Permission: Getting your head around Marshmallow's Runtime Permissions

Packt
17 Feb 2016
8 min read
In Android, each application runs with distinct system IDs known as Linux user ID and Group ID. The system parts are also separated into distinct IDs, forming isolated zones for applications—from each other and from the system. As part of this isolated life cycle scheme, accessing services or other applications' data requires that you declare this desire in advance by requesting a permission. This is done by adding the uses-permission element to your AndroidManifest.xml file. Your manifest may have zero or more uses-permission elements, and all of them must be the direct children of the root <manifest> element. Trying to access data or features without proper permission should give out a security exception (using a SecurityException class), informing you about the missing permission in most cases. The sendBroadcast(Intent) method is exceptional as it checks permissions after the method call has returned, so you will not receive an exception if there are permission failures. A permission failure should be printed to the system log. Note that in Android versions prior to Marshmallow, missing permissions were due to missing declarations in the manifest. Hence, it is important that you keep permissions in mind when you come up with the feature list for your app. (For more resources related to this topic, see here.) Understanding Android Marshmallow permissions Android Marshmallow introduces a new application permissions model, allowing a simpler process for users when installing and/or upgrading applications. Applications running on Marshmallow should work according to a new permissions model, where the user can grant or revoke permissions after the installation—permissions are not given until there is user acceptance. Supporting the new permissions model is backward-compatible, which means your apps can still An overview With the Android Marshmallow version, a new application permissions model has been introduced. Let's review it a bit more thoroughly: Declaring permissions: All permissions an app needs are declared in the manifest, which is done to preserve backward compatibility in a manner similar to earlier Android platform versions. Permission groups: As discussed previously, permissions are divided into permission groups based on their functionalities: PROTECTION_NORMAL permissions: Some of the permissions are granted when users install the app. Upon the installation, the system checks your app's manifest and automatically grants permissions that match the PROTECTION_NORMAL group. INTERNET permission: One important permission is the INTERNET permission, which will be granted upon the installation, and the user can't revoke it. App signature permissions granted: The user is not prompted to grant any permissions at the time of installation. Permissions granted by users at runtime: You as an app developer need to request a permission in your app; a system dialog is shown to the user, and the user response is passed back to your app, notifying whether the permission is granted. Permissions can be revoked: Users can revoke permissions that were granted previously. We must learn how to handle these cases, as we'll learn later on. If an app targets an Android Marshmallow version, it must use the new permissions model. Permission groups When working with permissions, we divide them into groups. This division is done for fast user interaction when reviewing and approving permissions. Granting is done only once per permission group. If you add a new permission or request a new permission from the same permission group and the user has already approved that group, the system will grant you the added permission without bothering the user about the approval. For more information on this, visit https://developer.android.com/reference/android/content/pm/PermissionInfo.html#constants[GS1] . When the user installs an app, the app is granted only those permissions that are listed in the manifest that belongs to the PROTECTION_NORMAL group. Requesting permissions from the PROTECTION_SIGNATURE group will be granted only if the application is signed with the same certificate as the app with the declared permission. Apps cannot request signature permissions at runtime. System components automatically receive all the permissions listed in their manifests. Runtime permissions Android Marshmallow showcased a new permissions model where users were able to directly manage app permissions at application runtime. Google has altered the old permissions model, mostly to enable easier and frictionless installations and auto-updates for users as well as for app developers. This allows users to install the app without the need to preapprove each permission the application needs. The user can install the app without going through the phase of checking each permission and declining the installation due to a single permission. Users can grant or revoke permissions for installed apps, leaving the tweaking and the freedom of choice in the users' hands. Most of the applications will need to address these issues when updating the target API to 23. Taking coding permissions into account Well, after all the explanations, we've reached the coding part, and this is where we will get our coding hands dirty. The following are the methods used for coding permissions: Context.checkSelfPermission(): This checks whether your app has been granted a permission Activity.requestPermission(): This requests a permission at runtime Even if your app is not yet targeting Android Marshmallow, you should test your app and prepare to support it. Testing permissions In the Android Marshmallow permissions model, your app must ask the user for individual permissions at runtime. There is limited compatibility support for legacy apps, and you should test your app and also test a version to make sure it's supported. You can use the following test guide and conduct app testing with the new behavior: Map your app's permissions Test flows with permissions granted and revoked The adb command shell can be quite helpful to check for permissions: Listing application permissions and status by group can be done using the following adb command: adb shell pm list permissions -g You can grant or revoke permissions using the following adb syntax: adb shell pm [grant|revoke] <permission.name> You can grant permissions and install apk using the following adb command: adb install -g <path_to_apk> Coding for runtime permissions When we want to adjust our application to the new model, we need to make sure that we organize our steps and leave no permission stranded: Check what platform the app is running on: When running a piece of code that is sensitive at the API level, we start by checking the version/API level that we are running on. By now, you should be familiar with Build.VERSION.SDK_INT. Check whether the app has the required permission: Here, we get ourselves a brand new API call: Context.checkSelfPermission(String permission_name) With this, we silently check whether permissions are granted or not. This method returns immediately, so any permission-related controls/flows should be dealt with by checking this first. Prompting for permissions: We have a new API call, Activity.requestPermissions (String[] permissions, int requestCode). This call triggers the system to show the dialog requesting a permission. This method functions asynchronously. You can request more than one permission at once. The second argument is a simple request code returned in the callback so that you can recognize the calls. This is just like how we've been dealing with startActivityForResult() and onActivityResult() for years. Another new API is Activity.shouldShowRequestPermissionRationale(String permission). This method returns true when you have requested a permission and the user denied the request. It's considered a good practice after verifying that you explain to the user why you need that exact permission. The user can decide to turn down the permission request and select the Don't ask again option; then, this method will return false. The following sample code checks whether the app has permission to read the user's contacts. It requests the permission if required, and the result callback returns to onRequestPermissionsResult: if (checkSelfPermission(Manifest.permission.READ_CONTACTS) != PackageManager.PERMISSION_GRANTED) { requestPermissions(new String[]{Manifest.permission.READ_CONTACTS}, SAMPLE_MATRIXY_READ_CONTACTS); } //Now this is our callback @Override public void onRequestPermissionsResult(int requestCode, String permissions[], int[] grantResults) { switch (requestCode) { case SAMPLE_MATRIXY_READ_CONTACTS: if (grantResults[0] == PackageManager.PERMISSION_GRANTED) { // permission granted - we can continue the feature flow. } else { // permission denied! - we should disable the functionality that depends on this permission. } } } Just to make sure we all know the constants used, here's the explanation: public static final int PERMISSION_DENIED=-1: Since it's API level 1, permission has not been granted to the given package public static final int PERMISSION_GRANTED=0: Since it's API level 1. permission has been granted to the given package. If the user denies your permission request, your app should take the appropriate action, such as notifying the user why this permission is required or explaining that the feature can't work without it. Your app cannot assume user interaction has taken place because the user can choose to reject granting a permission along with the do not show again option; your permission request is automatically rejected and onRequestPermissionsResult gets the result back. Summary To learn more about Android 6 Essentials, the following books published by Packt Publishing (https://www.packtpub.com/) are recommended: Android User Interface Development: Beginner's Guide (https://www.packtpub.com/application-development/android-user-interface-development-beginners-guide) Android Native Development Kit Cookbook (https://www.packtpub.com/application-development/android-native-development-kit-cookbook) Resources for Article: Further resources on this subject: Practical How-To Recipes for Android [article] Working with Xamarin.Android [article] Detecting Shapes Employing Hough Transform [article]
Read more
  • 0
  • 0
  • 2227

article-image-scraping-web-python-quick-start
Packt
17 Feb 2016
9 min read
Save for later

Scraping the Web with Python - Quick Start

Packt
17 Feb 2016
9 min read
In this article we're going to acquire intelligence data from a variety of sources. We might interview people. We might steal files from a secret underground base. We might search the World Wide Web (WWW). (For more resources related to this topic, see here.) Accessing data from the Internet The WWW and Internet are based on a series of agreements called Request for Comments (RFC). The RFCs define the standards and protocols to interconnect different networks, that is, the rules for internetworking. The WWW is defined by a subset of these RFCs that specifies the protocols, behaviors of hosts and agents (servers and clients), and file formats, among other details. In a way, the Internet is a controlled chaos. Most software developers agree to follow the RFCs. Some don't. If their idea is really good, it can catch on, even though it doesn't precisely follow the standards. We often see this in the way some browsers don't work with some websites. This can cause confusion and questions. We'll often have to perform both espionage and plain old debugging to figure out what's available on a given website. Python provides a variety of modules that implement the software defined in the Internet RFCs. We'll look at some of the common protocols to gather data through the Internet and the Python library modules that implement these protocols. Background briefing – the TCP/IP protocols The essential idea behind the WWW is the Internet. The essential idea behind the Internet is the TCP/IP protocol stack. The IP part of this is the internetworking protocol. This defines how messages can be routed between networks. Layered on top of IP is the TCP protocol to connect two applications to each other. TCP connections are often made via a software abstraction called a socket. In addition to TCP, there's also UDP; it's not used as much for the kind of WWW data we're interested in. In Python, we can use the low-level socket library to work with the TCP protocol, but we won't. A socket is a file-like object that supports open, close, input, and output operations. Our software will be much simpler if we work at a higher level of abstraction. The Python libraries that we'll use will leverage the socket concept under the hood. The Internet RFCs defines a number of protocols that build on TCP/IP sockets. These are more useful definitions of interactions between host computers (servers) and user agents (clients). We'll look at two of these: Hypertext Transfer Protocol (HTTP) and File Transfer Protocol (FTP). Using http.client for HTTP GET The essence of web traffic is HTTP. This is built on TCP/IP. HTTP defines two roles: host and user agent, also called server and client, respectively. We'll stick to server and client. HTTP defines a number of kinds of request types, including GET and POST. A web browser is one kind of client software we can use. This software makes GET and POST requests, and displays the results from the web server. We can do this kind of client-side processing in Python using two library modules. The http.client module allows us to make GET and POST requests as well as PUT and DELETE. We can read the response object. Sometimes, the response is an HTML page. Sometimes, it's a graphic image. There are other things too, but we're mostly interested in text and graphics. Here's a picture of a mysterious device we've been trying to find. We need to download this image to our computer so that we can see it and send it to our informant from http://upload.wikimedia.org/wikipedia/commons/7/72/IPhone_Internals.jpg: Here's a picture of the currency we're supposed to track down and pay with: We need to download this image. Here is the link: http://upload.wikimedia.org/wikipedia/en/c/c1/1drachmi_1973.jpg Here's how we can use http.client to get these two image files: import http.client import contextlib path_list = [ "/wikipedia/commons/7/72/IPhone_Internals.jpg", "/wikipedia/en/c/c1/1drachmi_1973.jpg", ] host = "upload.wikimedia.org" with contextlib.closing(http.client.HTTPConnection( host )) as connection: for path in path_list: connection.request( "GET", path ) response= connection.getresponse() print("Status:", response.status) print("Headers:", response.getheaders()) _, _, filename = path.rpartition("/") print("Writing:", filename) with open(filename, "wb") as image: image.write( response.read() ) We're using http.client to handle the client side of the HTTP protocol. We're also using the contextlib module to politely disentangle our application from network resources when we're done using them. We've assigned a list of paths to the path_list variable. This example introduces list objects without providing any background. It's important that lists are surrounded by [] and the items are separated by ,. Yes, there's an extra , at the end. This is legal in Python. We created an http.client.HTTPConnection object using the host computer name. This connection object is a little like a file; it entangles Python with operating system resources on our local computer plus a remote server. Unlike a file, an HTTPConnection object isn't a proper context manager. As we really like context managers to release our resources, we made use of the contextlib.closing() function to handle the context management details. The connection needs to be closed; the closing() function assures that this will happen by calling the connection's close() method. For all of the paths in our path_list, we make an HTTP GET request. This is what browsers do to get the image files mentioned in an HTML page. We print a few things from each response. The status, if everything worked, will be 200. If the status is not 200, then something went wrong and we'll need to read up on the HTTP status code to see what happened. If you use a coffee shop Wi-Fi connection, perhaps you're not logged in. You might need to open a browser to set up a connection. An HTTP response includes headers that provide some additional details about the request and response. We've printed the headers because they can be helpful in debugging any problems we might have. One of the most useful headers is ('Content-Type', 'image/jpeg'). This confirms that we really did get an image. We used _, _, filename = path.rpartition("/") to locate the right-most / character in the path. Recall that the partition() method locates the left-most instance. We're using the right-most one here. We assigned the directory information and separator to the variable _. Yes, _ is a legal variable name. It's easy to ignore, which makes it a handy shorthand for we don't care. We kept the filename in the filename variable. We create a nested context for the resulting image file. We can then read the body of the response—a collection of bytes—and write these bytes to the image file. In one quick motion, the file is ours. The HTTP GET request is what underlies much of the WWW. Programs such as curl and wget are expansions of this example. They execute batches of GET requests to locate one or more pages of content. They can do quite a bit more, but this is the essence of extracting data from the WWW. Changing our client information An HTTP GET request includes several headers in addition to the URL. In the previous example, we simply relied on the Python http.client library to supply a suitable set of default headers. There are several reasons why we might want to supply different or additional headers. First, we might want to tweak the User-Agent header to change the kind of browser that we're claiming to be. We might also need to provide cookies for some kinds of interactions. For information on the user agent string, see http://en.wikipedia.org/wiki/User_agent_string#User_agent_identification. This information may be used by the web server to determine if a mobile device or desktop device is being used. We can use something like this: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_2) AppleWebKit/537.75.14 (KHTML, like Gecko) Version/7.0.3 Safari/537.75.14 This makes our Python request appear to come from the Safari browser instead of a Python application. We can use something like this to appear to be a different browser on a desktop computer: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:28.0) Gecko/20100101 Firefox/28.0 We can use something like this to appear to be an iPhone instead of a Python application: Mozilla/5.0 (iPhone; CPU iPhone OS 7_1_1 like Mac OS X) AppleWebKit/537.51.2 (KHTML, like Gecko) Version/7.0 Mobile/11D201 Safari/9537.53 We make this change by adding headers to the request we're making. The change looks like this: connection.request( "GET", path, headers= { 'User-Agent': 'Mozilla/5.0 (iPhone; CPU iPhone OS 7_1_1 like Mac OS X) AppleWebKit/537.51.2 (KHTML, like Gecko) Version/7.0 Mobile/11D201 Safari/9537.53', }) This will make the web server treat our Python application like it's on an iPhone. This might lead to a more compact page of data than might be provided to a full desktop computer that makes the same request. The header information is a structure with the { key: value, } syntax. It's important that dictionaries are surrounded by {}, the keys and values are separated by :, and each key-value pair is separated by ,. Yes, there's an extra , at the end. This is legal in Python. There are many more HTTP headers we can provide. The User-Agent header is perhaps most important to gather different kinds of intelligence data from web servers. You can refer more book related to this topic on the following links: Python for Secret Agents - Volume II: (https://www.packtpub.com/application-development/python-secret-agents-volume-ii) Expert Python Programming: (https://www.packtpub.com/application-development/expert-python-programming) Raspberry Pi for Secret Agents: (https://www.packtpub.com/hardware-and-creative/raspberry-pi-secret-agents) Resources for Article: Further resources on this subject: Python Libraries[article] Optimization in Python[article] Introduction to Object-Oriented Programming using Python, JavaScript, and C#[article]
Read more
  • 0
  • 0
  • 15858

article-image-using-collider-based-system
Packt
17 Feb 2016
10 min read
Save for later

Using a collider-based system

Packt
17 Feb 2016
10 min read
In this article by Jorge Palacios, the author of the book Unity 5.x Game AI Programming Cookbook, you will learn how to implement agent awareness using a mixed approach that considers the previous learnt sensory-level algorithms. (For more resources related to this topic, see here.) Seeing using a collider-based system This is probably the easiest way to simulate vision. We take a collider, be it a mesh or a Unity primitive, and use it as the tool to determine whether an object is inside the agent's vision range or not. Getting ready It's important to have a collider component attached to the same game object using the script on this recipe, as well as the other collider-based algorithms in this chapter. In this case, it's recommended that the collider be a pyramid-based one in order to simulate a vision cone. The lesser the polygons, the faster it will be on the game. How to do it… We will create a component that is able to see enemies nearby by performing the following steps: Create the Visor component, declaring its member variables. It is important to add the corresponding tags into Unity's configuration: using UnityEngine; using System.Collections; public class Visor : MonoBehaviour { public string tagWall = "Wall"; public string tagTarget = "Enemy"; public GameObject agent; } Implement the function for initializing the game object in case the component is already assigned to it: void Start() { if (agent == null) agent = gameObject; } Declare the function for checking collisions for every frame and build it in the following steps: public void OnCollisionStay(Collision coll) { // next steps here } Discard the collision if it is not a target: string tag = coll.gameObject.tag; if (!tag.Equals(tagTarget)) return; Get the game object's position and compute its direction from the Visor: GameObject target = coll.gameObject; Vector3 agentPos = agent.transform.position; Vector3 targetPos = target.transform.position; Vector3 direction = targetPos - agentPos; Compute its length and create a new ray to be shot soon: float length = direction.magnitude; direction.Normalize(); Ray ray = new Ray(agentPos, direction); Cast the created ray and retrieve all the hits: RaycastHit[] hits; hits = Physics.RaycastAll(ray, length); Check for any wall between the visor and target. If none, we can proceed to call our functions or develop our behaviors to be triggered: int i; for (i = 0; i < hits.Length; i++) { GameObject hitObj; hitObj = hits[i].collider.gameObject; tag = hitObj.tag; if (tag.Equals(tagWall)) return; } // TODO // target is visible // code your behaviour below How it works… The collider component checks every frame to know whether it is colliding with any game object in the scene. We leverage the optimizations to Unity's scene graph and engine, and focus only on how to handle valid collisions. After checking whether a target object is inside the vision range represented by the collider, we cast a ray in order to check whether it is really visible or there is a wall in between. Hearing using a collider-based system In this recipe, we will emulate the sense of hearing by developing two entities; a sound emitter and a sound receiver. It is based on the principles proposed by Millington for simulating a hearing system, and uses the power of Unity colliders to detect receivers near an emitter. Getting ready As with the other recipes based on colliders, we will need collider components attached to every object to be checked and rigid body components attached to either emitters or receivers. How to do it… We will create the SoundReceiver class for our agents and SoundEmitter for things such as alarms: Create the class for the SoundReceiver object: using UnityEngine; using System.Collections; public class SoundReceiver : MonoBehaviour { public float soundThreshold; } We define the function for our own behavior to handle the reception of sound: public virtual void Receive(float intensity, Vector3 position) { // TODO // code your own behavior here } Now, let's create the class for the SoundEmitter object: using UnityEngine; using System.Collections; using System.Collections.Generic; public class SoundEmitter : MonoBehaviour { public float soundIntensity; public float soundAttenuation; public GameObject emitterObject; private Dictionary<int, SoundReceiver> receiverDic; } Initialize the list of receivers nearby and emitterObject in case the component is attached directly: void Start() { receiverDic = new Dictionary<int, SoundReceiver>(); if (emitterObject == null) emitterObject = gameObject; } Implement the function for adding new receivers to the list when they enter the emitter bounds: public void OnCollisionEnter(Collision coll) { SoundReceiver receiver; receiver = coll.gameObject.GetComponent<SoundReceiver>(); if (receiver == null) return; int objId = coll.gameObject.GetInstanceID(); receiverDic.Add(objId, receiver); } Also, implement the function for removing receivers from the list when they are out of reach: public void OnCollisionExit(Collision coll) { SoundReceiver receiver; receiver = coll.gameObject.GetComponent<SoundReceiver>(); if (receiver == null) return; int objId = coll.gameObject.GetInstanceID(); receiverDic.Remove(objId); } Define the function for emitting sound waves to nearby agents: public void Emit() { GameObject srObj; Vector3 srPos; float intensity; float distance; Vector3 emitterPos = emitterObject.transform.position; // next step here } Compute sound attenuation for every receiver: foreach (SoundReceiver sr in receiverDic.Values) { srObj = sr.gameObject; srPos = srObj.transform.position; distance = Vector3.Distance(srPos, emitterPos); intensity = soundIntensity; intensity -= soundAttenuation * distance; if (intensity < sr.soundThreshold) continue; sr.Receive(intensity, emitterPos); } How it works… The collider triggers help register agents in the list of agents assigned to an emitter. The sound emission function then takes into account the agent's distance from the emitter in order to decrease its intensity using the concept of sound attenuation. There is more… We can develop a more flexible algorithm by defining different types of walls that affect sound intensity. It works by casting rays and adding up their values to the sound attenuation: Create a dictionary to store wall types as strings (using tags) and their corresponding attenuation: public Dictionary<string, float> wallTypes; Reduce sound intensity this way: intensity -= GetWallAttenuation(emitterPos, srPos); Define the function called in the previous step: public float GetWallAttenuation(Vector3 emitterPos, Vector3 receiverPos) { // next steps here } Compute the necessary values for ray casting: float attenuation = 0f; Vector3 direction = receiverPos - emitterPos; float distance = direction.magnitude; direction.Normalize(); Cast the ray and retrieve the hits: Ray ray = new Ray(emitterPos, direction); RaycastHit[] hits = Physics.RaycastAll(ray, distance); For every wall type found via tags, add up its value (stored in the dictionary): int i; for (i = 0; i < hits.Length; i++) { GameObject obj; string tag; obj = hits[i].collider.gameObject; tag = obj.tag; if (wallTypes.ContainsKey(tag)) attenuation += wallTypes[tag]; } return attenuation; Smelling using a collider-based system Smelling can be simulated by computing collision between an agent and odor particles, scattered throughout the game level. Getting ready In this recipe based on colliders, we will need collider components attached to every object to be checked, which can be simulated by computing a collision between an agent and odor particles. How to do it… We will develop the scripts needed to represent odor particles and agents able to smell: Create the particle's script and define its member variables for computing its lifespan: using UnityEngine; using System.Collections; public class OdorParticle : MonoBehaviour { public float timespan; private float timer; } Implement the Start function for proper validations: void Start() { if (timespan < 0f) timespan = 0f; timer = timespan; } Implement the timer and destroy the object after its life cycle: void Update() { timer -= Time.deltaTime; if (timer < 0f) Destroy(gameObject); } Create the class for representing the sniffer agent: using UnityEngine; using System.Collections; using System.Collections.Generic; public class Smeller : MonoBehaviour { private Vector3 target; private Dictionary<int, GameObject> particles; } Initialize the dictionary for storing odor particles: void Start() { particles = new Dictionary<int, GameObject>(); } Add to the dictionary the colliding objects that have the odor-particle component attached: public void OnCollisionEnter(Collision coll) { GameObject obj = coll.gameObject; OdorParticle op; op = obj.GetComponent<OdorParticle>(); if (op == null) return; int objId = obj.GetInstanceID(); particles.Add(objId, obj); UpdateTarget(); } Release the odor particles from the local dictionary when they are out of the agent's range or are destroyed: public void OnCollisionExit(Collision coll) { GameObject obj = coll.gameObject; int objId = obj.GetInstanceID(); bool isRemoved; isRemoved = particles.Remove(objId); if (!isRemoved) return; UpdateTarget(); } Create the function for computing the odor centroid according to the current elements in the dictionary: private void UpdateTarget() { Vector3 centroid = Vector3.zero; foreach (GameObject p in particles.Values) { Vector3 pos = p.transform.position; centroid += pos; } target = centroid; } Implement the function for retrieving the odor centroid, if any: public Vector3? GetTargetPosition() { if (particles.Keys.Count == 0) return null; return target; } How it works… Just like the hearing recipe based on colliders, we use the trigger colliders to register odor particles to an agent's perception (implemented using a dictionary). When a particle is included or removed, the odor centroid is computed. However, we implement a function to retrieve that centroid because when no odor particle is registered, the internal centroid position is not updated. There is more… The particle emission logic is left behind to be implemented according to our game's needs and it basically instantiates odor-particle prefabs. Also, it is recommended to attach the rigid body components to the agents. Odor particles are prone to be massively instantiated, reducing the game's performance. Seeing using a graph-based system We will start a recipe oriented to use graph-based logic in order to simulate sense. Again, we will start by developing the sense of vision. Getting ready It is important to grasp the chapter regarding path finding in order to understand the inner workings of the graph-based recipes. How to do it… We will just implement a new file: Create the class for handling vision: using UnityEngine; using System.Collections; using System.Collections.Generic; public class VisorGraph : MonoBehaviour { public int visionReach; public GameObject visorObj; public Graph visionGraph; } Validate the visor object: void Start() { if (visorObj == null) visorObj = gameObject; } Define and start building the function needed to detect visibility of a given set of nodes: public bool IsVisible(int[] visibilityNodes) { int vision = visionReach; int src = visionGraph.GetNearestVertex(visorObj); HashSet<int> visibleNodes = new HashSet<int>(); Queue<int> queue = new Queue<int>(); queue.Enqueue(src); } Implement a breath-first search algorithm: while (queue.Count != 0) { if (vision == 0) break; int v = queue.Dequeue(); List<int> neighbours = visionGraph.GetNeighbors(v); foreach (int n in neighbours) { if (visibleNodes.Contains(n)) continue; queue.Enqueue(v); visibleNodes.Add(v); } } Compare the set of visible nodes with the set of nodes reached by the vision system: foreach (int vn in visibleNodes) { if (visibleNodes.Contains(vn)) return true; } Return false if there is no match between the two sets of nodes: return false; How it works… The recipe uses the breath-first search algorithm in order to discover nodes within its vision reach, and then compares this set of nodes with the set of nodes where the agents reside. Summary In this article, we explained some algorithms for simulating senses and agent awareness. Resources for Article: Further resources on this subject: Animation and Unity3D Physics[article] Unity 3-0 Enter the Third Dimension[article] Animation features in Unity 5[article]
Read more
  • 0
  • 0
  • 31157
Unlock access to the largest independent learning library in Tech for FREE!
Get unlimited access to 7500+ expert-authored eBooks and video courses covering every tech area you can think of.
Renews at ₹800/month. Cancel anytime
article-image-python-data-analysis-utilities
Packt
17 Feb 2016
13 min read
Save for later

Python Data Analysis Utilities

Packt
17 Feb 2016
13 min read
After the success of the book Python Data Analysis, Packt's acquisition editor Prachi Bisht gauged the interest of the author, Ivan Idris, in publishing Python Data Analysis Cookbook. According to Ivan, Python Data Analysis is one of his best books. Python Data Analysis Cookbook is meant for a bit more experienced Pythonistas and is written in the cookbook format. In the year after the release of Python Data Analysis, Ivan has received a lot of feedback—mostly positive, as far as he is concerned. Although Python Data Analysis covers a wide range of topics, Ivan still managed to leave out a lot of subjects. He realized that he needed a library as a toolbox. Named dautil for data analysis utilities, the API was distributed by him via PyPi so that it is installable via pip/easy_install. As you know, Python 2 will no longer be supported after 2020, so dautil is based on Python 3. For the sake of reproducibility, Ivan also published a Docker repository named pydacbk (for Python Data Analysis Cookbook). The repository represents a virtual image with preinstalled software. For practical reasons, the image doesn't contain all the software, but it still contains a fair percentage. This article has the following sections: Data analysis, data science, big data – what is the big deal? A brief history of data analysis with Python A high-level overview of dautil IPython notebook utilities Downloading data Plotting utilities Demystifying Docker Future directions (For more resources related to this topic, see here.) Data analysis, data science, big data – what is the big deal? You've probably seen Venn diagrams depicting data science as the intersection of mathematics/statistics, computer science, and domain expertise. Data analysis is timeless and was there before data science and computer science. You could perform data analysis with a pen and paper and, in more modern times, with a pocket calculator. Data analysis has many aspects with goals such as making decisions or coming up with new hypotheses and questions. The hype, status, and financial rewards surrounding data science and big data remind me of the time when data warehousing and business intelligence were the buzzwords. The ultimate goal of business intelligence and data warehousing was to build dashboards for management. This involved a lot of politics and organizational aspects, but on the technical side, it was mostly about databases. Data science, on the other hand, is not database-centric, and leans heavily on machine learning. Machine learning techniques have become necessary because of the bigger volumes of data. Data growth is caused by the growth of the world's population and the rise of new technologies such as social media and mobile devices. Data growth is in fact probably the only trend that we can be sure will continue. The difference between constructing dashboards and applying machine learning is analogous to the way search engines evolved. Search engines (if you can call them that) were initially nothing more than well-organized collections of links created manually. Eventually, the automated approach won. Since more data will be created in time (and not destroyed), we can expect an increase in automated data analysis. A brief history of data analysis with Python The history of the various Python software libraries is quite interesting. I am not a historian, so the following notes are written from my own perspective: 1989: Guido van Rossum implements the very first version of Python at the CWI in the Netherlands as a Christmas hobby project. 1995: Jim Hugunin creates Numeric, the predecessor to NumPy. 1999: Pearu Peterson writes f2py as a bridge between Fortran and Python. 2000: Python 2.0 is released. 2001: The SciPy library is released. Also, Numarray, a competing library of Numeric, is created. Fernando Perez releases IPython, which starts out as an afternoon hack. NLTK is released as a research project. 2002: John Hunter creates the matplotlib library. 2005: NumPy is released by Travis Oliphant. Initially, NumPy is Numeric extended with features inspired by Numarray. 2006: NumPy 1.0 is released. The first version of SQLAlchemy is released. 2007: The scikit-learn project is initiated as a Google Summer of Code project by David Cournapeau. Cython is forked from Pyrex. Cython is later intensively used in pandas and scikit-learn to improve performance. 2008: Wes McKinney starts working on pandas. Python 3.0 is released. 2011: The IPython 0.12 release introduces the IPython notebook. Packt releases NumPy 1.5 Beginner's Guide. 2012: Packt releases NumPy Cookbook. 2013: Packt releases NumPy Beginner's Guide - Second Edition. 2014: Fernando Perez announces Project Jupyter, which aims to make a language-agnostic notebook. Packt releases Learning NumPy Array and Python Data Analysis. 2015: Packt releases NumPy Beginner's Guide - Third Edition and NumPy Cookbook - Second Edition. A high-level overview of dautil The dautil API that Ivan made for this book is a humble toolbox, which he found useful. It is released under the MIT license. This license is very permissive, so you could in theory use the library in a production system. He doesn't recommend doing this currently (as of January, 2016), but he believes that the unit tests and documentation are of acceptable quality. The library has 3000+ lines of code and 180+ unit tests with a reasonable coverage. He has fixed as many issues reported by pep8 and flake8 as possible. Some of the functions in dautil are on the short side and are of very low complexity. This is on purpose. If there is a second edition (knock on wood), dautil will probably be completely transformed. The API evolved as Ivan wrote the book under high time pressure, so some of the decisions he made may not be optimal in retrospect. However, he hopes that people find dautil useful and, ideally, contribute to it. The dautil modules are summarized in the following table: Module Description LOC dautil.collect Contains utilities related to collections 331 dautil.conf Contains configuration utilities 48 dautil.data Contains utilities to download and load data 468 dautil.db Contains database-related utilities 98 dautil.log_api Contains logging utilities 204 dautil.nb Contains IPython/Jupyter notebook widgets and utilities 609 dautil.options Configures dynamic options of several libraries related to data analysis 71 dautil.perf Contains performance-related utilities 162 dautil.plotting Contains plotting utilities 382 dautil.report Contains reporting utilities 232 dautil.stats Contains statistical functions and utilities 366 dautil.ts Contains Utilities for time series and dates 217 dautil.web Contains utilities for web mining and HTML processing 47 IPython notebook utilities The IPython notebook has become a standard tool for data analysis. The dautil.nb has several interactive IPython widgets to help with Latex rendering, the setting of matplotlib properties, and plotting. Ivan has defined a Context class, which represents the configuration settings of the widgets. The settings are stored in a pretty-printed JSON file in the current working directory, which is named dautil.json. This could be extended, maybe even with a database backend. The following is an edited excerpt (so that it doesn't take up a lot of space) of an example dautil.json: { ... "calculating_moments": { "figure.figsize": [ 10.4, 7.7 ], "font.size": 11.2 }, "calculating_moments.latex": [ 1, 2, 3, 4, 5, 6, 7 ], "launching_futures": { "figure.figsize": [ 11.5, 8.5 ] }, "launching_futures.labels": [ [ {}, { "legend": "loc=best", "title": "Distribution of Means" } ], [ { "legend": "loc=best", "title": "Distribution of Standard Deviation" }, { "legend": "loc=best", "title": "Distribution of Skewness" } ] ], ... }  The Context object can be constructed with a string—Ivan recommends using the name of the notebook, but any unique identifier will do. The dautil.nb.LatexRenderer also uses the Context class. It is a utility class, which helps you number and render Latex equations in an IPython/Jupyter notebook, for instance, as follows: import dautil as dl lr = dl.nb.LatexRenderer(chapter=12, context=context) lr.render(r'delta! = x - m') lr.render(r'm' = m + frac{delta}{n}') lr.render(r'M_2' = M_2 + delta^2 frac{ n-1}{n}') lr.render(r'M_3' = M_3 + delta^3 frac{ (n - 1) (n - 2)}{n^2}/ - frac{3delta M_2}{n}') lr.render(r'M_4' = M_4 + frac{delta^4 (n - 1) / (n^2 - 3n + 3)}{n^3} + frac{6delta^2 M_2}/ {n^2} - frac{4delta M_3}{n}') lr.render(r'g_1 = frac{sqrt{n} M_3}{M_2^{3/2}}') lr.render(r'g_2 = frac{n M_4}{M_2^2}-3.') The following is the result:   Another widget you may find useful is RcWidget, which sets matplotlib settings, as shown in the following screenshot: Downloading data Sometimes, we require sample data to test an algorithm or prototype a visualization. In the dautil.data module, you will find many utilities for data retrieval. Throughout this book, Ivan has used weather data from the KNMI for the weather station in De Bilt. A couple of the utilities in the module add a caching layer on top of existing pandas functions, such as the ones that download data from the World Bank and Yahoo! Finance (the caching depends on the joblib library and is currently not very configurable). You can also get audio, demographics, Facebook, and marketing data. The data is stored under a special data directory, which depends on the operating system. On the machine used in the book, it is stored under ~/Library/Application Support/dautil. The following example code loads data from the SPAN Facebook dataset and computes the clique number: import networkx as nx import dautil as dl fb_file = dl.data.SPANFB().load() G = nx.read_edgelist(fb_file, create_using=nx.Graph(), nodetype=int) print('Graph Clique Number', nx.graph_clique_number(G.subgraph(list(range(2048)))))  To understand what is going on in detail, you will need to read the book. In a nutshell, we load the data and use the NetworkX API to calculate a network metric. Plotting utilities Ivan visualizes data very often in the book. Plotting helps us get an idea about how the data is structured and helps you form hypotheses or research questions. Often, we want to chart multiple variables, but we want to easily see what is what. The standard solution in matplotlib is to cycle colors. However, Ivan prefers to cycle line widths and line styles as well. The following unit test demonstrates his solution to this issue: def test_cycle_plotter_plot(self): m_ax = Mock() cp = plotting.CyclePlotter(m_ax) cp.plot([0], [0]) m_ax.plot.assert_called_with([0], [0], '-', lw=1) cp.plot([0], [1]) m_ax.plot.assert_called_with([0], [1], '--', lw=2) cp.plot([1], [0]) m_ax.plot.assert_called_with([1], [0], '-.', lw=1) The dautil.plotting module currently also has a helper tool for subplots, histograms, regression plots, and dealing with color maps. The following example code (the code for the labels has been omitted) demonstrates a bar chart utility function and a utility function from dautil.data, which downloads stock price data: import dautil as dl import numpy as np import matplotlib.pyplot as plt ratios = [] STOCKS = ['AAPL', 'INTC', 'MSFT', 'KO', 'DIS', 'MCD', 'NKE', 'IBM'] for symbol in STOCKS: ohlc = dl.data.OHLC() P = ohlc.get(symbol)['Adj Close'].values N = len(P) mu = (np.log(P[-1]) - np.log(P[0]))/N var_a = 0 var_b = 0 for k in range(1, N): var_a = (np.log(P[k]) - np.log(P[k - 1]) - mu) ** 2 var_a = var_a / N for k in range(1, N//2): var_b = (np.log(P[2 * k]) - np.log(P[2 * k - 2]) - 2 * mu) ** 2 var_b = var_b / N ratios.append(var_b/var_a - 1) _, ax = plt.subplots() dl.plotting.bar(ax, STOCKS, ratios) plt.show() Refer to the following screenshot for the end result: The code performs a random walk test and calculates the corresponding ratio for a list of stock prices. The data is retrieved whenever you run the code, so you may get different results. Some of you have a finance aversion, but rest assured that this book has very little finance-related content. The following script demonstrates a linear regression utility and caching downloader for World Bank data (the code for the watermark and plot labels has been omitted): import dautil as dl import matplotlib.pyplot as plt import numpy as np wb = dl.data.Worldbank() countries = wb.get_countries()[['name', 'iso2c']] inf_mort = wb.get_name('inf_mort') gdp_pcap = wb.get_name('gdp_pcap') df = wb.download(country=countries['iso2c'], indicator=[inf_mort, gdp_pcap], start=2010, end=2010).dropna() loglog = df.applymap(np.log10) x = loglog[gdp_pcap] y = loglog[inf_mort] dl.options.mimic_seaborn() fig, [ax, ax2] = plt.subplots(2, 1) ax.set_ylim([0, 200]) ax.scatter(df[gdp_pcap], df[inf_mort]) ax2.scatter(x, y) dl.plotting.plot_polyfit(ax2, x, y) plt.show()  The following image should be displayed by the code: The program downloads World Bank data for 2010 and plots the infant mortality rate against the GDP per capita. Also shown is a linear fit of the log-transformed data. Demystifying Docker Docker uses Linux kernel features to provide an extra virtualization layer. It was created in 2013 by Solomon Hykes. Boot2Docker allows us to install Docker on Windows and Mac OS X as well. Boot2Docker uses a VirtualBox VM that contains a Linux environment with Docker. Ivan's Docker image, which is mentioned in the introduction, is based on the continuumio/miniconda3 Docker image. The Docker installation docs are at https://docs.docker.com/index.html. Once you install Boot2Docker, you need to initialize it. This is only necessary once, and Linux users don't need this step: $ boot2docker init The next step for Mac OS X and Windows users is to start the VM: $ boot2docker start Check the Docker environment by starting a sample container: $ docker run hello-world Docker images are organized in a repository, which resembles GitHub. A producer pushes images and a consumer pulls images. You can pull Ivan's repository with the following command. The size is currently 387 MB. $ docker pull ivanidris/pydacbk Future directions The dautil API consists of items Ivan thinks will be useful outside of the context of this book. Certain functions and classes that he felt were only suitable for a particular chapter are placed in separate per-chapter modules, such as ch12util.py. In retrospect, parts of those modules may need to be included in dautil as well. In no particular order, Ivan has the following ideas for future dautil development: He is playing with the idea of creating a parallel library with "Cythonized" code, but this depends on how dautil is received Adding more data loaders as required There is a whole range of streaming (or online) algorithms that he thinks should be included in dautil as well The GUI of the notebook widgets should be improved and extended The API should have more configuration options and be easier to configure Summary In this article, Ivan roughly sketched what data analysis, data science, and big data are about. This was followed by a brief of history of data analysis with Python. Then, he started explaining dautil—the API he made to help him with this book. He gave a high-level overview and some examples of the IPython notebook utilities, features to download data, and plotting utilities. He used Docker for testing and giving readers a reproducible data analysis environment, so he spent some time on that topic too. Finally, he mentioned the possible future directions that could be taken for the library in order to guide anyone who wants to contribute. Resources for Article:   Further resources on this subject: Recommending Movies at Scale (Python) [article] Python Data Science Up and Running [article] Making Your Data Everything It Can Be [article]
Read more
  • 0
  • 0
  • 7911

Packt
17 Feb 2016
72 min read
Save for later

Speaking Java – Your First Game

Packt
17 Feb 2016
72 min read
In this article, we will start writing our very own Java code at the same time as we begin understanding Java syntax. We will learn how to store, retrieve, and manipulate different types of values stored in the memory. We will also look at making decisions and branching the flow of our code based on the values of this data. In this order, we will: Learn some Java syntax and see how it is turned into a running app by the compiler Store data and use it with variables Express yourself in Java with expressions Continue with the math game by asking a question Learn about decisions in Java Continue with the math game by getting and checking the answer (For more resources related to this topic, see here.) Acquiring the preceding Java skills will enable us to build the next two phases of our math game. This game will be able to ask the player a question on multiplication, check the answer and give feedback based on the answer given, as shown in the following screenshot: Java syntax Throughout this book, we will use plain English to discuss some fairly technical things. You will never be asked to read a technical explanation of a Java or Android concept that has not been previously explained in a non-technical way. Occasionally, I might ask or imply that you accept a simplified explanation in order to offer a fuller explanation at a more appropriate time, like the Java class as a black box; however, never will you need to scurry to Google in order to get your head around a big word or a jargon-filled sentence. Having said that, the Java and Android communities are full of people who speak in technical terms and to join in and learn from these communities, you need to understand the terms they use. So the approach this book takes is to learn a concept or appreciate an idea using an entirely plain speaking language, but at the same time, it introduces the jargon as part of the learning. Then, much of the jargon will begin to reveal its usefulness, usually as a way of clarification or keeping the explanation/discussion from becoming longer than it needs to be. The very term, "Java syntax," could be considered technical or jargon. So what is it? The Java syntax is the way we put together the language elements of Java in order to produce code that works in the Java/Dalvik virtual machine. Syntax should also be as clear as possible to a human reader, not least ourselves when we revisit our programs in the future. The Java syntax is a combination of the words we use and the formation of those words into sentence like structures. These Java elements or words are many in number, but when taken in small chunks are almost certainly easier to learn than any human-spoken language. The reason for this is that the Java language and its syntax were specifically designed to be as straightforward as possible. We also have Android Studio on our side which will often let us know if we make a mistake and will even sometimes think ahead and prompt us. I am confident that if you can read, you can learn Java; because learning Java is much easy. What then separates someone who has finished an elementary Java course from an expert programmer? The same things that separate a student of language from a master poet. Mastery of the language comes through practice and further study. The compiler The compiler is what turns our human-readable Java code into another piece of code that can be run in a virtual machine. This is called compiling. The Dalvik virtual machine will run this compiled code when our players tap on our app icon. Besides compiling Java code, the compiler will also check for mistakes. Although we might still have mistakes in our released app, many are stopped discovered at the when our code is compiled. Making code clear with comments As you become more advanced in writing Java programs, the solutions you use to create your programs will become longer and more complicated. Java was designed to manage complexity by having us divide our code into separate chunks, very often across multiple files. Comments are a part of the Java program that do not have any function in the program itself. The compiler ignores them. They serve to help the programmer to document, explain, and clarify their code to make it more understandable to themselves at a later date or to other programmers who might need to use or modify the code. So, a good piece of code will be liberally sprinkled with lines like this: //this is a comment explaining what is going on The preceding comment begins with the two forward slash characters, //. The comment ends at the end of the line. It is known as a single-line comment. So anything on that line is for humans only, whereas anything on the next line (unless it's another comment) needs to be syntactically correct Java code: //I can write anything I like here but this line will cause an error We can use multiple single-line comments: //Below is an important note //I am an important note //We can have as many single line comments like this as we like Single-line comments are also useful if we want to temporarily disable a line of code. We can put // in front of the code and it will not be included in the program. Recall this code, which tells Android to load our menu UI: //setContentView(R.layout.activity_main); In the preceding situation, the menu will not be loaded and the app will have a blank screen when run, as the entire line of code is ignored by the compiler. There is another type of comment in Java—the multiline comment. This is useful for longer comments and also to add things such as copyright information at the top of a code file. Also like the single-line comment, it can be used to temporarily disable code, in this case usually multiple lines. Everything in between the leading /* signs and the ending */ signs is ignored by the compiler. Here are some examples: /* This program was written by a Java expert You can tell I am good at this because my code has so many helpful comments in it. */ There is no limit to the number of lines in a multiline comment. Which type of comment is best to use will depend upon the situation. In this book, I will always explain every line of code explicitly but you will often find liberally sprinkled comments within the code itself that add further explanation, insight or clarification. So it's always a good idea to read all of the code: /* The winning lottery numbers for next Saturday are 9,7,12,34,29,22 But you still want to learn Java? Right? */ All the best Java programmers liberally sprinkle their code with comments. Storing data and using it with variables We can think of a variable as a labeled storage box. They are also like a programmer's window to the memory of the Android device, or whatever device we are programming. Variables can store data in memory (the storage box), ready to be recalled or altered when necessary by using the appropriate label. Computer memory has a highly complex system of addressing that we, fortunately, do not need to interact with in Java. Java variables allow us to make up convenient names for all the data that we want our program to work with; the JVM will handle all the technicalities that interact with the operating system, which in turn, probably through several layers of buck passing, will interact with the hardware. So we can think of our Android device's memory as a huge warehouse. When we assign names to our variables, they are stored in the warehouse, ready when we need them. When we use our variable's name, the device knows exactly what we are referring to. We can then tell it to do things such as "get box A and add it to box C, delete box B," and so on. In a game, we will likely have a variable named as something along the lines of score. It would be this score variable using which we manage anything related to the user's score, such as adding to it, subtracting or perhaps just showing it to the player. Some of the following situations that might arise: The player gets a question right, so add 10 to their existing score The player views their stats screen, so print score on the screen The player gets the best score ever, so make hiScore the same as their current score These are fairly arbitrary examples of names for variables and as long as you don't use any of the characters keywords that Java restricts, you can actually call your variables whatever you like. However, in practice, it is best to adopt a naming convention so that your variable names will be consistent. In this book, we will use a loose convention of variable names starting with a lowercase letter. When there is more than one word in the variable's name, the second word will begin with an uppercase letter. This is called "camel casing." Here are some examples of camel casing: score hiScore playersPersonalBest Before we look at some real Java code with some variables, we need to first look at the types of variables we can create and use. Types of variables It is not hard to imagine that even a simple game will probably have quite a few variables. What if the game has a high score table that remembers the names of the top 10 players? Then we might need variables for each player. And what about the case when a game needs to know if a playable character is dead or alive, or perhaps has any lives/retries left? We might need code that tests for life and then ends the game with a nice blood spurt animation if the playable character is dead. Another common requirement in a computer program, including games, is the right or wrong calculation: true or false. To cover almost these and many other types of information you might want to keep track of, every possibility Java has types. There are many types of variables and, we can also invent our own types or use other people's types. But for now, we will look at the built-in Java types. To be fair, they cover just about every situation we are likely to run into for a while. Some examples are the best way to explain this type of stuff. We have already discussed the hypothetical but highly likely score variable. The sWell score is likely to be a number, so we have to convey this (that the score is a number) to the Java compiler by giving the score an appropriate type. The hypothetical but equally likely playerName will of course, hold the characters that make up the player's name. Jumping ahead a couple of paragraphs, the type that holds a regular number is called an int, and the type that holds name-like data is called a string. And if we try and store a player name, perhaps "Ada Lovelace" in score, which is meant for numbers, we will certainly run into trouble. The compiler says no! Actually, the error would say this: As we can see, Java was designed to make it impossible for such errors to make it to a running program. Did you also spot in the previous screenshot that I had forgotten the semicolon at the end of the line? With this compiler identifying our errors, what could possibly go wrong? Here are the main types in Java. Later, we will see how to start using them: int: This type is used to store integers. It uses 32 pieces (bits) of memory and can therefore store values with a magnitude a little in excess of 2 billion, including negative values. long: As the name hints at, this data types can be used when even larger numbers are required. A long data type uses 64 bits of memory and 2 to the power of 63 is what we can store in this type. If you want to see what that looks like, try this: 9,223,372,036,854,775,807. Perhaps surprisingly, there are uses for long variables but if a smaller variable will do, we should use it so that our program uses less memory. You might be wondering when you might use numbers of this magnitude. The obvious examples would be math or science applications that do complex calculations but another use might be for timing. When you time how long something takes, the Java Date class uses the number of milliseconds since January 1, 1970. The long data type could be useful to subtract a start time from an end time to determine an elapsed time. float: This is for floating-point numbers, that is, numbers where there is precision beyond the decimal point. As the fractional part of a number takes memory space just as the whole number portion, the range of numbers possible in a float is therefore decreased compared to non-floating-point numbers. So, unless our variable will definitely use the extra precision, float would not be our data type of choice. double: When the precision in float is not enough we have double. short: When even an int data type is overkill, the super-skinny short fits into the tiniest of storage boxes, but we can only store around 64,000 values, from -32,768 to 32,767. byte: This is an even smaller storage box than a short type. There is plenty of room for these in memory but a byte can only store values from -128 to 127. boolean: We will be using plenty of Booleans throughout the book. A Boolean variable can be either true or false—nothing else. Perhaps Boolean answer questions such as: Is the player alive? Has a new high score been reached? Are two examples for a Boolean variable enough? char: This stores a single alphanumeric character. It's not going to change anything on its own but it could be useful if we put lots of them together. I have kept this discussion of data types to a practical level that is useful in the context of this book. If you are interested in how a data type's value is stored and why the limits are what they are, visit the Oracle Java tutorials site at http://docs.oracle.com/javase/tutorial/java/nutsandbolts/datatypes.html. Note that you do not need any more information than we have already discussed to continue with this book. As we just learned, each type of data that we might want to store will require a specific amount of memory. So we must let the Java compiler know the type of the variable before we begin to use it. The preceding variables are known as the primitive types. They use predefined amounts of memory and so, using our storage analogy, fit into predefined sizes of the storage box. As the "primitive" label suggests, they are not as sophisticated as the reference types. Reference types You might have noticed that we didn't cover the string variable type that we previously used to introduce the concept of variables. Strings are a special type of variable known as a reference type. They quite simply refer to a place in memory where the storage of the variable begins, but the reference type itself does not define a specific amount of memory. The reason for this is fairly straightforward: we don't always know how much data will need to be stored until the program is actually run. We can think of strings and other reference types as continually expanding and contracting storage boxes. So won't one of these string reference types bump into another variable eventually? If you think about the devices memory as a huge warehouse full of racks of labeled storage boxes, then you can think of the Dalvik virtual machine as a super-efficient forklift truck driver that puts the different types of storage boxes in the most appropriate place. And if it becomes necessary, the virtual machine will quickly move stuff around in a fraction of a second to avoid collisions. It will even incinerate unwanted storage boxes when appropriate. This happens at the same time as constantly unloading new storage boxes of all types and placing them in the best place, for that type of variable. Dalvik tends to keep reference variables in a part of the warehouse that is different from the part for the primitive variables. So strings can be used to store any keyboard character, like a char data type but of almost any length. Anything from a player's name to an entire book can be stored in a single string. There are a couple more reference types we will explore. Arrays are a way to store lots of variables of the same type, ready for quick and efficient access. Think of an array as an aisle in our warehouse with all the variables of a certain type lined up in a precise order. Arrays are reference types, so Dalvik keeps these in the same part of the warehouse as strings. So we know that each type of data that we might want to store will require an amount of memory. Hence, we must let the Java compiler know the type of the variable before we begin to use it. Declaration That's enough of theory. Let's see how we would actually use our variables and types. Remember that each primitive type requires a specific amount of real device memory. This is one of the reasons that the compiler needs to know what type a variable will be of. So we must first declare a variable and its type before we attempt to do anything with it. To declare a variable of type int with the name score, we would type: int score; That's it! Simply state the type, in this case int, then leave a space, and type the name you want to use for this variable. Also note the semicolon on the end of the line as usual to show the compiler that we are done with this line and what follows, if anything, is not part of the declaration. For almost all the other variable types, declaration would occur in the same way. Here are some examples. The variable names are arbitrary. This is like reserving a labeled storage box in the warehouse: long millisecondsElapsed; float gravity; double accurrateGravity; boolean isAlive; char playerInitial; String playerName; Initialization Here, for each type, we initialize a value to the variable. Think about placing a value inside the storage box, as shown in the following code: score = 0; millisecondsElapsed = 1406879842000;//1st Aug 2014 08:57:22 gravity = 1.256; double accurrateGravity =1.256098; isAlive = true; playerInitial = 'C'; playerName = "Charles Babbage"; Notice that the char variable uses single quotes (') around the initialized value while the string uses double quotes ("). We can also combine the declaration and initialization steps. In the following snippet of code, we declare and initialize the same variables as we did previously, but in one step each: int score = 0; long millisecondsElapsed = 1406879842000;//1st Aug 2014 08:57:22 float gravity = 1.256; double accurrateGravity =1.256098; boolean isAlive = true; char playerInitial = 'C'; String playerName = "Charles Babbage"; Whether we declare and initialize separately or together is probably dependent upon the specific situation. The important thing is that we must do both: int a; //The line below attempts to output a to the console Log.i("info", "int a = " + a); The preceding code would cause the following result: Compiler Error: Variable a might not have been initialized There is a significant exception to this rule. Under certain circumstances variables can have default values. Changing variables with operators Of course, in almost any program, we are going to need to do something with these values. Here is a list of perhaps the most common Java operators that allow us to manipulate variables. You do not need to memorize them as we will look at every line of code when we use them for the first time: The assignment operator (=): This makesthe variable to the left of the operator the same as the value to the right. For example, hiScore = score; or score = 100;. The addition operator (+): This adds the values on either side of the operator. It is usually used in conjunction with the assignment operator, such as score = aliensShot + wavesCleared; or score = score + 100;. Notice that it is perfectly acceptable to use the same variable simultaneously on both sides of an operator. The subtraction operator (-): This subtracts the value on the right side of the operator from the value on the left. It is usually used in conjunction with the assignment operator, such as lives = lives - 1; or balance = income - outgoings;. The division operator (/): This divides the number on the left by the number on the right. Again, it is usually used in conjunction with the assignment operator, as shown in fairShare = numSweets / numChildren; or recycledValueOfBlock = originalValue / .9;. The multiplication operator (*): This multiplies variables and numbers, such as answer = 10 * 10; or biggerAnswer = 10 * 10 * 10;. The increment operator (++): This is a really neat way to add 1 to the value of a variable. The myVariable = myVariable + 1; statement is the same as myVariable++;. The decrement operator (--): You guessed it: a really neat way to subtract 1 from something. The myVariable = myVariable -1; statement is the same as myVariable--;. The formal names for these operators are slightly different from the names used here for explanation. For example, the division operator is actually one of the multiplicative operators. But the preceding names are far more useful for the purpose of learning Java and if you used the term "division operator", while conversing with someone from the Java community, they would know exactly what you mean. There are actually many more operators than these in Java. If you are curious about operators there is a complete list of them on the Java website at http://docs.oracle.com/javase/tutorial/java/nutsandbolts/operators.html. All the operators required to complete the projects in this book will be fully explained in this book. The link is provided for the curious among us. Expressing yourself in Java Let's try using some declarations, assignments and operators. When we bundle these elements together into some meaningful syntax, we call it an expression. So let's write a quick app to try some out. Here we will make a little side project so we can play with everything we have learned so far. Instead, we will simply write some Java code and examine its effects by outputting the values of variables to the Android console, called LogCat. We will see exactly how this works by building the simple project and examining the code and the console output: The following is a quick reminder of how to create a new project. Close any currently open projects by navigating to File | Close Project. Click on Start a new Android Studio project. The Create New Project configuration window will appear. Fill in the Application name field and Company Domain with packtpub.com or you could use your own company website name here instead. Now click on the Next button. On the next screen, make sure the Phone and Tablet checkbox has a tick in it. Now we have to choose the earliest version of Android we want to build our app for. Go ahead and play with a few options in the drop-down selector. You will see that the earlier the version we select, the greater is the percentage of devices our app can support. However, the trade-off here is that the earlier the version we select, the less are cutting-edge Android features available in our apps. A good balance is to select API 8: Android 2.2 (Froyo). Go ahead and do that now as shown in the next screenshot. Click on Next. Now select Blank Activity as shown in the next screenshot and click on Next again. On the next screen, simply change Activity Name to MainActivity and click on Finish. To keep our code clear and simple, you can delete the two unneeded methods (onCreateOptionsMenu and onOptionsItemSelected) and their associated @override and @import statements. However, this is not necessary for the example to work. As with all the examples and projects in this book, you can copy or review the code from the download bundle. Just create the project as described previously and paste the code from MainActivity.java file from the download bundle to the MainActivity.java file that was generated when you created the project in Android Studio. Just ensure that the package name is the same as the one you chose when the project was created. However, I strongly recommend going along with the tutorial so that we can learn how to do everything for ourselves. As this app uses the LogCat console to show its output, you should run this app on the emulator only and not on a real Android device. The app will not harm a real device, but you just won't be able to see anything happening. Create a new blank project called Expressions In Java. Now, in the onCreate method just after the line where we use the setContentView method, add this code to declare and initialize some variables: //first we declare and initialize a few variables int a = 10; String b = "Alan Turing"; boolean c = true; Now add the following code. This code simply outputs the value of our variables in a form where we can closely examine them in a minute: //Let's look at how Android 'sees' these variables //by outputting them, one at a time to the console Log.i("info", "a = " + a); Log.i("info", "b = " + b); Log.i("info", "c = " + c); Now let's change our variables using the addition operator and another new operator. See if you can work out the output values for variables a, b, and c before looking at the output and the code explanation: //Now let's make some changes a++; a = a + 10; b = b + " was smarter than the average bear Booboo"; b = b + a; c = (1 + 1 == 3);//1 + 1 is definitely 2! So false. Let's output the values once more in the same way we did in step 3, but this time, the output should be different: //Now to output them all again Log.i("info", "a = " + a); Log.i("info", "b = " + b); Log.i("info", "c = " + c); Run the program on an emulator in the usual way. You can see the output by clicking on the Android tab from our "useful tabs" area below the Project Explorer. Here is the output, with some of the unnecessary formatting stripped off: info? a = 10 info? b = Alan Turing info? c = true info? a = 21 info? b = Alan Turing was smarter than the average bear Booboo21 info? c = false Now let's discuss what happened. In step 2, we declared and initialized three variables: a: This is an int that holds the value 10 b: This is a string that holds the name of an eminent computer scientist. c: This is a Boolean that holds the value false So when we output the values in step 3, it should be no surprise that we get the following: info? a = 10 info? b = Alan Turing info? c = true In step 4, all the fun stuff happens. We add 1 to the value of our int a using the increment operator like this: a++;. Remember that a++ is the same as a = a + 1. We then add 10 to a. Note we are adding 10 to a after having already added 1. So we get this output for a 10 + 1 + 10 operation: info? a = 21 Now let's examine our string, b. We appear to be using the addition operator on our eminent scientist. What is happening is what you could probably guess. We are adding together two strings "Alan Turing" and "was smarter than the average bear Booboo." When you add two strings together it is called concatenating and the + symbol doubles as the concatenation operator. Finally, for our string, we appear to be adding int a to it. This is allowed and the value of a is concatenated to the end of b. info? b = Alan Turing was smarter than the average bear Booboo21 This does not work the other way round; you cannot add a string to an int. This makes sense as there is no logical answer. a = a + b Finally, let's look at the code that changes our Boolean, c, from true to false: c = (1+1=3);. Here, we are assigning to c the value of the expression contained within the brackets. This would be straightforward, but why the double equals (==)? We have jumped ahead of ourselves a little. The double equals sign is another operator called the comparison operator. So we are really asking, does 1+1 equal 3? Clearly the answer is false. You might ask, "why use == instead of =?" Simply to make it clear to the compiler when we mean to assign and when we mean to compare. Inadvertently using = instead of == is a very common error. The assignment operator (=) assigns the value on the right to the value on the left, while the comparison operator (==) compares the values on either side. The compiler will warn us with an error when we do this but at first glance you might swear the compiler is wrong. Now let's use everything we know and a bit more to make our Math game project. Math game – asking a question Now that we have all that knowledge under our belts, we can use it to improve our math game. First, we will create a new Android activity to be the actual game screen as opposed to the start menu screen. We will then use the UI designer to lay out a simple game screen so that we can use our Java skills with variables, types, declaration, initialization, operators, and expressions to make our math game generate a question for the player. We can then link the start menu and game screens together with a push button. If you want to save typing and just review the finished project, you can use the code downloaded from the Packt website. If you have any trouble getting any of the code to work, you can review, compare, or copy and paste the code from the already completed code provided in the download bundle. The completed code is in the following files that correspond to the filenames we will be using in this tutorial: Chapter 3/MathGameChapter3a/java/MainActivity.java Chapter 3/MathGameChapter3a/java/GameActivity.java Chapter 3/MathGameChapter3a/layout/activity_main.xml Chapter 3/MathGameChapter3a/layout/activity_game.xml As usual, I recommend following this tutorial to see how we can create all of the code for ourselves. Creating the new game activity We will first need to create a new Java file for the game activity code and a related layout file to hold the game Activity UI. Run Android Studio and select your Math Game Chapter 2 project. It might have been opened by default. Now we will create the new Android Activity that will contain the actual game screen, which will run when the player taps the Play button on our main menu screen. To create a new activity, we know need another layout file and another Java file. Fortunately Android Studio will help us do this. To get started with creating all the files we need for a new activity, right-click on the src folder in the Project Explorer and then go to New | Activity. Now click on Blank Activity and then on Next. We now need to tell Android Studio a little bit about our new activity by entering information in the above dialog box. Change the Activity Name field to GameActivity. Notice how the Layout Name field is automatically changed for us to activity_game and the Title field is automatically changed to GameActivity. Click on Finish. Android Studio has created two files for us and has also registered our new activity in a manifest file, so we don't need to concern ourselves with it. If you look at the tabs at the top of the editor window, you will see that GameActivity.java has been opened up ready for us to edit, as shown in the following screenshot: Ensure that GameActivity.java is active in the editor window by clicking on the GameActivity.java tab shown previously. Android overrides some methods for us by default, and that most of them were not necessary. Here again, we can see the code that is unnecessary. If we remove it, then it will make our working environment simpler and cleaner. To avoid this here, we will simply use the code from MainActivity.java as a template for GameActivity.java. We can then make some minor changes. Click on the MainActivity.java tab in the editor window. Highlight all of the code in the editor window using Ctrl + A on the keyboard. Now copy all of the code in the editor window using the Ctrl + C on the keyboard. Now click on the GameActivity.java tab. Highlight all of the code in the editor window using Ctrl + A on the keyboard. Now paste the copied code and overwrite the currently highlighted code using Ctrl + V on the keyboard. Notice that there is an error in our code denoted by the red underlining as shown in the following screenshot. This is because we pasted the code referring to MainActivity in our file which is called GameActivity. Simply change the text MainActivity to GameActivity and the error will disappear. Take a moment to see if you can work out what the other minor change is necessary, before I tell you. Remember that setContentView loads our UI design. Well what we need to do is change setContentView to load the new design (that we will build next) instead of the home screen design. Change setContentView(R.layout.activity_main); to setContentView(R.layout.activity_game);. Save your work and we are ready to move on. Note the Project Explorer where Android Studio puts the two new files it created for us. I have highlighted two folders in the next screenshot. In future, I will simply refer to them as our java code folder or layout files folder. You might wonder why we didn't simply copy and paste the MainActivity.java file to begin with and saved going through the process of creating a new activity? The reason is that Android Studio does things behind the scenes. Firstly, it makes the layout template for us. It also registers the new Activity for use through a file we will see later, called AndroidManifest.xml. This is necessary for the new activity to be able to work in the first place. All things considered, the way we did it is probably the quickest. The code at this stage is exactly the same as the code for the home menu screen. We state the package name and import some useful classes provided by Android: package com.packtpub.mathgamechapter3a.mathgamechapter3a; import android.app.Activity; import android.os.Bundle; We create a new activity, this time called GameActivity: public class GameActivity extends Activity { Then we override the onCreate method and use the setContentView method to set our UI design as the contents of the player's screen. Currently, however, this UI is empty: super.onCreate(savedInstanceState); setContentView(R.layout.activity_main); We can now think about the layout of our actual game screen. Laying out the game screen UI As we know, our math game will ask questions and offer the player some multiple choices to choose answers from. There are lots of extra features we could add, such as difficulty levels, high scores, and much more. But for now, let's just stick to asking a simple, predefined question and offering a choice of three predefined possible answers. Keeping the UI design to the bare minimum suggests a layout. Our target UI will look somewhat like this: The layout is hopefully self-explanatory, but let's ensure that we are really clear; when we come to building this layout in Android Studio, the section in the mock-up that displays 2 x 2 is the question and will be made up of three text views (both numbers, and the = sign is also a separate view). Finally, the three options for the answer are made up of Button layout elements. We used all of these UI elements, but this time, as we are going to be controlling them using our Java code, there are a few extra things we need to do to them. So let's go through it step by step: Open the file that will hold our game UI in the editor window. Do this by double-clicking on activity_game.xml. This is located in our UI layout folder which can be found in the project explorer. Delete the Hello World TextView, as it is not required. Find the Large Text element on the palette. It can be found under the Widgets section. Drag three elements onto the UI design area and arrange them near the top of the design as shown in the next screenshot. It does not have to be exact; just ensure that they are in a row and not overlapping, as shown in the following screenshot: Notice in the Component Tree window that each of the three TextViews has been assigned a name automatically by Android Studio. They are textView , textView2 and textView3: Android Studio refers to these element names as an id. This is an important concept that we will be making use of. So to confirm this, select any one of the textViews by clicking on its name (id), either in the component tree as shown in the preceding screenshot or directly on it in the UI designer shown previously. Now look at the Properties window and find the id property. You might need to scroll a little to do this: Notice that the value for the id property is textView. It is this id that we will use to interact with our UI from our Java code. So we want to change all the IDs of our TextViews to something useful and easy to remember. If you look back at our design, you will see that the UI element with the textView id is going to hold the number for the first part of our math question. So change the id to textPartA. Notice the lowercase t in text, the uppercase P in Part, and the uppercase A. You can use any combination of cases and you can actually name the IDs anything you like. But just as with naming conventions with Java variables, sticking to conventions here will make things less error-prone as our program gets more complicated. Now select textView2 and change id to textOperator. Select the element currently with id textView3 and change it to textPartB. This TextView will hold the later part of our question. Now add another Large Text from the palette. Place it after the row of the three TextViews that we have just been editing. This Large Text will simply hold our equals to sign and there is no plan to ever change it. So we don't need to interact with it in our Java code. We don't even need to concern ourselves with changing the ID or knowing what it is. If this situation changed, we could always come back at a later time and edit its ID. However, this new TextView currently displays Large Text and we want it to display an equals to sign. So in the Properties window, find the text property and enter the value =. We have changed the text property, and you might also like to change the text property for textPartA, textPartB, and textOperator. This is not absolutely essential because we will soon see how we can change it via our Java code; however, if we change the text property to something more appropriate, then our UI designer will look more like it will when the game runs on a real device. So change the text property of textPartA to 2, textPartB to 2, and textOperator to x. Your UI design and Component tree should now look like this: For the buttons to contain our multiple choice answers, drag three buttons in a row, below the = sign. Line them up neatly like our target design. Now, just as we did for the TextViews, find the id properties of each button, and from left to right, change the id properties to buttonChoice1, buttonChoice2, and buttonChoice3. Why not enter some arbitrary numbers for the text property of each button so that the designer more accurately reflects what our game will look like, just as we did for our other TextViews? Again, this is not absolutely essential as our Java code will control the button appearance. We are now actually ready to move on. But you probably agree that the UI elements look a little lost. It would look better if the buttons and text were bigger. All we need to do is adjust the textSize property for each TextView and for each Button. Then, we just need to find the textSize property for each element and enter a number with the sp syntax. If you want your design to look just like our target design from earlier, enter 70sp for each of the TextView textSize properties and 40sp for each of the Buttons textSize properties. When you run the game on your real device, you might want to come back and adjust the sizes up or down a bit. But we have a bit more to do before we can actually try out our game. Save the project and then we can move on. As before, we have built our UI. This time, however, we have given all the important parts of our UI a unique, useful, and easy to identify ID. As we will see we are now able to communicate with our UI through our Java code. Coding a question in Java With our current knowledge of Java, we are not yet able to complete our math game but we can make a significant start. We will look at how we can ask the player a question and offer them some multiple choice answers (one correct and two incorrect). At this stage, we know enough of Java to declare and initialize some variables that will hold the parts of our question. For example, if we want to ask the times tables question 2 x 2, we could have the following variable initializations to hold the values for each part of the question: int partA = 2; int partB = 2; The preceding code declares and initializes two variables of the int type, each to the value of 2. We use int because we will not be dealing with any decimal fractions. Remember that the variable names are arbitrary and were just chosen because they seemed appropriate. Clearly, any math game worth downloading is going to need to ask more varied and advanced questions than 2 x 2, but it is a start. Now we know that our math game will offer multiple choices as answers. So, we need a variable for the correct answer and two variables for two incorrect answers. Take a look at these combined declarations and initializations: int correctAnswer = partA * partB; int wrongAnswer1 = correctAnswer - 1; int wrongAnswer2 = correctAnswer + 1; Note that the initialization of the variables for the wrong answers depends on the value of the correct answer, and the variables for the wrong answers are initialized after initializing the correctAnswer variable. Now we need to put these values, held in our variables, into the appropriate elements on our UI. The question variables (partA and partB) need to be displayed in our UI elements, textPartA and textPartB, and the answer variables (correctAnswer, wrongAnswer1, and wrongAnswer2) need to be displayed in our UI elements with the following IDs: buttonChoice1, buttonChoice2, and buttonChoice3. We will see how we do this in the next step-by-step tutorial. We will also implement the variable declaration and initialization code that we discussed a moment ago: First, open GameActivity.java in the editor window. Remember that you can do this by double-clicking on GameActivity in our java folder or clicking on its tab above the editor window if GameActivity.java is already open. All of our code will go into the onCreate method. It will go after the setContentView(R.layout.activity_game); line but before the closing curly brace } of the onCreate method. Perhaps, it's a good idea to leave a blank line for clarity and a nice explanatory comment as shown in the following code. We can see the entire onCreate method as it stands after the latest amendments. The parts in bold are what you need to add. Feel free to add helpful comments like mine if you wish: @Override     protected void onCreate(Bundle savedInstanceState) {         super.onCreate(savedInstanceState);         //The next line loads our UI design to the screen         setContentView(R.layout.activity_game);           //Here we initialize all our variables         int partA = 9;         int partB = 9;         int correctAnswer = partA * partB;         int wrongAnswer1 = correctAnswer - 1;         int wrongAnswer2 = correctAnswer + 1;       }//onCreate ends here Now we need to add the values contained within the variables to the TextView and Button of our UI. But first, we need to get access to the UI elements we created. We do that by creating a variable of the appropriate class and linking it via the ID property of the appropriate UI element. We already know the class of our UI elements: TextView and Button. Here is the code that creates our special class variables for each of the necessary UI elements. Take a close look at the code, but don't worry if you don't understand all of it now. We will dissect the code in detail once everything is working. Enter the code immediately after the code entered in the previous step. You can leave a blank line for clarity if you wish. Just before you proceed, note that at two points while typing in this code, you will be prompted to import another class. Go ahead and do so on both occasions: /*Here we get a working object based on either the button or TextView class and base as well as link our new   objects directly to the appropriate UI elements that we created previously*/   TextView textObjectPartA =   (TextView)findViewById(R.id.textPartA);   TextView textObjectPartB =   (TextView)findViewById(R.id.textPartB);   Button buttonObjectChoice1 =   (Button)findViewById(R.id.buttonChoice1);   Button buttonObjectChoice2 =   (Button)findViewById(R.id.buttonChoice2);   Button buttonObjectChoice3 =   (Button)findViewById(R.id.buttonChoice3); In the preceding code, if you read the multiline comment, you will see that I used the term object. When we create a variable type based on a class, we call it an object. Once we have an object of a class, we can do anything that that class was designed to do. Now we have five new objects linked to the elements of our UI that we need to manipulate. What precisely are we going to do with them? We need to display the values of our variables in the text of the UI elements. We can use the objects we just created combined with a method provided by the class, and use our variables as values for that text. As usual, we will dissect this code further at the end of this tutorial. Here is the code to enter directly after the code in the previous step. Try and work out what is going on before we look at it together: //Now we use the setText method of the class on our objects //to show our variable values on the UI elements. //Just like when we output to the console in the exercise - //Expressions in Java, only now we use setText method //to put the values in our variables onto the actual UI. textObjectPartA.setText("" + partA); textObjectPartB.setText("" + partB);   //which button receives which answer, at this stage is arbitrary.   buttonObjectChoice1.setText("" + correctAnswer); buttonObjectChoice2.setText("" + wrongAnswer1); buttonObjectChoice3.setText("" + wrongAnswer2); Save your work. If you play with the assignment values for partA and partB, you can make them whatever you like and the game adjusts the answers accordingly. Obviously, we shouldn't need to reprogram our game each time we want a new question and we will solve that problem soon. All we need to do now is link the game section we have just made to the start screen menu. We will do that in the next tutorial. Now lets explore the trickier and newer parts of our code in more detail. In step 2, we declared and initialized the variables required so far: //Here we initialize all our variables int partA = 2; int partB = 2; int correctAnswer = partA * partB; int wrongAnswer1 = correctAnswer - 1; int wrongAnswer2 = correctAnswer + 1; Then in step 3, we got a reference to our UI design through our Java code. For the TextViews, it was done like this: TextView textObjectPartA = (TextView)findViewById(R.id.textPartA); For each of the buttons, a reference to our UI design was obtained like this: Button buttonObjectChoice1 =   Button)findViewById(R.id.buttonChoice1); In step 4, we did something new. We used a the setText method to show the values of our variables on our UI elements (TextView and Button) to the player. Let's break down one line completely to see how it works. Here is the code that shows the correctAnswer variable being displayed on buttonObjectChoice1. buttonObjectChoice1.setText("" + correctAnswer); By typing buttonObjectChoice1 and adding a period, as shown in the following line of code, we have access to all the preprogrammed methods of that object's class type that are provided by Android: The power of Button and the Android API There are actually lots of methods that we can perform on an object of the Button type. If you are feeling brave, try this to get a feeling of just how much functionality there is in Android. Type the following code: buttonObjectChoice1 Be sure to type the period on the end. Android Studio will pop up a list of possible methods to use on this object. Scroll through the list and get a feel of the number and variety of options: If a mere button can do all of this, think of the possibilities for our games once we have mastered all the classes contained in Android. A collection of classes designed to be used by others is collectively known as an Application Programmers Interface (API). Welcome to the Android API! In this case, we just want to set the button's text. So, we use setText and concatenate the value stored in our correctAnswer variable to the end of an empty string, like this: setText("" + correctAnswer); We do this for each of the UI elements we require to show our variables. Playing with autocomplete If you tried the previous tip, The power of Button and the Android API, and explored the methods available for objects of the Button type, you will already have some insight into autocomplete. Note that as you type, Android Studio is constantly making suggestions for what you might like to type next. If you pay attention to this, you can save a lot of time. Simply select the correct code completion statement that is suggested and press Enter. You can even see how much time you saved by selecting Help | Productivity Guide from the menu bar. Here you will see statistics for every aspect of code completion and more. Here are a few entries from mine: As you can see, if you get used to using shortcuts early on, you can save a lot of time in the long run. Linking our game from the main menu At the moment, if we run the app, we have no way for the player to actually arrive at our new game activity. We want the game activity to run when the player clicks on the Play button on the main MainActivity UI. Here is what we need to do to make that happen: Open the file activity_main.xml, either by double-clicking on it in the project explorer or by clicking its tab in the editor window. Now, just like we did when building the game UI, assign an ID to the Play button. As a reminder, click on the Play button either on the UI design or in the component tree. Find the id property in the Properties window. Assign the buttonPlay value to it. We can now make this button do stuff by referring to it in our Java code. Open the file MainActivity.java, either by double-clicking on it in the project explorer or clicking on its tab in the editor window. In our onCreate method, just after the line where we setContentView, add the following highlighted line of code: setContentView(R.layout.activity_main); Button buttonPlay = (Button)findViewById(R.id.buttonPlay); We will dissect this code in detail once we have got this working. Basically we are making a connection to the Play button by creating a reference variable to a Button object. Notice that both words are highlighted in red indicating an error. Just as before, we need to import the Button class to make this code work. Use the Alt + Enter keyboard combination. Now click on Import class from the popped-up list of options. This will automatically add the required import directive at the top of our MainActivity.java file. Now for something new. We will give the button the ability to listen to the user clicking on it. Type this immediately after the last line of code we entered: buttonPlay.setOnClickListener(this); Notice how the this keyword is highlighted in red indicating an error. Setting that aside, we need to make a modification to our code now in order to allow the use of an interface that is a special code element that allows us to add a functionality, such as listening for button clicks. Edit the line as follows. When prompted to import another class, click on OK: public class MainActivity extends Activity { to public class MainActivity extends Activity implements View.OnClickListener{ Now we have the entire line underlined in red. This indicates an error but it's where we should be at this point. We mentioned that by adding implements View.OnClickListener, we have implemented an interface. We can think of this like a class that we can use but with extra rules. The rules of the OnClickListener interface state that we must implement/use one of its methods. Notice that until now, we have optionally overridden/used methods as and when they have suited us. If we wish to use the functionality this interface provides, namely listening for button presses, then we have to add/implement the onClick method. This is how we do it. Notice the opening curly brace, {, and the closing curly brace, }. These denote the start and end of the method. Notice that the method is empty and it doesn't do anything, but an empty method is enough to comply with the rules of the OnClickListener interface, and the red line indicating an that our code has an error has gone. Make sure that you type the following code, outside the closing curly brace (}) of the onCreate method but inside the closing curly brace of our MainActivity class: @Override     public void onClick(View view) {               } Notice that we have an empty line between { and } of the onClick method. We can now add code in here to make the button actually do something. Type the following highlighted code between { and } of onClick: @Override     public void onClick(View view) {         Intent i;         i = new Intent(this, GameActivity.class);         startActivity(i);     } OK, so that code is a bit of a mouthful to comprehend all at once. See if you can guess what is happening. The clue is in the method named startActivity and the hopefully familiar term, GameActivity. Notice that we are assigning something to i. We will quickly get our app working and then diagnose the code in full. Notice that we have an error: all instances of the word Intent are red. We can solve this by importing the classes required to make Intent work. As before press Alt + Enter. Run the game in the emulator or on your device. Our app will now work. This is what the new game screen looks like after pressing Play on the menu screen: Almost every part of our code has changed a little and we have added a lot to it as well. Let's go over the contents of MainActivity.java and look at it line by line. For context, here it is in full: package com.packtpub.mathgamechapter3a.mathgamechapter3a;   import android.app.Activity; import android.content.Intent; import android.os.Bundle; import android.view.View; import android.widget.Button;     public class MainActivity extends Activity implements View.OnClickListener{       @Override     protected void onCreate(Bundle savedInstanceState) {         super.onCreate(savedInstanceState);         setContentView(R.layout.activity_main);         final Button buttonPlay =           (Button)findViewById(R.id.buttonPlay);         buttonPlay.setOnClickListener(this);     }       @Override     public void onClick(View view) {         Intent i;         i = new Intent(this, GameActivity.class);         startActivity(i);     }   } We have seen much of this code before, but let's just go over it a chunk at a time before moving on so that it is absolutely clear. The code works like this: package com.packtpub.mathgamechapter3a.mathgamechapter3a; import android.app.Activity; import android.content.Intent; import android.os.Bundle; import android.view.View; import android.widget.Button; You would probably remember that this first block of code defines what our package is called and makes available all the Android API stuff we need for Button, TextView, and Activity. From our MainActivity.java file, we have this: public class MainActivity extends Activity implements View.OnClickListener{ Our MainActivity declaration with our new bit of code implements View.OnClickListener that gives us the ability to detect button clicks. Next in our code is this: @Override     protected void onCreate(Bundle savedInstanceState) {         super.onCreate(savedInstanceState);         setContentView(R.layout.activity_main); It is at the start of our onCreate method where we first ask the hidden code of onCreate to do its stuff using super.onCreate(savedInstanceState);. Then we set our UI to the screen with setContentView(R.layout.activity_main);. Next, we get a reference to our button with an ID of buttonPlay: Button buttonPlay = (Button)findViewById(R.id.buttonPlay); buttonPlay.setOnClickListener(this); Finally, our onClick method uses the Intent class to send the player to our GameActivity class and the related UI when the user clicks on the Play button: @Override     public void onClick(View view) {         Intent i;         i = new Intent(this, GameActivity.class);         startActivity(i);     } If you run the app, you will notice that we can now click on the Play button and our math game will ask us a question. Of course, we can't answer it yet. Although we have very briefly looked at how to deal with button presses, we need to learn more of Java in order to intelligently react to them. We will also reveal how to write code to handle presses from several buttons. This will be necessary to receive input from our multiple-choice-centric game_activity UI. Decisions in Java We can now summon enough of Java prowess to ask a question but a real math game must obviously do much more than this. We need to capture the player's answer, and we are nearly there with that—we can detect button presses. From there, we need to be able to decide whether their answer is right or wrong. Then, based on this decision, we have to choose an appropriate course of action. Let's leave the math game aside for now and look at how Java might help us by learning some more fundamentals and syntax of the Java language. More operators Let's look at some more operators: we can already add (+), take away (-), multiply (*), divide (/), assign (=), increment (++), compare (==), and decrement (--) with operators. Let's introduce some more super-useful operators, and then we will go straight to actually understanding how to use them in Java. Don't worry about memorizing every operator given here. Glance at them and their explanations. There, we will put some operators to use and they will become much clearer as we see a few examples of what they allow us to do. They are presented here in a list just to make the variety and scope of operators plain from the start. The list will also be more convenient to refer back to when not intermingled with the discussion about implementation that follows it. ==: This is a comparison operator we saw this very briefly before. It tests for equality and is either true or false. An expression like (10 == 9);, for example, is false. !: The logical NOT operator. The expression, ! (2+2==5).), is true because 2+2 is NOT 5. !=: This is another comparison operator which tests if something is NOT equal. For example, the expression, (10 != 9);), is true, that is, 10 is not equal to 9. >: This is another comparison operator which tests if something is greater than something else. The expression, (10 > 9);), is true. There are a few more comparison operators as well. <: You guessed it. This tests whether the value to the left is less than the value to the right or not. The expression, (10 < 9);, is false. >=: This operator tests whether one value is greater than or equal to the other, and if either is true, the result is true. For example, the expression, (10 >= 9);, is true. The expression, (10 >= 10);, is also true. <=: Like the preceding operator, this operator tests for two conditions but this time, less than and equal to. The expression, (10 <= 9);, is false. The expression, (10 <= 10);, is true. &&: This operator is known as logical AND. It tests two or more separate parts of an expression and all parts must be true in order for the result to be true. Logical AND is usually used in conjunction with the other operators to build more complex tests. The expression, ((10 > 9) && (10 < 11));, is true because both parts are true. The expression, ((10 > 9) && (10 < 9));, is false because only one part of the expression is true and the other is false. ||: This operator is called logical OR. It is just like logical AND except that only one of two or more parts of an expression need to be true for the expression to be true. Let's look at the last example we used but replace the && sign with ||. The expression, ((10 > 9) || (10 < 9));, is now true because one part of the expression is true. All of these operators are virtually useless without a way of properly using them to make real decisions that affect real variables and code. Let's look at how to make decisions in Java. Decision 1 – If they come over the bridge, shoot them As we saw, operators serve hardly any purpose on their own but it was probably useful to see just a part of the wide and varied range available to us. Now, when we look at putting the most common operator, ==, to use, we can start to see the powerful yet fine control that operators offer us. Let's make the previous examples less abstract using the Java if keyword, a few conditional operators, and a small story: in use with some code and a made up military situation that will hopefully make the following examples less abstract. The captain is dying and, knowing that his remaining subordinates are not very experienced, he decides to write a Java program to convey his last orders after he has died. The troops must hold one side of a bridge while awaiting reinforcements. The first command the captain wants to make sure his troops understand is this: If they come over the bridge, shoot them. So how do we simulate this situation in Java? We need a Boolean variable isComingOverBridge. The next bit of code assumes that the isComingOverBridge variable has been declared and initialized. We can then use it like this: if(isComingOverBridge){   //Shoot them } If the isComingOverBridge Boolean is true, the code inside the opening and closing curly braces will run. If not, the program continues after the if block without running it. Decision 2 – Else, do this The captain also wants to tell his troops what to do (stay put) if the enemy is not coming over the bridge. Now we introduce another Java keyword, else. When we want to explicitly do something and the if block does not evaluate to true, we can use else. For example, to tell the troops to stay put if the enemy is not coming over the bridge, we use else: if(isComingOverBridge){   //Shoot them }else{   //Hold position } The captain then realized that the problem wasn't as simple as he first thought. What if the enemy comes over the bridge and has more troops? His squad will be overrun. So, he came up with this code (we'll use some variables as well this time): boolean isComingOverTheBridge; int enemyTroops; int friendlyTroops; //Code that initializes the above variables one way or another   //Now the if if(isComingOverTheBridge && friendlyTroops > enemyTroops){   //shoot them }else if(isComingOverTheBridge && friendlyTroops < enemyTroops) {   //blow the bridge }else{   //Hold position } Finally, the captain's last concern was that if the enemy came over the bridge waving the white flag of surrender and were promptly slaughtered, then his men would end up as war criminals. The Java code needed was obvious. Using the wavingWhiteFlag Boolean variable he wrote this test: if (wavingWhiteFlag){   //Take prisoners } But where to put this code was less clear. In the end, the captain opted for the following nested solution and changing the test for wavingWhiteFlag to logical NOT, like this: if (!wavingWhiteFlag){//not surrendering so check everything else   if(isComingOverTheBridge && friendlyTroops > enemyTroops){     //shoot them   }else if(isComingOverTheBridge && friendlyTroops <   enemyTroops) {     //blow the bridge   } }else{//this is the else for our first if   //Take prisoners { //Holding position This demonstrates that we can nest if and else statements inside of one another to create even deeper decisions. We could go on making more and more complicated decisions but what we have seen is more than sufficient as an introduction. Take the time to reread this if anything is unclear. It is also important to point out that very often, there are two or more ways to arrive at the solution. The right way will usually be the way that solves the problem in the clearest and simplest manner. Switching to make decisions We have seen the vast and virtually limitless possibilities of combining the Java operators with if and else statements. But sometimes a decision in Java can be better made in other ways. When we have to make a decision based on a clear list of possibilities that doesn't involve complex combinations, then switch is usually the way to go. We start a switch decision like this: switch(argument){   } In the previous example, an argument could be an expression or a variable. Then within the curly braces, we can make decisions based on the argument with case and break elements: case x:   //code to for x   break;   case y:   //code for y   break; You can see that in the previous example, each case states a possible result and each break denotes the end of that case as well as the point at which no further case statements should be evaluated. The first break encountered takes us out of the switch block to proceed with the next line of code. We can also use default without a value to run some code if none of the case statements evaluate to true, like this: default://Look no value   //Do something here if no other case statements are true break; Supposing we are writing an old-fashioned text adventure game—the kind of game where the player types commands such as "Go East", "Go West", "Take Sword", and so on. In this case, switch could handle that situation like this example code and we could use default to handle the case of the player typing a command that is not specifically handled: //get input from user in a String variable called command switch(command){     case "Go East":":   //code to go east   break;     case "Go West":   //code to go west   break;   case "Take sword":   //code to take the sword   break;     //more possible cases     default:   //Sorry I don't understand your command   break;   } We will use switch so that our onClick method can handle the different multiple-choice buttons of our math game. Java has even more operators than we have covered here. We have looked at all the operators we are going to need in this book and probably the most used in general. If you want the complete lowdown on operators, take a look at the official Java documentation at http://docs.oracle.com/javase/tutorial/java/nutsandbolts/operators.html. Math game – getting and checking the answer Here we will detect the right or wrong answer and provide a pop-up message to the player. Our Java is getting quite good now, so let's dive in and add these features. I will explain things as we go and then, as usual, dissect the code thoroughly at the end. The already completed code is in the download bundle, in the following files that correspond to the filenames we will create/autogenerate in Android Studio in a moment: Chapter 3/MathGameChapter3b/java/MainActivity.java Chapter 3/MathGameChapter3b/java/GameActivity.java Chapter 3/MathGameChapter3b/layout/activity_main.xml Chapter 3/MathGameChapter3b/layout/activity_game.xml As usual, I recommend following this tutorial step by step to see how we can create all of the code for ourselves. Open the GameActivity.java file visible in the editor window. Now we need to add the click detection functionality to our GameActivity, just as we did for our MainActivity. However, we will go a little further than the last time. So let's do it step by step as if it is totally new. Once again, we will give the buttons the ability to listen to the user clicking on them. Type this immediately after the last line of code we entered in the onCreate method but before the closing }. This time of course, we need to add some code to listen to three buttons: buttonObjectChoice1.setOnClickListener(this); buttonObjectChoice2.setOnClickListener(this); buttonObjectChoice3.setOnClickListener(this); Notice how the this keyword is highlighted in red indicating an error. Again, we need to make a modification to our code in order to allow the use of an interface, the special code element that allows us to add functionalities such as listening to button clicks. Edit the line as follows. When prompted to import another class, click on OK. Consider this line of code: public class GameActivity extends Activity { Change it to the following line: public class GameActivity extends Activity implements View.OnClickListener{   Now we have the entire preceding line underlined in red. This indicates an error but it is where we should be at this point. We mentioned that by adding implements View.OnClickListener, we have implemented an interface. We can think of this like a class that we can use, but with extra rules. One of the rules of the OnClickListener interface is that we must implement one of its methods, as you might remember. If we wish to use the useful functionality this interface provides (listening to button presses). Now we will add the onClick method. Type the following code. Notice the opening curly brace, {, and the closing curly brace, }. These denote the start and end of the method. Notice that the method is empty; it doesn't do anything but an empty method is enough to comply with the rules of the OnClickListener interface and the red line that indicated an error has gone. Make sure that you type the following code outside the closing curly brace (}) of the onCreate method but inside the closing curly brace of our MainActivity class: @Override     public void onClick(View view) {     } Notice that we have an empty line between the { and } braces of our onClick method. We can now put some code in here to make the buttons actually do something. Type the following in between { and } of onClick. This is where things get different from our code in MainActivity. We need to differentiate between the three possible buttons that could be pressed. We will do this with the switch statement that we discussed earlier. Look at the case criteria; they should look familiar. Here is the code that uses the switch statements: switch (view.getId()) {               case R.id.buttonChoice1:             //button 1 stuff goes here                 break;               case R.id.buttonChoice2:             //button 2 stuff goes here                 break;               case R.id.buttonChoice3:            //button 3 stuff goes here                 break;           }   Each case element handles a different button. For each button case, we need to get the value stored in the button that was just pressed and see if it matches our correctAnswer variable. If it does, we must tell the player they got it right, and if not, we must tell them they got it wrong. However, there is still one problem we have to solve. The onClick method is separate from the onCreate method and the Button objects. In fact, all the variables are declared in the onCreate method. If you try typing the code from step 9 now, you will get lots of errors. We need to make all the variables that we need in onClick available in onClick. To do this, we will move their declarations from above the onCreate method to just below the opening { of GameActivity. This means that these variables become variables of the GameActivity class and can be seen anywhere within GameActivity. Declare the following variables like this: int correctAnswer; Button buttonObjectChoice1; Button buttonObjectChoice2; Button buttonObjectChoice3; Now change the initialization of these variables within onCreate as follows. The actual parts of code that need to be changed are highlighted. The rest is shown for context: //Here we initialize all our variables int partA = 9; int partB = 9; correctAnswer = partA * partB; int wrongAnswer1 = correctAnswer - 1; int wrongAnswer2 = correctAnswer + 1; and TextView textObjectPartA =   (TextView)findViewById(R.id.textPartA);   TextView textObjectPartB =   (TextView)findViewById(R.id.textPartB);   buttonObjectChoice1 = (Button)findViewById(R.id.buttonChoice1);         buttonObjectChoice2 = (Button)findViewById(R.id.buttonChoice2);         buttonObjectChoice3 = (Button)findViewById(R.id.buttonChoice3);   Here is the top of our onClick method as well as the first case statement for our onClick method: @Override     public void onClick(View view) {         //declare a new int to be used in all the cases         int answerGiven=0;         switch (view.getId()) {               case R.id.buttonChoice1:             //initialize a new int with the value contained in buttonObjectChoice1             //Remember we put it there ourselves previously                 answerGiven = Integer.parseInt("" +                     buttonObjectChoice1.getText());                   //is it the right answer?                 if(answerGiven==correctAnswer) {//yay it's the right answer                     Toast.makeText(getApplicationContext(),                       "Well done!",                       Toast.LENGTH_LONG).show();                 }else{//uh oh!                     Toast.makeText(getApplicationContext(),"Sorry that's     wrong", Toast.LENGTH_LONG).show();                 }                 break;   Here are the rest of the case statements that do the same steps as the code in the previous step except handling the last two buttons. Enter the following code after the code entered in the previous step:   case R.id.buttonChoice2:                 //same as previous case but using the next button                 answerGiven = Integer.parseInt("" +                   buttonObjectChoice2.getText());                 if(answerGiven==correctAnswer) {                     Toast.makeText(getApplicationContext(), "Well done!", Toast.LENGTH_LONG).show();                 }else{                     Toast.makeText(getApplicationContext(),"Sorry that's wrong", Toast.LENGTH_LONG).show();                 }                 break;               case R.id.buttonChoice3:                 //same as previous case but using the next button                 answerGiven = Integer.parseInt("" +                     buttonObjectChoice3.getText());                 if(answerGiven==correctAnswer) {                     Toast.makeText(getApplicationContext(), "Well done!", Toast.LENGTH_LONG).show();                 }else{                     Toast.makeText(getApplicationContext(),"Sorry that's wrong", Toast.LENGTH_LONG).show();                 }                 break;           } Run the program, and then we will look at the code carefully, especially that odd-looking Toast thing. Here is what happens when we click on the leftmost button: This is how we did it: In steps 1 through 6, we set up handling for our multi-choice buttons, including adding the ability to listen to clicks using the onClick method and a switch block to handle decisions depending on the button pressed. In steps 7 and 8, we had to alter our code to make our variables available in the onClick method. We did this by making them member variables of our GameActivity class. When we make a variable a member of a class, we call it a field. In steps 9 and 10, we implemented the code that actually does the work in our switch statement in onClick. Let's take a line-by-line look at the code that runs when button1 is pressed. case R.id.buttonChoice1: First, the case statement is true when the button with an id of buttonChoice1 is pressed. Then the next line of code to execute is this: answerGiven = Integer.parseInt(""+ buttonObjectChoice1.getText()); The preceding line gets the value on the button using two methods. First, getText gets the number as a string and then Integer.parseInt converts it to an int. The value is stored in our answerGiven variable. The following code executes next: if(answerGiven==correctAnswer) {//yay it's the right answer   Toast.makeText(getApplicationContext(), "Well done!",     Toast.LENGTH_LONG).show(); }else{//uh oh!     Toast.makeText(getApplicationContext(),"Sorry that's wrong",       Toast.LENGTH_LONG).show();                 } The if statement tests to see if the answerGiven variable is the same as correctAnswer using the == operator. If so, the makeText method of the Toast object is used to display a congratulatory message. If the values of the two variables are not the same, the message displayed is a bit more negative one. The Toast line of code is possibly the most evil thing we have seen thus far. It looks exceptionally complicated and it does need a greater knowledge of Java than we have at the moment to understand. All we need to know for now is that we can use the code as it is and just change the message, and it is a great tool to announce something to the player. If you really want an explanation now, you can think of it like this: when we made button objects, we got to use all the button methods. But with Toast, we used the class directly to access its setText method without creating an object first. We can do this process when the class and its methods are designed to allow it. Finally, we break out of the whole switch statement as follows: break; Self-test questions Q1) What does this code do? // setContentView(R.layout.activity_main); Q2) Which of these lines causes an error? String a = "Hello"; String b = " Vinton Cerf"; int c = 55; a = a + b c = c + c + 10; a = a + c; c = c + a; Q3) We talked a lot about operators and how different operators can be used together to build complicated expressions. Expressions, at a glance, can sometimes make the code look complicated. However, when looked at closely, they are not as tough as they seem. Usually, it is just a case of splitting the expressions into smaller pieces to work out what is going on. Here is an expression that is more convoluted than anything else you will ever see in this book. As a challenge, can you work out: what will x be? int x = 10; int y = 9; boolean isTrueOrFalse = false; isTrueOrFalse = (((x <=y)||(x == 10))&&((!isTrueOrFalse) || (isTrueOrFalse))); Summary We went from knowing nothing about Java syntax to learning about comments, variables, operators, and decision making. As with any language, mastery of Java can be achieved by simply practicing, learning, and increasing our vocabulary. At this point, the temptation might be to hold back until mastery of the current Java syntax has been achieved, but the best way is to move on to new syntax at the same time as revisiting what we have already begun to learn. We will finally finish our math game by adding random questions of multiple difficulties as well as using more appropriate and random wrong answers for the multiple choice buttons. To enable us to do this, we will first learn some more new on Java and syntax. Resources for Article: Further resources on this subject: Asking Permission: Getting your head around Marshmallow's Runtime Permissions [article] Android and iOS Apps Testing at a Glance [article] Practical How-To Recipes for Android [article]
Read more
  • 0
  • 0
  • 8977

article-image-understanding-php-basics
Packt
17 Feb 2016
27 min read
Save for later

Understanding PHP basics

Packt
17 Feb 2016
27 min read
In this article by Antonio Lopez Zapata, the author of the book Learning PHP 7, you need to understand not only the syntax of the language, but also its grammatical rules, that is, when and why to use each element of the language. Luckily, for you, some languages come from the same root. For example, Spanish and French are romance languages as they both evolved from spoken Latin; this means that these two languages share a lot of rules, and learning Spanish if you already know French is much easier. (For more resources related to this topic, see here.) Programming languages are quite the same. If you already know another programming language, it will be very easy for you to go through this chapter. If it is your first time though, you will need to understand from scratch all the grammatical rules, so it might take some more time. But fear not! We are here to help you in this endeavor. In this chapter, you will learn about these topics: PHP in web applications Control structures Functions PHP in web applications Even though the main purpose of this chapter is to show you the basics of PHP, doing so in a reference-manual way is not interesting enough. If we were to copy paste what the official documentation says, you might as well go there and read it by yourself. Instead, let's not forget the main purpose of this book and your main goal—to write web applications with PHP. We will show you how can you apply everything you are learning as soon as possible, before you get too bored. In order to do that, we will go through the journey of building an online bookstore. At the very beginning, you might not see the usefulness of it, but that is just because we still haven't seen all that PHP can do. Getting information from the user Let's start by building a home page. In this page, we are going to figure out whether the user is looking for a book or just browsing. How do we find this out? The easiest way right now is to inspect the URL that the user used to access our application and extract some information from there. Save this content as your index.php file: <?php $looking = isset($_GET['title']) || isset($_GET['author']); ?> <!DOCTYPE html> <html lang="en"> <head> <meta charset="UTF-8"> <title>Bookstore</title> </head> <body> <p>You lookin'? <?php echo (int) $looking; ?></p> <p>The book you are looking for is</p> <ul> <li><b>Title</b>: <?php echo $_GET['title']; ?></li> <li><b>Author</b>: <?php echo $_GET['author']; ?></li> </ul> </body> </html> And now, access http://localhost:8000/?author=Harper Lee&title=To Kill a Mockingbird. You will see that the page is printing some of the information that you passed on to the URL. For each request, PHP stores in an array—called $_GET- all the parameters that are coming from the query string. Each key of the array is the name of the parameter, and its associated value is the value of the parameter. So, $_GET contains two entries: $_GET['author'] contains Harper Lee and $_GET['title'] contains To Kill a Mockingbird. On the first highlighted line, we are assigning a Boolean value to the $looking variable. If either $_GET['title'] or $_GET['author'] exists, this variable will be true; otherwise, false. Just after that, we close the PHP tag and then we start printing some HTML, but as you can see, we are actually mixing HTML with PHP code. Another interesting line here is the second highlighted line. We are printing the content of $looking, but before that, we cast the value. Casting means forcing PHP to transform a type of value to another one. Casting a Boolean to an integer means that the resultant value will be 1 if the Boolean is true or 0 if the Boolean is false. As $looking is true since $_GET contains valid keys, the page shows 1. If we try to access the same page without sending any information as in http://localhost:8000, the browser will say "Are you looking for a book? 0". Depending on the settings of your PHP configuration, you will see two notice messages complaining that you are trying to access the keys of the array that do not exist. Casting versus type juggling We already knew that when PHP needs a specific type of variable, it will try to transform it, which is called type juggling. But PHP is quite flexible, so sometimes, you have to be the one specifying the type that you need. When printing something with echo, PHP tries to transform everything it gets into strings. Since the string version of the false Boolean is an empty string, this would not be useful for our application. Casting the Boolean to an integer first assures that we will see a value, even if it is just "0". HTML forms HTML forms are one of the most popular ways to collect information from users. They consist a series of fields called inputs in the HTML world and a final submit button. In HTML, the form tag contains two attributes: the action points, where the form will be submitted and method that specifies which HTTP method the form will use—GET or POST. Let's see how it works. Save the following content as login.html and go to http://localhost:8000/login.html: <!DOCTYPE html> <html lang="en"> <head> <meta charset="UTF-8"> <title>Bookstore - Login</title> </head> <body> <p>Enter your details to login:</p> <form action="authenticate.php" method="post"> <label>Username</label> <input type="text" name="username" /> <label>Password</label> <input type="password" name="password" /> <input type="submit" value="Login"/> </form> </body> </html> This form contains two fields, one for the username and one for the password. You can see that they are identified by the name attribute. If you try to submit this form, the browser will show you a Page Not Found message, as it is trying to access http://localhost:8000/authenticate.phpand the web server cannot find it. Let's create it then: <?php $submitted = !empty($_POST); ?> <!DOCTYPE html> <html lang="en"> <head> <meta charset="UTF-8"> <title>Bookstore</title> </head> <body> <p>Form submitted? <?php echo (int) $submitted; ?></p> <p>Your login info is</p> <ul> <li><b>username</b>: <?php echo $_POST['username']; ?></li> <li><b>password</b>: <?php echo $_POST['password']; ?></li> </ul> </body> </html> As with $_GET, $_POST is an array that contains the parameters received by POST. In this piece of code, we are first asking whether that array is not empty—note the ! operator. Afterwards, we just display the information received, just as in index.php. Note that the keys of the $_POST array are the values for the name argument of each input field. Control structures So far, our files have been executed line by line. Due to that, we are getting some notices on some scenarios, such as when the array does not contain what we are looking for. Would it not be nice if we could choose which lines to execute? Control structures to the rescue! A control structure is like a traffic diversion sign. It directs the execution flow depending on some predefined conditions. There are different control structures, but we can categorize them in conditionals and loops. A conditional allows us to choose whether to execute a statement or not. A loop will execute a statement as many times as you need. Let's take a look at each one of them. Conditionals A conditional evaluates a Boolean expression, that is, something that returns a value. If the expression is true, it will execute everything inside its block of code. A block of code is a group of statements enclosed by {}. Let's see how it works: <?php echo "Before the conditional."; if (4 > 3) { echo "Inside the conditional."; } if (3 > 4) { echo "This will not be printed."; } echo "After the conditional."; In this piece of code, we are using two conditionals. A conditional is defined by the keyword if followed by a Boolean expression in parentheses and by a block of code. If the expression is true, it will execute the block; otherwise, it will skip it. You can increase the power of conditionals by adding the keyword else. This tells PHP to execute a block of code if the previous conditions were not satisfied. Let's see an example: if (2 > 3) { echo "Inside the conditional."; } else { echo "Inside the else."; } This will execute the code inside else as the condition of if was not satisfied. Finally, you can also add an elseif keyword followed by another condition and block of code to continue asking PHP for more conditions. You can add as many elseif as you need after if. If you add else, it has to be the last one of the chain of conditions. Also keep in mind that as soon as PHP finds a condition that resolves to true, it will stop evaluating the rest of the conditions: <?php if (4 > 5) { echo "Not printed"; } elseif (4 > 4) { echo "Not printed"; } elseif (4 == 4) { echo "Printed."; } elseif (4 > 2) { echo "Not evaluated."; } else { echo "Not evaluated."; } if (4 == 4) { echo "Printed"; } In this last example, the first condition that evaluates to true is the one that is highlighted. After that, PHP does not evaluate any more conditions until a new if starts. With this knowledge, let's try to clean up a bit of our application, executing statements only when needed. Copy this code to your index.php file: <!DOCTYPE html> <html lang="en"> <head> <meta charset="UTF-8"> <title>Bookstore</title> </head> <body> <p> <?php if (isset($_COOKIE[username'])) { echo "You are " . $_COOKIE['username']; } else { echo "You are not authenticated."; } ?> </p> <?php if (isset($_GET['title']) && isset($_GET['author'])) { ?> <p>The book you are looking for is</p> <ul> <li><b>Title</b>: <?php echo $_GET['title']; ?></li> <li><b>Author</b>: <?php echo $_GET['author']; ?></li> </ul> <?php } else { ?> <p>You are not looking for a book?</p> <?php } ?> </body> </html> In this new code, we are mixing conditionals and HTML code in two different ways. The first one opens a PHP tag and adds an if-else clause that will print whether we are authenticated or not with echo. No HTML is merged within the conditionals, which makes it clear. The second option—the second highlighted block—shows an uglier solution, but this is sometimes necessary. When you have to print a lot of HTML code, echo is not that handy, and it is better to close the PHP tag; print all the HTML you need and then open the tag again. You can do that even inside the code block of an if clause, as you can see in the code. Mixing PHP and HTML If you feel like the last file we edited looks rather ugly, you are right. Mixing PHP and HTML is confusing, and you have to avoid it by all means. Let's edit our authenticate.php file too, as it is trying to access $_POST entries that might not be there. The new content of the file would be as follows: <?php $submitted = isset($_POST['username']) && isset($_POST['password']); if ($submitted) { setcookie('username', $_POST['username']); } ?> <!DOCTYPE html> <html lang="en"> <head> <meta charset="UTF-8"> <title>Bookstore</title> </head> <body> <?php if ($submitted): ?> <p>Your login info is</p> <ul> <li><b>username</b>: <?php echo $_POST['username']; ?></li> <li><b>password</b>: <?php echo $_POST['password']; ?></li> </ul> <?php else: ?> <p>You did not submitted anything.</p> <?php endif; ?> </body> </html> This code also contains conditionals, which we already know. We are setting a variable to know whether we've submitted a login or not and to set the cookies if we have. However, the highlighted lines show you a new way of including conditionals with HTML. This way, tries to be more readable when working with HTML code, avoiding the use of {} and instead using : and endif. Both syntaxes are correct, and you should use the one that you consider more readable in each case. Switch-case Another control structure similar to if-else is switch-case. This structure evaluates only one expression and executes the block depending on its value. Let's see an example: <?php switch ($title) { case 'Harry Potter': echo "Nice story, a bit too long."; break; case 'Lord of the Rings': echo "A classic!"; break; default: echo "Dunno that one."; break; } The switch case takes an expression; in this case, a variable. It then defines a series of cases. When the case matches the current value of the expression, PHP executes the code inside it. As soon as PHP finds break, it will exit switch-case. In case none of the cases are suitable for the expression, if there is a default case         , PHP will execute it, but this is optional. You also need to know that breaks are mandatory if you want to exit switch-case. If you do not specify any, PHP will keep on executing statements, even if it encounters a new case. Let's see a similar example but without breaks: <?php $title = 'Twilight'; switch ($title) { case 'Harry Potter': echo "Nice story, a bit too long."; case 'Twilight': echo 'Uh...'; case 'Lord of the Rings': echo "A classic!"; default: echo "Dunno that one."; } If you test this code in your browser, you will see that it is printing "Uh...A classic!Dunno that one.". PHP found that the second case is valid so it executes its content. But as there are no breaks, it keeps on executing until the end. This might be the desired behavior sometimes, but not usually, so we need to be careful when using it! Loops Loops are control structures that allow you to execute certain statements several times—as many times as you need. You might use them on several different scenarios, but the most common one is when interacting with arrays. For example, imagine you have an array with elements but you do not know what is in it. You want to print all its elements so you loop through all of them. There are four types of loops. Each of them has their own use cases, but in general, you can transform one type of loop into another. Let's see them closely While While is the simplest of the loops. It executes a block of code until the expression to evaluate returns false. Let's see one example: <?php $i = 1; while ($i < 4) { echo $i . " "; $i++; } Here, we are defining a variable with the value 1. Then, we have a while clause in which the expression to evaluate is $i < 4. This loop will execute the content of the block of code until that expression is false. As you can see, inside the loop we are incrementing the value of $i by 1 each time, so after 4 iterations, the loop will end. Check out the output of that script, and you will see "0 1 2 3". The last value printed is 3, so by that time, $i was 3. After that, we increased its value to 4, so when the while was evaluating whether $i < 4, the result was false. Whiles and infinite loops One of the most common problems with while loops is creating an infinite loop. If you do not add any code inside while, which updates any of the variables considered in the while expression so it can be false at some point, PHP will never exit the loop! For This is the most complex of the four loops. For defines an initialization expression, an exit condition, and the end of the iteration expression. When PHP first encounters the loop, it executes what is defined as the initialization expression. Then, it evaluates the exit condition, and if it resolves to true, it enters the loop. After executing everything inside the loop, it executes the end of the iteration expression. Once this is done, it will evaluate the end condition again, going through the loop code and the end of iteration expression until it evaluates to false. As always, an example will help clarify this: <?php for ($i = 1; $i < 10; $i++) { echo $i . " "; } The initialization expression is $i = 1 and is executed only the first time. The exit condition is $i < 10, and it is evaluated at the beginning of each iteration. The end of the iteration expression is $i++, which is executed at the end of each iteration. This example prints numbers from 1 to 9. Another more common usage of the for loop is with arrays: <?php $names = ['Harry', 'Ron', 'Hermione']; for ($i = 0; $i < count($names); $i++) { echo $names[$i] . " "; } In this example, we have an array of names. As it is defined as a list, its keys will be 0, 1, and 2. The loop initializes the $i variable to 0, and it will iterate until the value of $i is not less than the amount of elements in the array 3 The first iteration $i is 0, the second will be 1, and the third one will be 2. When $i is 3, it will not enter the loop as the exit condition evaluates to false. On each iteration, we are printing the content of the $i position of the array; hence, the result of this code will be all three names in the array. We careful with exit conditions It is very common to set an exit condition. This is not exactly what we need, especially with arrays. Remember that arrays start with 0 if they are a list, so an array of 3 elements will have entries 0, 1, and 2. Defining the exit condition as $i <= count($array) will cause an error on your code, as when $i is 3, it also satisfies the exit condition and will try to access to the key 3, which does not exist. Foreach The last, but not least, type of loop is foreach. This loop is exclusive for arrays, and it allows you to iterate an array entirely, even if you do not know its keys. There are two options for the syntax, as you can see in these examples: <?php $names = ['Harry', 'Ron', 'Hermione']; foreach ($names as $name) { echo $name . " "; } foreach ($names as $key => $name) { echo $key . " -> " . $name . " "; } The foreach loop accepts an array; in this case, $names. It specifies a variable, which will contain the value of the entry of the array. You can see that we do not need to specify any end condition, as PHP will know when the array has been iterated. Optionally, you can specify a variable that will contain the key of each iteration, as in the second loop. Foreach loops are also useful with maps, where the keys are not necessarily numeric. The order in which PHP will iterate the array will be the same order in which you used to insert the content in the array. Let's use some loops in our application. We want to show the available books in our home page. We have the list of books in an array, so we will have to iterate all of them with a foreach loop, printing some information from each one. Append the following code to the body tag in index.php: <?php endif; $books = [ [ 'title' => 'To Kill A Mockingbird', 'author' => 'Harper Lee', 'available' => true, 'pages' => 336, 'isbn' => 9780061120084 ], [ 'title' => '1984', 'author' => 'George Orwell', 'available' => true, 'pages' => 267, 'isbn' => 9780547249643 ], [ 'title' => 'One Hundred Years Of Solitude', 'author' => 'Gabriel Garcia Marquez', 'available' => false, 'pages' => 457, 'isbn' => 9785267006323 ], ]; ?> <ul> <?php foreach ($books as $book): ?> <li> <i><?php echo $book['title']; ?></i> - <?php echo $book['author']; ?> <?php if (!$book['available']): ?> <b>Not available</b> <?php endif; ?> </li> <?php endforeach; ?> </ul> The highlighted code shows a foreach loop using the : notation, which is better when mixing it with HTML. It iterates all the $books arrays, and for each book, it will print some information as a HTML list. Also note that we have a conditional inside a loop, which is perfectly fine. Of course, this conditional will be executed for each entry in the array, so you should keep the block of code of your loops as simple as possible. Functions A function is a reusable block of code that, given an input, performs some actions and optionally returns a result. You already know several predefined functions, such as empty, in_array, or var_dump. These functions come with PHP so you do not have to reinvent the wheel, but you can create your own very easily. You can define functions when you identify portions of your application that have to be executed several times or just to encapsulate some functionality. Function declaration Declaring a function means to write it down so that it can be used later. A function has a name, takes arguments, and has a block of code. Optionally, it can define what kind of value is returning. The name of the function has to follow the same rules as variable names; that is, it has to start by a letter or underscore and can contain any letter, number, or underscore. It cannot be a reserved word. Let's see a simple example: function addNumbers($a, $b) { $sum = $a + $b; return $sum; } $result = addNumbers(2, 3); Here, the function's name is addNumbers, and it takes two arguments: $a and $b. The block of code defines a new variable $sum that is the sum of both the arguments and then returns its content with return. In order to use this function, you just need to call it by its name, sending all the required arguments, as shown in the highlighted line. PHP does not support overloaded functions. Overloading refers to the ability of declaring two or more functions with the same name but different arguments. As you can see, you can declare the arguments without knowing what their types are, so PHP would not be able to decide which function to use. Another important thing to note is the variable scope. We are declaring a $sum variable inside the block of code, so once the function ends, the variable will not be accessible any more. This means that the scope of variables declared inside the function is just the function itself. Furthermore, if you had a $sum variable declared outside the function, it would not be affected at all since the function cannot access that variable unless we send it as an argument. Function arguments A function gets information from outside via arguments. You can define any number of arguments—including 0. These arguments need at least a name so that they can be used inside the function, and there cannot be two arguments with the same name. When invoking the function, you need to send the arguments in the same order as we declared them. A function may contain optional arguments; that is, you are not forced to provide a value for those arguments. When declaring the function, you need to provide a default value for those arguments, so in case the user does not provide a value, the function will use the default one: function addNumbers($a, $b, $printResult = false) { $sum = $a + $b; if ($printResult) { echo 'The result is ' . $sum; } return $sum; } $sum1 = addNumbers(1, 2); $sum1 = addNumbers(3, 4, false); $sum1 = addNumbers(5, 6, true); // it will print the result This new function takes two mandatory arguments and an optional one. The default value is false, and is used as a normal value inside the function. The function will print the result of the sum if the user provides true as the third argument, which happens only the third time that the function is invoked. For the first two times, $printResult is set to false. The arguments that the function receives are just copies of the values that the user provided. This means that if you modify these arguments inside the function, it will not affect the original values. This feature is known as sending arguments by a value. Let's see an example: function modify($a) { $a = 3; } $a = 2; modify($a); var_dump($a); // prints 2 We are declaring the $a variable with the value 2, and then we are calling the modify method, sending $a. The modify method modifies the $a argument, setting its value to 3. However, this does not affect the original value of $a, which reminds to 2 as you can see in the var_dump function. If what you want is to actually change the value of the original variable used in the invocation, you need to pass the argument by reference. To do that, you add & in front of the argument when declaring the function: function modify(&$a) { $a = 3; } Now, after invoking the modify function, $a will be always 3. Arguments by value versus by reference PHP allows you to do it, and in fact, some native functions of PHP use arguments by reference—remember the array sorting functions; they did not return the sorted array; instead, they sorted the array provided. But using arguments by reference is a way of confusing developers. Usually, when someone uses a function, they expect a result, and they do not want their provided arguments to be modified. So, try to avoid it; people will be grateful! The return statement You can have as many return statements as you want inside your function, but PHP will exit the function as soon as it finds one. This means that if you have two consecutive return statements, the second one will never be executed. Still, having multiple return statements can be useful if they are inside conditionals. Add this function inside your functions.php file: function loginMessage() { if (isset($_COOKIE['username'])) { return "You are " . $_COOKIE['username']; } else { return "You are not authenticated."; } } Let's use it in your index.php file by replacing the highlighted content—note that to save some tees, I replaced most of the code that was not changed at all with //…: //... <body> <p><?php echo loginMessage(); ?></p> <?php if (isset($_GET['title']) && isset($_GET['author'])): ?> //... Additionally, you can omit the return statement if you do not want the function to return anything. In this case, the function will end once it reaches the end of the block of code. Type hinting and return types With the release of PHP7, the language allows developers to be more specific about what functions get and return. You can—always optionally—specify the type of argument that the function needs, for example, type hinting, and the type of result the function will return—return type. Let's first see an example: <?php declare(strict_types=1); function addNumbers(int $a, int $b, bool $printSum): int { $sum = $a + $b; if ($printSum) { echo 'The sum is ' . $sum; } return $sum; } addNumbers(1, 2, true); addNumbers(1, '2', true); // it fails when strict_types is 1 addNumbers(1, 'something', true); // it always fails This function states that the arguments need to be an integer, and Boolean, and that the result will be an integer. Now, you know that PHP has type juggling, so it can usually transform a value of one type to its equivalent value of another type, for example, the string 2 can be used as integer 2. To stop PHP from using type juggling with the arguments and results of functions, you can declare the strict_types directive as shown in the first highlighted line. This directive has to be declared on the top of each file, where you want to enforce this behavior. The three invocations work as follows: The first invocation sends two integers and a Boolean, which is what the function expects. So, regardless of the value of strict_types, it will always work. The second invocation sends an integer, a string, and a Boolean. The string has a valid integer value, so if PHP was allowed to use type juggling, the invocation would resolve to just normal. But in this example, it will fail because of the declaration on top of the file. The third invocation will always fail as the something string cannot be transformed into a valid integer. Let's try to use a function within our project. In our index.php file, we have a foreach loop that iterates the books and prints them. The code inside the loop is kind of hard to understand as it is mixing HTML with PHP, and there is a conditional too. Let's try to abstract the logic inside the loop into a function. First, create the new functions.php file with the following content: <?php function printableTitle(array $book): string { $result = '<i>' . $book['title'] . '</i> - ' . $book['author']; if (!$book['available']) { $result .= ' <b>Not available</b>'; } return $result; } This file will contain our functions. The first one, printableTitle, takes an array representing a book and builds a string with a nice representation of the book in HTML. The code is the same as before, just encapsulated in a function. Now, index.php will have to include the functions.php file and then use the function inside the loop. Let's see how this is done: <?php require_once 'functions.php' ?> <!DOCTYPE html> <html lang="en"> //... ?> <ul> <?php foreach ($books as $book): ?> <li><?php echo printableTitle($book); ?> </li> <?php endforeach; ?> </ul> //... Well, now our loop looks way cleaner, right? Also, if we need to print the title of the book somewhere else, we can reuse the function instead of duplicating code! Summary In this article, we went through all the basics of procedural PHP while writing simple examples in order to practice them. You now know how to use variables and arrays with control structures and functions and how to get information from HTTP requests among others. Resources for Article: Further resources on this subject: Getting started with Modernizr using PHP IDE[article] PHP 5 Social Networking: Implementing Public Messages[article] Working with JSON in PHP jQuery[article]
Read more
  • 0
  • 0
  • 16342

article-image-training-and-visualizing-a-neural-network-with-r
Oli Huggins
16 Feb 2016
8 min read
Save for later

Training and Visualizing a neural network with R

Oli Huggins
16 Feb 2016
8 min read
The development of a neural network is inspired by human brain activities. As such, this type of network is a computational model that mimics the pattern of the human mind. In contrast to this, support vector machines first, map input data into a high dimensional feature space defined by the kernel function, and find the optimum hyperplane that separates the training data by the maximum margin. In short, we can think of support vector machines as a linear algorithm in a high dimensional space. In this article, we will cover: Training a neural network with neuralnet Visualizing a neural network trained by neuralnet (For more resources related to this topic, see here.) Training a neural network with neuralnet The neural network is constructed with an interconnected group of nodes, which involves the input, connected weights, processing element, and output. Neural networks can be applied to many areas, such as classification, clustering, and prediction. To train a neural network in R, you can use neuralnet, which is built to train multilayer perceptron in the context of regression analysis, and contains many flexible functions to train forward neural networks. In this recipe, we will introduce how to use neuralnet to train a neural network. Getting ready In this recipe, we will use an iris dataset as our example dataset. We will first split the irisdataset into a training and testing datasets, respectively. How to do it... Perform the following steps to train a neural network with neuralnet: First load the iris dataset and split the data into training and testing datasets: > data(iris) > ind <- sample(2, nrow(iris), replace = TRUE, prob=c(0.7, 0.3)) > trainset = iris[ind == 1,]> testset = iris[ind == 2,] Then, install and load the neuralnet package: > install.packages("neuralnet")> library(neuralnet) Add the columns versicolor, setosa, and virginica based on the name matched value in the Species column: > trainset$setosa = trainset$Species == "setosa" > trainset$virginica = trainset$Species == "virginica" > trainset$versicolor = trainset$Species == "versicolor" Next, train the neural network with the neuralnet function with three hidden neurons in each layer. Notice that the results may vary with each training, so you might not get the same result: > network = neuralnet(versicolor + virginica + setosa~ Sepal.Length + Sepal.Width + Petal.Length + Petal.Width, trainset, hidden=3) > network Call: neuralnet(formula = versicolor + virginica + setosa ~ Sepal.Length + Sepal.Width + Petal.Length + Petal.Width, data = trainset, hidden = 3) 1 repetition was calculated. Error Reached Threshold Steps 1 0.8156100175 0.009994274769 11063 Now, you can view the summary information by accessing the result.matrix attribute of the built neural network model: > network$result.matrix 1 error 0.815610017474 reached.threshold 0.009994274769 steps 11063.000000000000 Intercept.to.1layhid1 1.686593311644 Sepal.Length.to.1layhid1 0.947415215237 Sepal.Width.to.1layhid1 -7.220058260187 Petal.Length.to.1layhid1 1.790333443486 Petal.Width.to.1layhid1 9.943109233330 Intercept.to.1layhid2 1.411026063895 Sepal.Length.to.1layhid2 0.240309549505 Sepal.Width.to.1layhid2 0.480654059973 Petal.Length.to.1layhid2 2.221435192437 Petal.Width.to.1layhid2 0.154879347818 Intercept.to.1layhid3 24.399329878242 Sepal.Length.to.1layhid3 3.313958088512 Sepal.Width.to.1layhid3 5.845670010464 Petal.Length.to.1layhid3 -6.337082722485 Petal.Width.to.1layhid3 -17.990352566695 Intercept.to.versicolor -1.959842102421 1layhid.1.to.versicolor 1.010292389835 1layhid.2.to.versicolor 0.936519720978 1layhid.3.to.versicolor 1.023305801833 Intercept.to.virginica -0.908909982893 1layhid.1.to.virginica -0.009904635231 1layhid.2.to.virginica 1.931747950462 1layhid.3.to.virginica -1.021438938226 Intercept.to.setosa 1.500533827729 1layhid.1.to.setosa -1.001683936613 1layhid.2.to.setosa -0.498758815934 1layhid.3.to.setosa -0.001881935696 Lastly, you can view the generalized weight by accessing it in the network: > head(network$generalized.weights[[1]]) How it works... The neural network is a network made up of artificial neurons (or nodes). There are three types of neurons within the network: input neurons, hidden neurons, and output neurons. In the network, neurons are connected; the connection strength between neurons is called weights. If the weight is greater than zero, it is in an excitation status. Otherwise, it is in an inhibition status. Input neurons receive the input information; the higher the input value, the greater the activation. Then, the activation value is passed through the network in regard to weights and transfer functions in the graph. The hidden neurons (or output neurons) then sum up the activation values and modify the summed values with the transfer function. The activation value then flows through hidden neurons and stops when it reaches the output nodes. As a result, one can use the output value from the output neurons to classify the data. Artificial Neural Network The advantages of a neural network are: firstly, it can detect a nonlinear relationship between the dependent and independent variable. Secondly, one can efficiently train large datasets using the parallel architecture. Thirdly, it is a nonparametric model so that one can eliminate errors in the estimation of parameters. The main disadvantages of neural network are that it often converges to the local minimum rather than the global minimum. Also, it might over-fit when the training process goes on for too long. In this recipe, we demonstrate how to train a neural network. First, we split the iris dataset into training and testing datasets, and then install the neuralnet package and load the library into an R session. Next, we add the columns versicolor, setosa, and virginica based on the name matched value in the Species column, respectively. We then use the neuralnet function to train the network model. Besides specifying the label (the column where the name equals to versicolor, virginica, and setosa) and training attributes in the function, we also configure the number of hidden neurons (vertices) as three in each layer. Then, we examine the basic information about the training process and the trained network saved in the network. From the output message, it shows the training process needed 11,063 steps until all the absolute partial derivatives of the error function were lower than 0.01 (specified in the threshold). The error refers to the likelihood of calculating Akaike Information Criterion (AIC). To see detailed information on this, you can access the result.matrix of the built neural network to see the estimated weight. The output reveals that the estimated weight ranges from -18 to 24.40; the intercepts of the first hidden layer are 1.69, 1.41 and 24.40, and the two weights leading to the first hidden neuron are estimated as 0.95 (Sepal.Length), -7.22 (Sepal.Width), 1.79 (Petal.Length), and 9.94 (Petal.Width). We can lastly determine that the trained neural network information includes generalized weights, which express the effect of each covariate. In this recipe, the model generates 12 generalized weights, which are the combination of four covariates (Sepal.Length, Sepal.Width, Petal.Length, Petal.Width) to three responses (setosa, virginica, versicolor). See also For a more detailed introduction on neuralnet, one can refer to the following paper: Günther, F., and Fritsch, S. (2010). neuralnet: Training of neural networks. The R journal, 2(1), 30-38. Visualizing a neural network trained by neuralnet The package, neuralnet, provides the plot function to visualize a built neural network and the gwplot function to visualize generalized weights. In following recipe, we will cover how to use these two functions. Getting ready You need to have completed the previous recipe by training a neural network and have all basic information saved in network. How to do it... Perform the following steps to visualize the neural network and the generalized weights: You can visualize the trained neural network with the plot function: > plot(network) Figure 10: The plot of trained neural network Furthermore, You can use gwplot to visualize the generalized weights: > par(mfrow=c(2,2)) > gwplot(network,selected.covariate="Petal.Width") > gwplot(network,selected.covariate="Sepal.Width") > gwplot(network,selected.covariate="Petal.Length") > gwplot(network,selected.covariate="Petal.Width") Figure 11: The plot of generalized weights How it works... In this recipe, we demonstrate how to visualize the trained neural network and the generalized weights of each trained attribute. Also, the plot includes the estimated weight, intercepts and basic information about the training process. At the bottom of the figure, one can find the overall error and number of steps required to converge. If all the generalized weights are close to zero on the plot, it means the covariate has little effect. However, if the overall variance is greater than one, it means the covariate has a nonlinear effect. See also For more information about gwplot, one can use the help function to access the following document: > ?gwplot Summary To learn more about machine learning with R, the following books published by Packt Publishing (https://www.packtpub.com/) are recommended: Machine Learning with R (Second Edition) - Read Online Mastering Machine Learning with R - Read Online Resources for Article: Further resources on this subject: Introduction to Machine Learning with R [article] Hive Security [article] Spark - Architecture and First Program [article]
Read more
  • 0
  • 0
  • 14821
article-image-writing-blog-application-nodejs-and-angularjs
Packt
16 Feb 2016
35 min read
Save for later

Writing a Blog Application with Node.js and AngularJS

Packt
16 Feb 2016
35 min read
In this article, we are going to build a blog application by using Node.js and AngularJS. Our system will support adding, editing, and removing articles, so there will be a control panel. The MongoDB or MySQL database will handle the storing of the information and the Express framework will be used as the site base. It will deliver the JavaScript, CSS, and the HTML to the end user, and will provide an API to access the database. We will use AngularJS to build the user interface and control the client-side logic in the administration page. (For more resources related to this topic, see here.) This article will cover the following topics: AngularJS fundamentals Choosing and initializing a database Implementing the client-side part of an application with AngularJS Exploring AngularJS AngularJS is an open source, client-side JavaScript framework developed by Google. It's full of features and is really well documented. It has almost become a standard framework in the development of single-page applications. The official site of AngularJS, http://angularjs.org, provides a well-structured documentation. As the framework is widely used, there is a lot of material in the form of articles and video tutorials. As a JavaScript library, it collaborates pretty well with Node.js. In this article, we will build a simple blog with a control panel. Before we start developing our application, let's first take a look at the framework. AngularJS gives us very good control over the data on our page. We don't have to think about selecting elements from the DOM and filling them with values. Thankfully, due to the available data-binding, we may update the data in the JavaScript part and see the change in the HTML part. This is also true for the reverse. Once we change something in the HTML part, we get the new values in the JavaScript part. The framework has a powerful dependency injector. There are predefined classes in order to perform AJAX requests and manage routes. You could also read Mastering Web Development with AngularJS by Peter Bacon Darwin and Pawel Kozlowski, published by Packt Publishing. Bootstrapping AngularJS applications To bootstrap an AngularJS application, we need to add the ng-app attribute to some of our HTML tags. It is important that we pick the right one. Having ng-app somewhere means that all the child nodes will be processed by the framework. It's common practice to put that attribute on the <html> tag. In the following code, we have a simple HTML page containing ng-app: <html ng-app> <head> <script src="angular.min.js"></script> </head> <body> ... </body> </html>   Very often, we will apply a value to the attribute. This will be a module name. We will do this while developing the control panel of our blog application. Having the freedom to place ng-app wherever we want means that we can decide which part of our markup will be controlled by AngularJS. That's good, because if we have a giant HTML file, we really don't want to spend resources parsing the whole document. Of course, we may bootstrap our logic manually, and this is needed when we have more than one AngularJS application on the page. Using directives and controllers In AngularJS, we can implement the Model-View-Controller pattern. The controller acts as glue between the data (model) and the user interface (view). In the context of the framework, the controller is just a simple function. For example, the following HTML code illustrates that a controller is just a simple function: <html ng-app> <head> <script src="angular.min.js"></script> <script src="HeaderController.js"></script> </head> <body> <header ng-controller="HeaderController"> <h1>{{title}}</h1> </header> </body> </html>   In <head> of the page, we are adding the minified version of the library and HeaderController.js; a file that will host the code of our controller. We also set an ng-controller attribute in the HTML markup. The definition of the controller is as follows: function HeaderController($scope) { $scope.title = "Hello world"; } Every controller has its own area of influence. That area is called the scope. In our case, HeaderController defines the {{title}} variable. AngularJS has a wonderful dependency-injection system. Thankfully, due to this mechanism, the $scope argument is automatically initialized and passed to our function. The ng-controller attribute is called the directive, that is, an attribute, which has meaning to AngularJS. There are a lot of directives that we can use. That's maybe one of the strongest points of the framework. We can implement complex logic directly inside our templates, for example, data binding, filtering, or modularity. Data binding Data binding is a process of automatically updating the view once the model is changed. As we mentioned earlier, we can change a variable in the JavaScript part of the application and the HTML part will be automatically updated. We don't have to create a reference to a DOM element or attach event listeners. Everything is handled by the framework. Let's continue and elaborate on the previous example, as follows: <header ng-controller="HeaderController"> <h1>{{title}}</h1> <a href="#" ng-click="updateTitle()">change title</a> </header>   A link is added and it contains the ng-click directive. The updateTitle function is a function defined in the controller, as seen in the following code snippet: function HeaderController($scope) { $scope.title = "Hello world"; $scope.updateTitle = function() { $scope.title = "That's a new title."; } }   We don't care about the DOM element and where the {{title}} variable is. We just change a property of $scope and everything works. There are, of course, situations where we will have the <input> fields and we want to bind their values. If that's the case, then the ng-model directive can be used. We can see this as follows: <header ng-controller="HeaderController"> <h1>{{title}}</h1> <a href="#" ng-click="updateTitle()">change title</a> <input type="text" ng-model="title" /> </header>   The data in the input field is bound to the same title variable. This time, we don't have to edit the controller. AngularJS automatically changes the content of the h1 tag. Encapsulating logic with modules It's great that we have controllers. However, it's not a good practice to place everything into globally defined functions. That's why it is good to use the module system. The following code shows how a module is defined: angular.module('HeaderModule', []); The first parameter is the name of the module and the second one is an array with the module's dependencies. By dependencies, we mean other modules, services, or something custom that we can use inside the module. It should also be set as a value of the ng-app directive. The code so far could be translated to the following code snippet: angular.module('HeaderModule', []) .controller('HeaderController', function($scope) { $scope.title = "Hello world"; $scope.updateTitle = function() { $scope.title = "That's a new title."; } });   So, the first line defines a module. We can chain the different methods of the module and one of them is the controller method. Following this approach, that is, putting our code inside a module, we will be encapsulating logic. This is a sign of good architecture. And of course, with a module, we have access to different features such as filters, custom directives, and custom services. Preparing data with filters The filters are very handy when we want to prepare our data, prior to be displayed to the user. Let's say, for example, that we need to mention our title in uppercase once it reaches a length of more than 20 characters: angular.module('HeaderModule', []) .filter('customuppercase', function() { return function(input) { if(input.length > 20) { return input.toUpperCase(); } else { return input; } }; }) .controller('HeaderController', function($scope) { $scope.title = "Hello world"; $scope.updateTitle = function() { $scope.title = "That's a new title."; } });   That's the definition of the custom filter called customuppercase. It receives the input and performs a simple check. What it returns, is what the user sees at the end. Here is how this filter could be used in HTML: <h1>{{title | customuppercase}}</h1> Of course, we may add more than one filter per variable. There are some predefined filters to limit the length, such as the JavaScript to JSON conversion or, for example, date formatting. Dependency injection Dependency management can be very tough sometimes. We may split everything into different modules/components. They have nicely written APIs and they are very well documented. However, very soon, we may realize that we need to create a lot of objects. Dependency injection solves this problem by providing what we need, on the fly. We already saw this in action. The $scope parameter passed to our controller, is actually created by the injector of AngularJS. To get something as a dependency, we need to define it somewhere and let the framework know about it. We do this as follows: angular.module('HeaderModule', []) .factory("Data", function() { return { getTitle: function() { return "A better title."; } } }) .controller('HeaderController', function($scope, Data) { $scope.title = Data.getTitle(); $scope.updateTitle = function() { $scope.title = "That's a new title."; } });   The Module class has a method called factory. It registers a new service that could later be used as a dependency. The function returns an object with only one method, getTitle. Of course, the name of the service should match the name of the controller's parameter. Otherwise, AngularJS will not be able to find the dependency's source. The model in the context of AngularJS In the well-known Model-View-Controller pattern, the model is the part that stores the data in the application. AngularJS doesn't have a specific workflow to define models. The $scope variable could be considered a model. We keep the data in properties attached to the current scope. Later, we can use the ng-model directive and bind a property to the DOM element. We already saw how this works in the previous sections. The framework may not provide the usual form of a model, but it's made like that so that we can write our own implementation. The fact that AngularJS works with plain JavaScript objects, makes this task easily doable. Final words on AngularJS AngularJS is one of the leading frameworks, not only because it is made by Google, but also because it's really flexible. We could use just a small piece of it or build a solid architecture using the giant collection of features. Selecting and initializing the database To build a blog application, we need a database that will store the published articles. In most cases, the choice of the database depends on the current project. There are factors such as performance and scalability and we should keep them in mind. In order to have a better look at the possible solutions, we will have a look at the two of the most popular databases: MongoDB and MySQL. The first one is a NoSQL type of database. According to the Wikipedia entry (http://en.wikipedia.org/wiki/ NoSQL) on NoSQL databases: "A NoSQL or Not Only SQL database provides a mechanism for storage and retrieval of data that is modeled in means other than the tabular relations used in relational databases." In other words, it's simpler than a SQL database, and very often stores information in the key value type. Usually, such solutions are used when handling and storing large amounts of data. It is also a very popular approach when we need flexible schema or when we want to use JSON. It really depends on what kind of system we are building. In some cases, MySQL could be a better choice, while in some other cases, MongoDB. In our example blog, we're going to use both. In order to do this, we will need a layer that connects to the database server and accepts queries. To make things a bit more interesting, we will create a module that has only one API, but can switch between the two database models. Using NoSQL with MongoDB Let's start with MongoDB. Before we start storing information, we need a MongoDB server running. It can be downloaded from the official page of the database https://www.mongodb.org/downloads. We are not going to handle the communication with the database manually. There is a driver specifically developed for Node.js. It's called mongodb and we should include it in our package.json file. After successful installation via npm install, the driver will be available in our scripts. We can check this as follows: "dependencies": { "mongodb": "1.3.20" }   We will stick to the Model-View-Controller architecture and the database-related operations in a model called Articles. We can see this as follows: var crypto = require("crypto"), type = "mongodb", client = require('mongodb').MongoClient, mongodb_host = "127.0.0.1", mongodb_port = "27017", collection; module.exports = function() { if(type == "mongodb") { return { add: function(data, callback) { ... }, update: function(data, callback) { ... }, get: function(callback) { ... }, remove: function(id, callback) { ... } } } else { return { add: function(data, callback) { ... }, update: function(data, callback) { ... }, get: function(callback) { ... }, remove: function(id, callback) { ... } } } }   It starts with defining a few dependencies and settings for the MongoDB connection. Line number one requires the crypto module. We will use it to generate unique IDs for every article. The type variable defines which database is currently accessed. The third line initializes the MongoDB driver. We will use it to communicate with the database server. After that, we set the host and port for the connection and at the end a global collection variable, which will keep a reference to the collection with the articles. In MongoDB, the collections are similar to the tables in MySQL. The next logical step is to establish a database connection and perform the needed operations, as follows: connection = 'mongodb://'; connection += mongodb_host + ':' + mongodb_port; connection += '/blog-application'; client.connect(connection, function(err, database) { if(err) { throw new Error("Can't connect"); } else { console.log("Connection to MongoDB server successful."); collection = database.collection('articles'); } });   We pass the host and the port, and the driver is doing everything else. Of course, it is a good practice to handle the error (if any) and throw an exception. In our case, this is especially needed because without the information in the database, the frontend has nothing to show. The rest of the module contains methods to add, edit, retrieve, and delete records: return { add: function(data, callback) { var date = new Date(); data.id = crypto.randomBytes(20).toString('hex'); data.date = date.getFullYear() + "-" + date.getMonth() + "-" + date.getDate(); collection.insert(data, {}, callback || function() {}); }, update: function(data, callback) { collection.update( {ID: data.id}, data, {}, callback || function(){ } ); }, get: function(callback) { collection.find({}).toArray(callback); }, remove: function(id, callback) { collection.findAndModify( {ID: id}, [], {}, {remove: true}, callback ); } }   The add and update methods accept the data parameter. That's a simple JavaScript object. For example, see the following code: { title: "Blog post title", text: "Article's text here ..." }   The records are identified by an automatically generated unique id. The update method needs it in order to find out which record to edit. All the methods also have a callback. That's important, because the module is meant to be used as a black box, that is, we should be able to create an instance of it, operate with the data, and at the end continue with the rest of the application's logic. Using MySQL We're going to use an SQL type of database with MySQL. We will add a few more lines of code to the already working Articles.js model. The idea is to have a class that supports the two databases like two different options. At the end, we should be able to switch from one to the other, by simply changing the value of a variable. Similar to MongoDB, we need to first install the database to be able use it. The official download page is http://www.mysql.com/downloads. MySQL requires another Node.js module. It should be added again to the package. json file. We can see the module as follows: "dependencies": { "mongodb": "1.3.20", "mysql": "2.0.0" }   Similar to the MongoDB solution, we need to firstly connect to the server. To do so, we need to know the values of the host, username, and password fields. And because the data is organized in databases, a name of the database. In MySQL, we put our data into different databases. So, the following code defines the needed variables: var mysql = require('mysql'), mysql_host = "127.0.0.1", mysql_user = "root", mysql_password = "", mysql_database = "blog_application", connection;   The previous example leaves the password field empty but we should set the proper value of our system. The MySQL database requires us to define a table and its fields before we start saving data. So, the following code is a short dump of the table used in this article: CREATE TABLE IF NOT EXISTS `articles` ( `id` int(11) NOT NULL AUTO_INCREMENT, `title` longtext NOT NULL, `text` longtext NOT NULL, `date` varchar(100) NOT NULL, PRIMARY KEY (`id`) ) ENGINE=InnoDB DEFAULT CHARSET=utf8 AUTO_INCREMENT=1;   Once we have a database and its table set, we can continue with the database connection, as follows: connection = mysql.createConnection({ host: mysql_host, user: mysql_user, password: mysql_password }); connection.connect(function(err) { if(err) { throw new Error("Can't connect to MySQL."); } else { connection.query("USE " + mysql_database, function(err, rows, fields) { if(err) { throw new Error("Missing database."); } else { console.log("Successfully selected database."); } }) } });   The driver provides a method to connect to the server and execute queries. The first executed query selects the database. If everything is ok, you should see Successfully selected database as an output in your console. Half of the job is done. What we should do now is replicate the methods returned in the first MongoDB implementation. We need to do this because when we switch to the MySQL usage, the code using the class will not work. And by replicating them we mean that they should have the same names and should accept the same arguments. If we do everything correctly, at the end our application will support two types of databases. And all we have to do is change the value of the type variable: return { add: function(data, callback) { var date = new Date(); var query = ""; query += "INSERT INTO articles (title, text, date) VALUES ("; query += connection.escape(data.title) + ", "; query += connection.escape(data.text) + ", "; query += "'" + date.getFullYear() + "-" + date.getMonth() + "-" + date.getDate() + "'"; query += ")"; connection.query(query, callback); }, update: function(data, callback) { var query = "UPDATE articles SET "; query += "title=" + connection.escape(data.title) + ", "; query += "text=" + connection.escape(data.text) + " "; query += "WHERE id='" + data.id + "'"; connection.query(query, callback); }, get: function(callback) { var query = "SELECT * FROM articles ORDER BY id DESC"; connection.query(query, function(err, rows, fields) { if (err) { throw new Error("Error getting."); } else { callback(rows); } }); }, remove: function(id, callback) { var query = "DELETE FROM articles WHERE id='" + id + "'"; connection.query(query, callback); } }   The code is a little longer than the one generated in the first MongoDB variant. That's because we needed to construct MySQL queries from the passed data. Keep in mind that we have to escape the information, which comes to the module. That's why we use connection.escape(). With these lines of code, our model is completed. Now we can add, edit, remove, or get data. Let's continue with the part that shows the articles to our users. Developing the client side with AngularJS Let's assume that there is some data in the database and we are ready to present it to the users. So far, we have only developed the model, which is the class that takes care of the access to the information. To simplify the process, we will Express here. We need to first update the package.json file and include that in the framework, as follows: "dependencies": { "express": "3.4.6", "jade": "0.35.0", "mongodb": "1.3.20", "mysql": "2.0.0" }   We are also adding Jade, because we are going to use it as a template language. The writing of markup in plain HTML is not very efficient nowadays. By using the template engine, we can split the data and the HTML markup, which makes our application much better structured. Jade's syntax is kind of similar to HTML. We can write tags without the need to close them: body p(class="paragraph", data-id="12"). Sample text here footer a(href="#"). my site   The preceding code snippet is transformed to the following code snippet: <body> <p data-id="12" class="paragraph">Sample text here</p> <footer><a href="#">my site</a></footer> </body>   Jade relies on the indentation in the content to distinguish the tags. Let's start with the project structure, as seen in the following screenshot: We placed our already written class, Articles.js, inside the models directory. The public directory will contain CSS styles, and all the necessary client-side JavaScript: the AngularJS library, the AngularJS router module, and our custom code. We will skip some of the explanations about the following code. Our index.js file looks as follows: var express = require('express'); var app = express(); var articles = require("./models/Articles")(); app.set('views', __dirname + '/views'); app.set('view engine', 'jade'); app.use(express.static(__dirname + '/public')); app.use(function(req, res, next) { req.articles = articles; next(); }); app.get('/api/get', require("./controllers/api/get")); app.get('/', require("./controllers/index")); app.listen(3000); console.log('Listening on port 3000');   At the beginning, we require the Express framework and our model. Maybe it's better to initialize the model inside the controller, but in our case this is not necessary. Just after that, we set up some basic options for Express and define our own middleware. It has only one job to do and that is to attach the model to the request object. We are doing this because the request object is passed to all the route handlers. In our case, these handlers are actually the controllers. So, Articles.js becomes accessible everywhere via the req.articles property. At the end of the script, we placed two routes. The second one catches the usual requests that come from the users. The first one, /api/get, is a bit more interesting. We want to build our frontend on top of AngularJS. So, the data that is stored in the database should not enter the Node.js part but on the client side where we use Google's framework. To make this possible, we will create routes/controllers to get, add, edit, and delete records. Everything will be controlled by HTTP requests performed by AngularJS. In other words, we need an API. Before we start using AngularJS, let's take a look at the /controllers/api/get.js controller: module.exports = function(req, res, next) { req.articles.get(function(rows) { res.send(rows); }); }   The main job is done by our model and the response is handled by Express. It's nice because if we pass a JavaScript object, as we did, (rows is actually an array of objects) the framework sets the response headers automatically. To test the result, we could run the application with node index.js and open http://localhost:3000/api/ get. If we don't have any records in the database, we will get an empty array. If not, the stored articles will be returned. So, that's the URL, which we should hit from within the AngularJS controller in order to get the information. The code of the /controller/index.js controller is also just a few lines. We can see the code as follows: module.exports = function(req, res, next) { res.render("list", { app: "" }); }   It simply renders the list view, which is stored in the list.jade file. That file should be saved in the /views directory. But before we see its code, we will check another file, which acts as a base for all the pages. Jade has a nice feature called blocks. We may define different partials and combine them into one template. The following is our layout.jade file: doctype html html(ng-app="#{app}") head title Blog link(rel='stylesheet', href='/style.css') script(src='/angular.min.js') script(src='/angular-route.min.js') body block content   There is only one variable passed to this template, which is #{app}. We will need it later to initialize the administration's module. The angular.min.js and angular-route.min.js files should be downloaded from the official AngularJS site, and placed in the /public directory. The body of the page contains a block placeholder called content, which we will later fill with the list of the articles. The following is the list.jade file: extends layout block content .container(ng-controller="BlogCtrl") section.articles article(ng-repeat="article in articles") h2 {{article.title}} br small published on {{article.date}} p {{article.text}} script(src='/blog.js')   The two lines in the beginning combine both the templates into one page. The Express framework transforms the Jade template into HTML and serves it to the browser of the user. From there, the client-side JavaScript takes control. We are using the ng-controller directive saying that the div element will be controlled by an AngularJS controller called BlogCtrl. The same class should have variable, articles, filled with the information from the database. ng-repeat goes through the array and displays the content to the users. The blog.js class holds the code of the controller: function BlogCtrl($scope, $http) { $scope.articles = [ { title: "", text: "Loading ..."} ]; $http({method: 'GET', url: '/api/get'}) .success(function(data, status, headers, config) { $scope.articles = data; }) .error(function(data, status, headers, config) { console.error("Error getting articles."); }); }   The controller has two dependencies. The first one, $scope, points to the current view. Whatever we assign as a property there is available as a variable in our HTML markup. Initially, we add only one element, which doesn't have a title, but has text. It is shown to indicate that we are still loading the articles from the database. The second dependency, $http, provides an API in order to make HTTP requests. So, all we have to do is query /api/get, fetch the data, and pass it to the $scope dependency. The rest is done by AngularJS and its magical two-way data binding. To make the application a little more interesting, we will add a search field, as follows: // views/list.jade header .search input(type="text", placeholder="type a filter here", ng-model="filterText") h1 Blog hr   The ng-model directive, binds the value of the input field to a variable inside our $scope dependency. However, this time, we don't have to edit our controller and can simply apply the same variable as a filter to the ng-repeat: article(ng-repeat="article in articles | filter:filterText") As a result, the articles shown will be filtered based on the user's input. Two simple additions, but something really valuable is on the page. The filters of AngularJS can be very powerful. Implementing a control panel The control panel is the place where we will manage the articles of the blog. Several things should be made in the backend before continuing with the user interface. They are as follows: app.set("username", "admin"); app.set("password", "pass"); app.use(express.cookieParser('blog-application')); app.use(express.session());   The previous lines of code should be added to /index.js. Our administration should be protected, so the first two lines define our credentials. We are using Express as data storage, simply creating key-value pairs. Later, if we need the username we can get it with app.get("username"). The next two lines enable session support. We need that because of the login process. We added a middleware, which attaches the articles to the request object. We will do the same with the current user's status, as follows: app.use(function(req, res, next) { if (( req.session && req.session.admin === true ) || ( req.body && req.body.username === app.get("username") && req.body.password === app.get("password") )) { req.logged = true; req.session.admin = true; }; next(); });   Our if statement is a little long, but it tells us whether the user is logged in or not. The first part checks whether there is a session created and the second one checks whether the user submitted a form with the correct username and password. If these expressions are true, then we attach a variable, logged, to the request object and create a session that will be valid during the following requests. There is only one thing that we need in the main application's file. A few routes that will handle the control panel operations. In the following code, we are defining them along with the needed route handler: var protect = function(req, res, next) { if (req.logged) { next(); } else { res.send(401, 'No Access.'); } } app.post('/api/add', protect, require("./controllers/api/add")); app.post('/api/edit', protect, require("./controllers/api/edit")); app.post('/api/delete', protect, require("./controllers/api/ delete")); app.all('/admin', require("./controllers/admin"));   The three routes, which start with /api, will use the model Articles.js to add, edit, and remove articles from the database. These operations should be protected. We will add a middleware function that takes care of this. If the req.logged variable is not available, it simply responds with a 401 - Unauthorized status code. The last route, /admin, is a little different because it shows a login form instead. The following is the controller to create new articles: module.exports = function(req, res, next) { req.articles.add(req.body, function() { res.send({success: true}); }); }   We transfer most of the logic to the frontend, so again, there are just a few lines. What is interesting here is that we pass req.body directly to the model. It actually contains the data submitted by the user. The following code, is how the req.articles.add method looks for the MongoDB implementation: add: function(data, callback) { data.ID = crypto.randomBytes(20).toString('hex'); collection.insert(data, {}, callback || function() {}); } And the MySQL implementation is as follows: add: function(data, callback) { var date = new Date(); var query = ""; query += "INSERT INTO articles (title, text, date) VALUES ("; query += connection.escape(data.title) + ", "; query += connection.escape(data.text) + ", "; query += "'" + date.getFullYear() + "-" + date.getMonth() + "-" + date.getDate() + "'"; query += ")"; connection.query(query, callback); } In both the cases, we need title and text in the passed data object. Thankfully, due to Express' bodyParser middleware, this is what we have in the req.body object. We can directly forward it to the model. The other route handlers are almost the same: // api/edit.js module.exports = function(req, res, next) { req.articles.update(req.body, function() { res.send({success: true}); }); } What we changed is the method of the Articles.js class. It is not add but update. The same technique is applied in the route to delete an article. We can see it as follows: // api/delete.js module.exports = function(req, res, next) { req.articles.remove(req.body.id, function() { res.send({success: true}); }); }   What we need for deletion is not the whole body of the request but only the unique ID of the record. Every API method sends {success: true} as a response. While we are dealing with API requests, we should always return a response. Even if something goes wrong. The last thing in the Node.js part, which we have to cover, is the controller responsible for the user interface of the administration panel, that is, the. / controllers/admin.js file: module.exports = function(req, res, next) { if(req.logged) { res.render("admin", { app: "admin" }); } else { res.render("login", { app: "" }); } }   There are two templates that are rendered: /views/admin.jade and /views/login. jade. Based on the variable, which we set in /index.js, the script decides which one to show. If the user is not logged in, then a login form is sent to the browser, as follows: extends layout block content .container header h1 Administration hr section.articles article form(method="post", action="/admin") span Username: br input(type="text", name="username") br span Password: br input(type="password", name="password") br br input(type="submit", value="login")   There is no AngularJS code here. All we have is the good old HTML form, which submits its data via POST to the same URL—/admin. If the username and password are correct, the .logged variable is set to true and the controller renders the other template: extends layout block content .container header h1 Administration hr a(href="/") Public span | a(href="#/") List span | a(href="#/add") Add section(ng-view) script(src='/admin.js')   The control panel needs several views to handle all the operations. AngularJS has a great router module, which works with hashtags-type URLs, that is, URLs such as / admin#/add. The same module requires a placeholder for the different partials. In our case, this is a section tag. The ng-view attribute tells the framework that this is the element prepared for that logic. At the end of the template, we are adding an external file, which keeps the whole client-side JavaScript code that is needed by the control panel. While the client-side part of the applications needs only loading of the articles, the control panel requires a lot more functionalities. It is good to use the modular system of AngularJS. We need the routes and views to change, so the ngRoute module is needed as a dependency. This module is not added in the main angular.min.js build. It is placed in the angular-route.min.js file. The following code shows how our module starts: var admin = angular.module('admin', ['ngRoute']); admin.config(['$routeProvider', function($routeProvider) { $routeProvider .when('/', {}) .when('/add', {}) .when('/edit/:id', {}) .when('/delete/:id', {}) .otherwise({ redirectTo: '/' }); } ]);   We configured the router by mapping URLs to specific routes. At the moment, the routes are just empty objects, but we will fix that shortly. Every controller will need to make HTTP requests to the Node.js part of the application. It will be nice if we have such a service and use it all over our code. We can see an example as follows: admin.factory('API', function($http) { var request = function(method, url) { return function(callback, data) { $http({ method: method, url: url, data: data }) .success(callback) .error(function(data, status, headers, config) { console.error("Error requesting '" + url + "'."); }); } } return { get: request('GET', '/api/get'), add: request('POST', '/api/add'), edit: request('POST', '/api/edit'), remove: request('POST', '/api/delete') } });   One of the best things about AngularJS is that it works with plain JavaScript objects. There are no unnecessary abstractions and no extending or inheriting special classes. We are using the .factory method to create a simple JavaScript object. It has four methods that can be called: get, add, edit, and remove. Each one of them calls a function, which is defined in the helper method request. The service has only one dependency, $http. We already know this module; it handles HTTP requests nicely. The URLs that we are going to query are the same ones that we defined in the Node.js part. Now, let's create a controller that will show the articles currently stored in the database. First, we should replace the empty route object .when('/', {}) with the following object: .when('/', { controller: 'ListCtrl', template: ' <article ng-repeat="article in articles"> <hr /> <strong>{{article.title}}</strong><br /> (<a href="#/edit/{{article.id}}">edit</a>) (<a href="#/delete/{{article.id}}">remove</a>) </article> ' })   The object has to contain a controller and a template. The template is nothing more than a few lines of HTML markup. It looks a bit like the template used to show the articles on the client side. The difference is the links used to edit and delete. JavaScript doesn't allow new lines in the string definitions. The backward slashes at the end of the lines prevent syntax errors, which will eventually be thrown by the browser. The following is the code for the controller. It is defined, again, in the module: admin.controller('ListCtrl', function($scope, API) { API.get(function(articles) { $scope.articles = articles; }); });   And here is the beauty of the AngularJS dependency injection. Our custom-defined service API is automatically initialized and passed to the controller. The .get method fetches the articles from the database. Later, we send the information to the current $scope dependency and the two-way data binding does the rest. The articles are shown on the page. The work with AngularJS is so easy that we could combine the controller to add and edit in one place. Let's store the route object in an external variable, as follows: var AddEditRoute = { controller: 'AddEditCtrl', template: ' <hr /> <article> <form> <span>Title</spna><br /> <input type="text" ng-model="article.title"/><br /> <span>Text</spna><br /> <textarea rows="7" ng-model="article.text"></textarea> <br /><br /> <button ng-click="save()">save</button> </form> </article> ' };   And later, assign it to the both the routes, as follows: .when('/add', AddEditRoute) .when('/edit/:id', AddEditRoute)   The template is just a form with the necessary fields and a button, which calls the save method in the controller. Notice that we bound the input field and the text area to variables inside the $scope dependency. This comes in handy because we don't need to access the DOM to get the values. We can see this as follows: admin.controller( 'AddEditCtrl', function($scope, API, $location, $routeParams) { var editMode = $routeParams.id ? true : false; if (editMode) { API.get(function(articles) { articles.forEach(function(article) { if (article.id == $routeParams.id) { $scope.article = article; } }); }); } $scope.save = function() { API[editMode ? 'edit' : 'add'](function() { $location.path('/'); }, $scope.article); } })   The controller receives four dependencies. We already know about $scope and API. The $location dependency is used when we want to change the current route, or, in other words, to forward the user to another view. The $routeParams dependency is needed to fetch parameters from the URL. In our case, /edit/:id is a route with a variable inside. Inside the code, the id is available in $routeParams.id. The adding and editing of articles uses the same form. So, with a simple check, we know what the user is currently doing. If the user is in the edit mode, then we fetch the article based on the provided id and fill the form. Otherwise, the fields are empty and new records will be created. The deletion of an article can be done by using a similar approach, which is adding a route object and defining a new controller. We can see the deletion as follows: .when('/delete/:id', { controller: 'RemoveCtrl', template: ' ' })   We don't need a template in this case. Once the article is deleted from the database, we will forward the user to the list page. We have to call the remove method of the API. Here is how the RemoveCtrl controller looks like: admin.controller( 'RemoveCtrl', function($scope, $location, $routeParams, API) { API.remove(function() { $location.path('/'); }, $routeParams); } );   The preceding code depicts same dependencies like in the previous controller. This time, we simply forward the $routeParams dependency to the API. And because it is a plain JavaScript object, everything works as expected. Summary In this article, we built a simple blog by writing the backend of the application in Node.js. The module for database communication, which we wrote, can work with the MongoDB or MySQL database and store articles. The client-side part and the control panel of the blog were developed with AngularJS. We then defined a custom service using the built-in HTTP and routing mechanisms. Node.js works well with AngularJS, mainly because both are written in JavaScript. We found out that AngularJS is built to support the developer. It removes all those boring tasks such as DOM element referencing, attaching event listeners, and so on. It's a great choice for the modern client-side coding stack. You can refer to the following books to learn more about Node.js: Node.js Essentials Learning Node.js for Mobile Application Development Node.js Design Patterns Resources for Article: Further resources on this subject: Node.js Fundamentals [Article] AngularJS Project [Article] Working with Live Data and AngularJS [Article]
Read more
  • 0
  • 2
  • 13484

article-image-object-oriented-programming
Packt
16 Feb 2016
17 min read
Save for later

Object-Oriented Programming

Packt
16 Feb 2016
17 min read
S3 classes are the most popular classes in R programming language. These classes are simple and easy to implement. Most of the classes that come predefined in R are of this type. (For more resources related to this topic, see here.) This article is divided into three parts: Defining classes and methods: This section will give a general idea of how methods are defined whose function depends on the class name of the primary argument Objects and inheritance: In this section, we will discuss the way in which objects of a given class can be defined; we will also introduce the idea of inheritance in the context of S3 classes Encapsulation: In this section, we will discuss the importance of encapsulation with respect to a class and how it is handled within the context of an R class At first glance, S3 objects do not appear to behave like objects as defined in other languages. The definition is an odd implementation compared to Java or C++. On the plus side, S3 objects are relatively simple and can offer a powerful way to deal with a wide variety of circumstances. We have seen a variety of data structures as well as functions, and in this article, we will see how the class attribute can be used to dictate how a function responds when a list is passed to a function. The idea is that the class attribute for an object is a vector of names, and the vector represents an ordered set of names to search when deciding what action a function should take. We will build on and extend one example throughout this article. The idea is that we wish to create a set of classes that can be used to simulate a random variable, which follows a geometric distribution. There will be two classes. The first class is for a fair coin, in which we flip the coin until heads is tossed. The second class is for a fair, six-sided die, in which we roll until a 1 is rolled. Defining classes and methods The class command is similar to other attribute commands, and it can be used to either set or get information about an object's class. An object's class is a vector, and each item in the vector is the name of a class. The first element in the class vector is the object's base class, and it inherits from the other classes as you read from left to right. We first focus on the situation where an object has a single class and will examine inheritance in the section that follows. The example examined throughout this article is used to simulate one experiment that follows a geometric distribution. The idea is that you repeat some experiment and stop when the first success occurs. First, we examine two classes, and we construct a function that will take an action depending on the class name. The first class is used to represent a fair, six-sided die. The die will be rolled, giving an integer between 1 and 6 inclusive, and the experiment stops when a 1 is returned. The second class represents a fair coin. The coin will be flipped returning either an H or a T, and the experiment stops when H is returned. The two class definitions are illustrated in the following figure. Each class keeps track of the trials, and the results are kept in a vector. The two methods include a method to reset the history, but more will be added when we examine inheritance. In this example, we are not creating methods in the traditional sense but are creating functions that take appropriate action based on the class name of the argument passed to them. Have a look at the following diagram: The methods associated with the die and coin classes First, we define the two classes. Each class is composed of a list, and the class names are set to Die and Coin respectively. (The names are strings that we make up.) Each class consists of a list with a single numeric vector that initially has a length of zero. In each of the following cases, the list is created manually, and a class name is defined. We could have used a vector, but we used a list so that the examples are consistent with the way we extend the classes later: > oneDie <- list(trials=character(0)) > class(oneDie) <- "Die" > oneCoin <- list(trials=character(0)) > class(oneCoin) <- "Coin" First, we define two sets of functions. The first set of functions resets the history, and the second set performs a single Bernoulli trial. We first focus on a routine to reset and initialize the history, and define a function called reset. The reset function makes use of three different functions. The first uses the UseMethod command, which will tell R to search for the appropriate function to call. The decision is based on the class name of the object passed to it as the first argument. The UseMethod command looks for other functions whose names have the form resetTrial.class_name, where the class_name suffix must exactly match the name of the class. The exception is the default suffix that is executed if no other function is found: reset <- function(theObject) { UseMethod("reset",theObject) print("Reset the Trials") } reset.default <- function(theObject) { print("Uh oh, not sure what to do here!n") return(theObject) } reset.Die <- function(theObject) { theObject$trials <- character(0) print("Reset the dien") return(theObject) } reset.Coin <- function(theObject) { theObject$trials <- character(0) print("Reset the coinn") return(theObject) } Note that the functions return the object passed to them. Recall that R passes arguments as values. Any changes you make to the variable are local to the function, so the new value must be returned. We can now call the resetTrial function, and it will decide which function to call, given the argument passed to it. Have a look at the following code: > oneDie$trials = c("3","4","1") > oneDie$trials [1] "3" "4" "1" > oneDie <- reset(oneDie) Reset the die > oneDie $trials character(0) attr(,"class") [1] "Die" > oneCoin$trials = c("H","H","T") > oneCoin <- reset(oneCoin) Reset the coin > oneDie$trials character(0) > # Look at an example that will fail and use the default function. > v <- c(1,2,3) > v <- reset(v) [1] "Uh oh, not sure what to do here!n" > v [1] 1 2 3 Note that the print command after the UseMethod command in the function resetTrial is not executed. When the return function is called, any commands that follow the UseMethod command are not executed. Defining objects and inheritance The examples given in the previous section should invoke a twinge of shame for those familiar with object-oriented principles, and you should be assured that I felt appropriately embarrassed to share them. It was done, though, to keep the introduction to S3 classes as simple as possible. One issue is that the two classes are closely related, and the functions include a great deal of repeated code. We will now examine how inheritance can be used to avoid this problem. In this section, we define a base class, GeometricTrial, and then redefine the routines so that the Die and Coin classes can be derived from the base class. In doing so, we can demonstrate how inheritance is implemented in the context of an S3 class. Additionally, we respect the idea of encapsulation, which is the principle that an object of a given class should update its own elements using methods from within the class. We explore this issue in greater detail in the section that follows. We will now rethink the whole class structure. The die and the coin are closely related, and the only difference is the result returned from a single trial. We reimagine the classes to take advantage of the commonalities between the coin and the die. The new class structure is shown in the following diagram: In addition to the change in the classes, we also change the way in which the classes are defined. In this case, we define functions that will act as constructors for each class. Each constructor will use the class command to append the name of the class to the object's class attribute. As previously mentioned, the class attribute for an object is a vector. When you call the UseMethod command, R will search for a function whose class matches the first element in the vector. If it does not find that function, it looks for a function that matches the second element, and it proceeds until it reaches the last element in the vector. If it does not find anything, it calls the default function. With this in mind, we now examine new definitions of the classes. Rather than manually creating the class, we define functions that will create a list representing the class, append a class name to the class attribute, and then return the list. There are three classes, and we will define one function for each class. The first function is used to define a constructor for an object of the GeometricTrial class: GeometricTrial <- function() { # Create the basic data structure - a list that keeps track of # a set of trials. # Create the basic methods as part of a list to be returned. me = list( # Define the history to keep track of the trials. history = character(0) ) # Define my class identifier and return the list. class(me) <- append(class(me),"GeometricTrial") return(me) } Prior to returning the list, the append function is used to add the new class name to the end of the current class attribute. This idea is used in classes that are derived from the GeometricTrial classes as well. The constructor for the Die and Coin classes can now be defined, and both constructors explicitly call the constructor for the parent class, perform any actions associated with the current class, and then append the current class name to the class attribute: Die <- function() { # Define the object by first calling the constructor for the base class me <- GeometricTrial() # Add the class name to the end of the list of class names class(me) <- append(class(me),"Die") return(me) } Coin <- function() { # Define the object by calling the constructor for the base class me <- GeometricTrial() # Add the class name to the end of the list of class names class(me) <- append(class(me),"Coin") return(me) } The GeometricTrial class includes four methods. The reset method behaves exactly like the reset method discussed in the previous section. The getHistory method is an accessor for a data element and is discussed in the following section. We will now discuss the simulation method, and a discussion on the singleTrial method will follow. The simulation method is used to simulate a single experiment. The history is first cleared, and the singleTrial method is repeatedly called until a successful result is returned. We first define the base simulation function, the default simulation function, and then the simulation function used by the GeometricTrial class, as follows: simulation <- function(theObject) { UseMethod("simulation",theObject) } simulation.default <- function(theObject) { warning("Default simulation method called on unrecognized object.") return(theObject) } ## Define a method to run a simulation of a geometric trial. simulation.GeometricTrial = function(theObject) { theObject <- reset(theObject) # Reset the history # before the trial. repeat { ## perform a single trial and add it to the history thisTrial <- singleTrial(theObject) theObject <- appendEvent(theObject,thisTrial$result) if(thisTrial$success) { break # The trial resulted in a success. Time # to stop! } } # The trial was not a success. Keep going. return(theObject) } The effort to define a default function may not appear to be a worthwhile endeavor. However, this practice is generally employed to ensure that the system can responsibly react if the methods you define are called by mistake. The final step is to define the singleTrial methods. This method is executed by the child classes, Die and Coin. Again, the base and default methods are created. In this case, though, there are also methods for each of the three classes. The base function calls the UseMethod function, which scrolls through the class attribute for the first function to call. We use a method for the GeometricTrial class to demonstrate the order of the calls as well as the NextMethod function. The NextMethod function continues the search in the class attribute and will call the next function based on the class names that follow the current class: singleTrial.default = function(theObject) { ## Just generate a default success warning("Unrecognized object found for the singleTrial method") return(list(result="1",success=TRUE)) } singleTrial.GeometricTrial = function(theObject) { NextMethod("singleTrial",theObject) } singleTrial.Coin = function(theObject) { ## Perform a single coin flip value <- as.character( cut(as.integer(1+trunc(runif(1,0,2))),c(0,1,2),labels=c("H","T"))) return(list(result=value,success=(value=="H"))) } singleTrial.Die = function(theObject) { ## Perform a single die roll value <- as.integer(1+trunc(runif(1,0,6))) return(list(result=value,success=(value==1))) } With these methods defined and the getHistory method defined in the following section, the class will be complete. Objects of the Coin and Die class can be created, and simulations can be executed, as follows: > coin <- Coin() > coin <- simulation(coin) > getHistory(coin) [1] H Levels: H > coin <- simulation(coin) > getHistory(coin) [1] T T H Levels: H T > > die <- Die() > die <- simulation(die) > getHistory(die) [1] 1 Levels: 1 > die <- simulation(die) > getHistory(die) [1] 6 5 5 6 2 1 Levels: 1 2 5 6 Encapsulation The final method for the getHistory class will now be defined. It is defined in a separate section to stress an important point. An S3 object is generally a basic data structure, such as a vector or a list that has an additional class attribute defined. The functions that are defined for the class react to the class attribute in a predictable way. One side effect is that every element of an object from a given class is public data. The elements contained within an object can always be accessed. The result is that when programming in R, we must take extra steps to maintain discipline with respect to accessing the data elements maintained by an object. Code that directly accesses data elements within an object may work when first written, but any change to the class constructor risks breaking code in the other methods defined for a class. With respect to our previous example, we have an accessor, the getHistory method. If we have an object, called oneDie, from the Die class, we can easily get the history using oneDie$history. If we later decide to change the data structure used to store the history, then any code directly accessing this variable is likely to fail. Instead, we write an accessor method, getHistory, which is designed to return a vector that has the history in the form of a vector of factors. It is important to maintain discipline and only use this method to get a copy of the history. Have a look at the following code: getHistory <- function(theObject) { UseMethod("getHistory",theObject) } getHistory.default <- function(theObject) { return(factor()) # Just return an empty vector of factors } getHistory.GeometricTrial <- function(theObject) { return(as.factor(theObject$history)) } A final note There is one final note to share about S3 classes. If you have used R, you most likely have used them. Many functions are defined to react according to the class name of their first argument. Have a look at the following diagram: A common example of this is the plot command. If you type the plot command without arguments, you can see its definition, as follows: > plot function (x, y, ...) UseMethod("plot") <bytecode: 0x32fdd50> <environment: namespace:graphics> > The plot command will react differently depending on what kind of object you passed to it. If you wish to see what classes the plot command can handle, you can use the methods command to list them: > methods(plot) [1] plot.HoltWinters* plot.TukeyHSD* plot.acf* [4] plot.data.frame* plot.decomposed.ts* plot.default [7] plot.dendrogram* plot.density* plot.ecdf [10] plot.factor* plot.formula* plot.function [13] plot.hclust* plot.histogram* plot.isoreg* [16] plot.lm* plot.medpolish* plot.mlm* [19] plot.ppr* plot.prcomp* plot.princomp* [22] plot.profile.nls* plot.spec* plot.stepfun [25] plot.stl* plot.table* plot.ts [28] plot.tskernel* Non-visible functions are asterisked > One of the greatest advantages of the S3 class definition is that it is simple to build on what is already available. In the example from the previous section, I would like to have the plot command react appropriately according to whether or not I pass it a class of the type Die or Coin. Assuming that I have the previous classes, Die and Coin, defined, I merely have to define two new plot functions, as follows: > plot.Die <- function(theDie,theTitle) + { + plot(theDie$getHistory(), + xlab="Value After A Die Roll",ylab="Frequency", + main=theTitle) + } > > plot.Coin <- function(theCoin,theTitle) + { + plot(theCoin$getHistory(), + xlab="Value After Coin Flip",ylab="Frequency", + main=theTitle) + } > plot(aCoin,"This Here Trial") > plot(aDie,"A More Better Trial") It is common to use this idea to extend a number of commands. Some common examples include the print and the format functions. Summary We have explored how to create S3 classes, and we did so in the context of two examples. The first example focused on how to define functions that will react based on the class name of the first argument given to the function. The first example did not make full use of basic object-oriented principles, as it is an attempt to simply introduce the idea of S3 classes. The second example extended the first example to provide a simple example of how inheritance is implemented. It demonstrated how inheritance is implemented in the context of an S3 class. It also provided a demonstration of how encapsulation is implemented under the framework of an S3 class. One downside to the approach is that there is little type checking. It is possible to make changes to an object that can make it inconsistent with the original definition. When a change is made to an object, no checks are implemented to ensure that an object has the properties that are expected of it. One way to avoid this issue is to make use of S4 classes. Another advantage is that the S4 approach will look more familiar to those already familiar with object-oriented approaches to programming. To learn more about R, the following books published by Packt Publishing (https://www.packtpub.com/) are recommended: Instant R Starter (https://www.packtpub.com/big-data-and-business-intelligence/instant-r-starter-instant) R Object-oriented Programming (https://www.packtpub.com/big-data-and-business-intelligence/r-object-oriented-programming) R High Performance Programming (https://www.packtpub.com/application-development/r-high-performance-programming) Resources for Article: Further resources on this subject: Big Data Analytics[article] Data Analytics[article] Learning Data Analytics with R and Hadoop[article]
Read more
  • 0
  • 0
  • 1471

article-image-creating-our-first-app-ionic
Packt
16 Feb 2016
20 min read
Save for later

Creating Our First App with Ionic

Packt
16 Feb 2016
20 min read
There are many options for developing mobile applications today. Native applications require a unique implementation for each platform, such as iOS, Android, and Windows Phone. It's required for some use cases such as high-performance CPU and GPU processing with lots of memory consumption. Any application that does not need over-the-top graphics and intensive CPU processing could benefit greatly from a cost-effective, write once, and run everywhere HTML5 mobile implementation. In this article, we will cover: Setting up a development environment Creating a HelloWorld app via CLI Creating a HelloWorld app via Ionic Creator Copying examples from Ionic Codepen Demos Viewing the app using your web browser Viewing the app using iOS Simulator Viewing the app using Xcode for iOS Viewing the app using Genymotion for Android Viewing the app using Ionic View Customizing the app folder structure (For more resources related to this topic, see here.) For those who choose the HTML5 route, there are many great choices in this active market. Some options may be very easy to start but could be very hard to scale or could face performance problems. Commercial options are generally expensive for small developers to discover product and market fit. It's a best practice to think of the users first. There are instances where a simple responsive design website is a better choice; for example, the business has mainly fixed content with minimal updating required or the content is better off on the web for SEO purposes. Ionic has several advantages over its competitors: It's written on top of AngularJS UI performance is strong because of its use of the requestAnimationFrame() technique It offers a beautiful and comprehensive set of default styles, similar to a mobile-focused Twitter Bootstrap Sass is available for quick, easy, and effective theme customization You will go through several HelloWorld examples to bootstrap your Ionic app. This process will give you a quick skeleton to start building more comprehensive apps. The majority of apps have similar user experience flows such as tabs and a side menu. Setting up a development environment Before you create the first app, your environment must have the required components ready. Those components ensure a smooth process of development, build, and test. The default Ionic project folder is based on Cordova's. Therefore you will need the Ionic CLI to automatically add the correct platform (that is, iOS, Android, or Windows Phone) and build the project. This will ensure all Cordova plugins are included properly. The tool has many options to run your app in the browser or simulator with live reload. Getting ready You need to install Ionic and its dependencies to get started. Ionic itself is just a collection of CSS styles and AngularJS Directives and Services. It also has a command-line tool to help manage all of the technologies such as Cordova and Bower. The installation process will give you a command line to generate initial code and build the app. Ionic uses npm as the installer, which is included when installing Node.js. Please install the latest version of Node.js from http://nodejs.org/download/. You will need Cordova, ios-sim (iOS Simulator), and Ionic: $ npm install -g cordova ionic ios-sim This single command line will install all three components instead of issuing three command lines separately. The -g parameter is to install the package globally (not just in the current directory). For Linux and Mac, you may need to use the sudo command to allow system access: $ sudo npm install -g cordova ionic ios-sim There are a few common options for an integrated development environment: Xcode for iOS Eclipse or Android Studio for Android Microsoft Visual Studio Express or Visual Studio for Windows Phone Sublime Text (http://www.sublimetext.com/) for web development All of those have a free license. Sublime Text is free for non-commercial use only but you have to purchase a license if you are a commercial developer. Most frontend developers would prefer to use Sublime Text for coding HTML and JavaScript because it's very lightweight and comes with a well-supported developer community. You could code directly in Xcode, Eclipse, or Visual Studio Express, but those are somewhat heavy duty for web apps, especially when you have a lot of windows open and just need something simple to code. How to do it… If you decide to use Sublime Text, you will need Package Control (https://packagecontrol.io/installation), which is similar to a Plugin Manager. Since Ionic uses Sass, it's optional to install the Sass Syntax Highlighting package: Select Sublime Text | Preferences | Package Control: Select Package Control: Install Package. You could also just type the commands partially (that is, inst) and it will automatically select the right option. Type Sass and the search results will show one option for TextMate & Sublime Text. Select that item to install. See also There are tons of packages that you may want to use, such as Haml, JSHint, JSLint, Tag, ColorPicker, and so on. You can browse around this website: https://sublime.wbond.net/browse/popular, for more information. Creating a HelloWorld app via CLI It's quickest to start your app using existing templates. Ionic gives you three standard templates out of the box via the command line: Blank: This template has a simple one page with minimal JavaScript code. Tabs: This template has multiple pages with routes. A route URL goes to one tab or tabs. Sidemenu: This is template with left and/or right menu and with center content area. There are two other additional templates: maps and salesforce. But these are very specific to apps using Google Maps or for integration with the Salesforce.com API. How to do it… To set up the app with a blank template from Ionic, use this command: $ ionic start HelloWorld_Blank blank If you don't have an account in http://ionic.io/, the command line will ask for it. You could either press y or n to continue. It's not required to have an account at this step. If you replace blank with tabs, it will create a tab template: $ ionic start HelloWorld_Tabs tabs Similarly, this command will create an app with a sidemenu: $ ionic start HelloWorld_Sidemenu sidemenu The sidemenu template is the most common template as it provides a very nice routing example with different pages in the templates folder under /www. Additional guidance for the Ionic CLI is available on the GitHub page: https://github.com/driftyco/ionic-cli How it works… This article will show you how to quickly start your codebase and visually see the result. However, the following are the core concepts: Controller: Manage variables and models in the scope and trigger others, such as services or states. Directive: Where you manipulate the DOM, since the directive is bound to a DOM object. Service: Abstraction to manage models or collections of complex logic beside get/set required. Filter: Mainly used to process an expression in the template and return some data (that is, rounding number, add currency) by using the format {{ expression | filter }}. For example, {{amount | currency}} will return $100 if the amount variable is 100. The project folder structure will look like the following:   You will spend most of your time in the /www folder, because that's where your application logic and views will be placed. By default from the Ionic template, the AngularJS module name is called starter. You will see something like this in app.js, which is the bootstrap file for the entire app: angular.module('starter', ['ionic', 'ngCordova', 'starter.controllers', 'starter.services', 'starter.directives', 'starter.filters']) This basically declares starter to be included in ng-app="starter" of index.html. We would always have ionic and ngCordova. The other modules are required and listed in the array of string [...] as well. They can be defined in separate files. Note that if you double click on the index.html file to open in the browser, it will show a blank page. This doesn't mean the app isn't working. The reason is that the AngularJS component of Ionic dynamically loads all the .js files and this behavior requires server access via a http protocol (http://). If you open a file locally, the browser automatically treats it as a file protocol (file://) and therefore AngularJS will not have the ability to load additional .js modules to run the app properly. There are several methods of running the app that will be discussed. Creating a HelloWorld app via Ionic Creator Another way to start your app codebase is to use Ionic Creator. This is a great interface builder to accelerate your app development with a drag-and-drop style. You can quickly take existing components and position them to visualize how it should look in the app via a web-based interface. Most common components like buttons, images, checkboxes, and so on are available. Ionic Creator allows the user to export everything as a project with all .html, .css, and .js files. You should be able edit content in the /www folder to build on top of the interface. Getting ready Ionic Creator requires registration for a free account at https://creator.ionic.io/ to get started. How to do it… Create a new project called myApp:   You will see this simple screen:   The center area is your app interface. The left side gives you a list of pages. Each page is a single route. You also have access to a number of UI components that you would normally have to code by hand in an HTML file. The right panel shows the properties of any selected component. You're free to do whatever you need to do here by dropping components to the center screen. If you need to create a new page, you have to click the plus sign in the Pages panel. Each page is represented as a link, which is basically a route in AngularJS UI Router's definition. To navigate to another page (for example, after clicking a button), you can just change the Link property and point to that page. There is an Edit button on top where you can toggle back and forth between Edit Mode and Preview Mode. It's very useful to see how your app will look and behave. Once completed, click on the Export button on the top navigation. You have three options: Use the Ionic CLI tool to get the code Download the project as a zip file Review the raw HTML The best way to learn Ionic Creator is to play with it. You can add a new page and pick out any existing templates. This example shows a Login page template:   Here is how it should look out of the box:   There's more... To switch to Preview Mode where you can see the UI in a device simulator, click the switch button on the top right to enable Test:   In this mode, you should be able to interact with the components in the web browser as if it's actually deployed on the device. If you break something, it's very simple to start a new project. It's a great tool to use for "prototyping" and to get initial template or project scaffolding. You should continue to code in your regular IDE for the rest of the app. Ionic Creator doesn't do everything for you, yet. For example, if you want to access specific Cordova plugin features, you have to write that code separately. Also, if you want to tweak the interface outside of what is allowed within Ionic Creator, it will also require specific modifications to the .html and .css files. Copying examples from Ionic Codepen Demos Sometimes it's easier to just get snippets of code from the example library. Ionic Codepen Demos (http://codepen.io/ionic/public-list/) is a great website to visit. Codepen.io is a playground (or sandbox) to demonstrate and learn web development. There are other alternatives such as http://plnkr.com or http://jsfiddle.com. It's just a developer's personal preference which one to choose. However, all Ionic's demos are already available on Codepen, where you can experiment and clone to your own account. http://plnkr.com has an existing AngularJS boilerplate and could be used to just practice specific AngularJS areas because you can copy the link of sample code and post on http://stackoverflow.com/ if you have questions. How to do it… There are several tags of interest to browse through if you want specific UI component examples: You don't need a Codepen account to view. However, if there is a need to save a custom pen and share with others, free registration will be required. The Ionic Codepen Demos site has more collections of demos comparing to the CLI. Some are based on a nightly build of the platform so they could be unstable to use. There's more... You can find the same side menu example on this site: Navigate to http://codepen.io/ionic/public-list/ from your browser. Select Tag: menus and then click on Side Menu and Navigation: Nightly. Change the layout to fit a proper mobile screen by clicking on the first icon of the layout icons row on the bottom right of the screen. Viewing the app using your web browser In order to "run" the web app, you need to turn your /www folder into a web server. Again there are many methods to do this and people tend to stick with one or two ways to keep things simple. A few other options are unreliable such as Sublime Text's live watch package or static page generator (for example, Jekyll, Middleman App, and so on). They are slow to detect changes and may freeze your IDE so these won't be mentioned here. Getting ready The recommended method is to use the ionic serve command line. It basically launches an HTTP server so you can open your app in a desktop browser. How to do it… First you need to be in the project folder. Let's assume it is the Side Menu HelloWorld: $ cd HelloWorld_Sidemenu From there, just issue the simple command line: $ ionic serve  That's it! There is no need to go into the /www folder or figure out which port to use. The command line will provide these options while the web server is running: The most common option to use here is r to restart or q to quit when you are done. There is an additional step to view the app with the correct device resolution: Install Google Chrome if it's not already on your computer. Open the link (for example, http://localhost:8100/#/app/playlists) from ionic serve in Google Chrome. Turn on Developer Tools. For example, in Mac's Google Chrome, select View | Developer | Developer Tools: Click on the small mobile icon in the Chrome Developer Tools area: There will be a long list of devices to pick from: After selecting a device, you need to refresh the page to ensure the UI is updated. Chrome should give you the exact view resolution of the device. Most developers would prefer to use this method to code as you can debug the app using Chrome Developer Tools. It works exactly like any web application. You can create breakpoints or output variables to the console. How it works... Note that ionic serve is actually watching everything under the /www folder except the JavaScript modules in the /lib folder. This makes sense because there is no need for the system to scan through every single file when the probability for it to change is very small. People don't code directly in the /lib folder but only update when there is a new version of Ionic. However, there is some flexibility to change this. You can specify a watchPatterns property in the ionic.project file located in your project root to watch (or not watch) for specific changes: { "name": "myApp", "app_id": "", "watchPatterns": [ "www/**/*", "!www/css/**/*", "your_folder_here/**/*" ] } While the web server is running, you can go back to the IDE and continue coding. For example, let's open the playlists.html file under /www/templates and change the first line to this: <ion-view view-title="Updated Playlists"> Go back to the web browser where Ionic opened the new page; the app interface will change the title bar right away without requiring you to refresh the browser. This is a very nice feature when there is a lot of back and between code changes and allows checking on how it works or looks in the app instantly. Viewing the app using iOS Simulator So far you have been testing the web-app portion of Ionic. In order to view the app in the simulator, follow the next steps. How to do it... Add the specific platform using: $ ionic platform add ios Note that you need to do the "platform add" before building the app. The last step is to emulate the app: $ ionic emulate ios Viewing the app using Xcode for iOS Depending on personal preference, you may find it more convenient to just deploy the app using ionic ios --device on a regular basis. This command line will push the app to your physical device connected via USB without ever running Xcode. However, you could run the app using Xcode (in Mac), too. How to do it... Go to the /platforms/ios folder. Look for the folder with .xcodeproj and open in Xcode. Click on the iOS Device icon and select your choice of iOS Simulator. Click on the Run button and you should be able to see the app running in the simulator. There's more... You can connect a physical device via a USB port and it will show up in the iOS Device list for you to pick. Then you can deploy the app directly on your device. Note that iOS Developer Membership is required for this. This method is more complex than just viewing the app via a web browser. However, it's a must when you want to test your code related to device features such as camera or maps. If you change code in the /www folder and want to run it again in Xcode, you have to do ionic build ios first, because the running code is in the Staging folder of your Xcode project: For debugging, the Xcode Console can output JavaScript logs as well. However, you could use the more advanced features of Safari's Web Inspector (which is similar to Google Chrome's Developer Tools) to debug your app. Note that only Safari can debug a web app running on a connected physical iOS device because Chrome does not support this on a Mac. It's simple to enable this capability: Allow remote debugging for an iOS device by going to Settings | Safari | Advanced and enable Web Inspector. Connect the physical iOS device to your Mac via USB and run the app. Open the Safari browser. Select Develop, click on your device's name (or iOS Simulator), and click on index.html. Note: If you don't see the Develop menu in Safari, you need to navigate to menu Preferences | Advanced and check on Show Develop menu in menu bar. Safari will open a new console just for that specific device just as it's running within the computer's Safari. Viewing the app using Genymotion for Android Although it's possible to install the Google Android simulator, many developers have inconsistent experiences on a Mac computer. There are many commercial and free alternatives that offer more convenience and a wide range of device support. Genymotion provides some unique advantages such as allowing users to switch Android model and version, supporting networking from within the app, and allowing SD card simulation. You will learn how to set up an Android developer environment (on a Mac in this case) first. Then you will install and configure Genymotion for mobile app development. How to do it... The first step is to set up the Android environment properly for development. Download and install Android Studio from https://developer.android.com/sdk/index.html. Run Android Studio. You need to install all required packages such as the Android SDK. Just click on Next twice at the Setup Wizard screen and select the Finish button to start packages installation. After installation is complete, you need to install additional packages and other SDK versions. At the Quick Start screen, select Configure: Then select SDK Manager: It's a good practice to install a previous version such as Android 5.0.1 and 5.1.1. You may also want to install all Tools and Extras for later use. Select the Install packages... button. Check the box on Accept License and click on Install. The SDK Manager will give you SDK Path on the top. Make a copy of this path because you need to modify the environment path. Go to Terminal and type: $ touch ~/.bash_profile; open ~/.bash_profile It will open a text editor to edit your bash profile file. Insert the following line where /YOUR_PATH_TO/android-sdk should be the SDK Path that you copied earlier: export ANDROID_HOME=/YOUR_PATH_TO/android-sdk export PATH=$ANDROID_HOME/platform-tools:$PATH export PATH=$ANDROID_HOME/tools:$PATH Save and close that text editor. Go back to Terminal and type: $ source ~/.bash_profile $ echo $ANDROID_HOME You should see the output as your SDK Path. This verifies that you have correctly configured the Android developer environment. The second step is to install and configure Genymotion. Download and install Genymotion and Genymotion Shell from http://Genymotion.com. Run Genymotion. Select the Add button to start adding a new Android device. Select a device you want to simulate. In this case, let's select Samsung Galaxy S5: You will see the device being added to "Your virtual devices". Click on that device: Then click on Start. The simulator will take a few seconds to start and will show another window. This is just a blank simulator without your app running inside yet. Run Genymotion Shell. From Genymotion Shell, you need to get a device list and keep the IP address of the device attached, which is Samsung Galaxy S5. Type devices list: Type adb connect 192.168.56.101 (or whatever the IP address was you saw earlier from the devices list command line). Type adb devices to confirm that it is connected. Type ionic platform add android to add Android as a platform for your app. Finally, type ionic run android. You should be able to see the Genymotion window showing your app. Although there are many steps to get this working, it's a lot less likely that you will have to go through the same process again. Once your environment is set up, all you need to do is to leave Genymotion running while writing code. If there is a need to test the app in different Android devices, it's simple just to add another virtual device in Genymotion and connect to it. Summary In this article, we learned how to create your first ionic App. We also covered various ways in which we can view the App on Various platforms, that is, web browser, iOS Simulator, Xcode, Genymotion . You can also refer the following books on the similar topics: Learning Ionic: https://www.packtpub.com/application-development/learning-ionic Getting Started with Ionic: https://www.packtpub.com/application-development/getting-started-ionic Ionic Framework By Example: https://www.packtpub.com/application-development/ionic-framework-example Resources for Article: Further resources on this subject: Directives and Services of Ionic [article] First Look at Ionic [article] Ionic JS Components [article]
Read more
  • 0
  • 0
  • 20031
article-image-python-data-science-and-running
Packt
16 Feb 2016
10 min read
Save for later

Python Data Science Up and Running

Packt
16 Feb 2016
10 min read
In this article we will learn how to build NumPy, SciPy, matplotlib, and IPython and using Ipython as shell and then to create a simple application using it. (For more resources related to this topic, see here.) Building NumPY, SciPy, matplotlib, and IPython from source As a last resort or if we want to have the latest code, we can build from source.  In practice, it shouldn't be that hard, although depending on your operating system, you might run into problems. As operating systems and related software are rapidly evolving, in such cases, the best you can do is search online or ask for help. In this chapter, we give pointers on good places to look for help. The source code can be retrieved with git or as an archive from GitHub. The steps  to install NumPy from source are straightforward and given here. We can retrieve the source code for NumPy with git as follows: $ git clone git://github.com/numpy/numpy.git numpy There are similar commands for SciPy, matplotlib, and IPython (refer to the table that follows after this piece of information). The IPython source code can be downloaded from https://github.com/ipython/ipython/releases as a source archive or ZIP file. You can then unpack it with your favorite tool or with the following command: $ tar -xzf ipython.tar.gz Please refer to the following table for the git commands and source archive/zip links: Library Git command Tarball/zip URL NumPy git clone git://github.com/numpy/numpy.git numpy https://github.com/numpy/numpy/releases SciPy git clone http://github.com/scipy/scipy.git scipy https://github.com/scipy/scipy/releases matplotlib git clone git://github.com/matplotlib/matplotlib.git https://github.com/matplotlib/matplotlib/releases IPython git clone --recursive https://github.com/ipython/ipython.git https://github.com/ipython/ipython/releases Install on /usr/local with the following command from the source code directory: $ python setup.py build $ sudo python setup.py install --prefix=/usr/local To build, we need a C compiler such as GCC and the Python header files in the python-dev or python-devel package. Installing with setuptools If you have setuptools or pip, you can install NumPy, SciPy, matplotlib, and IPython with the following commands. For each library, we give two commands, one for setuptools and one for pip. You only need to choose one command per pair: $ easy_install numpy $ pip install numpy $ easy_install scipy $ pip install scipy $ easy_install matplotlib $ pip install matplotlib $ easy_install ipython $ pip install ipython It may be necessary to prepend sudo to these commands if your current user doesn't have sufficient rights on your system. NumPy arrays After going through the installation of NumPy, it's time to have a look at NumPy arrays. NumPy arrays are more efficient than Python lists when it comes to numerical operations. NumPy arrays are, in fact, specialized objects with extensive optimizations. NumPy code requires less explicit loops than equivalent Python code. This is based  on vectorization. If we go back to highschool mathematics, then we should remember the concepts of scalars and vectors. The number 2, for instance, is a scalar. When we add 2 to 2, we are performing scalar addition. We can form a vector out of a group of scalars. In Python programming terms, we will then have a one-dimensional array. This concept can, of course, be extended to higher dimensions. Performing an operation on two arrays, such as addition, can be reduced to a group of scalar operations. In straight Python, we will do that with loops going through each element in the first array and adding it to the corresponding element in the second array. However, this is more verbose than the way it is done in mathematics. In mathematics, we treat the addition of two vectors as a single operation. That's the way NumPy arrays do it too, and there are certain optimizations using low-level C routines, which make these basic operations more efficient. Simple application Imagine that we want to add two vectors called a and b. The word vector is used here in the mathematical sense, which means a one-dimensional array. The vector a holds the squares of integers 0 to n; for instance, if n is equal to 3, a contains 0, 1, or 4. The vector b holds the cubes of integers 0 to n, so if n is equal to 3, then the vector b is equal to 0, 1, or 8. How would you do that using plain Python? After we come up with a solution, we will compare it with the NumPy equivalent. The following function solves the vector addition problem using pure Python without NumPy: def pythonsum(n): a = range(n) b = range(n) c = [] for i in range(len(a)): a[i] = i ** 2 b[i] = i ** 3 c.append(a[i] + b[i]) return c The following is a function that solves the vector addition problem with NumPy: def numpysum(n): a = numpy.arange(n) ** 2 b = numpy.arange(n) ** 3 c = a + b return c Notice that numpysum() does not need a for loop. Also, we used the arange() function from NumPy, which creates a NumPy array for us with integers from  0 to n. The arange() function was imported; that is why it is prefixed with numpy. Now comes the fun part. Remember that it was mentioned in the Preface that NumPy is faster when it comes to array operations. How much faster is Numpy, though? The following program will show us by measuring the elapsed time in microseconds for the numpysum() and pythonsum() functions. It also prints the last two elements of the vector sum. Let's check that we get the same answers using Python and NumPy: #!/usr/bin/env/python import sys from datetime import datetime import numpy as np """ This program demonstrates vector addition the Python way. Run from the command line as follows python vectorsum.py n where n is an integer that specifies the size of the vectors. The first vector to be added contains the squares of 0 up to n. The second vector contains the cubes of 0 up to n. The program prints the last 2 elements of the sum and the elapsed time. """ def numpysum(n): a = np.arange(n) ** 2 b = np.arange(n) ** 3 c = a + b return c def pythonsum(n): a = range(n) b = range(n) c = [] for i in range(len(a)): a[i] = i ** 2 b[i] = i ** 3 c.append(a[i] + b[i]) return c size = int(sys.argv[1]) start = datetime.now() c = pythonsum(size) delta = datetime.now() - start print "The last 2 elements of the sum", c[-2:] print "PythonSum elapsed time in microseconds", delta.microseconds start = datetime.now() c = numpysum(size) delta = datetime.now() - start print "The last 2 elements of the sum", c[-2:] print "NumPySum elapsed time in microseconds", delta.microseconds The output of the program for 1000, 2000, and 3000 vector elements is as follows: $ python vectorsum.py 1000 The last 2 elements of the sum [995007996, 998001000] PythonSum elapsed time in microseconds 707 The last 2 elements of the sum [995007996 998001000] NumPySum elapsed time in microseconds 171 $ python vectorsum.py 2000 The last 2 elements of the sum [7980015996, 7992002000] PythonSum elapsed time in microseconds 1420 The last 2 elements of the sum [7980015996 7992002000] NumPySum elapsed time in microseconds 168 $ python vectorsum.py 4000 The last 2 elements of the sum [63920031996, 63968004000] PythonSum elapsed time in microseconds 2829 The last 2 elements of the sum [63920031996 63968004000] NumPySum elapsed time in microseconds 274 Clearly, NumPy is much faster than the equivalent normal Python code. One thing is certain; we get the same results whether we are using NumPy or not. However, the result that is printed differs in representation. Notice that the result from the numpysum() function does not have any commas. How come? Obviously, we are not dealing with a Python list but with a NumPy array. Using IPython as a shell Scientists, data analysts, and engineers are used to experimenting. IPython was created by scientists with experimentation in mind. The interactive environment that IPython provides is viewed by many as a direct answer to MATLAB, Mathematica, and Maple. The following is a list of features of the IPython shell: Tab completion, which helps you find a command History mechanism Inline editing Ability to call external Python scripts with %run Access to system commands The pylab switch Access to the Python debugger and profiler The following list describes how to use the IPython shell: The pylab switch: The pylab switch automatically imports all the Scipy, NumPy, and matplotlib packages. Without this switch, we would have to import these packages ourselves. All we need to do is enter the following instruction on the command line: $ ipython -pylab Type "copyright", "credits" or "license" for more information. IPython 2.0.0-dev -- An enhanced Interactive Python. ? -> Introduction and overview of IPython's features. %quickref -> Quick reference. help -> Python's own help system. object? -> Details about 'object', use 'object??' for extra details. Welcome to pylab, a matplotlib-based Python environment [backend: MacOSX]. For more information, type 'help(pylab)'. In [1]: quit() The quit() function or Ctrl + D quits the IPython shell. Saving a session: We might want to be able to go back to our experiments. In IPython, it is easy to save a session for later use, with the following command: In [1]: %logstart Activating auto-logging. Current session state plus future input saved. Filename : ipython_log.py Mode : rotate Output logging : False Raw input log : False Timestamping : False State : active Logging can be switched off as follows: In [9]: %logoff Switching logging OFF Executing system shell command: Execute a system shell command in  the default IPython profile by prefixing the command with the ! symbol.  For instance, the following input will get the current date: In [1]: !date In fact, any line prefixed with ! is sent to the system shell. Also, we can store the command output as shown here: In [2]: thedate = !date In [3]: thedate Displaying history: We can show the history of commands with the %hist command, for example: In [1]: a = 2 + 2 In [2]: a Out[2]: 4 In [3]: %hist a = 2 + 2 a %hist This is a common feature in Command Line Interface (CLI) environments. We can also search through the history with the -g switch as follows: In [5]: %hist -g a = 2    1: a = 2 + 2 Downloading the example code You can download the example code files for all the Packt books you have purchased from your account at https://www.packtpub.com. If you purchased this book elsewhere, you can visit https://www.packtpub.com/books/content/support and register to have the files e-mailed directly to you. We saw a number of so-called magic functions in action. These functions start with  the % character. If the magic function is used on a line by itself, the % prefix is optional. Summary In this article, we installed NumPy, SciPy, matplotlib, and IPython. We got a vector addition program working and convinced ourselves that NumPy offers superior performance. In addition, we explored the available documentation and online resources. You can also refer the following books on the similar topics: Mastering Python Data Analysis: (https://www.packtpub.com/big-data-and-business-intelligence/mastering-python-data-analysis) Python Data Analysis Cookbook: (https://www.packtpub.com/big-data-and-business-intelligence/python-data-analysis-cookbook) Getting Started with Python Data Analysis: (https://www.packtpub.com/big-data-and-business-intelligence/getting-started-python-data-analysis) Resources for Article: Further resources on this subject: Python Design Patterns in Depth – The Observer Pattern [article] Python Design Patterns in Depth: The Factory Pattern [article] Customizing IPython [article]
Read more
  • 0
  • 0
  • 1765

article-image-r-vs-pandas
Packt
16 Feb 2016
1 min read
Save for later

R vs Pandas

Packt
16 Feb 2016
1 min read
This article focuses on comparing pandas with R, the statistical package on which much of pandas' functionality is modeled. It is intended as a guide for R users who wish to use pandas, and for users who wish to replicate functionality that they have seen in the R code in pandas. It focuses on some key features available to R users and shows how to achieve similar functionality in pandas by using some illustrative examples. This article assumes that you have the R statistical package installed. If not, it can be downloaded and installed from here: http://www.r-project.org/. By the end of the article, data analysis users should have a good grasp of the data analysis capabilities of R as compared to pandas, enabling them to transition to or use pandas, should they need to. The various topics addressed in this article include the following: R data types and their pandas equivalents Slicing and selection Arithmetic operations on datatype columns Aggregation and GroupBy Matching Split-apply-combine Melting and reshaping Factors and categorical data
Read more
  • 0
  • 0
  • 4657
Modal Close icon
Modal Close icon