Search icon CANCEL
Subscription
0
Cart icon
Your Cart (0 item)
Close icon
You have no products in your basket yet
Save more on your purchases! discount-offer-chevron-icon
Savings automatically calculated. No voucher code required.
Arrow left icon
Explore Products
Best Sellers
New Releases
Books
Videos
Audiobooks
Learning Hub
Newsletter Hub
Free Learning
Arrow right icon
timer SALE ENDS IN
0 Days
:
00 Hours
:
00 Minutes
:
00 Seconds

How-To Tutorials

7012 Articles
article-image-create-configure-azure-virtual-machine
Gebin George
25 May 2018
13 min read
Save for later

How to create and configure an Azure Virtual Machine

Gebin George
25 May 2018
13 min read
Creating virtual machines on Azure gives you on-demand, high-scale, secure, virtualized infrastructure using Windows Server. Virtual machine helps you deploy and scale applications easily. In this article, we will learn how to run an Azure Virtual Machine. This tutorial is an excerpt from the book, Hands-On Networking with Azure, written by Mohamed Waly. This book will help you efficiently monitor, diagnose, and troubleshoot Azure Networking. Creating an Azure VM is a very straightforward process – all you have to do is follow the given steps: Navigate to the Azure portal and search for Virtual Machines, as shown in the following screenshot: Figure 3.1: Searching for Virtual Machines Once the VM blade is opened, you can click on +Add to create a new VM, as shown in the following screenshot: Figure 3.2: Virtual Machines blade Once you have clicked on +Add, a new blade will pop up where you have to search for and select the desired OS for the VM, as shown in the following screenshot: Figure 3.3: Searching for Windows Server 2016 OS for the VM Once the OS is selected, you need to select the deployment model, whether that be Resource Manager or Classic, as shown in the following screenshot: Figure 3.4: Selecting the deployment model Once the deployment model is selected, a new blade will pop up where you have to specify the following: Name: Specify the name of the VM. VM disk type: Specify whether the disk type will be SSD or HDD. Consider that SSD will offer consistent, low-latency performance, but will incur more charges. Note that this option is not available for the Classic model in this blade, but is available in the Configure optional features blade. User name: Specify the username that will be used to log on the VM. Password: Specify the password, which must be between 12 and 123 characters long and must contain three of the following: one lowercase character, one uppercase character, one number, and one special character that is not or -. Subscription: This specifies the subscription that will be charged for the VM usage. Resource group: This specifies the resource group within which the VM will exist. Location: Specify the location in which the VM will be created. It is recommended that you select the nearest location to you. Save money: Here, you specify whether you own Windows Server Licenses with active Software Assurance (SA). If you do, Azure Hybrid Benefit is recommended to save compute costs. For more information about Azure Hybrid Benefit, you can check this page. Figure 3.5: Configure the VM basic settings Once you have clicked on OK, a new blade will pop up where you have to specify the VM size so that the VMs series can select the one that will fulfil your needs, as shown in the following screenshot: Figure 3.6: Select the VM size Once the VM size has been specified, you need to specify the following settings: Availability set: This option provides High availability for the VM by setting the VMs within the same application and availability set. Here, the VMs will be in different fault and update domains, granting the VMs high availability (up to 99.95% of Azure SLA). Use managed disks: Enable this feature to have Azure automatically manage the availability of disks to provide data redundancy and fault tolerance without creating and managing storage accounts on your own. This setting is not available in the Classic model. Virtual network: Specify the virtual network to which you want to assign the VM. Subnet: Select the subnet within the virtual network that you specified earlier to assign the VM to. Public IP address: Either select an existing public IP address or create a new one. Network security group (firewall): Select the NSG you want to assign to the VM NIC. This is called endpoints in the Classic model. Extensions: You can add more features to the VM using extensions, such as configuration management, antivirus protection, and so on. Auto-shutdown: Specify whether you want to shut down your VM daily or not; if you do, you can set a schedule. Considering that this option will help you saving compute cost especially for dev and test scenarios. This is not available in the Classic model. Notification before shutdown: Check this if you enabled Auto-shutdown and want to subscribe for notifications before the VM shuts down. This is not available in the Classic model. Boot diagnostics: This captures serial console output and screenshots of the VM running on a host to help diagnose start up issues. Guest OS diagnostics: This obtains metrics for the VM every minute; you can use these metrics to create alerts and stay informed of your applications. Diagnostics storage account: This is where metrics are written, so you can analyze them with your own tools. Figure 3.7: Specify more settings for the VM Enabling Boot diagnostics and Guest diagnostics will incur more charges since the diagnostics will need a dedicated storage account to store their data. Finally, once you are done with its settings, Azure will validate those you have specified and summarize them, as shown in the following screenshot: Figure 3.8: VM Settings Summary Once clicked on, Create the VM will start the creation process, and within minutes the VM will be created. Once the VM is created, you can navigate to the Virtual Machines blade to open the VM that has been created, as shown in the following screenshot: Figure 3.9: The created VM overview To connect to the VM, click on Connect, where a pre-configured RDP file with the required VM information will be downloaded. Open the RDP file. You will be asked to enter the username and password you specified for the VM during its configuration, as shown in the following screenshot: Figure 3.10: Entering the VM credentials Voila! You should now be connected to the VM. Azure VMs networking There are many network configurations that can be done for the VM. You can add additional NICs, change the private IP address, set a private or public IP address to be either static or dynamic, and you can change the inbound and outbound security rules. Adding inbound and outbound rules Adding inbound and outbound security rules to the VM NIC is a very simple process; all you need to do is follow these steps: Navigate to the desired VM. Scroll down to Networking, under SETTINGS, as shown in the following screenshot: Figure 3.11: VM networking settings To add inbound and outbound security rules, you have to click on either Add inbound or Add outbound. Once clicked on, a new blade will pop up where you have to specify settings using the following fields: Service: The service specifies the destination protocol and port range for this rule. Here, you can choose a predefined service, such as RDP or SSH, or provide a custom port range. Port ranges: Here, you need to specify a single port, a port range, or a comma-separated list of single ports or port ranges. Priority: Here, you enter the desired priority value. Name: Specify a name for the rule here. Description: Write a description for the rule that relates to it here. Figure 3.12: Adding an inbound rule Once you have clicked OK, the rule will be applied. Note that the same process applies when adding an outbound rule. Adding an additional NIC to the VM Adding an additional NIC starts from the same blade as adding inbound and outbound rules. To add an additional NIC, you have to follow the given steps: Before adding an additional NIC to the VM, you need to make sure that the VM is in a Stopped (Deallocated) status. Navigate to Networking on the desired VM. Click on Attach network interface, and a new blade will pop up. Here, you have to either create a network interface or select an existing one. If you are selecting an existing interface, simply click on OK and you are done. If you are creating a new interface, click on Create network interface, as shown in the following screenshot: Figure 3.13: Attaching network interface A new blade will pop up where you have to specify the following: Name: The name of the new NIC. Virtual network: This field will be grayed out because you cannot attach a VM's NIC to different virtual networks. Subnet: Select the desired subnet within the virtual network. Private IP address assignment: Specify whether you want to allocate this IP dynamically or statically. Network security group: Specify an NSG to be assigned to this NIC. Private IP address (IPv6): If you want to assign an IPv6 to this NIC, check this setting. Subscription: This field will be grayed out because you cannot have a VM's NIC in a different subscription. Resource group: Specify the resource group to which the NIC will exist. Location: This field will be grayed out because you cannot have VM NICs in different locations. Figure 3.14: Specify the NIC settings Once you are done, click Create. Once the network interface is created, you will return to the previous blade. Here, you need to specify the NIC you just created and click on OK, as shown in the following screenshot: Figure 3.15: Attaching the NIC Configuring the NICs The Network Interface Cards (NICs) include some configuration that you might be interested in. They are as follows: To navigate to the desired NIC, you can search for the network interfaces blade, as shown in the following screenshot: Figure 3.16: Searching for network interfaces blade Then, the blade will pop up, from which you can select the desired NIC, as shown in the following screenshot: Figure 3.17: Select the desired NIC You can also navigate back to the VM via | Networking | and then click on the desired NIC, as shown in the following screenshot: Figure 3.18: The VM NIC To configure the NIC, you need to follow the given steps: Once the NIC blade is opened, navigate to IP configurations, as shown in the following screenshot: Figure 3.19: NIC blade overview To enable IP forwarding, click on Enabled and then click Save. Enabling this feature will cause the NIC to receive traffic that is not destined to its own IP address. Traffic will be sent with a different source IP. To add another IP to the NIC, click on Add, and a new blade will pop up, for which you have to specify the following: Name: The name of the IP. Type: This field will be grayed out because a primary IP already exists. Therefore, this one will be secondary. Allocation: Specify whether the allocation method is static or dynamic. IP address: Enter the static IP address that belongs to the same subnet that the NIC belongs to. If you have selected dynamic allocation, you cannot enter the IP address statically. Public IP address: Specify whether or not you need a public IP address for this IP configuration. If you do, you will be asked to configure the required settings. Figure 3.20: Configure the IP configuration settings Click on Configure required settings for the public IP address and a new blade will pop up from which you can select an existing public IP address or create a new one, as shown in the following screenshot: Figure 3.21: Create a new public IP address Click on OK and you will return to the blade, as shown in Figure 3.20, with the following warning: Figure 3.22: Warning for adding a new IP address In this case, you need to plan for the addition of a new IP address to ensure that the time the VM is restarted is not during working hours. Azure VNets considerations for Azure VMs Building VMs in Azure is a common task, but to do this task well, and to make it operate properly, you need to understand the considerations of Azure VNets for Azure VMs. These considerations are as follows: Azure VNets enable you to bring your own IPv4/IPv6 addresses and assign them to Azure VMs, statically or dynamically You do not have access to the role that acts as DHCP or provides IP addresses; you can only control the ranges you want to use in the form of address ranges and subnets Installing a DHCP role on one of the Azure VMs is currently unsupported; this is because Azure does not use traditional Layer-2 or Layer-3 topology, and instead uses Layer-3 traffic with tunneling to emulate a Layer-2 LAN Private IP addresses can be used for internal communication; external communication can be done via public IP addresses You can assign multiple private and public IP addresses to a single VM You can assign multiple NICs to a single VM By default, all the VMs within the same virtual network can communicate with each other, unless otherwise specified by an NSG on a subnet within this virtual network The network security group (NSG) can sometimes cause an overhead; without this overhead, however, all VMs within the same subnet would communicate with each other By default, an inbound security rule is created for remote desktops for Windows-based VMs, and SSH for Linux-based VMs The inbound security rules are first applied on the NSG of the subnet and then the VM NIC NSG – for example, if the subnet's NSG allows HTTP traffic, it will pass through it; however, it may not reach its destination if the VM NIC NSG does not allow it The outbound security rules are applied for the VM NIC NSG first, and then applied on the subnet NSG Multiple NICs assigned to a VM can exist in different subnets Azure VMs with multiple NICs in the same availability set do not have to have the same number of NICs, but the VMs must have at least two NICs When you attach an NIC to a VM, you need to ensure that they exist in the same location and subscription The NIC and the VNet must exist in the same subscription and location The NIC's MAC address cannot be changed until the VM to which the NIC is assigned is deleted Once the VM is created, you cannot change the VNet to which it is assigned; however, you can change the subnet to which the VM is assigned You cannot attach an existing NIC to a VM during its creation, but you can add an existing NIC as an additional NIC By default, a dynamic public IP address is assigned to the VM during creation, but this address will change if the VM is stopped or deleted; to ensure it will not change, you need to ensure its IP address is static In a multi-NIC VM, the NSG that is applied to one NIC does not affect the others If you found this post useful, do check out the book Hands-On Networking with Azure, to design and implement Azure Networking for Azure VMs. Read More Introducing Azure Sphere – A secure way of running your Internet of Things devices Learn Azure Serverless computing for free: Download e-book
Read more
  • 0
  • 0
  • 26931

article-image-digital-forensics-using-autopsy
Savia Lobo
24 May 2018
10 min read
Save for later

Getting started with Digital forensics using Autopsy

Savia Lobo
24 May 2018
10 min read
Digital forensics involves the preservation, acquisition, documentation, analysis, and interpretation of evidence from various storage media types. It is not only limited to laptops, desktops, tablets, and mobile devices but also extends to data in transit which is transmitted across public or private networks. In this tutorial, we will cover how one can carry out digital forensics with Autopsy. Autopsy is a digital forensics platform and graphical interface to the sleuth kit and other digital forensics tools. This article is an excerpt taken from the book, 'Digital Forensics with Kali Linux', written by Shiva V.N. Parasram. Let's proceed with the analysis using the Autopsy browser by first getting acquainted with the different ways to start Autopsy. Starting Autopsy Autopsy can be started in two ways. The first uses the Applications menu by clicking on Applications | 11 - Forensics | autopsy: Alternatively, we can click on the Show applications icon (last item in the side menu) and type autopsy into the search bar at the top-middle of the screen and then click on the autopsy icon: Once the autopsy icon is clicked, a new terminal is opened showing the program information along with connection details for opening The Autopsy Forensic Browser. In the following screenshot, we can see that the version number is listed as 2.24 with the path to the Evidence Locker folder as /var/lib/autopsy: To open the Autopsy browser, position the mouse over the link in the terminal, then right-click and choose Open Link, as seen in the following screenshot: Creating a new case To create a new case, follow the given steps: When the Autopsy Forensic Browser opens, investigators are presented with three options. Click on NEW CASE: Enter details for the Case Name, Description, and Investigator Names. For the Case Name, I've entered SP-8-dftt, as it closely matches the image name (8-jpeg-search.dd), which we will be using for this investigation. Once all information is entered, click NEW CASE: Several investigator name fields are available, as there may be instances where several investigators may be working together. The locations of the Case directory and Configuration file are displayed and shown as created.  It's important to take note of the case directory location, as seen in the screenshot: Case directory (/var/lib/autopsy/SP-8-dftt/) created. Click ADD HOST to continue: Enter the details for the Host Name (name of the computer being investigated) and the Description of the host. Optional settings: Time zone: Defaults to local settings, if not specified Timeskew Adjustment: Adds a value in seconds to compensate for time differences Path of Alert Hash Database: Specifies the path of a created database of known bad hashes Path of Ignore Hash Database: Specifies the path of a created database of known good hashes similar to the NIST NSRL: Click on the ADD HOST button to continue. Once the host is added and directories are created, we add the forensic image we want to analyze by clicking the ADD IMAGE button: Click on the ADD IMAGE FILE button to add the image file: To import the image for analysis, the full path must be specified. On my machine, I've saved the image file (8-jpeg-search.dd) to the Desktop folder. As such, the location of the file would be /root/Desktop/ 8-jpeg-search.dd. For the Import Method, we choose Symlink. This way the image file can be imported from its current location (Desktop) to the Evidence Locker without the risks associated with moving or copying the image file. If you are presented with the following error message, ensure that the specified image location is correct and that the forward slash (/) is used: Upon clicking Next, the Image File Details are displayed. To verify the integrity of the file, select the radio button for Calculate the hash value for this image, and select the checkbox next to Verify hash after importing? The File System Details section also shows that the image is of a ntfs partition. Click on the ADD button to continue: After clicking the ADD button in the previous screenshot, Autopsy calculates the MD5 hash and links the image into the evidence locker. Press OK to continue: At this point, we're just about ready to analyze the image file. If there are multiple cases listed in the gallery area from any previous investigations you may have worked on, be sure to choose the 8-jpeg-search.dd file and case: Before proceeding, we can click on the IMAGE DETAILS option. This screen gives detail such as the image name, volume ID, file format, file system, and also allows for the extraction of ASCII, Unicode, and unallocated data to enhance and provide faster keyword searches. Click on the back button in the browser to return to the previous menu and continue with the analysis: Before clicking on the ANALYZE button to start our investigation and analysis, we can also verify the integrity of the image by creating an MD5 hash, by clicking on the IMAGE INTEGRITY button: Several other options exist such as FILE ACTIVITY TIMELINES, HASH DATABASES, and so on. We can return to these at any point in the investigation. After clicking on the IMAGE INTEGRITY button, the image name and hash are displayed. Click on the VALIDATE button to validate the MD5 hash: The validation results are displayed in the lower-left corner of the Autopsy browser window: We can see that our validation was successful, with matching MD5 hashes displayed in the results. Click on the CLOSE button to continue. To begin our analysis, we click on the ANALYZE button: Analysis using Autopsy Now that we've created our case, added host information with appropriate directories, and added our acquired image, we get to the analysis stage. After clicking on the ANALYZE button (see the previous screenshot), we're presented with several options in the form of tabs, with which to begin our investigation: Let's look at the details of the image by clicking on the IMAGE DETAILS tab. In the following snippet, we can see the Volume Serial Number and the operating system (Version) listed as Windows XP: Next, we click on the FILE ANALYSIS tab. This mode opens into File Browsing Mode, which allows the examination of directories and files within the image. Directories within the image are listed by default in the main view area: In File Browsing Mode, directories are listed with the Current Directory specified as C:/. For each directory and file, there are fields showing when the item was WRITTEN, ACCESSED, CHANGED, and CREATED, along with its size and META data: WRITTEN: The date and time the file was last written to ACCESSED: The date and time the file was last accessed (only the date is accurate) CHANGED: The date and time the descriptive data of the file was modified CREATED: The data and time the file was created META: Metadata describing the file and information about the file: For integrity purposes, MD5 hashes of all files can be made by clicking on the GENERATE MD5 LIST OF FILES button. Investigators can also make notes about files, times, anomalies, and so on, by clicking on the ADD NOTE button: The left pane contains four main features that we will be using: Directory Seek: Allows for the searching of directories File Name Search: Allows for the searching of files by Perl expressions or filenames ALL DELETED FILES: Searches the image for deleted files EXPAND DIRECTORIES: Expands all directories for easier viewing of contents By clicking on EXPAND DIRECTORIES, all contents are easily viewable and accessible within the left pane and main window. The + next to a directory indicates that it can be further expanded to view subdirectories (++) and their contents: To view deleted files, we click on the ALL DELETED FILES button in the left pane. Deleted files are marked in red and also adhere to the same format of WRITTEN, ACCESSED, CHANGED, and CREATED times. From the following screenshot, we can see that the image contains two deleted files: We can also view more information about this file by clicking on its META entry. By viewing the metadata entries of a file (last column to the right), we can also view the hexadecimal entries for the file, which may give the true file extensions, even if the extension was changed. In the preceding screenshot, the second deleted file (file7.hmm) has a peculiar file extension of .hmm. Click on the META entry (31-128-3) to view the metadata: Under the Attributes section, click on the first cluster labelled 1066 to view header information of the file: We can see that the first entry is .JFIF, which is an abbreviation for JPEG File Interchange Format. This means that the file7.hmm file is an image file but had its extension changed to .hmm. Sorting files Inspecting the metadata of each file may not be practical with large evidence files. For such an instance, the FILE TYPE feature can be used. This feature allows for the examination of existing (allocated), deleted (unallocated), and hidden files. Click on the FILE TYPE tab to continue: Click Sort files into categories by type (leave the default-checked options as they are) and then click OK to begin the sorting process: Once sorting is complete, a results summary is displayed. In the following snippet, we can see that there are five Extension Mismatches: To view the sorted files, we must manually browse to the location of the output folder, as Autopsy 2.4 does not support viewing of sorted files. To reveal this location, click on View Sorted Files in the left pane: The output folder locations will vary depending on the information specified by the user when first creating the case, but can usually be found at /var/lib/autopsy/<case name>/<host name>/output/sorter-vol#/index.html. Once the index.html file has been opened, click on the Extension Mismatch link: The five listed files with mismatched extensions should be further examined by viewing metadata content, with notes added by the investigator. Reopening cases in Autopsy Cases are usually ongoing and can easily be restarted by starting Autopsy and clicking on OPEN CASE: In the CASE GALLERY, be sure to choose the correct case name and, from there, continue your examination: To recap, we looked at forensics using the Autopsy Forensic Browser with The Sleuth Kit. Compared to individual tools, Autopsy has case management features and supports various types of file analysis, searching, and sorting of allocated, unallocated, and hidden files. Autopsy can also perform hashing on a file and directory levels to maintain evidence integrity. If you enjoyed reading this article, do check out, 'Digital Forensics with Kali Linux' to take your forensic abilities and investigations to a professional level, catering to all aspects of a digital forensic investigation from hashing to reporting. What is Digital Forensics? IoT Forensics: Security in an always connected world where things talk Working with Forensic Evidence Container Recipes
Read more
  • 0
  • 0
  • 72212

article-image-what-matters-on-an-engineering-resume-hacker-rank-report-says-skills-not-certifications
Richard Gall
24 May 2018
6 min read
Save for later

What matters on an engineering resume? Hacker Rank report says skills, not certifications

Richard Gall
24 May 2018
6 min read
Putting together an engineering resume can be a real headache. What should you include? How can you best communicate your experience and skills? Software engineers are constantly under pressure to deliver new projects and fix problems while learning new skills. Documenting the complexity of developer life in a straightforward and marketable manner is a challenge to say the least. Luckily, hiring managers and tech recruiters today recognize just how difficult communicating skill and competency in an engineering resume can be. A report by Hacker Rank revealed that the things that feature on a resume aren't that highly valued by recruiters and hiring managers. However, skills does remain top of the agenda: the question, really, is about how we demonstrate and communicate those skills. The quality of your previous experience matters on an engineering resume Hacker Rank found that hiring managers and tech recruiters value previous experience over everything else. 77% of survey respondents said previous experience was one of the 3 most important qualifications before a formal interview. In second place was years of experience with 46%. The difference between the two is subtle but important; it's offers a useful takeaway for engineers creating an engineering resume. Essentially, the quality of your experience is more important than the quantity of your experience. You need to make sure you communicate the details of your employment experiences. It sounds obvious but it's worth stating: applying for an engineering job isn't just a competition based on who has the most experience. You should explain the nature of the projects you are working on. The skills you used are essential, but being clear about how the project supported wider strategic or tactical goals is also important. This demonstrates not only your skills, but also your contextual awareness. It suggests to a hiring manager or recruiter you not only have the competence, but that you are also a team player with commercial awareness. Certifications aren't that important on your resume One of the most interesting insights from the Hacker Rank report was that both hiring managers and recruiters don't really care about certifications any more. Less than 16% listed it as one of the 3 most important things they look at during the recruitment process. Does this mean, then, that the certification is well and truly over? At this stage, it's hard to tell. But it does point to a wider cultural change that probably has a lot to do with open source. Because change is built into the reality of open source software, certifications are never going to be able to keep up with what's new and important. The things you learned to pass one year will likely be out of date the next. It probably also says something about the nature of technical roles today. Years ago, engineers would start a job knowing what they were going to be using. The toolchains and tech stacks would be relatively stable and consistent. In this context, certification was like a license, proving you understood the various components of a given tool or suite of tools. But today, it's more important for engineers to prove that they are both adaptable and capable of solving a range of different problems. With that in mind, it's essential to demonstrate your flexibility on your engineering resume. Make it clear that you're able to learn new things quickly, and that you can adapt your skill set to the problems you need to solve. You don't need to look good on paper to get the job... but it's going to help Hacker Rank's research also revealed that 75% of recruiters and hiring managers have hired people they initially thought didn't look good on paper. But that doesn't necessarily mean you should stop working on your resume. If anything, what this shows is that if you get your resume right, you could really catch someone's attention. You need to consider everything in your resume. Traditional resumes have a pretty clear structure, whatever job you're applying for, but if Hacker Rank's research tells us anything, it's that a an engineering resume requires a slightly different approach. Personal projects are more important than your portfolio on an engineering resume A further insight from Hacker Rank's report suggests one way you might adopt a different approach to your resume. Responding to the same question as the one we looked at above, 37% said personal projects were one of the 3 most important factors in determining whether to invite a candidate to interview. By contrast, only 22% said portfolio. This seems strange - surely a portfolio offers a deeper insight into someone's professional experience. Personal projects are more like hobbies, right? Personal projects actually tell you so much more about a candidate than a portfolio. A portfolio is largely determined by the work you have been doing. What's more, it's not always that easy to communicate value in a portfolio. Equally, if you've been badly managed, or faced a lack of support, your portfolio might not actually be a good reflection of how good you really are. Personal projects give you an insight into how a person thinks. It shows recruiters what makes an engineer tick. In the workplace your scope for creativity and problem solving might well be limited. With personal projects you're free to test out ideas try new tools. You're able to experiment. So, when you're putting together an engineering resume, make sure you dedicate some time outlining your personal projects. Consider these sorts of questions: Why did you start a project? What did you find interesting? What did you learn? Engineering skills still matter Just because the traditional resume appears to have fallen out of favor, it doesn't mean your skills don't matter. In fact, skill matters more than ever. For a third of hiring managers, skill assessments are the area they want to invest in. This would allow them to judge a candidate's competencies and skills much more effectively than simply looking at a resume. As we've seen, things like personal projects are valuable because they demonstrate skills in a way that is often difficult. They not only prove you have the technical skills you say you have, they also provide a good indication of how you think and how you might approach solving problems. They can help illustrate how you deploy those skills. And when its so easy to learn how to write lines of code (no bad thing, true), showing how you think and apply your skills is a sure fire way to make sure you stand out from the crowd. Read next: How to assess your tech team's skills Are technical skills overrated when hiring tech pros?  ‘Soft’ Skills Every Data Pro Needs
Read more
  • 0
  • 0
  • 42946

article-image-build-progressive-web-apps-firebase
Savia Lobo
24 May 2018
19 min read
Save for later

Build powerful progressive web apps with Firebase

Savia Lobo
24 May 2018
19 min read
Progressive Web Apps describe a collection of technologies, design concepts, and Web APIs. They work in tandem to provide an app-like experience on the mobile web. In this tutorial, we'll discuss diverse recipes and show what it takes to turn any application into a powerful, fully-optimized progressive web application using Firebase magic. This article is an excerpt taken from the book,' Firebase Cookbook', written by Houssem Yahiaoui. In this book, you will learn how to create cross-platform mobile apps, integrate Firebase in native platforms, and learn how to monetize your mobile applications using Admob for Android and iOS. Integrating Node-FCM in NodeJS server We'll see how we can fully integrate the FCM (Firebase Cloud Messaging) with any Nodejs application. This process is relatively easy and straightforward, and we will see how it can be done in a matter of minutes. How to do it... Let's ensure that our working environment is ready for our project, so let's install a couple of dependencies in order to ensure that everything will run smoothly: Open your terminal--in case you are using macOS/Linux--or your cmd--if you're using Windows--and write down the following command: ~> npm install fcm-push --save By using this command, we are downloading and installing the fcm-push library locally. If this is executed, it will help with managing the process of sending web push notification to our users. The NodeJS ecosystem has npm as part of it. So in order to have it on your working machine, you will have to download NodeJS and configure your system within. Here's the official NodeJS link; download and install the suitable version to your system: https://nodejs.org/en/download/. Now that we have successfully installed the library locally, in order to integrate it with our applications and start using it, we will just need another line of code. Within your local development project, create a new file that will host the functionality and simply implement the following code: const FCM = require('fcm-push'); Congratulations! We're typically done. In subsequent recipes, we'll see how to manage our way through the sending process and how we can successfully send our user's web push notification. Implementing service workers Service workers were, in fact, the missing piece in the web scenes of the past. They are what give us the feel of reactivity to any state that a web application, after integrating, can have from, for example, a notification message, an offline-state (no internet connection) and more. In this recipe, we'll see how we can integrate service worker into our application. Service workers files are event-driven, so everything that will happen inside them will be event based. Since it's JavaScript, we can always hook up listeners to any event. To do that, we will want to give a special logic knowing that if you will use this logic, the event will behave in its default way. You can read more about service workers and know how you can incorporate them--to achieve numerous awesome features to your application that our book won't cover--from https://developers.google.com/web/fundamentals/getting-started/primers/service-workers. How to do it... Services workers will live in the browser, so including them within your frontend bundle is the most suitable place for them. Keeping that in mind, let's create a file called firebase-messaging-sw.js and manifest.json file. The JavaScript file will be our service worker file and will host all major workload, and the JSON file will simply be a metadata config file. After that, ensure that you also create app.js file, where this file will be the starting point for our Authorization and custom UX.  We will look into the importance of each file individually later on, but for now, go back to the firebase-messaging-sw.js file and write the following: //[*] Importing Firebase Needed Dependencies importScripts('https://www.gstatic.com/firebasejs/ 3.5.2/firebase-app.js'); importScripts('https://www.gstatic.com/firebasejs/ 3.5.2/firebase-messaging.js'); // [*] Firebase Configurations var config = { apiKey: "", authDomain: "", databaseURL: "", storageBucket: "", messagingSenderId: "" }; //[*] Initializing our Firebase Application. firebase.initializeApp(config); // [*] Initializing the Firebase Messaging Object. const messaging = firebase.messaging(); // [*] SW Install State Event. self.addEventListener('install', function(event) { console.log("Install Step, let's cache some files =D"); }); // [*] SW Activate State Event. self.addEventListener('activate', function(event) { console.log('Activated!', event);}); Within any service worker file, the install event is always fired first. Within this event, we can handle and add custom logic to any event we want. This can range from saving a local copy of our application in the browser cache to practically anything we want. Inside your metadata file, which will be the manifest.json files, write the following line: { "name": "Firebase Cookbook", "gcm_sender_id": "103953800507" } How it works... For this to work, we're doing the following: Importing using importScripts while considering this as the script tag with the src attribute in HTML, the firebase app, and the messaging libraries. Then, we're introducing our Firebase config object; we've already discussed where you can grab that object content in the past chapters. Initializing our Firebase app with our config file. Creating a new reference from the firebase.messaging library--always remember that everything in Firebase starts with a reference. We're listening to the install and activate events and printing some stdout friendly message to the browser debugger. Also, within our manifest.json file, we're adding the following metadata: The application name (optional). The gcm_sender_id with the given value. Keep in mind that this value will not change for any new project you have or will create in the future. The gcm_sender_id added line might get deprecated in the future, so keep an eye on that. Implementing sending/receiving registration using Socket.IO By now, we've integrated our FCM server and made our service worker ready to host our awesome custom logic. Like we mentioned, we're about to send web push notifications to our users to expand and enlarge their experience without application. Lately, web push notification is being considered as an engagement feature that any cool application nowadays, ranging from Facebook and Twitter to numerous e-commerce sites, is making good use of. So in the first approach, let's see how we can make it happen with Socket.IO. In order to make the FCM server aware of any client--basically, a browser--Browser OEM has what we call a registration_id. This token is a unique token for every browser that will represent our clients by their browsers and needs to be sent to the FCM server. Each browser generates its own registration_id token. So if your user uses, for instance, chrome for their first interaction with the server and they used firefox for their second experience, the web push notification message won't be sent, and they need to send another token so that they can be notified. How to do it... Now, go back to your NodeJS project that was created in the first recipe. Let's download the node.js socket.io dependency: ~> npm install express socket.io --save Socket.io is also event-based and lets us create our custom event for everything we have, plus some native one for connections. Also, we've installed ExpressJS for configuration sake in order to create a socket.io server. Now after doing that, we need to configure our socket.io server using express. In this case, do as shown in the following code: const express = require('express'); const app = express(); app.io = require('socket.io')(); // [*] Configuring our static files. app.use(express.static('public/')); // [*] Configuring Routes. app.get('/', (req, res) => { res.sendFile(__dirname + '/public/index.html'); }); // [*] Configuring our Socket Connection. app.io.on('connection', socket => { console.log('Huston ! we have a new connection ...'); }) So let's discuss the preceding code, where we are simply doing the following: Importing express and creating a new express app. Using the power of object on the fly, we've included the socket.io package over a new sub-object over the express application. This integration makes our express application now support the usage of  socket.io over the Application. We're indicating that we want to use the public folder as our static files folder, which will host our HTML/CSS/Javascript/IMG resources. We're listening to the connection, which will be fired once we have a new upcoming client. Once a new connection is up, we're printing a friendly message over our console. Let's configure our frontend. Head directly to your index.html files located in the public folder, and add the following line at the head of your page: <script src="/socket.io/socket.io.js"></script> The socket.io.js file will be served in application launch time, so don't worry much if you don't have one locally. Then, at the bottom of our index.html file before the closing of our <body> tag, add the following: <script> var socket = io.connect('localhost:3000'); </script> In the preceding script, we're connecting our frontend to our socket.io backend. Supposedly, our server is in port 3000; this way, we ensured that our two applications are running in sync. Head to your app.js files--created in the earlier recipes; create and import it if you didn't do so already--and introduce the following: //[*] Importing Firebase Needed Dependencies importScripts('https://www.gstatic.com/firebasejs/ 3.5.2/firebase-app.js'); importScripts('https://www.gstatic.com/firebasejs/ 3.5.2/firebase-messaging.js'); // [*] Firebase Configurations var config = { apiKey: "", authDomain: "", databaseURL: "", storageBucket: "", messagingSenderId: "" }; //[*] Initializing our Firebase Application. firebase.initializeApp(config); // [*] Initializing the Firebase Messaging Object. const messaging = firebase.messaging(); Everything is great; don't forget to import the app.js file within your index.html file. We will now see how we can grab our registration_id token next: As I explained earlier, the registration token is unique per browser. However, in order to get it, you should know that this token is considered a privacy issue. To ensure that they do not fall into the wrong hands, it's not available everywhere and to get it, you need to ask for the Browser User Permission. Since you can use it in any particular application, let's see how we can do that. The registration_id token will be considered as security and privacy threat to the user in case this token has been compromised, because once the attacker or the hacker gets the tokens, they can send or spam users with push messages that might have a malicious intent within, so safely keeping and saving those tokens is a priority. Now, within your app.js files that we created early on, let's add the following lines of code underneath the Firebase messaging reference: messaging.requestPermission() .then(() => { console.log("We have permission !"); return messaging.getToken(); }) .then((token) => { console.log(token); socket.emit("new_user", token); }) .catch(function(err) { console.log("Huston we have a problem !",err); }); We've sent out token using the awesome feature of socket.io. In order to get it now, let's simply listen to the same event and hope that we will get some data over our NodeJS backend. We will now proceed to learn about receiving registration token: Back to the app.js file, inside our connection event, let's add the following code: socket.on('new_user', (endpoint) => { console.log(endpoint); //TODO : Add endpoint aka.registration_token, to secure place. }); Our socket.io logic will look pretty much as shown in the following code block: // [*] Configuring our Socket Connection. app.io.on('connection', socket => { console.log('Huston ! we have a new connection ...'); socket.on('new_user', (endpoint) => { console.log(endpoint); //TODO : Add endpoint aka.registration_token, to secure place. }); }); Now, we have our bidirectional connection set. Let's grab that regitration_token and save it somewhere safe for further use. How it works... In the first part of the recipe, we did the following: Importing using importScripts considering it a script tag with src attribute in HTML, the firebase app, and messaging libraries. Then, we're introducing our Firebase Config object. We've already discussed where you can grab that object content in the previous chapters. Initializing our Firebase app with our config file. Creating a new reference from firebase.messaging library--always remember that everything in Firebase starts with a reference. Let's discuss what we did previously in the section where we talked about getting the registration_token value: We're using the Firebase messaging reference that we created earlier and executing the requestPermission() function, which returns a promise. Next--assuming that you're following along with this recipe--you will do the following over your page. After launching the development server, you will get a request from your page (Figure 1): Now, let's get back to the code. If we allow the notification, the promise resolver will be executed and then we will return the registration_token value from themessaging.getToken(). Then, we're sending that token over a socket and emitting that with an even name of new_user. Socket.io uses web sockets and like we explained before, sockets are event based. So in order to have a two-way connection between nodes, we need to emit, that is, send an event after giving it a name and listening to the same event with that name. Remember the socket.io event's name because we'll incorporate that into our further recipes. Implementing sending/receiving registration using post requests In a different approach from the one used in Socket.io, let's explore the other way around using post requests. This means that we'll use a REST API that will handle all that for us. At the same time, it will handle saving the registration_token value in a secure place as well. So let's see how we can configure that. How to do it... First, let's start writing our REST API. We will do that by creating an express post endpoint. This endpoint will be porting our data to the server, but before that, let's install some dependencies using the following line: ~> npm install express body-parser --save Let's discuss what we just did: We're using npm to download ExpressJS locally to our development directory. Also, we're downloading body-parser. This is an ExpressJS middleware that will host all post requests data underneath the body subobject. This module is quite a common package in the NodeJS community, and you will find it pretty much everywhere. Now, let's configure our application. Head to the app.js file and add the following code lines there: const express = require('express'); const app = express(); const bodyParser = require('body-parser'); // [*] Configuring Body Parser. app.use(bodyParser.json()); // [*] Configuring Routes. app.post('/regtoken', (req, res) => { let reg_token =req.body.regtoken; console.log(reg_token); //TODO : Create magic while saving this token in secure place. }); In the preceding code, we're doing the following: Importing our dependencies including ExpressJS and BodyParser. In the second step, we're registering the body parser middleware. You can read more about body parser and how to properly configure it to suit your needs from https://github.com/expressjs/body-parser. Next, we're creating an express endpoint or a route. This route will host our custom logic to manage the retrieval of the registration token sent from our users. Now, let's see how we can send the registration token to form our user's side. In this step, you're free to use any HTTP client you seek. However, in order to keep things stable, we'll use the browser native fetch APIs. Since we've managed to fully configure our routes, it can host the functionality we want. Let's see how we can get the registration_token value and send it to our server using post request and the native browser HTTP client named fetch: messaging.requestPermission() .then(() => { console.log("We have permission !"); return messaging.getToken(); }) .then((token) => { console.log(token); //[*] Sending the token fetch("http://localhost:3000/regtoken", { method: "POST" }).then((resp) => { //[*] Handle Server Response. }) .catch(function(err) { //[*] Handle Server Error. }) }) .catch(function(err) { console.log("Huston we have a problem !", err); }); How it works... Let's discuss what we just wrote in the preceding code: We're using the Firebase messaging reference that we created earlier and executing the requestPermission() function that returns a promise. Next, supposing that you're following along with this recipe, after launching the development server you will get the following authorization request from your page: Moving back to the code, if we allow the notification, the promise resolver will be executed and then we will return the registration_token value from the messaging.getToken(). Next, we're using the Fetch API given it a URL as a first parameter and a method name that is post and handling the response and error. Since we saw the different approaches to exchange data between the client and the server, in the next recipe, we will see how we can receive web push notification messages from the server. Receiving web push notification messages We can definitely say that things are on a good path. We managed to configure our message to the server in the past recipes. Now, let's work on getting the message back from the server in a web push message. This is a proven way to gain more leads and regain old users. It is a sure way to re-engage your users, and success stories do not lie. Facebook, Twitter, and e-commerce websites are the living truth on how a web push message can make a difference in your ecosystem and your application in general. How to do it... Let's see how we can unleash the power of push messages. The API is simple and the way to do has never been easier, so let's how we can do that! Let's write down the following code over our firebase-messaging-sw.js file: // [*] Special object let us handle our Background Push Notifications messaging.setBackgroundMessageHandler(function(payload) { return self.registration.showNotification(payload.data.title, body: payload.data.body); }); Let's explain the preceding code: We're using the already created messaging object created using the Firebase messaging library, and we're calling. the setBackgroundMessageHandler() function. This will mean that we will catch all the messages that we will keep receiving in the background. We're using the service worker object represented in the self-object, and we're calling the showNotification() function and passing it some parameters. The first parameter is the title, and we're grabbing it from the server; we'll see how we can get it in just a second. The second parameter is the body of the message. Now, we've prepared our frontend to received messages. Let's send them from the server, and we will see how we can do that using the following code: var fcm = new FCM('<FCM_CODE>'); var message = { to: data.endpoint, // required fill with device token or topics notification: { title: data.payload.title, body: data.payload.body } }; fcm.send(message) .then(function(response) { console.log("Successfully sent with response: ", response); }) .catch(function(err) { console.log("Something has gone wrong!"); console.error(err); }) }); The most important part is FCM_CODE. You can grab it in the Firebase console by going to the Firebase Project Console and clicking on the Overview tab (Figure 3): Then, go to the CLOUD MESSAGING tab and copy and paste the Server Key in the section (Figure 4): How it works... Now, let's discuss the code that what we just wrote: The preceding code can be put everywhere, which means that you can send push notifications in every part of your application. We're composing our notification message by putting the registration token and the information we want to send. We're using the fcm.send() method in order to send our notification to the wanted users. Congratulations! We're done; now go and test your awesome new functionality! Implementing custom notification messages In the previous recipes, we saw how we can send a normal notification. Let's add some controllers and also learn how to add some pictures to it so that we can prettify it a bit. How to do it... Now write the following code and add it to our earlier code over the messaging.setBackgroundMessageHandler() function. So, the end result will look something like this: // [*] Special object let us handle our Background Push Notifications messaging.setBackgroundMessageHandler(function(payload) { const notificationOptions = { body: payload.data.msg, icon: "images/icon.jpg", actions: [ { action : 'like', title: 'Like', image: '<link-to-like-img>' }, { action : 'dislike', title: 'Dislike', image: '<link-to-like-img>' } ] } self.addEventListener('notificationclick', function(event) { var messageId = event.notification.data; event.notification.close(); if (event.action === 'like') { console.log("Going to like something !"); } else if (event.action === 'dislike') { console.log("Going to dislike something !"); } else { console.log("wh00t !"); } }, false); return self.registration.showNotification( payload.data.title,notificationOptions); }); How it works... Let's discuss what we've done so far: We added the notificationOptions object that hosts some of the required metadata, such as the body of the message and the image. Also, in this case, we're adding actions, which means we'll add custom buttons to our notification message. These will range from title to image, the most important part is the action name. Next, we listened on notificationclick, which will be fired each time one of the actions will be selected. Remember the action field we added early on; it'll be the differentiation point between all actions we might add. Then, we returned the notification and showed it using the showNotification() function. We saw how to build powerful progressive applications using Firebase. If you've enjoyed reading this tutorial, do check out, 'Firebase Cookbook', for creating serverless applications with Firebase Cloud Functions or turning your traditional applications into progressive apps with Service workers. What is a progressive web app? Windows launches progressive web apps… that don’t yet work on mobile Hybrid Mobile apps: What you need to know
Read more
  • 0
  • 0
  • 28801

article-image-arduino-based-follow-me-drone
Vijin Boricha
23 May 2018
9 min read
Save for later

How to build an Arduino based 'follow me' drone

Vijin Boricha
23 May 2018
9 min read
In this tutorial, we will learn how to train the drone to do something or give the drone artificial intelligence by coding from scratch. There are several ways to build Follow Me-type drones. We will learn easy and quick ways in this article. Before going any further, let's learn the basics of a Follow Me drone. This is a book excerpt from Building Smart Drones with ESP8266 and Arduino written by Syed Omar Faruk Towaha. If you are a hardcore programmer and hardware enthusiast, you can build an Arduino drone, and make it a Follow Me drone by enabling a few extra features. For this section, you will need the following things: Motors ESCs Battery Propellers Radio-controller Arduino Nano HC-05 Bluetooth module GPS MPU6050 or GY-86 gyroscope. Some wires Connections are simple: You need to connect the motors to the ESCs, and ESCs to the battery. You can use a four-way connector (power distribution board) for this, like in the following diagram: Now, connect the radio to the Arduino Nano with the following pin configuration: Arduino pin Radio pin D3 CH1 D5 CH2 D2 CH3 D4 CH4 D12 CH5 D6 CH6 Now, connect the Gyroscope to the Arduino Nano with the following configuration: Arduino pin Gyroscope pin 5V 5V GND GND A4 SDA A5 SCL You are left with the four wires of the ESC signals; let's connect them to the Arduino Nano now, as shown in the following configuration: Arduino pin Motor signal pin D7 Motor 1 D8 Motor 2 D9 Motor 3 D10 Motor 4 Our connection is almost complete. Now we need to power the Arduino Nano and the ESCs. Before doing that, making common the ground means connecting both the wired to the ground. Before going any further, we need to upload the code to the brain of our drone, which is the Arduino Nano. The code is a little bit big. I am going to explain the code after installing the necessary library. You will need a library installed to the Arduino library folder before going to the programming part. The library's name is PinChangeInt. Install the library and write the code for the drone. The full code can be found at Github. Let's explain the code a little bit. In the code, you will find lots of functions with calculations. For our gyroscope, we needed to define all the axes, sensor data, pin configuration, temperature synchronization data, I2C data, and so on. In the following function, we have declared two structures for the accel and gyroscope data with all the directions: typedef union accel_t_gyro_union { struct { uint8_t x_accel_h; uint8_t x_accel_l; uint8_t y_accel_h; uint8_t y_accel_l; uint8_t z_accel_h; uint8_t z_accel_l; uint8_t t_h; uint8_t t_l; uint8_t x_gyro_h; uint8_t x_gyro_l; uint8_t y_gyro_h; uint8_t y_gyro_l; uint8_t z_gyro_h; uint8_t z_gyro_l; } reg; struct { int x_accel; int y_accel; int z_accel; int temperature; int x_gyro; int y_gyro; int z_gyro; } value; }; In the void setup() function of our code, we have declared the pins we have connected to the motors: myservoT.attach(7); //7-TOP myservoR.attach(8); //8-Right myservoB.attach(9); //9 - BACK myservoL.attach(10); //10 LEFT We also called our test_gyr_acc() and test_radio_reciev() functions, for testing the gyroscope and receiving data from the remote respectively. In our test_gyr_acc() function. In our test_gyr_acc() function, we have checked if it can detect our gyroscope sensor or not and set a condition if there is an error to get gyroscope data then to set our pin 13 high to get a signal: void test_gyr_acc() { error = MPU6050_read (MPU6050_WHO_AM_I, &c, 1); if (error != 0) { while (true) { digitalWrite(13, HIGH); delay(300); digitalWrite(13, LOW); delay(300); } } } We need to calibrate our gyroscope after testing if it connected. To do that, we need the help of mathematics. We will multiply both the rad_tilt_TB and rad_tilt_LR by 2.4 and add it to our x_a and y_a respectively. then we need to do some more calculations to get correct x_adder and the y_adder: void stabilize() { P_x = (x_a + rad_tilt_LR) * 2.4; P_y = (y_a + rad_tilt_TB) * 2.4; I_x = I_x + (x_a + rad_tilt_LR) * dt_ * 3.7; I_y = I_y + (y_a + rad_tilt_TB) * dt_ * 3.7; D_x = x_vel * 0.7; D_y = y_vel * 0.7; P_z = (z_ang + wanted_z_ang) * 2.0; I_z = I_z + (z_ang + wanted_z_ang) * dt_ * 0.8; D_z = z_vel * 0.3; if (P_z > 160) { P_z = 160; } if (P_z < -160) { P_z = -160; } if (I_x > 30) { I_x = 30; } if (I_x < -30) { I_x = -30; } if (I_y > 30) { I_y = 30; } if (I_y < -30) { I_y = -30; } if (I_z > 30) { I_z = 30; } if (I_z < -30) { I_z = -30; } x_adder = P_x + I_x + D_x; y_adder = P_y + I_y + D_y; } We then checked that our ESCs are connected properly with the escRead() function. We also called elevatorRead() and aileronRead() to configure our drone's elevator and the aileron. We called test_radio_reciev() to test if the radio we have connected is working, then we called check_radio_signal() to check if the signal is working. We called all the stated functions from the void loop() function of our Arduino code. In the void loop() function, we also needed to configure the power distribution of the system. We added a condition, like the following: if(main_power > 750) { stabilize(); } else { zero_on_zero_throttle(); } We also set a boundary; if main_power is greater than 750 (which is a stabling value for our case), then we stabilize the system or we call zero_on_zero_throttle(), which initializes all the values of all the directions. After uploading this, you can control your drone by sending signals from your remote control. Now, to make it a Follow Me drone, you need to connect a Bluetooth module or a GPS. You can connect your smartphone to the drone by using a Bluetooth module (HC-05 preferred) or another Bluetooth module as master-slave usage. And, of course, to make the drone follow you, you need the GPS. So, let's connect them to our drone. To connect the Bluetooth module, follow the following configuration: Arduino pin Bluetooth module pin TX RX RX TX 5V 5V GND GND See the following diagram for clarification: For the GPS, connect it as shown in the following configuration: Arduino pin GPS pin D11 TX D12 RX GND GND 5V 5V See the following diagram for clarification: Since all the sensors usages 5V power, I would recommend using an external 5V power supply for better communication, especially for the GPS. If we use the Bluetooth module, we need to make the drone's module the slave module and the other module the master module. To do that, you can set a pin mode for the master and then set the baud rate to at least 38,400, which is the minimum operating baud rate for the Bluetooth module. Then, we need to check if one module can hear the other module. For that, we can write our void loop() function as follows: if(Serial.available() > 0) { state = Serial.read(); } if (state == '0') { digitalWrite(Pin, LOW); state = 0; } else if (state == '1') { digitalWrite(Pin, HIGH); state = 0; } And do the opposite for the other module, connecting it to another Arduino. Remember, you only need to send and receive signals, so refrain from using other utilities of the Bluetooth module, for power consumption and swiftness. If we use the GPS, we need to calibrate the compass and make it able to communicate with another GPS module. We need to read the long value from the I2C, as follows: float readLongFromI2C() { unsigned long tmp = 0; for (int i = 0; i < 4; i++) { unsigned long tmp2 = Wire.read(); tmp |= tmp2 << (i*8); } return tmp; } float readFloatFromI2C() { float f = 0; byte* p = (byte*)&f; for (int i = 0; i < 4; i++) p[i] = Wire.read(); return f; } Then, we have to get the geo distance, as follows, where DEGTORAD is a variable that changes degree to radian: float geoDistance(struct geoloc &a, struct geoloc &b) { const float R = 6371000; // Earth radius float p1 = a.lat * DEGTORAD; float p2 = b.lat * DEGTORAD; float dp = (b.lat-a.lat) * DEGTORAD; float dl = (b.lon-a.lon) * DEGTORAD; float x = sin(dp/2) * sin(dp/2) + cos(p1) * cos(p2) * sin(dl/2) * sin(dl/2); float y = 2 * atan2(sqrt(x), sqrt(1-x)); return R * y; } We also need to write a function for the Geo bearing, where lat and lon are latitude and longitude respectively, gained from the raw data of the GPS sensor: float geoBearing(struct geoloc &a, struct geoloc &b) { float y = sin(b.lon-a.lon) * cos(b.lat); float x = cos(a.lat)*sin(b.lat) - sin(a.lat)*cos(b.lat)*cos(b.lon-a.lon); return atan2(y, x) * RADTODEG; } You can also use a mobile app to communicate with the GPS and make the drone move with you. Then the process is simple. Connect the GPS to your drone and get the TX and RX data from the Arduino and spread it through the radio and receive it through the telemetry, and then use the GPS from the phone with DroidPlanner or Tower. You also need to add a few lines in the main code to calibrate the compass. You can see the previous calibration code. The calibration of the compass varies from location to location. So, I would suggest you use the try-error method. In the following section, I will discuss how you can use an ESP8266 to make a GPS tracker that can be used with your drone. We learned to build a Follow Me-type drone and also used DroidPlanner 2 and Tower to configure it. Know more about using a smartphone to enable the follow me feature of ArduPilot and GPS tracker using ESP8266 from this book Building Smart Drones with ESP8266 and Arduino. Read More
Read more
  • 1
  • 0
  • 36561

article-image-is-web-development-dying
Richard Gall
23 May 2018
7 min read
Save for later

Is web development dying?

Richard Gall
23 May 2018
7 min read
It's not hard to find people asking whether web development is dying. A quick search throws up questions on Quora, Reddit, and other forums. "Is web development a dying profession or does it just smell funny?" asks one Reddit user. The usual suspects in the world of content (Forbes et al) have responded with their own takes and think pieces on whether web development is dead. And why wouldn't they? I, for one, would never miss out on an opportunity to write something with an outlandish and provocative headline for clicks. So, is web development dying or simply very unwell? Why do people think web development is dying? The question might seem a bit overwrought, but there are good reasons for people to ask the question. One reason is that getting a website has never been easier or cheaper. Think about it: if you want to create a content site, it doesn't take much to set one up with WordPress. You barely need to be technically literate, let alone a developer. Similarly, if you want an eCommerce store there are plenty of off-the-shelf solutions that allow people to start running an online business with very little work at all. Even if you do want a custom solution, you can now do that pretty cheaply. On the Treehouse forums, one user comments that thanks to sites like SquareSpace, businesses can now purchase a website for less than £100 (about $135). The commenter remarks that whereas he'd typically charge around £3000 for a complete website build, potential clients are coming back puzzled as to why he would think they'd spend so much when they could get the same result for a fraction of the price. From a professional perspective, this sort of anecdotal evidence indicates that it's becoming more and more difficult to be successful in web development. For all the talk around 'learning to code' and the digital economy, maybe building websites isn't the best area to get into. Web development is getting easier When people say web development is dying, they might actually be saying that there isn't as much money in it any more. If freelancers are struggling to charge the rates that they used to, that's because there is someone out there who is going to do it for a lot less money. The reason for this isn't that there's a new generation of web developers able to subsist on a paltry sum of money. It's actually getting a lot easier. Aside from solutions like WordPress and Shopify, the task of building websites from scratch (sort of scratch) is now easier than it has ever been. Are templates killing web development? Templates make everything easy for web developers and designers. Why would you want to do much more than drag and drop templates if you could? If the result looks good and does the job, then why spend time doing more? The more you do yourself, the more you're likely to break things. And the more you break things the more you've got to fix. Of course, templates are lowering the barrier to entry into web development and design. And while we shouldn't be precious about new web developers entering the industry, it is understandable that many experienced web developers are anxious about what the future might hold. From this perspective, templates aren't killing web development, but they are changing what the profession looks like. And without wishing to sound euphemistic, this is both a challenge and an opportunity for everyone in web development. Whether you're experienced or new to the industry, these changes mean people are going to have to adapt. Web development isn't dying, it's fragmenting The way web developers are going to have to adapt is by choosing what path they want to take in their career. Web development as we've always known it is, perhaps well and truly dead. Instead, it's fragmenting into specialized areas; design on the one hand, and full-stack on the other. This means your skill set needs to be unique. In a world where building websites takes very little skill or technical knowledge, specific expertise is vital. This is something journalist Andrew Pierno noted in a blog post on Medium. Pierno writes:  ...we are in a scenario where the web developer no longer has the skill set to build that interesting differentiator anymore, particularly if the main value prop is around A.I, computer vision, machine learning, AR, VR, blockchain, etc. Building websites is no longer remarkable - as we've seen, people that can do it are ubiquitous. But building a native application; that's not quite so easy. Building a mobile app that uses computer vision to compare you to Renaissance paintings - that's even harder to do. These are the sorts of things that are going to be valuable - and these are the sorts of things that web developers are going to need to learn how to do. Full-stack development and the expansion of the developer skill set In his piece, Pierno argues that the scope of the web developers role is shrinking. However, I don't think that's quite right. Yes, it might be fragmenting, but the scope of, say, full-stack development, is huge. In fact, full-stack developers need to know a huge range of technologies and tools. If they're to differentiate themselves in the job market, as Pierno suggests they should, they need to know machine learning, they need to know mobile, databases, and maybe even Blockchain. From this perspective, it's not hard to see how the 'web' part of web development might be dying. To some extent, as the web becomes more ubiquitous and less of a rarefied 'space' in people's lives, the more we have to get into the detail of how we utilize the technologies around it. Web development's decline is design's gain If web development as a discipline is dying, that's only going to make design more important. If, as we saw earlier, building websites is going to become a free for all for just about anyone with an internet connection and enough confidence, standards and quality might start to slip. That means the value of someone who understands good design will be higher than ever. As a web developer you might disappear into the ether of everyone else out there. But if you market yourself as a designer, someone who understands the intricacies of UI and UX implicitly, you immediately start to look a little different. Think of it like a sandwich shop - anyone can start making sandwiches. But to make a great sandwich shop, the type that wins awards and the type that people want to Instagram, requires extra attention to detail. It demands more skill and more culinary awareness. Maybe web development is dying, but maybe it just needs to change Clearly, what we call web development is very different in 2018 than what it was 5 years ago. There are a huge number of reasons for this, but perhaps the most important is that it doesn't really make sense to talk about 'the web' any more. Because 'the web' is now an outdated concept, perhaps web development needs to die. Maybe we're holding on to something which is only going to play into the hands of poor design and poor quality software. It's might even damage the careers of talented engineers and designers. You could make a pretty good comparison between 'the web' and 'big data'. Even reading those words feels oddly outdated today, but they're still at the center of the tech landscape. Big data, for example, is everywhere - it's hard to imagine our lives outside of it, but it doesn't make sense to talk about it in the abstract. Instead, what's interesting is how it's applied, how engineers make data accessible, usable and secure. The same is true of the web. It's not dead, but it has certainly assumed a slightly different form. And web development might well be dying, but the world will always need developers and designers. It's simply time to adapt. Read next Why is everyone talking about JavaScript fatigue? Is novelty ruining web development?
Read more
  • 0
  • 4
  • 51771
Unlock access to the largest independent learning library in Tech for FREE!
Get unlimited access to 7500+ expert-authored eBooks and video courses covering every tech area you can think of.
Renews at $19.99/month. Cancel anytime
article-image-unity-variables-script-unity-2017-games
Amarabha Banerjee
23 May 2018
12 min read
Save for later

Working with Unity Variables to script powerful Unity 2017 games

Amarabha Banerjee
23 May 2018
12 min read
In this tutorial, you will learn how to work with the different variables available with the Unity 2017 platform. We will show you how to use these variables through use cases in order to script powerful Unity games. This article is an excerpt from the book, Learning C# by Developing Games with Unity 2017 written by Micael DaGraca, Greg Lukosek. The most popular game engine of our generation i.e. Unity is a preferred choice among game developers. Due to the flexibility it provides to code and script a game in C#. To understand and leverage the power of C# in your games, it is utterly necessary to get a proper understanding of how C# coding works. We are going to show you exactly that in the section given below. Writing C# statements properly When you do normal writing, it's in the form of a sentence, with a period used to end the sentence. When you write a line of code, it's called a statement, with a semicolon used to end the statement. This is necessary because the console reads the code one line at a time and it's necessary to use a semicolon to tell the console that the line of code is over and that the console can jump to the next line. (This is happening so fast that it looks like the computer is reading all of them at the same time, but it isn't.) When we start learning how to code, forgetting about this detail is very common, so don't forget to check for this error if the code isn't working: The code for a C# statement does not have to be on a single line as shown in the following example: public int number1 = 2; The statement can be on several lines. Whitespace and carriage returns are ignored, so, if you really want to, you can write it as follows: public int number1 = 2; However, I do not recommend writing your code like this because it's terrible to read code that is formatted like the preceding code. Nevertheless, there will be times when you'll have to write long statements that are longer than one line. Unity won't care. It just needs to see the semicolon at the end. Understanding component properties in Unity's Inspector GameObjects have some components that make them behave in a certain way. For instance, select Main Camera and look at the Inspector panel. One of the components is the camera. Without that component, it will cease being a camera. It would still be a GameObject in your scene, just no longer a functioning camera. Variables become component properties When we refer to components, we are basically referring to the available functions of a GameObject, for example, the human body has many functions, such as talking, moving, and observing. Now let's say that we want the human body to move faster. What is the function linked to that action? Movement. So in order to make our body move faster, we would need to create a script that had access to the movement component and then we would use that to make the body move faster. Just like in real life, different GameObjects can also have different components, for example, the camera component can only be accessed from a camera. There are plenty of components that already exist that were created by Unity's programmers, but we can also write our own components. This means that all the properties that we see in Inspector are just variables of some type. They simply store data that will be used by some method. Unity changes script and variable names slightly When we create a script, one of the first things that we need to do is give a name to the script and it's always good practice to use a name that identifies the content of the script. For example, if we are creating a script that is used to control the player movement, ideally that would be the name of the script. The best practice is to write playerMovement, where the first word is uncapitalized and the second one is capitalized. This is the standard way Unity developers name scripts and variables. Now let's say that we created a script named playerMovement. After assigning that script to a GameObject, we'll see that in the Inspector panel we would see that Unity adds a space to separate the words of the name, Player Movement. Unity does this modification to variable names too where, for example, a variable named number1 is shown as Number 1 and number2 as Number 2. Unity capitalizes the first letter as well. These changes improve readability in Inspector. Changing a property's value in the Inspector panel There are two situations where you can modify a property value: During the Play mode During the development stage (not in the Play mode) When you are in the Play mode, you will see that your changes take effect immediately in real time. This is great when you're experimenting and want to see the results. Write down any changes that you want to keep because when you stop the Play mode, any changes you made will be lost. When you are in the Development mode, changes that you make to the property values will be saved by Unity. This means that if you quit Unity and start it again, the changes will be retained. Of course, you won't see the effect of your changes until you click Play. The changes that you make to the property values in the Inspector panel do not modify your script. The only way your script can be changed is by you editing it in the script editor (MonoDevelop). The values shown in the Inspector panel override any values you might have assigned in your script. If you want to undo the changes you've made in the Inspector panel, you can reset the values to the default values assigned in your script. Click on the cog icon (the gear) on the far right of the component script, and then select Reset, as shown in the following screenshot: Displaying public variables in the Inspector panel You might still be wondering what the word public at the beginning of a variable statement means: public int number1 = 2; We mentioned it before. It means that the variable will be visible and accessible. It will be visible as a property in the Inspector panel so that you can manipulate the value stored in the variable. The word also means that it can be accessed from other scripts using the dot syntax. Private variables Not all variables need to be public. If there's no need for a variable to be changed in the Inspector panel or be accessed from other scripts, it doesn't make sense to clutter the Inspector panel with needless properties. In the LearningScript, perform the following steps: Change line 6 to this: private int number1 = 2; Then change line 7 to the following: int number2 = 9; Save the file In Unity, select Main Camera You will notice in the Inspector panel that both properties, Number 1 and Number 2, are gone: Line 6: private int number1 = 2; The preceding line explicitly states that the number1 variable has to be private. Therefore, the variable is no longer a property in the Inspector panel. It is now a private variable for storing data: Line 7: int number2 = 9; The number2 variable is no longer visible as a property either, but you didn't specify it as private. If you don't explicitly state whether a variable will be public or private, by default, the variable will implicitly be private in C#. It is good coding practice to explicitly state whether a variable will be public or private. So now, when you click Play, the script works exactly as it did before. You just can't manipulate the values manually in the Inspector panel anymore. Naming Unity variables properly As we explored previously, naming a script or variable is a very important step. It won't change the way that the code runs, but it will help us to stay organized and, by using best practices, we are avoiding errors and saving time trying to find the piece of code that isn't working. Always use meaningful names to store your variables. If you don't do that, six months down the line, you will be lost. I'm going to exaggerate here a bit to make a point. Let's say you will name a variable as shown in this code: public bool areRoadConditionsPerfect = true; That's a descriptive name. In other words, you know what it means by just reading the variable. So 10 years from now, when you look at that name, you'll know exactly what I meant in the previous comment. Now suppose that instead of areRoadConditionsPerfect, you had named this variable as shown in the following code: public bool perfect = true; Sure, you know what perfect is, but would you know that it refers to perfect road conditions? I know that right now you'll understand it because you just wrote it, but six months down the line, after writing hundreds of other scripts for all sorts of different projects, you'll look at this word and wonder what you meant. You'll have to read several lines of code you wrote to try to figure it out. You may look at the code and wonder who in their right mind would write such terrible code. So, take your time to write descriptive code that even a stranger can look at and know what you mean. Believe me, in six months or probably less time, you will be that stranger. Using meaningful names for variables and methods is helpful not only for you but also for any other game developer who will be reading your code. Whether or not you work in a team, you should always write easy-to-read code. Beginning variable names with lowercase You should begin a variable name with a lowercase letter because it helps distinguish between a class name and a variable name in your code. There are some other guides in the C# documentation as well, but we don't need to worry about them at this stage. Component names (class names) begin with an uppercase letter. For example, it's easy to know that Transform is a class and transform is a variable. There are, of course, exceptions to this general rule, and every programmer has a preferred way of using lowercase, uppercase, and perhaps an underscore to begin a variable name. In the end, you will have to decide upon a naming convention that you like. If you read the Unity forums, you will notice that there are some heated debates on naming variables. In this book, I will show you my preferred way, but you can use whatever is more comfortable for you. Using multiword variable names Let's use the same example again, as follows: public bool areRoadConditionsPerfect = true; You can see that the variable name is actually four words squeezed together. Since variable names can be only one word, begin the first word with a lowercase and then just capitalize the first letter of every additional word. This greatly helps create descriptive names that the viewer is still able to read. There's a term for this, it's called camel casing. I have already mentioned that for public variables, Unity's Inspector will separate each word and capitalize the first word. Go ahead! Add the previous statement to the LearningScript and see what Unity does with it in the Inspector panel. Declaring a variable and its type Every variable that we want to use in a script must be declared in a statement. What does that mean? Well, before Unity can use a variable, we have to tell Unity about it first. Okay then, what are we supposed to tell Unity about the variable? There are only three absolute requirements to declare a variable and they are as follows: We have to specify the type of data that a variable can store We have to provide a name for the variable We have to end the declaration statement with a semicolon The following is the syntax we use to declare a variable: typeOfData nameOfTheVariable; Let's use one of the LearningScript variables as an example; the following is how we declare a variable with the bare minimum requirements: int number1; This is what we have: Requirement #1 is the type of data that number1 can store, which in this case is an int, meaning an integer Requirement #2 is a name, which is number1 Requirement #3 is the semicolon at the end The second requirement of naming a variable has already been discussed. The third requirement of ending a statement with a semicolon has also been discussed. The first requirement of specifying the type of data will be covered next. The following is what we know about this bare minimum declaration as far as Unity is concerned: There's no public modifier, which means it's private by default It won't appear in the Inspector panel or be accessible from other scripts The value stored in number1 defaults to zero We discussed working with the Unity 2017 variables and how you can start working with them to create fun-filled games effectively. If you liked this article, be sure to go through the book Learning C# by Developing games with Unity 2017 to create exciting games with C# and Unity 2017. Read More Unity 2D & 3D game kits simplify Unity game development for beginners Build a Virtual Reality Solar System in Unity for Google Cardboard Unity Machine Learning Agents: Transforming Games with Artificial Intelligence
Read more
  • 0
  • 0
  • 31671

article-image-restful-java-web-services-swagger
Fatema Patrawala
22 May 2018
14 min read
Save for later

Documenting RESTful Java Web Services using Swagger

Fatema Patrawala
22 May 2018
14 min read
In the earlier days, many software solution providers did not really pay attention to documenting their RESTful web APIs. However, many API vendors soon realized the need for a good API documentation solution. Today, you will find a variety of approaches to documenting RESTful web APIs. There are some popular solutions available today for describing, producing, consuming, and visualizing RESTful web services. In this tutorial, we will explore Swagger which offers a specification and a complete framework implementation for describing, producing, consuming, and visualizing RESTful web services. The Swagger framework works with many popular programming languages, such as Java, Scala, Clojure, Groovy, JavaScript, and .NET. This tutorial is an extract taken from the book RESTFul Java Web Services - Third Edition, written by Bogunuva Mohanram Balachandar. This book will help you master core REST concepts and create RESTful web services in Java. A glance at the market adoption of Swagger The greatest strength of Swagger is its powerful API platform, which satisfies the client, documentation, and server needs. The Swagger UI framework serves as the documentation and testing utility. Its support for different languages and its matured tooling support have really grabbed the attention of many API vendors, and it seems to be the one with the most traction in the community today. Swagger is built using Scala. This means that when you package your application, you need to have the entire Scala runtime into your build, which may considerably increase the size of your deployable artifact (the EAR or WAR file). That said, Swagger is however improving with each release. For example, the Swagger 2.0 release allows you to use YAML for describing APIs. So, keep a watch on this framework. The Swagger framework has the following three major components: Server: This component hosts the RESTful web API descriptions for the services that the clients want to use Client: This component uses the RESTful web API descriptions from the server to provide an automated interfacing mechanism to invoke the REST APIs User interface: This part of the framework reads a description of the APIs from the server, renders it as a web page, and provides an interactive sandbox to test the APIs A quick overview of the Swagger structure Let's take a quick look at the Swagger file structure before moving further. The Swagger 1.x file contents that describe the RESTful APIs are represented as the JSON objects. With Swagger 2.0 release onwards, you can also use the YAML format to describe the RESTful web APIs. This section discusses the Swagger file contents represented as JSON. The basic constructs that we'll discuss in this section for JSON are also applicable for the YAML representation of APIs, although the syntax differs. When using the JSON structure for describing the REST APIs, the Swagger file uses a Swagger object as the root document object for describing the APIs. Here is a quick overview of the various properties that you will find in a Swagger object: The following properties describe the basic information about the RESTful web application: swagger: This property specifies the Swagger version. info: This property provides metadata about the API. host: This property locates the server where the APIs are hosted. basePath: This property is the base path for the API. It is relative to the value set for the host field. schemes: This property transfers the protocol for a RESTful web API, such as HTTP and HTTPS. The following properties allow you to specify the default values at the application level, which can be optionally overridden for each operation: consumes: This property specifies the default internet media types that the APIs consume. It can be overridden at the API level. produces: This property specifies the default internet media types that the APIs produce. It can be overridden at the API level. securityDefinitions: This property globally defines the security schemes for the document. These definitions can be referred to from the security schemes specified for each API. The following properties describe the operations (REST APIs): paths: This property specifies the path to the API or the resources. The path must be appended to the basePath property in order to get the full URI. definitions: This property specifies the input and output entity types for operations on REST resources (APIs). parameters: This property specifies the parameter details for an operation. responses: This property specifies the response type for an operation. security: This property specifies the security schemes in order to execute this operation. externalDocs: This property links to the external documentation. A complete Swagger 2.0 specification is available at Github. The Swagger specification was donated to the Open API initiative, which aims to standardize the format of the API specification and bring uniformity in usage. Swagger 2.0 was enhanced as part of the Open API initiative, and a new specification OAS 3.0 (Open API specification 3.0) was released in July 2017. The tools are still being worked on to support OAS3.0. I encourage you to explore the Open API Specification 3.0 available at Github. Overview of Swagger APIs The Swagger framework consists of many sub-projects in the Git repository, each built with a specific purpose. Here is a quick summary of the key projects: swagger-spec: This repository contains the Swagger specification and the project in general. swagger-ui: This project is an interactive tool to display and execute the Swagger specification files. It is based on swagger-js (the JavaScript library). swagger-editor: This project allows you to edit the YAML files. It is released as a part of Swagger 2.0. swagger-core: This project provides the scala and java library to generate the Swagger specifications directly from code. It supports JAX-RS, the Servlet APIs, and the Play framework. swagger-codegen: This project provides a tool that can read the Swagger specification files and generate the client and server code that consumes and produces the specifications. In the next section, you will learn how to use the swagger-core project offerings to generate the Swagger file for a JAX-RS application. Generating Swagger from JAX-RS Both the WADL and RAML tools that we discussed in the previous sections use the JAX-RS annotations metadata to generate the documentation for the APIs. The Swagger framework does not fully rely on the JAX-RS annotations but offers a set of proprietary annotations for describing the resources. This helps in the following scenarios: The Swagger core annotations provide more flexibility for generating documentations compliant with the Swagger specifications It allows you to use Swagger for generating documentations for web components that do not use the JAX-RS annotations, such as servlet and the servlet filter The Swagger annotations are designed to work with JAX-RS, improving the quality of the API documentation generated by the framework. Note that the swagger-core project is currently supported on the Jersey and Restlet implementations. If you are considering any other runtime for your JAX-RS application, check the respective product manual and ensure the support before you start using Swagger for describing APIs. Some of the commonly used Swagger annotations are as follows: The @com.wordnik.swagger.annotations.Api annotation marks a class as a Swagger resource. Note that only classes that are annotated with @Api will be considered for generating the documentation. The @Api annotation is used along with class-level JAX-RS annotations such as @Produces and @Path. Annotations that declare an operation are as follows: @com.wordnik.swagger.annotations.ApiOperation: This annotation describes a resource method (operation) that is designated to respond to HTTP action methods, such as GET, PUT, POST, and DELETE. @com.wordnik.swagger.annotations.ApiParam: This annotation is used for describing parameters used in an operation. This is designed for use in conjunction with the JAX-RS parameters, such as @Path, @PathParam, @QueryParam, @HeaderParam, @FormParam, and @BeanParam. @com.wordnik.swagger.annotations.ApiImplicitParam: This annotation allows you to define the operation parameters manually. You can use this to override the @PathParam or @QueryParam values specified on a resource method with custom values. If you have multiple ImplicitParam for an operation, wrap them with @ApiImplicitParams. @com.wordnik.swagger.annotations.ApiResponse: This annotation describes the status codes returned by an operation. If you have multiple responses, wrap them by using @ApiResponses. @com.wordnik.swagger.annotations.ResponseHeader: This annotation describes a header that can be provided as part of the response. @com.wordnik.swagger.annotations.Authorization: This annotation is used within either Api or ApiOperation to describe the authorization scheme used on a resource or an operation. Annotations that declare API models are as follows: @com.wordnik.swagger.annotations.ApiModel: This annotation describes the model objects used in the application. @com.wordnik.swagger.annotations.ApiModelProperty: This annotation describes the properties present in the ApiModel object. A complete list of the Swagger core annotations is available at Github. Having learned the basics of Swagger, it is time for us to move on and build a simple example to get a feel of the real-life use of Swagger in a JAX-RS application. As always, this example uses the Jersey implementation of JAX-RS. Specifying dependency to Swagger To use Swagger in your Jersey 2 application, specify the dependency to swagger-jersey2-jaxrs jar. If you use Maven for building the source, the dependency to the swagger-core library will look as follows: <dependency> <groupId>com.wordnik</groupId> <artifactId>swagger-jersey2-jaxrs</artifactId> <version>1.5.1-M1</version> <!-use the appropriate version here, 1.5.x supports Swagger 2.0 spec --> </dependency> You should be careful while choosing the swagger-core version for your product. Note that swagger-core 1.3 produces the Swagger 1.2 definitions, whereas swagger-core 1.5 produces the Swagger 2.0 definitions. The next step is to hook the Swagger provider components into your Jersey application. This is done by configuring the Jersey servlet (org.glassfish.jersey.servlet.ServletContainer) in web.xml, as shown here: <servlet> <servlet-name>jersey</servlet-name> <servlet-class> org.glassfish.jersey.servlet.ServletContainer </servlet-class> <init-param> <param-name>jersey.config.server.provider.packages </param-name> <param-value> com.wordnik.swagger.jaxrs.json, com.packtpub.rest.ch7.swagger </param-value> </init-param> <load-on-startup>1</load-on-startup> </servlet> <servlet-mapping> <servlet-name>jersey</servlet-name> <url-pattern>/webresources/*</url-pattern> </servlet-mapping> To enable the Swagger documentation features, it is necessary to load the Swagger framework provider classes from the com.wordnik.swagger.jaxrs.listing package. The package names of the JAX-RS resource classes and provider components are configured as the value for the jersey.config.server.provider.packages init parameter. The Jersey framework scans through the configured packages for identifying the resource classes and provider components during the deployment of the application. Map the Jersey servlet to a request URI so that it responds to the REST resource calls that match the URI. If you prefer not to use web.xml, you can also use the custom application subclass for (programmatically) specifying all the configuration entries discussed here. To try this option, refer to Github. Configuring the Swagger definition After specifying the Swagger provider components, the next step is to configure and initialize the Swagger definition. This is done by configuring the com.wordnik.swagger.jersey.config.JerseyJaxrsConfig servlet in web.xml, as follows: <servlet> <servlet-name>Jersey2Config</servlet-name> <servlet-class> com.wordnik.swagger.jersey.config.JerseyJaxrsConfig </servlet-class> <init-param> <param-name>api.version</param-name> <param-value>1.0.0</param-value> </init-param> <init-param> <param-name>swagger.api.basepath</param-name> <param-value> http://localhost:8080/hrapp/webresources </param-value> </init-param> <load-on-startup>1</load-on-startup> </servlet> Here is a brief overview of the initialization parameters used for JerseyJaxrsConfig: api.version: This parameter specifies the API version for your application swagger.api.basepath: This parameter specifies the base path for your application With this step, we have finished all the configuration entries for using Swagger in a JAX-RS (Jersey 2 implementation) application. In the next section, we will see how to use the Swagger metadata annotation on a JAX-RS resource class for describing the resources and operations. Adding a Swagger annotation on a JAX-RS resource class Let's revisit the DepartmentResource class used in the previous sections. In this example, we will enhance the DepartmentResource class by adding the Swagger annotations discussed earlier. We use @Api to mark DepartmentResource as the Swagger resource. The @ApiOperation annotation describes the operation exposed by the DepartmentResource class: import com.wordnik.swagger.annotations.Api; import com.wordnik.swagger.annotations.ApiOperation; import com.wordnik.swagger.annotations.ApiParam; import com.wordnik.swagger.annotations.ApiResponse; import com.wordnik.swagger.annotations.ApiResponses; //Other imports are removed for brevity @Stateless @Path("departments") @Api(value = "/departments", description = "Get departments details") public class DepartmentResource { @ApiOperation(value = "Find department by id", notes = "Specify a valid department id", response = Department.class) @ApiResponses(value = { @ApiResponse(code = 400, message = "Invalid department id supplied"), @ApiResponse(code = 404, message = "Department not found") }) @GET @Path("{id}") @Produces("application/json") public Department findDepartment( @ApiParam(value = "The department id", required = true) @PathParam("id") Integer id){ return findDepartmentEntity(id); } //Rest of the codes are removed for brevity } To view the Swagger documentation, build the source and deploy it to the server. Once the application is deployed, you can navigate to http://<host>:<port>/<application-name>/<application-path>/swagger.json to view the Swagger resource listing in the JSON format. The Swagger URL for this example will look like the following: http://localhost:8080/hrapp/webresource/swagger.json The following sample Swagger representation is for the DepartmentResource class discussed in this section: { "swagger": "2.0", "info": { "version": "1.0.0", "title": "" }, "host": "localhost:8080", "basePath": "/hrapp/webresources", "tags": [ { "name": "user" } ], "schemes": [ "http" ], "paths": { "/departments/{id}": { "get": { "tags": [ "user" ], "summary": "Find department by id", "description": "", "operationId": "loginUser", "produces": [ "application/json" ], "parameters": [ { "name": "id", "in": "path", "description": "The department id", "required": true, "type": "integer", "format": "int32" } ], "responses": { "200": { "description": "successful operation", "schema": { "$ref": "#/definitions/Department" } }, "400": { "description": "Invalid department id supplied" }, "404": { "description": "Department not found" } } } } }, "definitions": { "Department": { "properties": { "departmentId": { "type": "integer", "format": "int32" }, "departmentName": { "type": "string" }, "_persistence_shouldRefreshFetchGroup": { "type": "boolean" } } } } } As mentioned at the beginning of this section, from the Swagger 2.0 release onward it supports the YAML representation of APIs. You can access the YAML representation by navigating to swagger.yaml. For instance, in the preceding example, the following URI gives you the YAML file: http://<host>:<port>/<application-name>/<application-path>/swagger.yaml The complete source code for this example is available at the Packt website. You can download the example from the Packt website link that we mentioned at the beginning of this book, in the Preface section. In the downloaded source code, see the rest-chapter7-service-doctools/rest-chapter7-jaxrs2swagger project. Generating a Java client from Swagger The Swagger framework is packaged with the Swagger code generation tool as well (swagger-codegen-cli), which allows you to generate client libraries by parsing the Swagger documentation file. You can download the swagger-codegen-cli.jar file from the Maven central repository by searching for swagger-codegen-cli in search maven. Alternatively, you can clone the Git repository and build the source locally by executing mvn install. Once you have swagger-codegen-cli.jar locally available, run the following command to generate the Java client for the REST API described in Swagger: java -jar swagger-codegen-cli.jar generate -i <Input-URI-or-File-location-for-swagger.json> -l <client-language-to-generate> -o <output-directory> The following example illustrates the use of this tool: java -jar swagger-codegen-cli-2.1.0-M2.jar generate -i http://localhost:8080/hrapp/webresources/swagger.json -l java -o generated-sources/java When you run this tool, it scans through the RESTful web API description available at http://localhost:8080/hrapp/webresources/swagger.json and generates a Java client source in the generated-sources/java folder. Note that the Swagger code generation process uses the mustache templates for generating the client source. If you are not happy with the generated source, Swagger lets you specify your own mustache template files. Use the -t flag to specify your template folder. To learn more, refer to the README.md file at Github. To learn more grab this latest edition of RESTful Java Web Services to build robust, scalable and secure RESTful web services using Java APIs. Getting started with Django RESTful Web Services How to develop RESTful web services in Spring Testing RESTful Web Services with Postman
Read more
  • 0
  • 0
  • 49304

article-image-sdlc-puts-process-at-the-center-of-software-engineering
Richard Gall
22 May 2018
7 min read
Save for later

SDLC puts process at the center of software engineering

Richard Gall
22 May 2018
7 min read
What is SDLC? SDLC stands for software development lifecycle. It refers to all of the different steps that software engineers need to take when building software. This includes planning, creating, building and then deploying software, but maintenance is also crucial too. In some instances you may need to change or replace software - that is part of the software development lifecycle as well. SDLC is about software quality and development efficiency SDLC is about more than just the steps in the software development process. It's also about managing that process in a way that improves quality while also improving efficiency. Ultimately, there are numerous ways of approaching the software development lifecycle - Waterfall and Agile are the too most well known methodologies for managing the development lifecycle. There are plenty of reasons you might choose one over another. What is most important is that you pay close attention to what the software development lifecycle looks like. It sounds obvious, but it is very difficult to build software without a plan in place. Things can get chaotic very quickly. If it does, that's bad news for you, as the developer, and bad news for users as well. When you don't follow the software development lifecycle properly, you're likely to miss user requirements, and faults will also find their way into your code. The stages of the software development lifecycle (SDLC) There are a number of ways you might see an SDLC presented but the core should always be the same. And yes, different software methodologies like Agile and Waterfall outline very different ways of working, but broadly the steps should be the same. What differs between different software methodologies is how each step fits together. Step 1: Requirement analysis This is the first step in any SDLC. This is about understanding everything that needs to be understood in as much practical detail as possible. It might mean you need to find out about specific problems that need to be solved. Or, there might be certain things that users need that you need to make sure are in the software. To do this, you need to do good quality research, from discussing user needs with a product manager to revisiting documentation on your current systems. It is often this step that is the most challenging in the software development lifecycle. This is because you need to involve a wide range of stakeholders. Some of these might not be technical, and sometimes you might simply use a different vocabulary. It's essential that you have a shared language to describe everything from the user needs to the problems you might be trying to solve. Step 2: Design the software Once you have done a requirement analysis you can begin designing the software. You do this by turning all the requirements and software specifications into a design document. This might feel like it slows down the development process, but if you don't do this, not only are you wasting the time taken to do your requirement analysis, you're also likely to build poor quality or even faulty software. While it's important not to design by committee or get slowed down by intensive nave-gazing, keeping stakeholders updated and requesting feedback and input where necessary can be incredibly important. Sometimes its worth taking that extra bit of time, as it could solve a lot of problems later in the SDLC. Step 3: Plan the project Once you have captured requirements and feel you have properly understood exactly what needs to be delivered - as well as any potential constraints - you need to plan out how you're going to build that software. To do this you'll need to have an overview of the resources at your disposal. These are the sorts of questions you'll need to consider at this stage: Who is available? Are there any risks? How can we mitigate them? What budget do we have for this project? Are there any other competing projects? In truth, you'll probably do this during the design stage. The design document you create should, of course, be developed with context in mind. It's pointless creating a stunning design document, outlining a detailed and extensive software development project if it's simply not realistic for your team to deliver it. Step 4: Start building the software Now you can finally get down to the business of actually writing code. With all the work you have done in the previous steps this should be a little easier. However, it's important to remember that imperfection is part and parcel of software engineering. There will always be flaws in your software. That doesn't necessarily mean bugs or errors, however; it could be small compromises that need to be made in order to ensure something works. The best approach here is to deliver rapidly. The sooner you can get software 'out there' the faster you can make changes and improvements if (or more likely when) they're needed. It's worth involving stakeholders at this stage - transparency in the development process is a good way to build collaboration and ensure the end result delivers on what was initially in the requirements. Step 5: Testing the software Testing is, of course, an essential step in the software development lifecycle. This is where you identify any problems. That might be errors or performance issues, but you may find you haven't quite been able to deliver what you said you would in the design document. The continuous integration server is important here, as the continuous integration server can help to detect any problems with the software. The rise of automated software testing has been incredibly valuable; it means that instead of spending time manually running tests, engineers can dedicate more time to fixing problems and optimizing code. Step 6: Deploy the software The next step is to deploy the software to production. All the elements of the software should now be in place, and you want it to simply be used. It's important to remember that there will be problems here. Testing can never capture every issue, and feedback and insight from users are going to be much more valuable than automated tests run on a server. Continuous delivery pipelines allow you to deploy software very efficiently. This makes the build-test-deploy steps of the software development lifecycle to be relatively frictionless. Okay, maybe not frictionless - there's going to be plenty of friction when you're developing software. But it does allow you to push software into production very quickly. Step 7: Maintaining software Software maintenance is a core part of the day-to-day life of a software engineer. Its a crucial step in the SDLC. There are two forms of software maintenance; both are of equal importance. Evolutive maintenance and corrective maintenance. Evolutive maintenance As the name suggests, evolutive maintenance is where you evolve software by adding in new functionality or making larger changes to the logic of the software. These changes should be a response to feedback from stakeholders or, more importantly, users. There may be times when business needs dictate this type of maintenance - this is never ideal, but it is nevertheless an important part of a software engineer's work. Corrective maintenance Corrective maintenance isn't quite as interesting or creative as evolutive maintenance - it's about fixing bugs and errors in the code. This sort of maintenance can feel like a chore, and ideally you want to minimize the amount of time you spend doing this. However, if you're following SDLC closely, you shouldn't find too many bugs in your software. The benefits of SDLC are obvious The benefits of SDLC are clear. It puts process at the center of software engineering. Without those processes it becomes incredibly difficult to build the software that stakeholders and users want. And if you don't care about users then, really, why build software at all. It's true that DevOps has done a lot to change SDLC. Arguably, it is an area that is more important and more hotly debated than ever before. It's not difficult to find someone with an opinion on the best way to build something. Equally, as software becomes more fragmented and mutable, thanks to the emergence of cloud and architectural trends like microservices and serverless, the way we design, build and deploy software has never felt more urgent. Read next DevOps Engineering and Full-Stack Development – 2 Sides of the Same Agile Coin
Read more
  • 0
  • 0
  • 14572

article-image-firebase-nativescript-cross-platform-app-development
Savia Lobo
22 May 2018
16 min read
Save for later

How to integrate Firebase with NativeScript for cross-platform app development

Savia Lobo
22 May 2018
16 min read
NativeScript is now considered as one of the hottest platforms attracting developers. By using XML, JavaScript (also Angular), minor CSS for the visual aspects, and the magical touch of incorporating native SDKs, the platform has adopted the best of both worlds. Plus, this allows applications to be cross-platform, which means your application can run on both Android and iOS! In this tutorial, we're going to see how we can use Firebase to create some awesome native applications. This article is an excerpt taken from the book,' Firebase Cookbook', written by Houssem Yahiaoui. Getting started with NativeScript project In order to start a NativeScript project, we will need to make our development environment Node.js ready. So, in order to do so, let's download Node.js. Head directly to https://nodejs.org/en/download/ and download the most suitable version for your OS. After you have successfully installed Node.js, you will have two utilities. One is the Node.js executable and the other will be npm or the node package manager. This will help us download our dependencies and install NativeScript locally. How to do it... In your terminal/cmd of choice, type the following command: ~> npm install -g nativescript After installing NativeScript, you will need to add some dependencies. To know what your system is missing, simply type the following command: ~> tns doctor This command will test your system and make sure that everything is in place. If not, you'll get the missing parts. In order to create a new project, you will have many options to choose from. Those options could be creating a new project with vanilla JavaScript, with Typescript, from a template, or even better, using Angular. Head directly to your working directory and type the following command: ~> tns create <application-name> This command will initialize your project and install the basic needed dependencies. With that done, we're good to start working on our NativeScript project. Adding the Firebase plugin to our application One of NativeScript's most powerful features is the possibility of incorporating truly native SDKs. So, in this context, we can install the Firebase NativeScript on Android using the normal Gradle installation command. You can also do it on iOS via a Podfile if you are running macOS and want to create an iOS application along the way. However, the NativeScript ecosystem is pluggable, which means the ecosystem has plugins that extend certain functionalities. Those plugins usually incorporate native SDKs and expose the functionalities using JavaScript so we can exploit them directly within our application. In this recipe, we're going to use the wonderfully easy-to-use Eddy Verbruggen Firebase plugin, so let's see how we can add it to our project. How to do it... Head to your terminal/cmd of choice, type the following command, and hit Return/Enter respectively: tns plugin add nativescript-plugin-firebase This command will install the necessary plugin and do the required configuration. To find out the id, open your package.json file where you will find the NativeScript value: "nativescript": { "id": "org.nativescript.<your-app-name>" } Copy the id that you found in the preceding step and head over to your Firebase project console. Create a new Android/iOS application and paste that ID over your bundle name. Download the google-service.json/GoogleServices-Info.plist files and paste google-server.json in your app/Application_Resources/Android folder if you created an Android project. If you've created an iOS project, then paste the GoogleServices-Info.plist in the app/Application_Resources/iOS folder. Pushing/retrieving data from the Firebase Real-time Database Firebase stores data in a link-based manner that allows you to add and query the information really simple. The NativeScript Firebase plugin makes the operation much simpler with an easy-to-use API. So, let's discover how we can perform such operations. In this recipe, we're going to see how we can add and retrieve data in NativeScript and Firebase. Before we begin, we need to make sure that our application is fully configured with Firebase. You will also need to initialize the Firebase plugin with the application. To do that, open your project, head to your app.js file, and add the following import code: var firebase = require("nativescript-plugin- firebase"); This will import the Firebase NativeScript plugin. Next, add the following lines of code: firebase.init({}).then((instance) => { console.log("[*] Firebase was successfully initialised"); }, (error) => { console.log("[*] Huston we've an initialization error: " + error); }); The preceding code will simply go and initialize Firebase within our application. How to do it... After initializing our application, let's see how we can push some data to our Firebase Realtime Database. Let's start first by adding our interface, which will look similar to the one (Figure 1): Figure 1: Ideas adding page. The code behind it is as follows, and you can use this to implement the addition of new data to your bucket: <Page xmlns="http://schemas.nativescript.org/tns.xsd" navigatingTo="onNavigatingTo" class="page"> <Page.actionBar> <ActionBar title="Firebase CookBook" class="action-bar"> </ActionBar> </Page.actionBar> <StackLayout class="p-20"> <TextField text="{{ newIdea }}" hint="Add here your shiny idea"/> <Button text="Add new idea to my bucket" tap="{{ addToMyBucket }}" </span>class</span>="btn btn-primary btn- active"/> </StackLayout> </Page> Now let's see the JavaScript related to this UI for our behavior. Head to your view-model and add the following snippets inside the createViewModel function: viewModel.addToMyBucket = () => { firebase.push('/ideas', { idea: viewModel.newIdea }).then((result) => { console.log("[*] Info : Your data was pushed !"); }, (error) => { console.log("[*] Error : While pushing your data to Firebase, with error: " + error); }); } If you check your Firebase database, you will find your new entry present there. Once your data is live, you will need to think of a way to showcase all your shiny, new ideas. Firebase gave us a lovely event that we shall listen to whenever a new child element is created. The following code teaches you how to create the event for showcasing the addition of new child elements: var onChildEvent = function(result) { console.log("Idea: " + JSON.stringify(result.value)); }; firebase.addChildEventListener(onChildEvent, "/ideas").then((snapshot) => { console.log("[*] Info : We've some data !"); }); After getting the newly-added child, it's up to you to find the proper way to bind your ideas. They are mainly going to be either lists or cards, but they could be any of the previously mentioned ones. To run and experience your new feature, use the following command: ~> tns run android # for android ~> tns run ios # for ios How it works... So what just happened? We defined a basic user interface that will serve us by adding those new ideas to our Firebase console application. Next, the aim was to save all that information inside our Firebase Realtime Database using the same schema that Firebase uses. This is done via specifying a URL where all your information will be stored and then specifying the data schema. This will finally hold and define the way our data will be stored. We then hooked a listener to our data URL using firebase.addChildEventListener. This will take a function where the next item will be held and the data URL that we want our listener hook to listen on. In case you're wondering how this module or service works in NativeScript, the answer is simple. It's due to the way NativeScript works; because one of NativeScript's powerful features is the ability to use native SDKs. So in this case, we're using and implementing the Firebase Database Android/iOS SDKs for our needs, and the plugin APIs we're using are the JavaScript abstraction of how we want to exploit our native calls. Authenticating using anonymous or password authentication As we all know, Firebase supports both anonymous and password-based authentication, each with its own, suitable use case. So in this recipe, we're going to see how we can perform both anonymous and password authentication. You will need to initialize the Firebase plugin with the application. To do that, open your project, head to your app.js file, and add the following import: var firebase = require("nativescript-plugin- firebase"); This will import the Firebase NativeScript plugin. Next, add the following lines of code: firebase.init({}).then((instance) => { console.log("[*] Firebase was successfully initialised"); }, (error) => { console.log("[*] Huston we've an initialization error: " + error); }); The preceding code will go and initialize Firebase within our application. How to do it... Before we start, we need to create some UI elements. Your page will look similar to this one after you finish (Figure 2): Figure 2: Application login page. Now open your login page and add the following code snippets there: <Page xmlns="http://schemas.nativescript.org/tns.xsd" navigatingTo="onNavigatingTo" class="page"> <Page.actionBar> <ActionBar title="Firebase CookBook" icon="" class="action-bar"> </ActionBar> </Page.actionBar> <StackLayout> <Label text="Login Page" textWrap="true" style="font-weight: bold; font-size: 18px; text-align: center; padding:20"/> <TextField hint="Email" text="{{ user_email }}" /> <TextField hint="Password" text="{{ user_password }}" secure="true"/&gt;<Button text="LOGIN" tap="{{ passLogin }}" class="btn btn-primary btn-active" /> <Button text="Anonymous login" tap="{{ anonLogin }}" class="btn btn-success"/> </StackLayout> </Page> Save that. Let's now look at the variables and functions in our view-model file. For that, let's implement the passLogin and anonLogin functions. The first one will be our normal email and password authentication, and the second will be our anonymous login function. To make this implementation come alive, type the following code lines on your page: viewModel.anonLogin = () => { firebase.login({ type: firebase.LoginType.ANONYMOUS }).then((result) => { console.log("[*] Anonymous Auth Response:" + JSON.stringify(result)); },(errorMessage) => { console.log("[*] Anonymous Auth Error: "+errorMessage); }); } viewModel.passLogin = () => { let email = viewModel.user_email; let pass = viewModel.user_password; firebase.login({ type: firebase.LoginType.PASSWORD, passwordOptions: { email: email, password: pass } }).then((result) => { console.log("[*] Email/Pass Response : " + JSON.stringify(result)); }, (error) => { console.log("[*] Email/Pass Error : " + error); }); } Now, simply save your file and run it using the following command: ~> tns run android # for android ~> tns run ios # for ios How it works... Let's quickly understand what we've just done in the recipe: We've built the UI we needed as per the authentication type. If we want the email and password one, we will need the respective fields, whereas, for anonymous authentication, all we need is a button. Then, for both functions, we call the Firebase login button specifying the connection type for both cases. After finishing that part, it's up to you to define what is next and to retrieve that metadata from the API for your own needs later on. Authenticating using Google Sign-In Google Sign-In is one of the most popular integrated services in Firebase. It does not require any extra hustle, has the most functionality, and is popular among many apps. In this recipe, we're going to see how we can integrate Firebase Google Sign-In with our NativeScript project. You will need to initialize the Firebase plugin within the application. To do that, open your project, head to your app.js file, and add the following line: var firebase = require("nativescript-plugin- firebase"); This will import the Firebase NativeScript plugin. Next, add the following lines of code: firebase.init({}).then((instance) => { console.log("[*] Firebase was successfully initialised"); }, (error) => { console.log("[*] Huston we've an initialization error: " + error); }); The preceding code will go and initialize Firebase within our application. We will also need to install some dependencies. For that, underneath the NativeScript-plugin-firebase folder | platform | Android | include.gradle file, uncomment the following entry for Android: compile "com.google.android.gms:play-services- auth:$googlePlayServicesVersion" Now save and build your application using the following command: ~> tns build android Or uncomment this entry if you're building an iOS application: pod 'GoogleSignIn' Then, build your project using the following command: ~> tns build ios How to do it... First, you will need to create your button. So for this to happen, please go to your login-page.xml file and add the following button declaration: <Button text="Google Sign-in" tap="{{ googleLogin }}" class="btn" style="color:red"/> Now let's implement the googleLogin() function by using the following code snippet: viewModel.googleLogin = () => { firebase.login({ type: firebase.LoginType.GOOGLE, }).then((result) => { console.log("[*] Google Auth Response: " + JSON.</span>stringify(result)); },(errorMessage) => { console.log("[*] Google Auth Error: " + errorMessage); }); } To build and experience your new feature, use the following command: ~> tns run android # for android ~> tns run ios # for ios Now, once you click on the Google authentication button, you should have the following (Figure 3): Figure 3: Account picking after clicking on Google Login button. Don't forget to add your SHA-1 fingerprint code or the authentication process won't finish. How it works... Let's explain what just happened in the preceding code: We added the new button for the Google authentication. Within the tap event of this button, we gave it the googleLogin() function. Within  googleLogin(), we used the Firebase login button giving it firebase.LoginType.GOOGLE as type. Notice that, similar to normal Google authentication on a web platform, we can also give the hd or the hostedDomain option. We could also use the option of filtering the connection hosting we want by adding the following option under the login type: googleOptions: { hostedDomain: "<your-host-name>" } The hd option or the hostedDomain is simply what's after the @ sign in an email address. So, for example, in the email ID cookbook@packtpub.com the hosted domain is packtpub.com. For some apps, you might want to limit the email ID used by users when they connect to your application to just that host. This can be done by providing only the hostedDomain parameter in the code line pertaining to the storage of the email address. When you look at the actual way we're making these calls, you will see that it's due to the powerful NativeScript feature that lets us exploit native SDK. If you remember the Getting ready section of this recipe, we uncommented a section where we installed the native SDK for both Android and iOS. Besides the NativeScript firebase plugin, you can also exploit the Firebase Auth SDK, which will let you exploit all supported Firebase authentication methods. Adding dynamic behavior using Firebase Remote Config Remote Config is one of the hottest features of Firebase and lets us play with all the different application configurations without too much of a headache. By a headache, we mean the process of building, testing, and publishing, which usually takes a lot of our time, even if it's just to fix a small float number that might be wrong. So in this recipe, we're going to see how we can use Firebase Remote Config within our application. In case you didn't choose the functionality by default when you created your application, please head to your build.gradle and Podfile and uncomment the Firebase Remote Config line in both files or in the environment you're using with your application. How to do it... Actually, the integration part of your application is quite easy. The tricky part is when you want to toggle states or alter some configuration. So think upon that heavily, because it will affect how your application works and will also affect the way you change properties. Let's suppose that within this NativeScript application we want to have a mode called Ramadan mode. We want to create this mode for a special month where we wish to offer discounts, help our users with new promos, or even change our user interface to suit the spirit of it. So, let's see how we can do that: firebase.getRemoteConfig({ developerMode: true, cacheExpirationSeconds: 1, properties: [{ key: "ramadan_promo_enabled", default: false } }).then(function (result) { console.log("Remote Config: " + JSON.stringify( result.properties.ramadan_promo_enabled)); //TODO : Use the value to make changes. }); In the preceding code, and because we are still in development mode, we set that we want the developerMode to be activated. We also set the cacheExpirationSeconds to be one second. This is important because we don't want our settings to take a long time until they affect our application during the development phase. This will set the throttled mode to true, which will make the application fetch or look for new data every second to our Firebase remote configurations. We can set the default values of each and every item within our Firebase remote configuration. This value will be the starting point for fetching any new values that might be present over the Firebase project console. Now, let's see how we can wire that value from the project console. To do this, head to your Firebase project Console | Remote Config Section | ADD YOUR FIRST PARAMETER Button (Figure 4): Figure 4: Firebase Remote Config Parameter adding section. Next, you will get a modal where you will add your properties and their values. Make sure to add the exact same one that's in your code otherwise it won't work. The following screenshot shows the PARAMETERS tab of the console where you will add the properties (Figure 5): Figure 5: While adding the new parameter After adding them, click on the PUBLISH CHANGES button (Figure 6): Figure 6: Publishing the new created Parameter. With that, you're done. Exit your application and open it back up again. Watch how your console and your application fetches the new values. Then, it's up to you and your application to make the needed changes once the values are changed. How it works... Let's explain what just happened: We added back our dependencies from the build.gradle and Podfile so we can support the functionality we want to use. We've selected and added the code that will be responsible for giving the default values and for fetching the new changes. We have also activated the developer mode, which will help out in our development and staging phases. This mode will be disabled once we're in production. We've set the cache expiration time, which is essential while being in development so we can retrieve those values in a fast way. This too will be changed in production, by giving the cache more expiration time, because we don't want to jeopardize our application with high-throttled operations every second. We've added our support config in our Firebase Remote Config parameters, gave it the necessary value, and published it. This final step will control the way our application feels and looks like each new change. We learned how to integrate Firebase with NativeScript using various recipes. If you've enjoyed this article, do check out 'Firebase Cookbook' to change the way you develop and make your app a first class citizen of the cloud.
Read more
  • 0
  • 0
  • 40139
article-image-unity-arcore-application-android
Sugandha Lahoti
21 May 2018
11 min read
Save for later

Build an ARCore app with Unity from scratch

Sugandha Lahoti
21 May 2018
11 min read
In this tutorial, we will learn to install, build, and deploy Unity ARCore apps for Android. Unity is a leading cross-platform game engine that is exceptionally easy to use for building game and graphic applications quickly. Unity has developed something of a bad reputation in recent years due to its overuse in poor-quality games. It isn't because Unity can't produce high-quality games, it most certainly can. However, the ability to create games quickly often gets abused by developers seeking to release cheap games for profit. This article is an excerpt from the book, Learn ARCore - Fundamentals of Google ARCore, written by Micheal Lanham. The following is a summary of the topics we will cover in this article: Installing Unity and ARCore Building and deploying to Android Remote debugging Exploring the code Installing Unity and ARCore Installing the Unity editor is relatively straightforward. However, the version of Unity we will be using may still be in beta. Therefore, it is important that you pay special attention to the following instructions when installing Unity: Navigate a web browser to https://unity3d.com/unity/beta. At the time of writing, we will use the most recent beta version of Unity since ARCore is also still in beta preview. Be sure to note the version you are downloading and installing. This will help in the event you have issues working with ARCore. Click on the Download installer button. This will download UnityDownloadAssistant. Launch UnityDownloadAssistant. Click on Next and then agree to the Terms of Service. Click on Next again. Select the components, as shown: Install Unity in a folder that identifies the version, as follows: Click on Next to download and install Unity. This can take a while, so get up, move around, and grab a beverage. Click on the Finish button and ensure that Unity is set to launch automatically. Let Unity launch and leave the window open. We will get back to it shortly. Once Unity is installed, we want to download the ARCore SDK for Unity. This will be easy now that we have Git installed. Follow the given instructions to install the SDK: Open a shell or Command Prompt. Navigate to your Android folder. On Windows, use this: cd C:Android Type and execute the following: git clone https://github.com/google-ar/arcore-unity-sdk.git After the git command completes, you will see a new folder called arcore-unity-sdk. If this is your first time using Unity, you will need to go online to https://unity3d.com/ and create a Unity user account. The Unity editor will require that you log in on first use and from time to time. Now that we have Unity and ARCore installed, it's time to open the sample project by implementing the following steps: If you closed the Unity window, launch the Unity editor. The path on Windows will be C:Unity 2017.3.0b8EditorUnity.exe. Feel free to create a shortcut with the version number in order to make it easier to launch the specific Unity version later. Switch to the Unity project window and click on the Open button. Select the Android/arcore-unity-sdk folder. This is the folder we used the git command to install the SDK to earlier, as shown in the following dialog: Click on the Select Folder button. This will launch the editor and load the project. Open the Assets/GoogleARCore/HelloARExample/Scenes folder in the Project window, as shown in the following excerpt: Double-click on the HelloAR scene, as shown in the Project window and in the preceding screenshot. This will load our AR scene into Unity. At any point, if you see red console or error messages in the bottom status bar, this likely means you have a version conflict. You will likely need to install a different version of Unity. Now that we have Unity and ARCore installed, we will build the project and deploy the app to an Android device in the next section. Building and deploying to Android With most Unity development, we could just run our scene in the editor for testing. Unfortunately, when developing ARCore applications, we need to deploy the app to a device for testing. Fortunately, the project we are opening should already be configured for the most part. So, let's get started by following the steps in the next exercise: Open up the Unity editor to the sample ARCore project and open the HelloAR scene. If you left Unity open from the last exercise, just ignore this step. Connect your device via USB. From the menu, select File | Build Settings. Confirm that the settings match the following dialog: Confirm that the HelloAR scene is added to the build. If the scene is missing, click on the Add Open Scenes button to add it. Click on Build and Run. Be patient, first-time builds can take a while. After the app gets pushed to the device, feel free to test it, as you did with the Android version. Great! Now we have a Unity version of the sample ARCore project running. In the next section, we will look at remotely debugging our app. Remote debugging Having to connect a USB all the time to push an app is inconvenient. Not to mention that, if we wanted to do any debugging, we would need to maintain a physical USB connection to our development machine at all times. Fortunately, there is a way to connect our Android device via Wi-Fi to our development machine. Use the following steps to establish a Wi-Fi connection: Ensure that a device is connected via USB. Open Command Prompt or shell. On Windows, we will add C:Androidsdkplatform-tools to the path just for the prompt we are working on. It is recommended that you add this path to your environment variables. Google it if you are unsure of what this means. Enter the following commands: //WINDOWS ONLY path C:Androidsdkplatform-tools //FOR ALL adb devices adb tcpip 5555 If it worked, you will see restarting in TCP mode port: 5555. If you encounter an error, disconnect and reconnect the device. Disconnect your device. Locate the IP address of your device by doing as follows: Open your phone and go to Settings and then About phone. Tap on Status. Note down the IP address. Go back to your shell or Command Prompt and enter the following: adb connect [IP Address] Ensure that you use the IP Address you wrote down from your device. You should see connected to [IP Address]:5555. If you encounter a problem, just run through the steps again. Testing the connection Now that we have a remote connection to our device, we should test it to ensure that it works. Let's test our connection by doing the following: Open up Unity to the sample AR project. Expand the Canvas object in the Hierarchy window until you see the SearchingText object and select it, just as shown in the following excerpt: Hierarchy window showing the selected SearchingText object Direct your attention to the Inspector window, on the right-hand side by default. Scroll down in the window until you see the text "Searching for surfaces…". Modify the text to read "Searching for ARCore surfaces…", just as we did in the last chapter for Android. From the menu, select File | Build and Run. Open your device and test your app. Remotely debugging a running app Now, building and pushing an app to your device this way will take longer, but it is far more convenient. Next, let's look at how we can debug a running app remotely by performing the following steps: Go back to your shell or Command Prompt. Enter the following command: adb logcat You will see a stream of logs covering the screen, which is not something very useful. Enter Ctrl + C (command + C on Mac) to kill the process. Enter the following command: //ON WINDOWS C:Androidsdktoolsmonitor.bat //ON LINUX/MAC cd android-sdk/tools/ monitor This will open Android Device Monitor. You should see your device on the list to the left. Ensure that you select it. You will see the log output start streaming in the LogCat window. Drag the LogCat window so that it is a tab in the main window, as illustrated: Android Device Monitor showing the LogCat window Leave the Android Device Monitor window open and running. We will come back to it later. Now we can build, deploy, and debug remotely. This will give us plenty of flexibility later when we want to become more mobile. Of course, the remote connection we put in place with adb will also work with Android Studio. Yet, we still are not actually tracking any log output. We will output some log messages in the next section. Exploring the code Unlike Android, we were able to easily modify our Unity app right in the editor without writing code. In fact, given the right Unity extensions, you can make a working game in Unity without any code. However, for us, we want to get into the nitty-gritty details of ARCore, and that will require writing some code. Jump back to the Unity editor, and let's look at how we can modify some code by implementing the following exercise: From the Hierarchy window, select the ExampleController object. This will pull up the object in the Inspector window. Select the Gear icon beside Hello AR Controller (Script) and from the context menu, select Edit Script, as in the following excerpt: This will open your script editor and load the script, by default, MonoDevelop. Unity supports a number of Integrated Development Environments (IDEs) for writing C# scripts. Some popular options are Visual Studio 2015-2017 (Windows), VS Code (All), JetBrains Rider (Mac), and even Notepad++(All). Do yourself a favor and try one of the options listed for your OS.   Scroll down in the script until you see the following block of code: public void Update () { _QuitOnConnectionErrors(); After the _QuitOnConnectionErrors(); line of code, add the following code: Debug.Log("Unity Update Method"); Save the file and then go back to Unity. Unity will automatically recompile the file. If you made any errors, you will see red error messages in the status bar or console. From the menu, select File | Build and Run. As long as your device is still connected via TCP/IP, this will work. If your connection broke, just go back to the previous section and reset it. Run the app on the device. Direct your attention to Android Device Monitor and see whether you can spot those log messages. Unity Update method The Unity Update method is a special method that runs before/during a frame update or render. For your typical game running at 60 frames per second, this means that the Update method will be called 60 times per second as well, so you should be seeing lots of messages tagged as Unity. You can filter these messages by doing the following: Jump to the Android Device Monitor window. Click on the green plus button in the Saved Filters panel, as shown in the following excerpt: Adding a new tag filter Create a new filter by entering a Filter Name (use Unity) and by Log Tag (use Unity), as shown in the preceding screenshot. Click on OK to add the filter. Select the new Unity filter. You will now see a list of filtered messages specific to Unity platform when the app is running on the device. If you are not seeing any messages, check your connection and try to rebuild. Ensure that you saved your edited code file in MonoDevelop as well. Good job. We now have a working Unity set up with remote build and debug support. In this post,  we installed Unity and the ARCore SDK for Unity. We then took a slight diversion by setting up a remote build and debug connection to our device using TCP/IP over Wi-Fi. Next, we tested out our ability to modify the C# script in Unity by adding some debug log output. Finally, we tested our code changes using the Android Device Monitor tool to filter and track log messages from the Unity app deployed to the device. To know how to setup web development with JavaScript in ARCore and look through the various sample ARCore templates, check out the book Learn ARCore - Fundamentals of Google ARCore. Getting started with building an ARCore application for Android Unity plugins for augmented reality application development Types of Augmented Reality targets
Read more
  • 0
  • 0
  • 30770

article-image-use-m-functions-within-power-bi-querying-data
Amarabha Banerjee
21 May 2018
10 min read
Save for later

How to use M functions within Microsoft Power BI for querying data

Amarabha Banerjee
21 May 2018
10 min read
Microsoft Power BI Desktop contains a rich set of data source connectors and transformation capabilities that support the integration and enhancement of source data. These features are all driven by a powerful functional language and query engine, M, which leverages source system resources when possible and can greatly extend the scope and robustness of the data retrieval process beyond the possibilities of the standard query editor interface alone. As with almost all BI projects, the design and development of the data access and retrieval process has great implications for the analytical value, scalability, and sustainability of the overall Power BI solution. [box type="note" align="" class="" width=""]Our article is an excerpt from the book Microsoft Power BI Cookbook, written by Brett Powell. This book shows how to leverage  Microsoft Power BI and the development tools to create better data driven analytics and visualizations. [/box] In this article, we dive into Power BI Desktop's Get Data experience and go through the process of establishing and managing data source connections and queries. Examples are provided of using the Query Editor interface and the M language directly to construct and refine queries to meet common data transformation and cleansing needs. In practice and as per the examples, a combination of both tools is recommended to aid the query development process. Viewing and analyzing M functions Every time you click on a button to connect to any of Power BI Desktop's supported data sources or apply any transformation to a data source object, such as changing a column's data type, one or multiple M expressions are created reflecting your choices. These M expressions are automatically written to dedicated M documents and, if saved, are stored within the Power BI Desktop file as Queries. M is a functional programming language like F#, and it's important that Power BI developers become familiar with analyzing and later writing and enhancing the M code that supports their queries. Getting ready Build a query through the user interface that connects to the AdventureWorksDW2016CTP3 SQL Server database on the ATLAS server and retrieves the DimGeography table, filtered by United States for English. Click on Get Data from the Home tab of the ribbon, select SQL Server from the list of database sources, and provide the server and database names. For the Data Connectivity mode, select Import. A navigation window will appear, with the different objects and schemas of the database. Select the DimGeography table from the Navigation window and click on Edit. In the Query Editor window, select the EnglishCountryRegionName column and then filter on United States from its dropdown. Figure 2: Filtering for United States only in the Query Editor At this point, a preview of the filtered table is exposed in the Query Editor and the Query Settings pane displays the previous steps. Figure 3: The Query Settings pane in the Query Editor How to do it Formula Bar With the Formula Bar visible in the Query Editor, click on the Source step under Applied Steps in the Query Settings pane. You should see the following formula expression: Figure 4: The SQL.Database() function created for the Source step Click on the Navigation step to expose the following expression: Figure 5: The metadata record created for the Navigation step The navigation expression (2) references the source expression (1) The Formula Bar in the Query Editor displays individual query steps, which are technically individual M expressions It's convenient and very often essential to view and edit all the expressions in a centralized window, and for this, there's the Advanced Editor M is a functional language, and it can be useful to think of query evaluation in M as similar to Excel spreadsheet formulas in which multiple formulas can reference each other. The M engine can determine which expressions are required by the final expression to return and evaluate only those expressions. Configuring Power BI Development Tools, the display setting for both the Query Settings pane and the Formula bar should be enabled as GLOBAL | Query Editor options. Figure 6: Global layout options for the Query Editor Alternatively, on a per file basis, you can control these settings and others from the View tab of the Query Editor toolbar. Figure 7: Property settings of the View tab in the Query Editor Advanced Editor window Given its importance to the query development process, the Advanced Editor dialog is exposed on both the Home and View tabs of the Query Editor. It's recommended to use the Query Editor when getting started with a new query and when learning the M language. After several steps have been applied, use the Advanced Editor to review and optionally enhance or customize the M query. As a rich, functional programming language, there are many M functions and optional parameters not exposed via the Query Editor; going beyond the limits of the Query Editor enables more robust data retrieval and integration processes. Figure 8: The Home tab of the Query Editor Click on Advanced Editor from either the View or Home tabs (Figure 8 and Figure 9, respectively). All M function expressions and any comments are exposed Figure 9: The Advanced Editor view of the DimGeography query When developing retrieval processes for Power BI models, consider these common ETL questions: How are our queries impacting the source systems? Can we make our retrieval queries more resilient to changes in source data such that they avoid failure? Is our retrieval process efficient and simple to follow and support or are there unnecessary steps and queries? Are our retrieval queries delivering sufficient performance to the BI application? Is our process flexible such that we can quickly apply changes to data sources and logic? M queries are not intended as a substitute for the workloads typically handled by enterprise ETL tools such as SSIS or Informatica. However, just as BI professionals would carefully review the logic and test the performance of SQL stored procedures and ETL packages supporting their various cubes and reports environment, they should also review the M queries created to support Power BI models and reports. How it works Two of the top performance and scalability features of M's engine are Query Folding and Lazy Evaluation. If possible, the M queries developed in Power BI Desktop are converted (folded) into SQL statements and passed to source systems for processing. M can also reduce the required resources for a given query by ignoring any unnecessary or redundant steps (variables). M is a case-sensitive language. This includes referencing variables in M expressions (RenameColumns versus Renamecolumns) as well as the values in M queries. For example, the values "Apple" and "apple" are considered unique values in an M query; the Table.Distinct() function will not remove rows for one of the values. Variable names in M expressions cannot have spaces without a hash sign and double quotes. Per Figure 10, when the Query Editor graphical interface is used to create M queries this syntax is applied automatically, along with a name describing the M transformation applied. Applying short, descriptive variable names (with no spaces) improves the readability of M queries.  Query folding The query from this recipe was "folded" into the following SQL statement and sent to the ATLAS server for processing. Figure 10: The SQL statement generated from the DimGeography M query Right-click on the Filtered Rows step and select View Native Query to access the Native Query window from Figure 11: Figure 11: View Native Query in Query Settings Finding and revising queries that are not being folded to source systems is a top technique for enhancing large Power BI datasets. See the Pushing Query Processing Back to Source Systems recipe of Chapter 11, Enhancing and Optimizing Existing Power BI Solutions for an example of this process. M query structure The great majority of queries created for Power BI will follow the let...in structure as per this recipe, as they contain multiple steps with dependencies among them. Individual expressions are separated by commas. The expression referred to following the in keyword is the expression returned by the query. The individual step expressions are technically "variables", and if the identifiers for these variables (the names of the query steps) contain spaces then the step is placed in quotes, and prefixed with a # sign as per the Filtered Rows step in Figure 10. Lazy evaluation The M engine also has powerful "lazy evaluation" logic for ignoring any redundant or unnecessary variables, as well as short-circuiting evaluation (computation) once a result is determinate, such as when one side (operand) of an OR logical operator is computed as True. The order of evaluation of the expressions is determined at runtime; it doesn't have to be sequential from top to bottom. In the following example, a step for retrieving Canada was added and the step for the United States was ignored. Since the CanadaOnly variable satisfies the overall let expression of the query, only the Canada query is issued to the server as if the United States row were commented out or didn't exist. Figure 12: Revised query that ignores Filtered Rows step to evaluate Canada only View Native Query (Figure 12) is not available given this revision, but a SQL Profiler trace against the source database server (and a refresh of the M query) confirms that CanadaOnly was the only SQL query passed to the source database. Figure 13: Capturing the SQL statement passed to the server via SQL Server Profiler trace There's more Partial query folding A query can be "partially folded", in which a SQL statement is created resolving only part of an overall query The results of this SQL statement would be returned to Power BI Desktop (or the on-premises data gateway) and the remaining logic would be computed using M's in-memory engine with local resources M queries can be designed to maximize the use of the source system resources, by using standard expressions supported by query folding early in the query process Minimizing the use of local or on-premises data gateway resources is a top consideration Limitations of query folding No folding will take place once a native SQL query has been passed to the source system. For example, passing a SQL query directly through the Get Data dialog. The following query, specified in the Get Data dialog, is included in the Source Step: Figure 14: Providing a user defined native SQL query Any transformations applied after this native query will use local system resources. Therefore, the general implication for query development with native or user-defined SQL queries is that if they're used, try to include all required transformations (that is, joins and derived columns), or use them to utilize an important feature of the source database not being utilized by the folded query, such as an index. Not all data sources support query folding, such as text and Excel files. Not all transformations available in the Query Editor or via M functions directly are supported by some data sources. The privacy levels defined for the data sources will also impact whether folding is used or not. SQL statements are not parsed before they're sent to the source system. The Table.Buffer() function can be used to avoid query folding. The table output of this function is loaded into local memory and transformations against it will remain local. We have discussed effective techniques for accessing and retrieving data using Microsoft Power BI. Do check out this book Microsoft Power BI Cookbook for more information on using Microsoft power BI for data analysis and visualization. Expert Interview: Unlocking the secrets of Microsoft Power BI Tutorial: Building a Microsoft Power BI Data Model Expert Insights:Ride the third wave of BI with Microsoft Power BI    
Read more
  • 0
  • 0
  • 22707

article-image-this-week-on-packt-hub-18-may-2018
Aarthi Kumaraswamy
18 May 2018
3 min read
Save for later

This week on Packt Hub – 18 May 2018

Aarthi Kumaraswamy
18 May 2018
3 min read
We've just completely revamped our weekly newsletter. We'd love to hear from us on hub@packtpub.com. If you like what you see, make sure to subscribe and tell others too. Here’s what you may have missed in the last 7 days – Tech news, insights and tutorials… Featured interview “The wider use of generator functions in Python 3 has made functional Python programming much more common.” — Steven F. Lott We recently interviewed Steven F. Lott, a programmer since the 70s, a true Python professional and best-selling author of a number of Python books. In this interview, Steven talks a bit about modern Python and how the language adapts well to the Functional paradigm, offering developers a range of solutions to build modern, cloud-based applications. Tech news Conferences covered this week PyCon US 2018 Highlights: Quantum computing, blockchains, and serverless rule! What Google, RedHat, Oracle, and others announced at KubeCon + CloudNativeCon 2018 Data news in depth Shiny 1.1.0 releasing soon pandas 0.23 released Development & programming news in depth What can you expect from the upcoming Java 11 JDK? Android P new features: artificial intelligence, digital wellbeing, and simplicity Firefox 60 arrives with exciting updates for web developers: Quantum CSS engine, new Web APIs and more Unity announces a new automotive division and two-day Unity AutoTech Summit SteamVR introduces new controllers for game developers, the SteamVR Input system Amazon open sources Amazon Sumerian, its popular AR/VR app toolkit Introducing Fuse Open, Fuse App Engine and apps-as-a-service Barracuda announces Cloud-Delivered Web Application Firewall service Cloud & networking news in depth Google Compute Engine Plugin makes it easy to use Jenkins on Google Cloud Platform Introducing Intel’s OpenVINO computer vision toolkit for edge computing Rackspace now supports Kubernetes-as-a-Service Verizon chooses Amazon Web Services(AWS) as its preferred cloud provider Other News Google employees quit over company’s continued AI ties with the Pentagon Twitter’s disdain for third-party clients gets real Tutorials Data tutorials Building a Microsoft Power BI Data Model How to Build TensorFlow Models for Mobile and Embedded devices What does the structure of a data mining architecture look like? Getting started with Google Data Studio: An intuitive tool for visualizing BigQuery Data Development & programming tutorials Development tutorials Building a chat app with Kotlin using Node.js Is your web design responsive? How to use arrays, lists, and dictionaries in Unity for 3D game development How to create a generic reusable section for a single page based website How to use Bootstrap grid system for responsive website design? Programming tutorials How to work with classes in Typescript That ’70s language: AWK programming Regular expressions in AWK programming: What, Why, and How Other tutorials How to secure an Azure Virtual Network Tips and tricks for troubleshooting and flying drones safely This week’s opinions, analysis, and insights Data Insights What can Google Duplex do for businesses? 30 common data science terms explained Development & Programming Insights Top frameworks for building your Progressive Web Apps (PWA) What is a multi-layered software architecture? Other Insights 5 pen testing rules of engagement: What to consider while performing Penetration testing Cognitive IoT: How Artificial Intelligence is remolding Industrial and Consumer IoT What are lightweight Architecture Decision Records? BeyondCorp is transforming enterprise security Polycloud: a better alternative to cloud agnosticism  
Read more
  • 0
  • 0
  • 3461
article-image-regular-expressions-awk-programming
Pavan Ramchandani
18 May 2018
8 min read
Save for later

Regular expressions in AWK programming: What, Why, and How

Pavan Ramchandani
18 May 2018
8 min read
AWK is a pattern-matching language. It searches for a pattern in a file and, upon finding the corresponding match, it performs the file's action on the input line. This pattern could consist of fixed strings or a pattern of text. This variable content or pattern is generally searched with the help of regular expressions. Hence, regular expressions form an important part of AWK programming language. Today we will introduce you to the regular expressions in AWK programming and will get started with string-matching patterns and basic constructs to use with AWK. This article is an excerpt from a book written by Shiwang Kalkhanda, titled Learning AWK Programming. What is a regular expression? A regular expression, or regexpr, is a set of characters used to describe a pattern. A regular expression is generally used to match lines in a file that contain a particular pattern. Many Unix utilities operate on plain text files line by line, such as grep, sed, and awk. Regular expressions search for a pattern on a single line in a file. A regular expression doesn't search for a pattern that begins on one line and ends on another. Other programming languages may support this, notably Perl. Why use regular expressions? Generally, all editors have the ability to perform search-and-replace operations. Some editors can only search for patterns, others can also replace them, and others can also print the line containing that pattern. A regular expression goes many steps beyond this simple search, replace, and printing functionality, and hence it is more powerful and flexible. We can search for a word of a certain size, such as a word that has four characters or numbers. We can search for a word that ends with a particular character, let's say e. You can search for phone numbers, email IDs, and so on, and can also perform validation using regular expressions. They simplify complex pattern-matching tasks and hence form an important part of AWK programming. Other regular expression variations also exist, notably those for Perl. Using regular expressions with AWK There are mainly two types of regular expressions in Linux: Basic regular expressions that are used by vi, sed, grep, and so on Extended regular expressions that are used by awk, nawk, gawk, and egrep Here, we will refer to extended regular expressions as regular expressions in the context of AWK. In AWK, regular expressions are enclosed in forward slashes, '/', (forming the AWK pattern) and match every input record whose text belongs to that set. The simplest regular expression is a string of letters, numbers, or both that matches itself. For example, here we use the ly regular expression string to print all lines that contain the ly pattern in them. We just need to enclose the regular expression in forward slashes in AWK: $ awk '/ly/' emp.dat The output on execution of this code is as follows: Billy Chabra 9911664321 bily@yahoo.com M lgs 1900 Emily Kaur 8826175812 emily@gmail.com F Ops 2100 In this example, the /ly/ pattern matches when the current input line contains the ly sub-string, either as ly itself or as some part of a bigger word, such as Billy or Emily, and prints the corresponding line. Regular expressions as string-matching patterns with AWK Regular expressions are used as string-matching patterns with AWK in the following three ways. We use the '~' and '! ~' match operators to perform regular expression comparisons: /regexpr/: This matches when the current input line contains a sub-string matched by regexpr. It is the most basic regular expression, which matches itself as a string or sub-string. For example, /mail/ matches only when the current input line contains the mail string as a string, a sub-string, or both. So, we will get lines with Gmail as well as Hotmail in the email ID field of the employee database as follows: $ awk '/mail/' emp.dat The output on execution of this code is as follows: Jack Singh 9857532312 jack@gmail.com M hr 2000 Jane Kaur 9837432312 jane@gmail.com F hr 1800 Eva Chabra 8827232115 eva@gmail.com F lgs 2100 Ana Khanna 9856422312 anak@hotmail.com F Ops 2700 Victor Sharma 8826567898 vics@hotmail.com M Ops 2500 John Kapur 9911556789 john@gmail.com M hr 2200 Sam khanna 8856345512 sam@hotmail.com F lgs 2300 Emily Kaur 8826175812 emily@gmail.com F Ops 2100 Amy Sharma 9857536898 amys@hotmail.com F Ops 2500 In this example, we do not specify any expression, hence it automatically matches a whole line, as follows: $ awk '$0 ~ /mail/' emp.dat The output on execution of this code is as follows: Jack Singh 9857532312 jack@gmail.com M hr 2000 Jane Kaur 9837432312 jane@gmail.com F hr 1800 Eva Chabra 8827232115 eva@gmail.com F lgs 2100 Ana Khanna 9856422312 anak@hotmail.com F Ops 2700 Victor Sharma 8826567898 vics@hotmail.com M Ops 2500 John Kapur 9911556789 john@gmail.com M hr 2200 Sam khanna 8856345512 sam@hotmail.com F lgs 2300 Emily Kaur 8826175812 emily@gmail.com F Ops 2100 Amy Sharma 9857536898 amys@hotmail.com F Ops 2500 expression ~ /regexpr /: This matches if the string value of the expression contains a sub-string matched by regexpr. Generally, this left-hand operand of the matching operator is a field. For example, in the following command, we print all the lines in which the value in the second field contains a /Singh/ string: $ awk '$2 ~ /Singh/{ print }' emp.dat We can also use the expression as follows: $ awk '{ if($2 ~ /Singh/) print}' emp.dat The output on execution of the preceding code is as follows: Jack Singh 9857532312 jack@gmail.com M hr 2000 Hari Singh 8827255666 hari@yahoo.com M Ops 2350 Ginny Singh 9857123466 ginny@yahoo.com F hr 2250 Vina Singh 8811776612 vina@yahoo.com F lgs 2300 expression !~ /regexpr /: This matches if the string value of the expression does not contain a sub-string matched by regexpr. Generally, this expression is also a field variable. For example, in the following example, we print all the lines that don't contain the Singh sub-string in the second field, as follows: $ awk '$2 !~ /Singh/{ print }' emp.dat The output on execution of the preceding code is as follows: Jane Kaur 9837432312 jane@gmail.com F hr 1800 Eva Chabra 8827232115 eva@gmail.com F lgs 2100 Amit Sharma 9911887766 amit@yahoo.com M lgs 2350 Julie Kapur 8826234556 julie@yahoo.com F Ops 2500 Ana Khanna 9856422312 anak@hotmail.com F Ops 2700 Victor Sharma 8826567898 vics@hotmail.com M Ops 2500 John Kapur 9911556789 john@gmail.com M hr 2200 Billy Chabra 9911664321 bily@yahoo.com M lgs 1900 Sam khanna 8856345512 sam@hotmail.com F lgs 2300 Emily Kaur 8826175812 emily@gmail.com F Ops 2100 Amy Sharma 9857536898 amys@hotmail.com F Ops 2500 Any expression may be used in place of /regexpr/ in the context of ~; and !~. The expression here could also be if, while, for, and do statements. Basic regular expression construct Regular expressions are made up of two types of characters: normal text characters, called literals, and special characters, such as the asterisk (*, +, ?, .), called metacharacters. There are times when you want to match a metacharacter as a literal character. In such cases, we prefix that metacharacter with a backslash (), which is called an escape sequence. The basic regular expression construct can be summarized as follows: Here is the list of metacharacters, also known as special characters, that are used in building regular expressions:     ^    $    .    [    ]    |    (    )    *    +    ? The following table lists the remaining elements that are used in building a basic regular expression, apart from the metacharacters mentioned before: Literal A literal character (non-metacharacter ), such as A, that matches itself. Escape sequence An escape sequence that matches a special symbol: for example t matches tab. Quoted metacharacter () In quoted metacharacters, we prefix metacharacter with a backslash, such as $ that matches the metacharacter literally. Anchor (^) Matches the beginning of a string. Anchor ($) Matches the end of a string. Dot (.) Matches any single character. Character classes (...) A character class [ABC] matches any one of the A, B, or C characters. Character classes may include abbreviations, such as [A-Za-z]. They match any single letter. Complemented character classes Complemented character classes [^0-9] match any character except a digit. These operators combine regular expressions into larger ones: Alternation (|) A|B matches A or B. Concatenation AB matches A immediately followed by B. Closure (*) A* matches zero or more As. Positive closure (+) A+ matches one or more As. Zero or one (?) A? matches the null string or A. Parentheses () Used for grouping regular expressions and back-referencing. Like regular expressions, (r) can be accessed using n digit in future. Do check out the book Learning AWK Programming to learn more about the intricacies of AWK programming language for text processing. Read More What is the difference between functional and object-oriented programming? What makes a programming language simple or complex?
Read more
  • 0
  • 0
  • 30605

article-image-bootstrap-grid-system-responsive-website
Savia Lobo
18 May 2018
12 min read
Save for later

How to use Bootstrap grid system for responsive website design?

Savia Lobo
18 May 2018
12 min read
Bootstrap Origins In 2011, Bootstrap was created by two Twitter employees (Mark Otto and Jacob Thornton) to address the issue of fragmentation of internal tools/platforms. Bootstrap aimed to provide consistency among different web applications that were internally developed at Twitter to reduce redundancy and increase adaptability and reusability. As digital creators, we should always aim to make our applications adaptable and reusable. This will help keep coherency between applications and speed up processes, as we won't need to create basic foundations over and over again. In today's tutorial, you will learn what a Bootstrap is, how it relates to Responsive Web Design and its importance to the web industry. When Twitter Blueprint was born, it provided a way to document and share common design patterns/assets within Twitter. This alone is an amazing feature that would make Bootstrap an extremely useful framework. With this more internal developers began contributing towards the Bootstrap project as part of Hackathon week, and the project just exploded. Not long after, it was renamed "Bootstrap" as we know and love it today, and was released as an open source project to the community. A core team led by Mark and Jacob along with a passionate and growing community of developers helped to accelerate the growth of Bootstrap. In early 2012 after a lot of contributions from the core team and the community, Bootstrap 2 was born. It had come a long way from being a framework for providing internal consistency among Twitter tools. It was now a responsive framework using a 12-column grid system. It also provided inbuilt support for Glyphicons and a plethora of other new components. In 2013, Bootstrap 3 was released with a mobile-first approach to design and a fully redesigned set of components using the immensely popular flat design. This is the version many websites use today and it is very suitable for most developers. Bootstrap 4 is the latest stable  release. This article is an excerpt taken from the book,' Responsive Web Design by Example', written by Frahaan Hussain. Why use Bootstrap? You probably have a reasonable idea of why you would use Bootstrap for developing websites after reading its history, but there is more to it. Simply put, it provides the following: A responsive grid, using the design philosophies. Cross browser compatibility, using Normalize.css to ensure elements render consistently across all browsers (which isn't a very easy task). You might be wondering why it's difficult. Simply put, there are several different browsers, each with a plethora of versions, which all render content differently. I've seen some browsers put a border around an image by default, whereas some browsers don't. This type of inconsistency will prove to be very bad for user experience. A plethora of UI components, by providing polished UI components as developers, we are going to bring our creativity to life in a much easier way. These components usually allow a team to increase their development velocity since they start from a solid tried and tested foundation. They not only provide good design, but they are usually implemented using best practices in terms of performance and accessibility. A very compact size with only a small footprint. Really fast to develop with, it doesn't get in the way like many other frameworks, but allows your creativity to shine through. Extremely easy to start using Bootstrap in your website. Bundles common JavaScript plugins such as jQuery. Excellent documentation. Customizable, allowing you to remove any unnecessary features. An amazing community that is always ready, 24/7, to help. It's pretty clear now that Bootstrap is an amazing framework and that it will help provide consistency among our projects and aid cross browser responsive design. But why use Bootstrap over other frameworks? There are endless responsive frameworks like Bootstrap out there, such as Foundation, W3.CSS, and Skeleton, to mention a few. Bootstrap, however, was one of the first responsive frameworks and is by far the most developed with an ever-growing community. It has documentation online, both official and unofficial, and other frameworks aren't able to boast about their resources as much as Bootstrap can. Constantly being updated, it makes it the right choice for any website developer. Also, most JavaScript frameworks, such as Angular and React, have bindings to Bootstrap components that will reduce the amount of code and time spent binding it with another framework. It can also be used with tools such as SASS to customize  the components provided further. Bootstrap's grid system First, let's cover what a grid system is in general, regardless of the framework you choose to develop your amazing website on top of. Without using a framework, CSS would be used to implement the grid. However, a framework like Bootstrap handles all of the CSS side and provides us with easy-to-use classes. A responsive grid system is composed of two main elements: Columns: These are the horizontal containers for storing content on a single row Rows: These are top level containers for storing columns Your website will have at least one row, but it can have more. Each row can contain containers that span a set number of columns. For example, if the grid system had 100 columns, then a container that spans 50 would be half the width of the browser and/or parent element. Basics of Bootstrap Bootstrap's grid system consists of 12 columns that can be used to display content. Bootstrap also uses containers (methods for storing the website's content), rows, and columns to aid in the layout and alignment of the web page's content. All of these employ HTML classes for usage and will be explained very shortly. The purpose of these are as follows: Columns are used to group snippets of the website's content, and they, in turn, allow manipulation without disrupting the internal content's flow. There are two different types of columns: .container: Used for a fixed width, which is set by Bootstrap .container fluid: Used for full width to span the entire browser Rows are used to horizontally group columns, which aids in lining up the site's content properly: .row: There is only one type of row Columns mentioned previously are a way of setting how wide content should be. The following are the classes used for columns: .col-xs: Designed to display the content only on extra-small screens Max container width—none Triggered when the browser width is below 576px .col-sm: Designed to display the content only on small screens Max container width—540px Triggered when the browser width is above or equal to 576px and below 768px .col-md: Designed to display the content only on medium screens Max container width—720px Triggered when the browser width is above or equal to 768 and below 992px .col-lg: Designed to display the content only on large screens Max container width—960px Triggered when the browser width is above or equal to 992px and below 1200px .col-xl: Designed to display the content only on extra-large screens Max container width—1140px Triggered when the browser width is above or equal to 1200px .col: Designed to be triggered on all screen sizes To set a column's width, we simply append an integer ranging from 1 to 12 at the end of the class, like so: .col-6: Spans six columns on all screen sizes .col-md-6: Spans six columns only on extra-small screen sizes Later in this chapter, we will run through some examples of how to use these features and how they work together. Usage and examples To use the aforementioned features, the structure is as follows: div with container class div with row class div with column class Content div with column class Content div with column class Content div with column class Content div with row class div with column class Content div with column class Content div with column class Content div with column class Content div with column class Content div with column class Content The following examples may have some CSS styling applied; this does not affect their usage. Equal width columns example We will start off with a simple example that consists of one row and three equal columns on all screen sizes. The following code produces the aforementioned result: You may be scratching your head in regards to the column classes, as they have no numbers appended. This is an amazing feature that will come in useful very often. It allows us, as web developers, to add columns easily, without having to update the numbers, if the width of the columns is equal. In this example, there are three columns, which means the three divs equally span their thirds of the row. Multi-row, equal-width columns example Now let's extend the previous example to multiple rows: The following code produces the aforementioned result: As you can see, by adding a new row, the columns automatically go to the next row. This is extremely useful for grouping content together. Multi-row, equal-width columns without multiple rows example The title of this example may seem confusing, but you need to read it correctly. We will now cover creating multiple rows using only a single row class. This can be achieved with the help of a display utility class called w-100. The following code produces the aforementioned result: The example shows multiple row divs are not required for multiple rows. But the result isn't exactly identical, as there is no gap between the rows. This is useful for separating content that is still similar. For example, on a social network, it is common to have posts, and each post will contain information such as its date, title, description, and so on. Each post could be its own row, but within the post, the individual pieces of information could be separated using this class. Differently sized columns Up until now, we have only created rows with equal-width columns. These are useful, but not as useful as being able to set individual sizes. As mentioned in the Bootstrap grid system section, we can easily change the column width by appending a number ranging from 1-12 at the end of the col class. The following code produces the aforementioned result: As you can see, setting the explicit width of a column is very easy, but this applies the width to all screen sizes. You may want it only to be applied on certain screen sizes. The next section will cover this. Differently sized columns with screen size restrictions Let's use the previous example and expand it to change size responsively on differently sized screens. On extra-large screens, the grid will look like the following: On all other screen sizes it will appear with equal-width columns: The following code produces the aforementioned result: Now we are beginning to use breakpoints that provide a way of creating multiple layouts with minimal extra code to make use of the available real estate fully. Mixing and matching We aren't restricted to choosing only one break-point, we are able to set breakpoints for all the available screen sizes. The following figures illustrate all screen sizes, from extra-small to extra-large: Extra-small: Small: Medium: Large: Extra-large: The following code produces the aforementioned results: It isn't necessary for all divs to have the same breakpoints or to have breakpoints at all. Vertical alignment The previous examples provide functionality for use cases, but sometimes the need may arise to align objects vertically. This could technically be done with empty divs, but this wouldn't be a very elegant solution. Instead, there are alignment classes to help with this as can be seen here: As we can see, you can align rows vertically in one of three positions. The following code produces the aforementioned result: We aren't restricted to only aligning rows, we can easily align columns relative to each other, as is demonstrated here: The following code produces the aforementioned result: Horizontal alignment As we vertically aligned content in the previous section, we will now cover how easy it is to align content horizontally. The following figures show the results of horizontal alignment:   The following code produces the aforementioned result: Column offsetting The need may arise to position content with a slight offset. If the content isn't centered or at the start or end, this can become problematic, but using column offsetting, we can overcome this issue. Simply add an offset class, with the screen size to target, and how many columns (1-12) the content should be offset by, as can be seen in the following example:   The following code produces the aforementioned result: Grid wrap up The examples covered so far will suffice for most websites. There are more techniques for manipulating the grid, which can be found on Bootstrap's website. If you tried any of the examples, you may have noticed cascading from smaller screen-size classes to larger screen-size classes. This occurs when there are no explicit classes set for a certain screen size. Bootstrap components There are plethora of amazing components that are provided with Bootstrap, thus saving time creating them from scratch. There are components for dropdowns, buttons, images, and so much more. The usage is very similar to that of the grid system, and the same HTML elements we know and love are used with CSS classes to modify and display Bootstrap constructs. I won't go over every component that Bootstrap offers as that would require an encyclopedia in itself, and many of the commonly used ones will be covered in future chapters through example projects. I would however recommend taking a look at some of the components on Bootstrap's website. If you have found this post useful, do check out this book, ' Responsive Web Design by Example' to build engaging responsive websites using frameworks like Bootstrap and upgrade your skills as a web designer. Get ready for Bootstrap v4.1; Web developers to strap up their boots Web Development with React and Bootstrap Bootstrap 4 Objects, Components, Flexbox, and Layout  
Read more
  • 0
  • 2
  • 45262
Modal Close icon
Modal Close icon