Search icon CANCEL
Subscription
0
Cart icon
Your Cart (0 item)
Close icon
You have no products in your basket yet
Save more on your purchases! discount-offer-chevron-icon
Savings automatically calculated. No voucher code required.
Arrow left icon
Explore Products
Best Sellers
New Releases
Books
Events
Videos
Audiobooks
Packt Hub
Free Learning
Arrow right icon
timer SALE ENDS IN
0 Days
:
00 Hours
:
00 Minutes
:
00 Seconds

How-To Tutorials

7019 Articles
article-image-how-to-set-up-a-deep-learning-system-on-amazon-web-services-aws
Gebin George
07 Mar 2018
5 min read
Save for later

How to set up a Deep Learning System on Amazon Web Services (AWS)

Gebin George
07 Mar 2018
5 min read
[box type="note" align="" class="" width=""]This article is an excerpt from the book, Deep Learning Essentials written by Wei Di, Anurag Bhardwaj, and Jianing Wei.  This book covers popular Python libraries such as Tensorflow, Keras, and more, along with tips to train, deploy and optimize deep learning models in the best possible manner.[/box] Today, we will learn two different methods of setting up a deep learning system using Amazon Web Services (AWS). Setup from scratch We will illustrate how to set up a deep learning environment on an AWS EC2 GPU instance g2.2xlarge running Ubuntu Server 16.04 LTS. For this example, we will use a pre-baked Amazon Machine Image (AMI) which already has a number of software packages installed—making it easier to set up an end-end deep learning system. We will use a publicly available AMI Image ami-b03ffedf, which has following pre-installed Packages: CUDA 8.0 Anaconda 4.20 with Python 3.0 Keras / Theano The first step to setting up the system is to set up an AWS account and spin a new EC2 GPU instance using the AWS web console as (http://console.aws.amazon.com/) shown in figure Choose EC2 AMI: 2. We pick a g2.2xlarge instance type from the next page as shown in figure Choose instance type: 3. After adding a 30 GB of storage as shown in figure Choose storage, we now launch a cluster and assign an EC2 key pair that can allow us to ssh and log in to the box using the provided key pair file: 4. Once the EC2 box is launched, next step is to install relevant software packages.To ensure proper GPU utilization, it is important to ensure graphics drivers are installed first. We will upgrade and install NVIDIA drivers as follows: $ sudo add-apt-repository ppa:graphics-drivers/ppa -y $ sudo apt-get update $ sudo apt-get install -y nvidia-375 nvidia-settings While NVIDIA drivers ensure that host GPU can now be utilized by any deep learning application, it does not provide an easy interface to application developers for easy programming on the device. Various different software libraries exist today that help achieve this task reliably. Open Computing Language (OpenCL) and CUDA are more commonly used in industry. In this book, we use CUDA as an application programming interface for accessing NVIDIA graphics drivers. To install CUDA driver, we first SSH into the EC2 instance and download CUDA 8.0 to our $HOME folder and install from there: $ wget https://developer.nvidia.com/compute/cuda/8.0/Prod2/local_installers/cuda-r epo-ubuntu1604-8-0-local-ga2_8.0.61-1_amd64-deb $ sudo dpkg -i cuda-repo-ubuntu1604-8-0-local_8.0.44-1_amd64-deb $ sudo apt-get update $ sudo apt-get install -y cuda nvidia-cuda-toolkit Once the installation is finished, you can run the following command to validate the installation: $ nvidia-smi Now your EC2 box is fully configured to be used for a deep learning development. However, for someone who is not very familiar with deep learning implementation details, building a deep learning system from scratch can be a daunting task. To ease this development, a number of advanced deep learning software frameworks exist, such as Keras and Theano. Both of these frameworks are based on a Python development environment, hence we first install a Python distribution on the box, such as Anaconda: $ wget https://repo.continuum.io/archive/Anaconda3-4.2.0-Linux-x86_64.sh $ bash Anaconda3-4.2.0-Linux-x86_64.sh Finally, Keras and Theanos are installed using Python’s package manager pip: $ pip install --upgrade --no-deps git+git://github.com/Theano/Theano.git $ pip install keras Once the pip installation is completed successfully, the box is now fully set up for a deep learning development. Setup using Docker The previous section describes getting started from scratch which can be tricky sometimes given continuous changes to software packages and changing links on the web. One way to avoid dependence on links is to use container technology like Docker. In this chapter, we will use the official NVIDIA-Docker image that comes pre-packaged with all the necessary packages and deep learning framework to get you quickly started with deep learning application development: $ sudo add-apt-repository ppa:graphics-drivers/ppa -y $ sudo apt-get update $ sudo apt-get install -y nvidia-375 nvidia-settings nvidia-modprobe We now install Docker Community Edition as follows: $ curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add - # Verify that the key fingerprint is 9DC8 5822 9FC7 DD38 854A E2D8 8D81 803C 0EBF CD88 $ sudo apt-key fingerprint 0EBFCD88 $ sudo add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/ubuntu $(lsb_release -cs) Stable" $ sudo apt-get update $ sudo apt-get install -y docker-ce 2. We then install NVIDIA-Docker and its plugin: $ wget -P /tmp https://github.com/NVIDIA/nvidia-docker/releases/download/v1.0.1/nv Idia-docker_1.0.1-1_amd64.deb $ sudo dpkg -i /tmp/nvidia-docker_1.0.1-1_amd64.deb && rm /tmp/nvidia-docker_1.0.1-1_amd64.deb 3. To validate if the installation happened correctly, we use the following command:  $ sudo nvidia-docker run --rm nvidia/cuda nvidia-smi 4. Once it’s setup correctly, we can use the official TensorFlow or Theano Docker Image: $ sudo nvidia-docker run -it tensorflow/tensorflow:latest-gpu bash 5. We can run a simple Python program to check if TensorFlow works properly: import tensorflow as tf a = tf.constant(5, tf.float32) b = tf.constant(5, tf.float32) with tf.Session() as sess: sess.run(tf.add(a, b)) # output is 10.0 print("Output of graph computation is = ",output) You should see the TensorFlow output on the screen now as shown in figure Tensorflow sample output: We saw how to set up deep learning system on AWS from scratch and on Docker. If you found our post useful, do check out this book Deep Learning Essentials  to optimize deep learning models for better performance output.  
Read more
  • 0
  • 0
  • 13127

article-image-zenoss-core-3-tips-and-tricks
Packt
10 May 2011
4 min read
Save for later

Zenoss Core 3: Tips and Tricks

Packt
10 May 2011
4 min read
Zenoss Core 3.x Network and System Monitoring Importing devices and attributes Tip: If you already have a list of the SNMP capable devices by hostname or IP address, Zenoss Core can import them via the zenbatchload command. In its simplest form, zebatchload will process a text file that lists one device per line. Here's a sample list of devices that I will call deviceList.txt: device01 router02 web03 Since zenbatchload is a Zenoss Core daemon, we need to run it as the zenoss user. Here are the commands: su - zenoss <enter zenoss user's password> zenbatchload deviceList.txt The zenbatchload import utility will not update devices that are already in the device inventory. Editing and moving organizers Tip: Editing organizers: You can change the name or description of an organizer by selecting the organizer and then clicking on Edit. You can find the Edit option by clicking on the Actions menu button (which looks like a sprocket) at the bottom of the Infrastructure sidebar. See the following screenshot: Moving organizers: You can nest organizers by dragging and dropping one organizer into another one to create a hierarchy. You can only nest similar organizers, which means that you cannot move a system into a group or a group into a location. Zeneventlog—unable to connect to Windows Tip: If you see an event that indicates zeneventlog is unable to connect to Windows, that's an indication Zenoss Core is not able to authenticate to the Windows server. Check the Configuration Properties (zProperties) of the device to ensure you have the correct username and password set. The following screenshot shows the WMI specific zProperties: Remember, the zWinUser and zWinPassword properties must be an administrator on the Windows server you're monitoring. It's possible that each of your Windows servers could require a different username and password. The zWinUser property can be either a local or a domain account. If you use a domain account, specify the domain for the zWinuser value (for example, mojoactiveadministrator). Zenoss Core does not collect WMI data Tip: It's possible that Zenoss Core isn't collecting any data from the Windows Servers via WMI, but there are no events reported. You can check the following zProperties at the device or class level: Make sure the device is using the zenoss.wmi.WinServiceMap collector plugin Set the zWmiMonitorIgnore property to False in the Configuration Properties for the device or device class If you want to collect event logs, set the zWinEventlog property to True in the Configuration Properties for the device or device class Acknowledging an event Tip: Acknowledging an event signals to other team members and to Zenoss Core that you are aware of the event and, presumably, taking action. Acknowledging the event is good communication among your team, but Zenoss Core can also escalate event severities or alerts based on an event status. A common way to escalate an event or an alert is by the event count. For example, we can instruct Zenoss Core to escalate the event severity from Error to Critical if the event hasn't been acknowledged after a specified number of monitoring cycles. Or if we're dealing with alerts, Zenoss Core can be configured to alert the next person on call, in the event you fall asleep on the roof at 3 in the morning and don't realize the database server has been down for 15 minutes. To acknowledge an event: Select the event from the Event Console by clicking on it. Click on the on Acknowledge Events button (the icon that looks like a check mark). A check mark will appear next to the event. See the following screenshot: In the screenshot all the events are acknowledged, except the first one.
Read more
  • 0
  • 0
  • 13125

article-image-creating-horizon-desktop-pools
Packt
16 May 2016
17 min read
Save for later

Creating Horizon Desktop Pools

Packt
16 May 2016
17 min read
A Horizon desktop pool is a collection of desktops that users select when they log in using the Horizon client. A pool can be created based on a subset of users, such as finance, but this is not explicitly required unless you will be deploying multiple virtual desktop master images. The pool can be thought of as a central point of desktop management within Horizon; from it you create, manage, and entitle access to Horizon desktops. This article by Jason Ventresco, author of the book Implementing VMware Horizon View 6.X, will discuss how to create a desktop pool using the Horizon Administrator console, an important administrative task. (For more resources related to this topic, see here.) Creating a Horizon desktop pool This section will provide an example of how to create two different Horizon dedicated assignment desktop pools, one based on Horizon Composer linked clones and another based on full clones. Horizon Instant Clone pools only support floating assignment, so they have fewer options compared to the other types of desktop pools. Also discussed will be how to use the Horizon Administrator console and the vSphere client to monitor the provisioning process. The examples provided for full clone and linked clone pools created dedicated assignment pools, although floating assignment may be created as well. The options will be slightly different for each, so refer the information provided in the Horizon documentation (https://www.vmware.com/support/pubs/view_pubs.html), to understand what each setting means. Additionally, the Horizon Administrator console often explains each setting within the desktop pool configuration screens. Creating a pool using Horizon Composer linked clones The following steps outline how to use the Horizon Administrator console to create a dedicated assignment desktop pool using Horizon Composer linked clones. As discussed previously, it is assumed that you already have a virtual desktop master image that you have created a snapshot of. During each stage of the pool creation process, a description of many of the settings is displayed in the right-hand side of the Add Desktop Pool window. In addition, a question mark appears next to some of the settings; click on it to read important information about the specified setting. Log on to the Horizon Administrator console using an AD account that has administrative permissions within Horizon. Open the Catalog | Desktop Pools window within the console. Click on the Add… button in the Desktop Pools window to open the Add Desktop Pool window. In the Desktop Pool Definition | Type window, select the Automated Desktop Pool radio button as shown in the following screenshot, and then click on Next >: In the Desktop Pool Definition | User Assignment window, select the Dedicated radio button and check the Enable automatic assignment checkbox as shown in the following screenshot, and then click on Next >: In the Desktop Pool Definition | vCenter Server window, select the View Composer linked clones radio button, highlight the vCenter server as shown in the following screenshot, and then click on Next >: In the Setting | Desktop Pool Identification window, populate the pool ID: as shown in the following screenshot, and then click on Next >. Optionally, configure the Display Name: field. When finished, click on Next >: In the Setting | Desktop Pool Settings window, configure the various settings for the desktop pool. These settings can also be adjusted later if desired. When finished, click on Next >: In the Setting | Provisioning Settings window, configure the various provisioning options for the desktop pool that include the desktop naming format, the number of desktops, and the number of desktops that should remain available during Horizon Composer maintenance operations. When finished, click on Next >: When creating a desktop naming pattern, use a {n} to instruct Horizon to insert a unique number in the desktop name. For example, using Win10x64{n} as shown in the preceding screenshot will name the first desktop Win10x641, the next Win10x642, and so on. In the Setting | View Composer Disks window, configure the settings for your optional linked clone disks. By default, both a Persistent Disk for user data and a non-persistent disk for Disposable File Redirection are created. When finished, click on Next >: In the Setting | Storage Optimization window, we configure whether or not our desktop storage is provided by VMware Virtual SAN, and if not whether or not to separate our Horizon desktop replica disks from the individual desktop OS disks. In our example, we have checked the Use VMware Virtual SAN radio button as that is what our destination vSphere cluster is using. When finished, click on Next >: As all-flash storage arrays or all-flash or flash-dependent Software Defined Storage (SDS) platforms become more common, there is less of a need to place the shared linked clone replica disks on separate, faster datastores than the individual desktop OS disks. In the Setting | vCenter Settings window, we will need to configure six different options that include selecting the parent virtual machine, which snapshot of that virtual machine to use, what vCenter folder to place the desktops in, what vSphere cluster and resource pool to deploy the desktops to, and what datastores to use. Click on the Browse… button next to the Parent VM: field to begin the process and open the Select Parent VM window: In the Select Parent VM window, highlight the virtual desktop master image that you wish to deploy desktops from, as shown in the following screenshot. Click on OK when the image is selected to return to the previous window: The virtual machine will only appear if a snapshot has been created. In the Setting | vCenter Settings window, click on the Browse… button next to the Snapshot: field to open the Select default image window. Select the desired snapshot, as shown in the following screenshot, and click on OK to return to the previous window: In the Setting | vCenter Settings window, click on the Browse… button next to the VM folder location: field to open the VM Folder Location window, as shown in the following screenshot. Select the folder within vCenter where you want the desktop virtual machines to be placed, and click on OK to return to the previous window: In the Setting | vCenter Settings window, click on the Browse… button next to the Host or cluster: field to open the Host or Cluster window, as shown in the following screenshot. Select the cluster or individual ESXi server within vCenter where you want the desktop virtual machines to be created, and click on OK to return to the previous window: In the Setting | vCenter Settings window, click on the Browse… button next to the Resource pool: field to open the Resource Pool window, as shown in the following screenshot. If you intend to place the desktops within a resource pool you would select that here; if not select the same cluster or ESXi server you chose in the previous step. Once finished, click on OK to return to the previous window: In the Setting | vCenter Settings window, click on the Browse… button next to the Datastores: field to open the Select Linked Clone Datastores window, as shown in the following screenshot. Select the datastore or datastores where you want the desktops to be created, and click on OK to return to the previous window: If you were using storage other than VMware Virtual SAN, and had opted to use separate datastores for your OS and replica disks in step 11, you would have had to select unique datastores for each here instead of just one. Additionally, you would have had the option to configure the storage overcommit level. The Setting | vCenter Settings window should now have all options selected, enabling the Next > button. When finished, click on Next >: In the Setting | Advanced Storage Options window, if desired select and configure the Use View Storage Accelerator and Other Options check boxes to enable those features. In our example, we have enabled both the Use View Storage Accelerator and Reclaim VM disk space options, and configured Blackout Times to ensure that these operations do not occur between 8 A.M. (08:00) and 5 P.M. (17:00) on weekdays. When finished, click on Next >: The Use native NFS snapshots (VAAI) feature enables Horizon to leverage features of the a supported NFS storage array to offload the creation of linked clone desktops. If you are using an external array with your Horizon ESXi servers, consult the product documentation to understand if it supports this feature. Since we are using VMware Virtual SAN, this and other options under Other Options are greyed out as these settings are not needed. Additionally, if View Storage Accelerator is not enabled in the vCenter Server settings the option to use it would be greyed out here. In the Setting | Guest Customization window, select the Domain: where the desktops will be created, the AD container: where the computer accounts will be placed, whether to Use QuickPrep or Use a customization specification (Sysprep), and any other options as required. When finished, click on Next >: In the Setting | Ready to Complete window, verify that the settings we selected were correct, using the < Back button if needed to go back and make changes. If all the settings are correct, click on Finish to initiate the creation of the desktop pool. The Horizon desktop pool and virtual desktops will now be created. Creating a pool using Horizon Instant Clones The process used to create an Instant Clone desktop pool is similar to that used to create a linked clone pool. As discussed previously, it is assumed that you already have a virtual desktop master image that has the Instant Clone option enabled in the Horizon agent, and that you have taken a snapshot of that master image. A master image can have either the Horizon Composer (linked clone) option or Instant Clone option enabled in the Horizon agent, but not both. To get around this restriction you can configure one snapshot of the master image with the View Composer option installed, and a second with the Instant Clone option installed. The following steps outline the process used to create the Instant Clone desktop pool. Screenshots are included only when the step differs significantly from the same step in the Creating a pool using Horizon Composer linked clones section. Log on to the Horizon Administrator console using an AD account that has administrative permissions within Horizon. Open the Catalog | Desktop Pools window within the console. Click on the Add… button in the Desktop Pools window to open the Add Desktop Pool window. In the Desktop Pool Definition | Type window, select the Automated Desktop Pool radio button as shown in the following screenshot, and then click on Next >. In the Desktop Pool Definition | User Assignment window, select the Floating radio button (mandatory for Instant Clone desktops), and then click on Next >. In the Desktop Pool Definition | vCenter Server window, select the View Composer linked clones radio button as shown in the following screenshot, highlight the vCenter server, and then click on Next >: If Instant Clones is greyed out here, it is usually because you did not select Floating in the previous step. In the Setting | Desktop Pool Identification window, populate the pool ID:, and then click on Next >. Optionally, configure the Display Name: field. In the Setting | Desktop Pool Settings window, configure the various settings for the desktop pool. These settings can also be adjusted later if desired. When finished, click on Next >. In the Setting | Provisioning Settings window, configure the various provisioning options for the desktop pool that include the desktop naming format, the number of desktops, and the number of desktops that should remain available during maintenance operations. When finished, click on Next >. Instant Clones are required to always be powered on, so some options available to linked clones will be greyed out here. In the Setting | Storage Optimization window, we configure whether or not our desktop storage is provided by VMware Virtual SAN, and if not whether or not to separate our Horizon desktop replica disks from the individual desktop OS disks. When finished, click on Next >. In the Setting | vCenter Settings window, we will need to configure six different options that include selecting the parent virtual machine, which snapshot of that virtual machine to use, what vCenter folder to place the desktops in, what vSphere cluster and resource pool to deploy the desktops to, and what datastores to use. Click on the Browse… button next to the Parent VM: Horizon Instant Clone field to begin the process and open the Select Parent VM window. In the Select Parent VM window, highlight the virtual desktop master image that you wish to deploy desktops from. Click on OK when the image is selected to return to the previous window. In the Setting | vCenter Settings window, click on the Browse… button next to the Snapshot: field to open the Select default image window. Select the desired snapshot, and click on OK to return to the previous window. In the Setting | vCenter Settings window, click on the Browse… button next to the VM folder location: field to open the VM Folder Location window. Select the folder within vCenter where you want the desktop virtual machines to be placed, and click on OK to return to the previous window. In the Setting | vCenter Settings window, click on the Browse… button next to the Host or cluster: field to open the Host or Cluster window. Select the cluster or individual ESXi server within vCenter where you want the desktop virtual machines to be created, and click on OK to return to the previous window. In the Setting | vCenter Settings window, click on the Browse… button next to the Resource pool: field to open the Resource Pool window. If you intend to place the desktops within a resource pool you would select that here; if not select the same cluster or ESXi server you chose in the previous step. Once finished, click on OK to return to the previous window. In the Setting | vCenter Settings window, click on the Browse… button next to the Datastores: field to open the Select Instant Clone Datastores window. Select the datastore or datastores where you want the desktops to be created, and click on OK to return to the previous window. The Setting | vCenter Settings window should now have all options selected, enabling the Next > button. When finished, click on Next >. In the Setting | Guest Customization window, select the Domain: where the desktops will be created, the AD container: where the computer accounts will be placed, and any other options as required. When finished, click on Next >. Instant Clones only support ClonePrep for customization, so there are fewer options here than seen when deploying a linked clone desktop pool. In the Setting | Ready to Complete window, verify that the settings we selected were correct, using the < Back button if needed to go back and make changes. If all the settings are correct, click on Finish to initiate the creation of the desktop pool. The Horizon desktop pool and Instant Clone virtual desktops will now be created. Creating a pool using full clones The process used to create full clone desktops pool is similar to that used to create a linked clone pool. As discussed previously, it is assumed that you already have a virtual desktop master image that you have converted to a vSphere template. In addition, if you wish for Horizon to perform the virtual machine customization, you will need to create a Customization Specification using the vCenter Customization Specifications Manager. The Customization Specification is used by the Windows Sysprep utility to complete the guest customization process. Visit the VMware vSphere virtual machine administration guide (http://pubs.vmware.com/vsphere-60/index.jsp) for instructions on how to create a Customization Specification. The following steps outline the process used to create the full clone desktop pool. Screenshots are included only when the step differs significantly from the same step in the Creating a pool using Horizon Composer linked clones section. Log on to the Horizon Administrator console using an AD account that has administrative permissions within Horizon. Open the Catalog | Desktop Pools window within the console. Click on the Add… button in the Desktop Pools window to open the Add Desktop Pool window. In the Desktop Pool Definition | Type window select the Automated Pool radio button and then click on Next. In the Desktop Pool Definition | User Assignment window, select the Dedicated radio button, check the Enable automatic assignment checkbox, and then click on Next. In the Desktop Pool Definition | vCenter Server window, click the Full virtual machines radio button, highlight the desired vCenter server, and then click on Next. In the Setting | Desktop Pool Identification window, populate the pool ID: and Display Name: fields and then click on Next. In the Setting | Desktop Pool Settings window, configure the various settings for the desktop pool. These settings can also be adjusted later if desired. When finished, click on Next >. In the Setting | Provisioning Settings window, configure the various provisioning options for the desktop pool that include the desktop naming format and number of desktops. When finished, click on Next >. In the Setting | Storage Optimization window, we configure whether or not our desktop storage is provided by VMware Virtual SAN. When finished, click on Next >. In the Setting | vCenter Settings window, we will need to configure settings that set the virtual machine template, what vSphere folder to place the desktops in, which ESXi server or cluster to deploy the desktops to, and which datastores to use. Other than the Template setting described in the next step, each of these settings is identical to those seen when creating a Horizon Composer linked clone pool. Click on the Browse… button next to each of the settings in turn and select the appropriate options. To configure the Template: setting, select the vSphere template that you created from your virtual desktop master image as shown in the following screenshot, and then click OK to return to the previous window: A template will only appear if one is present within vCenter. Once all the settings in the Setting | vCenter Settings window have been configured, click on Next >. In the Setting | Advanced Storage Options window, if desired select and configure the Use View Storage Accelerator radio buttons and configure Blackout Times. When finished, click on Next >. In the Setting | Guest Customization window, select either the None | Customization will be done manually or Use this customization specification radio button, and if applicable select a customization specification. When finished, click on Next >. In the following screenshot, we have selected the Win10x64-HorizonFC customization specification that we previously created within vCenter: Manual customization is typically used when the template has been configured to run Sysprep automatically upon start up, without requiring any interaction from either Horizon or VMware vSphere. In the Setting | Ready to Complete window, verify that the settings we selected were correct, using the < Back button if needed to go back and make changes. If all the settings are correct, click on Finish to initiate the creation of the desktop pool. The desktop pool and virtual desktops will now be created. Summary In this article, we have learned about Horizon desktop pools. In addition to learning how to create three different types of desktop pools, we were introduced to a number of key concepts that are part of the pool creation process. Resources for Article: Further resources on this subject: Essentials of VMware vSphere [article] Cloning and Snapshots in VMware Workstation [article] An Introduction to VMware Horizon Mirage [article]
Read more
  • 0
  • 0
  • 13120

article-image-we-are-not-going-to-withdraw-from-the-future-says-microsofts-brad-smith-on-the-ongoing-jedi-bid-amazon-concurs
Prasad Ramesh
29 Oct 2018
5 min read
Save for later

‘We are not going to withdraw from the future’ says Microsoft’s Brad Smith on the ongoing JEDI bid, Amazon concurs

Prasad Ramesh
29 Oct 2018
5 min read
The Pentagon has been trying to get a hold of AI and related technologies from tech giants. Google employees had quit over it, Microsoft employees had asked the company to withdraw from the JEDI project. Last Friday, Microsoft President Brad Smith wrote about Microsoft and the US Military and the company’s visions in this area. Amazon, Microsoft, IBM, and Oracle are the companies who have bid for the Joint Enterprise Defense Infrastructure (JEDI) project. JEDI is a department wide cloud computing infrastructure that will give the Pentagon access to weapons systems enhanced with artificial intelligence and cloud computing. Microsoft believes in defending USA “We are not going to withdraw from the future, in the most positive way possible, we are going to work to help shape it.” said Brad Smith, President at Microsoft indicating that Microsoft intends to provide their technology to the Pentagon. Microsoft did not shy away from bidding in the Pentagon’s JEDI project. This in contrast to Google, which opted out of the same program earlier this month citing ethical concerns. Smith expressed Microsoft’s intent towards providing AI and related technologies to the US defense department saying, “we want the people who defend USA to have access to the nation’s best technology, including from Microsoft”. Smith stated that Microsoft’s work in this area is based on three convictions: Microsoft believes in the strong defense of USA and wants the defenders to have access to USA’s best technology, this includes Microsoft They want to use their ‘knowledge and voice’ to address ethical AI issues via the nation’s ‘civic and democratic processes’. They are giving their employees to opt out of work on these projects given that as a global company they consist of employees from different countries. Smith shared that Microsoft has had a long standing history with the US Department of Defense (DOD). Their tech has been used throughout the US military from the front office to field operations. This includes bases, ships, aircraft and training facilities. Amazon shares Microsoft’s visions Amazon too shares these visions with Microsoft in empowering US law and defense institutions with the latest technology. Amazon already provides cloud services to power the Central Intelligence Agency (CIA). Amazon CEO, Jeff Bezos said: “If big tech companies are going to turn their back on the Department of Defense, this country is going to be in trouble.” Amazon also provides the US law enforcement with their facial recognition technology called Rekognition. This has been a bone of contention for not just civil rights groups but also for some Amazon’s employees. Rekognition will help in identifying and incarcerating undesirable people. But it does not really work with accuracy. In a study by ACLU, Rekognition identified 28 people from the US congress incorrectly. The American Civil Liberties Union (ACLU) has now filed a Freedom of Information Act (FOIA) request which demands the Department of Homeland Security (DHS) to disclose how DHS and Immigration and Customs Enforcement (ICE) use Rekognition for law enforcement and immigration checks. Google’s rationale for withdrawing from the JEDI project Last week, in an interview with the Fox Network, Oracle founder Larry Ellison stated that it was shocking how Google viewed this matter. Google withdrew from the JEDI project following strong backlash from many of its employees. In the official statement, they have stated the reason for dropping out of the JEDI contract bidding as an ethical value misalignment and also that they don’t fully have all necessary clearance to work on Government projects.’ However, Google is open to launching a customized search engine in China that complies with China’s rules of censorship including potential to surveil Chinese citizens. Should AI be used in weapons? This question is the at the heart of the contentious topic of the tech industry working with the military. It is a serious topic that has been debated over the years by educated scientists and experienced leaders. Elon Musk, researchers from DeepMind and other companies even pledged to not build lethal AI. Personally, I side with the researchers and believe AI should be used exclusively for the benefit of mankind, to enhance human lives and solve problems that would prosper people’s lives. And not against each other in a race to build weapons or to become a superpower. But then again what would I know? Leading nations are in an AI arms race as we speak, with sophisticated national AI plans and agendas. For more details on Microsoft’s interest in working with the US Military visit the Microsoft website. ‘Employees of Microsoft’ ask Microsoft not to bid on US Military’s Project JEDI in an open letter Google employees quit over company’s continued Artificial Intelligence ties with the Pentagon Oracle’s bid protest against U.S Defence Department’s(Pentagon) $10 billion cloud contract
Read more
  • 0
  • 0
  • 13109

article-image-downloading-pyrocms-and-its-pre-requisites
Packt
31 Oct 2013
6 min read
Save for later

Downloading PyroCMS and it's pre-requisites

Packt
31 Oct 2013
6 min read
(For more resources related to this topic, see here.) Getting started PyroCMS, like many other content management systems including WordPress, Typo3, or Drupal, comes with a pre-developed installation process. For PyroCMS, this installation process is easy to use and comes with a number of helpful hints just in case you hit a snag while installing the system. If, for example, your system files don't have the correct permissions profile (writeable versus write-protected), the PyroCMS installer will help you, along with all the other installation details, such as checking for required software and taking care of file permissions. Before you can install PyroCMS (the version used for examples in this article is 2.2) on a server, there are a number of server requirements that need to be met. If you aren't sure if these requirements have been met, the PyroCMS installer will check to make sure they are available before installation is complete. Following are the software requirements for a server before PyroCMS can be installed: HTTP Web Server MySQL 5.x or higher PHP 5.2.x or higher GD2 cURL Among these requirements, web developers interested in PyroCMS will be glad to know that it is built on CodeIgniter, a popular MVC patterned PHP framework. I recommend that the developers looking to use PyroCMS should also have working knowledge of CodeIgniter and the MVC programming pattern. Learn more about CodeIgniter and see their excellent system documentation online at http://ellislab.com/codeigniter. CodeIgniter If you haven't explored the Model-View-Controller (MVC) programming pattern, you'll want to brush up before you start developing for PyroCMS. The primary reason that CodeIgniter is a good framework for a CMS is that it is a well-documented framework that, when leveraged in the way PyroCMS has done, gives developers power over how long a project will take to build and the quality with which it is built. Add-on modules for PyroCMS, for example, follow the MVC method, a programming pattern that saves developers time and keeps their code dry and portable. Dry and portable programming are two different concepts. Dry is an acronym for "don't repeat yourself" code. Portable code is like "plug-and-play" code—write it once so that it can be shared with other projects and used quickly. HTTP web server Out of the PyroCMS software requirements, it is obvious, you can guess, that a good HTTP web server platform will be needed. Luckily, PyroCMS can run on a variety of web server platforms, including the following: Abyss Web Server Apache 2.x Nginx Uniform Server Zend Community Server If you are new to web hosting and haven't worked with web hosting software before, or this is your first time installing PyroCMS, I suggest that you use Apache as a HTTP web server. It will be the system for which you will find the most documentation and support online. If you'd prefer to avoid Apache, there is also good support for running PyroCMS on Nginx, another fairly-well documented web server platform. MySQL Version 5 is the latest major release of MySQL, and it has been in use for quite some time. It is the primary database choice for PyroCMS and is thoroughly supported. You don't need expert level experience with MySQL to run PyroCMS, but you'll need to be familiar with writing SQL queries and building relational databases if you plan to create add-ons for the system. You can learn more about MySQL at http://www.mysql.com. PHP Version 5.2 of PHP is no longer the officially supported release of PHP, which is, at the time of this article, Version 5.4. Version 5.2, which has been criticized as being a low server requirement for any CMS, is allowed with PyroCMS because it is the minimum version requirement for CodeIgniter, the framework upon which PyroCMS is built. While future versions of PyroCMS may upgrade this minimum requirement to PHP 5.3 or higher, you can safely use PyroCMS with PHP 5.2. Also, many server operating systems, like SUSE and Ubuntu, install PHP 5.2 by default. You can, of course, upgrade PHP to the latest version without causing harm to your instance of PyroCMS. To help future-proof your installation of PyroCMS, it may be wise to install PHP 5.3 or above, to maximize your readiness for when PyroCMS more strictly adopts features found in PHP 5.3 and 5.4, such as namespaceing. GD2 GD2, a library used in the manipulation and creation of images, is used by PyroCMS to dynamically generate images (where needed) and to crop and resize images used in many PyroCMS modules and add-ons. The image-based support offered by this library is invaluable. cURL As described on the cURL project website, cURL is "a command line tool for transferring data with URL syntax" using a large number of methods, including HTTP(S) GET, POST, PUT, and so on. You can learn more about the project and how to use cURL on their website http://curl.haxx.se. If you've never used cURL with PHP, I recommend taking time to learn how to use it, especially if you are thinking about building a web-based API using PyroCMS. Most popular web hosting companies meet the basic server requirements for PyroCMS. Downloading PyroCMS Getting your hands on a copy of PyroCMS is very simple. You can download the system files from one of two locations, the PryoCMS project website and GitHub. To download PyroCMS from the project website, visit http://www.pyrocms.com and click on the green button labeled Get PyroCMS! This will take you to a download page that gives you the choice between downloading the Community version of PyroCMS and buying the Professional version. If you are new to PyroCMS, you can start with the Community version, currently at Version 2.2.3. The following screenshot shows the download screen: To download PyroCMS from GitHub, visit https://github.com/pyrocms/pyrocms and click on the button labeled Download ZIP to get the latest Community version of PyroCMS, as shown in the following screenshot: If you know how to use Git, you can also clone a fresh version of PyroCMS using the following command. A word of warning, cloning PyroCMS from GitHub will usually give you the latest, stable release of the system, but it could include changes not described in this article. Make sure you checkout a stable release from PyroCMS's repository. git clone https://github.com/pyrocms/pyrocms.git As a side-note, if you've never used Git, I recommend taking some time to get started using it. PyroCMS is an open source project hosted in a Git repository on Github, which means that the system is open to being improved by any developer looking to contribute to the well-being of the project. It is also very common for PyroCMS developers to host their own add-on projects on Github and other online Git repository services. Summary In this article, we have covered the pre-requisites for using PyroCMS, and also how to download PyroCMS. Resources for Article : Further resources on this subject: Kentico CMS 5 Website Development: Managing Site Structure [Article] Kentico CMS 5 Website Development: Workflow Management [Article] Web CMS [Article]
Read more
  • 0
  • 0
  • 13109

article-image-python-ldap-applications-part-4-ldap-schema
Packt
23 Oct 2009
8 min read
Save for later

Python LDAP Applications: Part 4 - LDAP Schema

Packt
23 Oct 2009
8 min read
Using Schema Information As with most LDAP servers, OpenLDAP provides schema access to LDAP clients. An LDAP schema defines object classes, attributes, matching rules, and other LDAP structures. In this article, we will take a brief look at what might be the most complex module in the Python-LDAP API, the ldap.schema module. This module provides programmatic access to the schema, using the LDAP subschema record, and the subschema's subentries obtained from the LDAP server. The module has two major components. The first is the SubSchema object, which contains the schema definition, and provides numerous functions for navigating through the definitions stored in the schema. The second component is the model, which contains classes that describe structural components (Schema Elements) of the schema. For example, the model contains classes like ObjectClass, AttributeType, MatchingRule, and DITContentRule. Getting the Schema from the LDAP Server The ldap.schema module does not automatically retrieve the schema information. It must be fetched from the server with an LDAP search operation. The schema is always stored in a specific entry, almost always accessible with the DN cn=subschema. (If it is elsewhere, and is accessible, the Root DSE will note the location.) We can retrieve the record by doing a search with a base scope: >>> res = l.search_s('cn=subschema',... ldap.SCOPE_BASE,... '(objectclass=*)',... ['*','+']... )>>> subschema_entry = ldaphelper.get_search_results(res)[0]>>> subschema_subentry = subschema_entry.get_attributes()>>> The search configuration above should return only one record – the record for cn=subschema. Because most of the schema attributes are operational attributes, we need to specify, in the list of attributes, both * for all regular attributes and + for all operational attributes. The ldaphelper.get_search_results() function we created early in this series returns a list of LDAPSearchResult objects. Since we know that we want the first one (in a list of one), we can use the [0] notation at the end to return just the first item in the resulting list. Now, schema_entry contains the LDAPSearchResult object for the cn=subschema record. We need the list of attributes – namely, the schema-defining attributes, usually called the subschema subentry. We can use the get_attributes() method to retrieve the dict of attributes. Now we have the information necessary for creating a new SubSchema object. The SubSchema Object The SubSchema object provides access to the details of the schema definitions. The SubSchema() constructor takes one parameter: a dictionary of attributes that contains the subschema subentry information. This is the information we retrieved and stored in the subschema_subentry variable above. Creating a new SubSchema object is done like this: >>> subschema = ldap.schema.SubSchema( subschema_subentry )>>> Now we can access the schema information. We can, for instance, get the schema information for the cn attribute: >>> cn_attr = subschema.get_obj( ldap.schema.AttributeType, 'cn' )>>> cn_attr.names('cn', 'commonName')>>> cn_attr.desc'RFC2256: common name(s) for which the entity is known by'>>> cn_attr.oid'2.5.4.3'>>> The first line employs the get_obj() method to retrieve an AttributeType object. The call to get_obj() above uses two parameters. The first is the class (a subclass of SchemaElement) that represents an attribute. This is ldap.schema.AttributeType. If we were getting an object class instead of an attribute, we would use the same method, but pass an ldap.schema.ObjectClass as the first parameter. The second parameter is a string name (or OID) of the attribute. We could have used 'commonName' or '2.5.4.3' and attained the same result. The cn_attr object (an instance of an AttributeType class) has a number of properties representing schema statements. For example, in the example above, the names property contains a tuple of the attribute names for that attribute, and the desc property contains the value of the description, as specified in the schema. The oid attribute contains the Object Identifier (OID) for the CN attribute. Let's look at one more method of the SubSchema class before moving on to the final script in this article. Using the attribute_types() method of the SubSchema class, we can find out what attributes are required for an record, and what attributes are allowed. For example, consider a record that has the object classes account and simpleSecurityObject. The uid=authenticate,ou=system,dc=example,dc=com entry in our directory information tree is an example of such a user. We can use the attribute_types() method to get information about what attributes this record can or must have: >>> oc_list = ['account', 'simpleSecurityObject']>>> oc_attrs = subschema.attribute_types( oc_list )>>> must_attrs = oc_attrs[0]>>> may_attrs = oc_attrs[1]>>> >>> for ( oid, attr_obj ) in must_attrs.iteritems():... print "Must have %s" % attr_obj.names[0]... Must have userPasswordMust have objectClassMust have uid>>> for ( oid, attr_obj ) in may_attrs.iteritems():... print "May have %s" % attr_obj.names[0]... May have oMay have ouMay have seeAlsoMay have descriptionMay have lMay have host>>> The oc_list list has the names of the two object classes in which we are interested: account and simpleSecurityObject. Passing this list to the attribute_types() method, we get a two-item tuple. The first item in the tuple is a dictionary of required attributes. The key in the dictionary is the OID: >>> must_attrs.keys()['2.5.4.35', '2.5.4.0', '0.9.2342.19200300.100.1.1']>>> The value in the dictionary is an AttributeType object corresponding to the attribute defined for the OID key: >>> must_attrs['2.5.4.35'].oid'2.5.4.35'>>> In the code snippet above, we assigned each value in the two-item tuple to a different variable: must_attrs contains the first item in the tuple – the dictionary of must-have attributes. The may_attrs contains a dictionary of the attributes that are allowed, but not required. Iterating through the dictionaries and printing the output, we can see that the required attributes for a record that used both the account and the simpleSecurityObject object classes would be userPassword, objectclass, and uid. Several other attributes are allowed, but not required: o, ou, seeAlso, description, l, and host. We could find out which object class definitions required or allowed which of these attributes using the get_obj() method we looked at above: >>> oc_obj = subschema.get_obj( ldap.schema.ObjectClass, 'account' )>>> oc_obj.may('description', 'seeAlso', 'localityName', 'organizationName', 'organizationalUnitName', 'host')>>> oc_obj.must('userid',)>>>>>> oc_obj = subschema.get_obj( ldap.schema.ObjectClass,... 'simpleSecurityObject' )>>> oc_obj.must('userPassword',)>>> oc_obj.may()>>> From the above, we can see that most of the required and optional attributes come from the account definition, while only userPassword comes from the simpleSecurityObject definition. The requirement of the objectClass attribute comes from the top object class, the ultimate ancestor of all structural object classes. The schema support offered by the Python-LDAP API makes it possible to program schema-aware clients that can, for instance, perform client-side schema checking, dynamically build forms for creating records, or compare definitions between different LDAP servers on a network. Unfortunately, the ldap.schema module is poorly documented. With most of the module, the best source of information is the __doc__ strings embedded in the code: >>> print ldap.schema.SubSchema.attribute_types.__doc__ Returns a 2-tuple of all must and may attributes including all inherited attributes of superior object classes by walking up classes along the SUP attribute. The attributes are stored in a ldap.cidict.cidict dictionary. object_class_list list of strings specifying object class names or OIDs attr_type_filter list of 2-tuples containing lists of class attributes which has to be matched raise_keyerror All KeyError exceptions for non-existent schema elements are ignored ignore_dit_content_rule A DIT content rule governing the structural object class is ignored >>> In some cases, though, the best source of documentation is the code itself. The last script in this article will provide an example of how the schema information can be used. An Example Script: suggest_attributes.py This example script compares the attributes in a user-specified record with the possible attributes, and prints out an annotated list of “suggested” available attributes. This script is longer than the other scripts in this article, but it makes use of similar techniques, and we will be able to move through it quickly.
Read more
  • 0
  • 0
  • 13109
Unlock access to the largest independent learning library in Tech for FREE!
Get unlimited access to 7500+ expert-authored eBooks and video courses covering every tech area you can think of.
Renews at €18.99/month. Cancel anytime
article-image-code-igniter-email-and-html-table
Packt
22 Nov 2013
18 min read
Save for later

CodeIgniter Email and HTML Table

Packt
22 Nov 2013
18 min read
There might be times rather than just plain text, so you may wish to include images, text formatting, and URLs in the body of your e-mail. HTML e-mails will allow you to do this and CodeIgniter Email library can easily be set to do just that. How to do it... HTML e-mails can be sent by executing the following steps: Create a file email.php at /path/to/codeigniter/application/controllers/. Add the following code to the controller file email.php: <?php if ( ! defined('BASEPATH')) exit('No direct script access allowed'); class Email extends CI_Controller { function __construct() { parent::__construct(); $this->load->helper('url'); $this->load->library('email'); } public function index() { redirect('email/send_email'); } public function send_email() { $config['protocol'] = 'sendmail'; $config['mailpath'] = '/usr/sbin/sendmail'; $config['charset'] = 'iso-8859-1'; $config['wordwrap'] = TRUE; $config['mailtype'] = 'html'; $this->email->initialize($config); $this->email->from('from@domain.com', 'Your Name'); $this->email->to('to@domain.com'); $this->email->subject('This is a html email'); $html = 'This is an <b>HTML</b> email'; $this->email->message($html); $this->email->send(); echo $this->email->print_debugger(); } } How it works... In the constructor controller we load the Email library (highlighted in the following code), which provides support for us to send e-mails: function __construct() { parent::__construct(); $this->load->helper('url'); $this->load->library('email'); } Next, public function index() redirects us to the function public function send_mail(), which sets some initial configuration variables for CodeIgniter Email library to work with, such as the system used to send the e-mail (in this case, sendmail), the path to send e-mail on your system, the mailtype variable (text or HTML), and so on. Take a look at the following line of code: $config['mailtype'] = 'html'; Here, we're telling CodeIgniter to send the e-mail as HTML rather than as text. These configuration settings are initialized (that is, passed to the Email library) and we begin to build the e-mail by setting the to, from, subject, and message attributes: $this->email->from('from@domain.com', 'Your Name'); $this->email->to('to@domain.com'); $this->email->subject('This is a text email'); $this->email->message('And this is some content for the text email.'); Then, send the e-mail using the following code: $this->email->send(); If all works out as planned, you should see an output similar to the following code: Your message has been successfully sent using the following protocol: sendmail User-Agent: CodeIgniter Date: Fri, 4 Oct 2013 08:56:59 +0200 From: "Your Name" <from@domain.com> Return-Path: <from@domain.com> To: to@domain.com Subject: =?iso-8859-1?Q?This_is_a_html_email?= Reply-To: "from@domain.com" <from@domain.com> X-Sender: from@domain.com X-Mailer: CodeIgniter X-Priority: 3 (Normal) Message-ID: <524e66bbf282f@domain.com> Mime-Version: 1.0 Content-Type: multipart/alternative; boundary="B_ALT_524e66bbf2868" This is a multi-part message in MIME format. Your email application may not support this format. --B_ALT_524e66bbf2868 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: 8bit This is an HTML email --B_ALT_524e66bbf2868 Content-Type: text/html; charset=iso-8859-1 Content-Transfer-Encoding: quoted-printable This is an <b>HTML</b> email --B_ALT_524e66bbf2868-- Sending attachments with CodeIgniter Email There might be times when you wish to send an attachment along with the e-mail, such as an invoice to a customer for a recent purchase or perhaps an image. The CodeIgniter Email library can easily be set to do just that. How to do it... You can send attachments with CodeIgniter Email by executing the following steps: Create a file email.php at /path/to/codeigniter/application/controllers/. Add the following code to the controller file, email.php: <?php if ( ! defined('BASEPATH')) exit('No direct script access allowed'); class Email extends CI_Controller { function __construct() { parent::__construct(); $this->load->helper('url'); $this->load->library('email'); } public function index() { redirect('email/send_email'); } public function send_email() { $config['protocol'] = 'sendmail'; $config['mailpath'] = '/usr/sbin/sendmail'; $config['charset'] = 'iso-8859-1'; $config['wordwrap'] = TRUE; $config['mailtype'] = 'html'; $this->email->initialize($config); $this->email->from('from@domain.com', 'Your Name'); $this->email->to('to@domain.com'); $this->email->subject('This is a html email'); $html = 'This is an <b>HTML</b> email with an attachment, <i>lovely!</i>'; $this->email->message($html); $this->email->attach('/path/to/attachment'); $this->email->send(); echo $this->email->print_debugger(); } } How it works... In the constructor controller we load the Email library (highlighted in the following code), which provides support to send e-mails: function __construct() { parent::__construct(); $this->load->helper('url'); $this->load->library('email'); } Next, public function index() redirects us to the function, public function send_mail(), which sets some initial configuration variables for the CodeIgniter Email library to work with, such as the system used to send the e-mail (in this case, sendmail), the path to send mail on your system, the mailtype variable (text or HTML), and so on. These configuration settings are initialized (that is, passed to the Email library) and we begin to build the e-mail; setting the to, from, subject, and message attributes, as well as the path to the attachment we're sending in the e-mail (highlighted in the following code): $this->email->from('from@domain.com', 'Your Name'); $this->email->to('to@domain.com'); $this->email->subject('This is a html email'); $html = 'This is an <b>HTML</b> email with an attachment, <i>lovely!</i>'; $this->email->message($html); $this->email->attach('/path/to/attachment'); Then, send the e-mail using the following code: $this->email->send(); Sending bulk e-mails with CodeIgniter Email There may be times when you wish to send out bulk e-mails; perhaps to all the people who have paid to go on a tour. You may wish to send them each a personalized e-mail, and also add an attachment. You may also want to pull their e-mail preference (plain text or HTML) from the account on your database and send them the correct format of e-mail. That's what we're going to do here. Getting ready We need to know each person's preferences such as whether they want HTML e-mails or text, and also their individual reference number (or booking ID) for their trip. As per this requirement, we are going to have a database to hold all the information; so copy the following code into your database: CREATE TABLE `bookers` ( `id` int(11) NOT NULL AUTO_INCREMENT, `firstname` varchar(50) NOT NULL, `lastname` varchar(50) NOT NULL, `email` varchar(255) NOT NULL, `email_pref` varchar(4) NOT NULL, `booking_ref` varchar(10) NOT NULL, PRIMARY KEY (`id`) ) ENGINE=InnoDB DEFAULT CHARSET=latin1 AUTO_INCREMENT=3 ; INSERT INTO `bookers` (`id`, `firstname`, `lastname`, `email`, `email_pref`, `booking_ref`) VALUES (1, 'Robert', 'Foster', 'example1@domain1.com', 'html', 'ABC123'), (2, 'Lucy', 'Welsh', 'example2@domain2.com', 'html', 'DEF456'); How to do it... Create a file email.php at /path/to/codeigniter/application/controllers/. Add the following code to the controller file, email.php: <?php if ( ! defined('BASEPATH')) exit('No direct script access allowed'); class Email extends CI_Controller { function __construct() { parent::__construct(); $this->load->helper('url'); $this->load->library('email'); } public function index() { redirect('email/send_email'); } public function send_email() { $config['protocol'] = 'sendmail'; $config['mailpath'] = '/usr/sbin/sendmail'; $config['charset'] = 'iso-8859-1'; $config['wordwrap'] = TRUE; $query = "SELECT * FROM bookers "; $result = $this->db->query($query); foreach ($result->result() as $row) { $this->email->clear(); if ($row->email_pref == 'text') { $config['mailtype'] = 'text'; $body = 'Hi ' . $row->firstname . ', Thanks you for booking with us, please find attached the itinerary for your trip. This is your booking reference number: ' . $row->booking_ref . ' Thanks for booking with us, have a lovely trip.'; } else { $config['mailtype'] = 'html'; $body = 'Hi ' . $row->firstname . ',<br /><br />Thanks you for booking with us, please find attached the itinerary for your trip. </p>This is your booking reference number: <b>' . $row->booking_ref . '</b><br /><br />Thanks for booking with us, have a lovely trip.'; } $this->email->initialize($config); $this->email->to($row->email); $this->email->from('bookings@thecodeigniterholidaycompany.com'); $this->email->subject('Holiday booking details'); $this->email->message($body); $this->email->send(); } echo $this->email->print_debugger(); } } How it works... In the constructor controller we load the Email library (highlighted in the following code), which provides support for us to send e-mails: function __construct() { parent::__construct(); $this->load->helper('url'); $this->load->library('email'); } Next, public function index() redirects us to the function, public function send_mail(), which sets some initial configuration variables for CodeIgniter Email library to work with, such as the system used to send the e-mail (in this case, sendmail), the path to send mail from your system. We then query the database for each of the customer's booking details: $query = "SELECT * FROM bookers "; $result = $this->db->query($query); foreach ($result->result() as $row) { } The query will loop through each result and send a specific e-mail based on the values retrieved from the database in each loop. Firstly, we give ourselves a clean slate by clearing all the settings and variables from a previous loop iteration by using the CodeIgniter email function: $this->email->clear(); We then look at their e-mail preference and set the e-mail sending (mailtype) variable accordingly, along with the text for the body of the e-mails. So, if someone prefers HTML, we look for that preference and define the body of the HTML e-mail, otherwise for a text e-mail, we look for the text e-mail preference and define the body for the text e-mail: if ($row->email_pref == 'text') { $config['mailtype'] = 'text'; $body = 'Hi ' . $row->firstname . ', Thank you for booking with us, please find attached the itinerary for your trip. This is your booking reference number: ' . $row->booking_ref . ' Thanks for booking with us, have a lovely trip.'; } else { $config['mailtype'] = 'html'; $body = 'Hi ' . $row->firstname . ',<br /><br />Thank you for booking with us, please find attached the itinerary for your trip. </p>This is your booking reference number: <b>' . $row->booking_ref . '</b><br /><br />Thanks for booking with us, have a lovely trip.'; } After this, we initialize the configuration variables. Those of you who have looked at the previous few recipes will notice that the initialization takes place later in the code of this recipe than in others. This is because we cannot initialize the config variables earlier as some of the variables rely on the preferences of individual customers, which are fetched from the database. So, we have to wait until each user's details are fetched from a database to initialize each iteration of the configuration settings. And finally, we send the e-mail: $this->email->send(); If all goes well, you should see an output similar to the following: Your message has been successfully sent using the following protocol: sendmail User-Agent: CodeIgniter Date: Fri, 4 Oct 2013 20:06:13 +0200 To: to@domain.com From: <bookings@thecodeigniterholidaycompany.com> Return-Path: <bookings@thecodeigniterholidaycompany.com> Subject: =?iso-8859-1?Q?Holiday_booking_details?= Reply-To: "bookings@thecodeigniterholidaycompany.com" <bookings@thecodeigniterholidaycompany.com> X-Sender: bookings@thecodeigniterholidaycompany.com X-Mailer: CodeIgniter X-Priority: 3 (Normal) Message-ID: <524f0395942a2@thecodeigniterholidaycompany.com> Mime-Version: 1.0 Content-Type: multipart/alternative; boundary="B_ALT_524f0395942bb" This is a multi-part message in MIME format. Your email application may not support this format. --B_ALT_524f0395942bb Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: 8bit Hi RobertThanks you booking with us, please find attached the itinerary for your trip. This is your booking reference number: ABC123 Thanks for booking with us, have a lovely trip. --B_ALT_524f0395942bb Content-Type: text/html; charset=iso-8859-1 Content-Transfer-Encoding: quoted-printable Hi Robert<br /><br />Thanks you booking with us,=20 please find attached the itinerary for your trip. </p>This is your booking reference number:=20 <b>ABC123</b><br /><br /> Thanks for booking with us, have a lovely trip. --B_ALT_524f0395942bb-- Using an HTML table with DataTable DataTable is a free to use library that turns your normal looking html table into an interactive marvel with sortable and searchable columns and a whole lot more; we're going to use it with CodeIgniter, merging DataTable and CodeIgniter table functionality. It's simple to use and is able to handle most of the things you will need it for. Here, in this recipe, we're going to use it with DataTable to create an interactive HTML table that is sortable and searchable. It has pagination too! If you want database results, move on to the next recipe, Using an HTML table with DataTable and a database, where we'll look at populating a table from a database query. Getting ready For this recipe, you will need to follow the given procedure: Ensure that you've downloaded DataTable from the following link: https://datatables.net/download/ Unzip the downloaded .zip file, and move the files to a location on your web server or localhost, which will be accessible by CodeIgniter. For this recipe, I have put the folder at application/views; but you can make your own choice if you wish. How to do it... Create four files as given: /path/to/codeigniter/application/controllers/table.php /path/to/codeigniter/application/views/table_header.php /path/to/codeigniter/application/views/table_body.php /path/to/codeigniter/application/views/table_footer.php Create the controller file, table.php, and add the following code to it: <?php if ( ! defined('BASEPATH')) exit('No direct script access allowed'); class Table extends CI_Controller { function __construct() { parent::__construct(); $this->load->library('table'); } public function index() { $tmpl = array ( 'table_open' => '<table border="0" cellpadding="4" cellspacing="0" id="example">', 'heading_row_start' => '<tr>', 'heading_row_end' => '</tr>', 'heading_cell_start' => '<th>', 'heading_cell_end' => '</th>', 'row_start' => '<tr>', 'row_end' => '</tr>', 'cell_start' => '<td>', 'cell_end' => '</td>', 'row_alt_start' => '<tr>', 'row_alt_end' => '</tr>', 'cell_alt_start' => '<td>', 'cell_alt_end' => '</td>', 'table_close' => '</table>' ); $this->table->set_template($tmpl); $this->table->set_heading(array('ID', 'First Name', 'Last Name')); $this->table->add_row(array('1', 'Rob', 'Foster')); $this->table->add_row(array('2', 'Lucy', 'Welsh')); $this->table->add_row(array('3', 'George', 'Foster')); $this->table->add_row(array('4', 'Jackie', 'Foster')); $this->table->add_row(array('5', 'Antony', 'Welsh')); $this->table->add_row(array('6', 'Rowena', 'Welsh')); $this->table->add_row(array('7', 'Peter', 'Foster')); $this->table->add_row(array('8', 'Jenny', 'Foster')); $this->table->add_row(array('9', 'Oliver', 'Welsh')); $this->table->add_row(array('10', 'Felicity', 'Foster')); $this->table->add_row(array('11', 'Harrison', 'Foster')); $this->table->add_row(array('12', 'Mia', 'The Cat')); $data['table'] = $this->table->generate(); $this->load->view('tables/table_header'); $this->load->view('tables/table_body',$data); $this->load->view('tables/table_footer'); } } Create the view file, table_header.php, and add the following code to it: <html> <head> <style type="text/css" title="currentStyle"> @import "<?php echo $this->config->item('base_url') ; ?>application/views/DataTables-1.9.4/media/css/demo_page.css"; @import "<?php echo $this->config->item('base_url') ; ?>application/views/DataTables-1.9.4/media/css/jquery.dataTables.css"; </style> <script type="text/javascript" language="javascript" src="<?php echo $this->config->item('base_url') ; ?>application/views/DataTables-1.9.4/media/js/jquery.js"></script> <script type="text/javascript" language="javascript" src="<?php echo $this->config->item('base_url') ; ?>application/views/DataTables-1.9.4/media/js/jquery.dataTables.js"></script> <script type="text/javascript" charset="utf-8"> $(document).ready(function() { $('#example').dataTable(); } ); </script> </head> <body> Take a look at the <script> tag: <script type="text/javascript" charset="utf-8"> $(document).ready(function() { $('#example').dataTable(); } ); </script> The #example parameter is the ID of the table (detailed in the following How it works... section). Ensure that the value example in <script> and table markup is the same. Create the view file, table_body.php, and add the following code to it: <?php echo $table ; ?> Create the controller file, table_footer.php, and add the following code to it:   </body> </html> How it works... The constructor in the table controller loads the CodeIgniter's Table library: function __construct() { parent::__construct(); $this->load->library('table'); } The public function index() function is called. We then define how we want our HTML table markup to appear. This is where you can place any markup for the specific CSS, using which you can style the elements of the table: $tmpl = array ( 'table_open' => '<table border="0" cellpadding="4" cellspacing="0" id="example">', 'heading_row_start' => '<tr>', 'heading_row_end' => '</tr>', 'heading_cell_start' => '<th>', 'heading_cell_end' => '</th>', 'row_start' => '<tr>', 'row_end' => '</tr>', 'cell_start' => '<td>', 'cell_end' => '</td>', 'row_alt_start' => '<tr>', 'row_alt_end' => '</tr>', 'cell_alt_start' => '<td>', 'cell_alt_end' => '</td>', 'table_close' => '</table>' ); Take a closer look at the table_open element of the $tmpl array. Look for the item highlighted in the preceding code. The id="example" item is used by DataTable (in the <script> tag of the file table_header.php) to apply its CSS and functionality. You can, of course, name it anything you like, but be sure to reflect that change in the JavaScript. We then call $this->table->set_template() to apply the HTML table markup: $this->table->set_template($tmpl); We then set the table headers and apply the data for our rows. Ensure that the number of items in the table headers is the same as the number of items in the table data: $this->table->set_heading(array('ID', 'First Name', 'Last Name')); $this->table->add_row(array('1', 'Rob', 'Foster')); $this->table->add_row(array('2', 'Lucy', 'Welsh')); $this->table->add_row(array('3', 'George', 'Foster')); $this->table->add_row(array('4', 'Jackie', 'Foster')); $this->table->add_row(array('5', 'Antony', 'Welsh')); $this->table->add_row(array('6', 'Rowena', 'Welsh')); $this->table->add_row(array('7', 'Peter', 'Foster')); $this->table->add_row(array('8', 'Jenny', 'Foster')); $this->table->add_row(array('9', 'Oliver', 'Welsh')); $this->table->add_row(array('10', 'Felicity', 'Foster')); $this->table->add_row(array('11', 'Harrison', 'Foster')); $this->table->add_row(array('12', 'Mia', 'The Cat')); We then generate the table. The $this->table->generate() function will return a string of HTML, which we save in $data['table']. $data['table'] = $this->table->generate(); The $data array our view files for rendering to the browser: $this->load->view('tables/table_header'); $this->load->view('tables/table_body',$data); $this->load->view('tables/table_footer');
Read more
  • 0
  • 0
  • 13105

article-image-image-classification-and-feature-extraction-images
Packt
25 Oct 2013
3 min read
Save for later

Image classification and feature extraction from images

Packt
25 Oct 2013
3 min read
(For more resources related to this topic, see here.) Classifying images Automated Remote Sensing ( ARS ) is rarely ever done in the visible spectrum. The most commonly available wavelengths outside of the visible spectrum are infrared and near-infrared. The following scene is a thermal image (band 10) from a fairly recent Landsat 8 flyover of the US Gulf Coast from New Orleans, Louisiana to Mobile, Alabama. Major natural features in the image are labeled so you can orient yourself: Because every pixel in that image has a reflectance value, it is information. Python can "see" those values and pick out features the same way we intuitively do by grouping related pixel values. We can colorize pixels based on their relation to each other to simplify the image and view related features. This technique is called classification. Classifying can range from fairly simple groupings based only on some value distribution algorithm derived from the histogram to complex methods involving training data sets and even computer learning and artificial intelligence. The simplest forms are called unsupervised classifications, whereas methods involving some sort of training data to guide the computer are called supervised. It should be noted that classification techniques are used across many fields, from medical doctors trying to spot cancerous cells in a patient's body scan, to casinos using facial-recognition software on security videos to automatically spot known con-artists at blackjack tables. To introduce remote sensing classification we'll just use the histogram to group pixels with similar colors and intensities and see what we get. First you'll need to download the Landsat 8 scene here: http://geospatialpython.googlecode.com/files/thermal.zip Instead of our histogram() function from previous examples, we'll use the version included with NumPy that allows you to easily specify a number of bins and returns two arrays with the frequency as well as the ranges of the bin values. We'll use the second array with the ranges as our class definitions for the image. The lut or look-up table is an arbitrary color palette used to assign colors to classes. You can use any colors you want. import gdalnumeric # Input file name (thermal image) src = "thermal.tif" # Output file name tgt = "classified.jpg" # Load the image into numpy using gdal srcArr = gdalnumeric.LoadFile(src) # Split the histogram into 20 bins as our classes classes = gdalnumeric.numpy.histogram(srcArr, bins=20)[1] # Color look-up table (LUT) - must be len(classes)+1. # Specified as R,G,B tuples lut = [[255,0,0],[191,48,48],[166,0,0],[255,64,64], [255,115,115],[255,116,0],[191,113,48],[255,178,115], [0,153,153],[29,115,115],[0,99,99],[166,75,0], [0,204,0],[51,204,204],[255,150,64],[92,204,204],[38,153,38], [0,133,0],[57,230,57],[103,230,103],[184,138,0]] # Starting value for classification start = 1 # Set up the RGB color JPEG output image rgb = gdalnumeric.numpy.zeros((3, srcArr.shape[0], srcArr.shape[1],), gdalnumeric.numpy.float32) # Process all classes and assign colors for i in range(len(classes)): mask = gdalnumeric.numpy.logical_and(start <= srcArr, srcArr <= classes[i]) for j in range(len(lut[i])): rgb[j] = gdalnumeric.numpy.choose(mask, (rgb[j], lut[i][j])) start = classes[i]+1 # Save the image gdalnumeric.SaveArray(rgb.astype(gdalnumeric.numpy.uint8), tgt, format="JPEG") The following image is our classification output, which we just saved as a JPEG. We didn't specify the prototype argument when saving as an image, so it has no georeferencing information. This result isn't bad for a very simple unsupervised classification. The islands and coastal flats show up as different shades of green. The clouds were isolated as shades of orange and dark blues. We did have some confusion inland where the land features were colored the same as the Gulf of Mexico. We could further refine this process by defining the class ranges manually instead of just using the histogram.
Read more
  • 0
  • 0
  • 13101

article-image-aspnet-site-performance-improving-javascript-loading
Packt
28 Oct 2010
11 min read
Save for later

ASP.Net Site Performance: Improving JavaScript Loading

Packt
28 Oct 2010
11 min read
  ASP.NET Site Performance Secrets Simple and proven techniques to quickly speed up your ASP.NET website Speed up your ASP.NET website by identifying performance bottlenecks that hold back your site's performance and fixing them Tips and tricks for writing faster code and pinpointing those areas in the code that matter most, thus saving time and energy Drastically reduce page load times Configure and improve compression – the single most important way to improve your site's performance Written in a simple problem-solving manner – with a practical hands-on approach and just the right amount of theory you need to make sense of it all           Read more about this book       One approach to improving page performance is to shift functionality from the server to the browser. Instead of calculating a result or validating a form in C# on the server, you use JavaScript code on the browser. A drawback of this approach is that it involves physically moving code from the server to the browser. Because JavaScript is not compiled, it can be quite bulky. This can affect page load times, especially if you use large JavaScript libraries. You're effectively trading off increased page load times against faster response times after the page has loaded. In this article by Matt Perdeck, author of ASP.NET Site Performance Secret, you'll see how to reduce the impact on page load times by the need to load JavaScript files. It shows: How JavaScript files can block rendering of the page while they are being loaded and executed How to load JavaScript in parallel with other resources How to load JavaScript more quickly (For more resources on ASP.Net, see here.) Problem: JavaScript loading blocks page rendering JavaScript files are static files, just as images and CSS files. However, unlike images, when a JavaScript file is loaded or executed using a <script> tag, rendering of the page is suspended. This makes sense, because the page may contain script blocks after the <script> tag that are dependent on the JavaScript file. If loading of a JavaScript file didn't block page rendering, the other blocks could be executed before the file had loaded, leading to JavaScript errors. Confirming with a test site You can confirm that loading a JavaScript file blocks rendering of the page by running the website in the folder JavaScriptBlocksRendering in the downloaded code bundle. This site consists of a single page that loads a single script, script1.js. It also has a single image, chemistry.png, and a stylesheet style1.css. It uses an HTTP module that suspends the working thread for five seconds when a JavaScript file is loaded. Images and CSS files are delayed by about two seconds. When you load the page, you'll see that the page content appears after only about five seconds. Then after two seconds, the image appears, unless you use Firefox, which often loads images in parallel with the JavaScript. If you make a Waterfall chart, you can see how the image and stylesheet are loaded after the JavaScript file, instead of in parallel: To get the delays, run the test site on IIS 7 in integrated pipeline mode. Do not use the Cassini web server built into Visual Studio. If you find that there is no delay, clear the browser cache. If that doesn't work either, the files may be in kernel cache on the server—remove them by restarting IIS using Internet Information Services (IIS) Manager. To open IIS manager, click on Start | Control Panel, type "admin" in the search box, click on Administrative Tools, and then double-click on Internet Information Services (IIS) Manager. Integrated/Classic Pipeline Mode As in IIS 6, every website runs as part of an application pool in IIS 7. Each IIS 7 application pool can be switched between Integrated Pipeline Mode (the default) and Classic Pipeline Mode. In Integrated Pipeline Mode, the ASP.NET runtime is integrated with the core web server, so that the server can be managed for example, via web.config elements. In Classic Pipeline Mode, IIS 7 functions more like IIS 6, where ASP.NET runs within an ISAPI extension. Approaches to reduce the impact on load times Although it makes sense to suspend rendering the page while a <script> tag loads or executes JavaScript, it would still be good to minimize the time visitors have to wait for the page to appear, especially if there is a lot of JavaScript to load. Here are a few ways to do that: Start loading JavaScript after other components have started loading, such as images and CSS files. That way, the other components load in parallel with the JavaScript instead of after the JavaScript, and so are available sooner when page rendering resumes. Load JavaScript more quickly. Page rendering is still blocked, but for less time. Load JavaScript on demand. Only load the JavaScript upfront that you need to render the page. Load the JavaScript that handles button clicks, and so on, when you need it. Use specific techniques to prevent JavaScript loading from blocking rendering. This includes loading the JavaScript after the page has rendered, or in parallel with page rendering. These approaches can be combined or used on their own for the best tradeoff between development time and performance. Let's go through each approach. Approach: Start loading after other components This approach aims to render the page sooner by loading CSS stylesheets and images in parallel with the JavaScript rather than after the JavaScript. That way, when the JavaScript has finished loading, the CSS and images will have finished loading too and will be ready to use; or at least it will take less time for them to finish loading after the JavaScript has loaded. To load the CSS stylesheets and images in parallel with the JavaScript, you would start loading them before you start loading the JavaScript. In the case of CSS stylesheets that is easy—simply place their <link> tags before the <script> tags: <link rel="Stylesheet" type="text/css" href="css/style1.css" /><script type="text/javascript" src="js/script1.js"></script> Starting the loading of images is slightly trickier because images are normally loaded when the page body is evaluated, not as a part of the page head. In the test page you just saw with the image chemistry.png, you can use a bit of simple JavaScript to get the browser to start loading the image before it starts loading the JavaScript file. This is referred to as "image preloading" (page PreLoadWithJavaScript.aspx in the folder PreLoadImages in the downloaded code bundle): <script type="text/javascript"> var img1 = new Image(); img1.src = "images/chemistry.png";</script><link rel="Stylesheet" type="text/css" href="css/style1.css" /><script type="text/javascript" src="js/script1.js"></script> Run the page now and you'll get the following Waterfall chart: When the page is rendered after the JavaScript has loaded, the image and CSS files have already been loaded; so the image shows up right away. A second option is to use invisible image tags at the start of the page body that preload the images. You can make the image tags invisible by using the style display:none. You would have to move the <script> tags from the page head to the page body after the invisible image tags, as shown (page PreLoadWithCss.aspx in folder PreLoadImages in the downloaded code bundle): <body> <div style="display:none"> <img src="images/chemistry.png" /> </div> <script type="text/javascript" src="js/script1.js"></script> Although the examples we've seen so far preload only one image, chemistry.png, you could easily preload multiple images. When you do, it makes sense to preload the most important images first, so that they are most likely to appear right away when the page renders. The browser loads components, such as images, in the order in which they appear in the HTML, so you'd wind up with something similar to the following code: <script type="text/javascript"> var img1 = new Image(); img1.src = "images/important.png"; var img1 = new Image(); img2.src = "images/notsoimportant.png"; var img1 = new Image(); img3.src = "images/unimportant.png";</script> Approach: Loading JavaScript more quickly The second approach is to simply spend less time loading the same JavaScript, so that visitors spend less time waiting for the page to render. There are a number of ways to achieve just that: Techniques used with images, such as caching and parallel download Free Content Delivery Networks GZIP compression Minification Combining or breaking up JavaScript files Removing unused code Techniques used with images JavaScript files are static files, just like images and CSS files. This means that many techniques that apply to images apply to JavaScript files as well, including the use of cookie-free domains, caching, and boosting parallel loading. Free Content Delivery Networks Serving static files from a Content Delivery Network (CDN) can greatly reduce download times, by serving the files from a server that is close to the visitor. A CDN also saves you bandwidth because the files are no longer served from your own server. A number of companies now serve popular JavaScript libraries from their CDNs for free. Here are their details: Google AJAX Libraries API http://code.google.com/apis/ajaxlibs/ Serves a wide range of libraries including jQuery, jQuery UI, Prototype, Dojo, and Yahoo! User Interface Library (YUI) Microsoft Ajax Content Delivery Network http://www.asp.net/ajaxlibrary/cdn.ashx Serves libraries used by the ASP.NET and ASP.NET MVC frameworks including the jQuery library and the jQuery Validation plugin jQuery CDN http://docs.jquery.com/Downloading_jQuery Serves the jQuery library In ASP.NET 4.0 and later, you can get the ScriptManager control to load the ASP. NET AJAX script files from the Microsoft AJAX CDN instead of your web server, by setting the EnableCdn property to true: <asp:ScriptManager ID="ScriptManager1" EnableCdn="true" runat="server" /> One issue with loading libraries from a CDN is that it creates another point of failure—if the CDN goes down, your site is crippled. GZIP compression IIS has the ability to compress content sent to the browser, including JavaScript and CSS files. Compression can make a dramatic difference to a JavaScript file as it goes over the wire from the server to the browser. Take for example the production version of the jQuery library:   Uncompressed Compressed jQuery library 78 KB 26 KB   Compression for static files is enabled by default in IIS 7. This immediately benefits CSS files. It should also immediately benefit JavaScript files, but it doesn't because of a quirk in the default configuration of IIS 7. Not all static files benefit from compression; for example JPEG, PNG, and GIF files are already inherently compressed because of their format. To cater to this, the IIS 7 configuration file applicationHost.config contains a list of mime types that get compressed when static compression is enabled: <staticTypes> <add mimeType="text/*" enabled="true" /> <add mimeType="message/*" enabled="true" /> <add mimeType="application/javascript" enabled="true" /> <add mimeType="*/*" enabled="false" /></staticTypes> To allow IIS to figure out what mime type a particular file has, applicationHost.config also contains default mappings from file extensions to mime types, including this one: <staticContent lockAttributes="isDocFooterFileName"> ... <mimeMap fileExtension=".js" mimeType="application/x-javascript" /> ...</staticContent> If you look closely, you'll see that the .js extension is mapped by default to a mime type that isn't in the list of mime types to be compressed when static file compression is enabled. The easiest way to solve this is to modify your site's web.config, so that it maps the extension .js to mime type text/javascript. This matches text/* in the list of mime types to be compressed. So, IIS 7 will now compress JavaScript files with the extension .js (folder Minify in the downloaded code bundle): <system.webServer> <staticContent> <remove fileExtension=".js" /> <mimeMap fileExtension=".js" mimeType="text/javascript" /> </staticContent></system.webServer> Keep in mind that IIS 7 only applies static compression to files that are "frequently" requested. This means that the first time you request a file, it won't be compressed! Refresh the page a couple of times and compression will kick in.
Read more
  • 0
  • 0
  • 13093

article-image-dhtmlx-grid
Packt
30 Oct 2013
7 min read
Save for later

The DHTMLX Grid

Packt
30 Oct 2013
7 min read
(For more resources related to this topic, see here.) The DHTMLX grid component is one of the more widely used components of the library. It has a vast amount of settings and abilities that are so robust we could probably write an entire book on them. But since we have an application to build, we will touch on some of the main methods and get into utilizing it. Some of the cool features that the grid supports is filtering, spanning rows and columns, multiple headers, dynamic scroll loading, paging, inline editing, cookie state, dragging/ordering columns, images, multi-selection, and events. By the end of this article, we will have a functional grid where we will control the editing, viewing, adding, and removing of users. The grid methods and events When creating a DHTMLX grid, we first create the object; second we add all the settings and then call a method to initialize it. After the grid is initialized data can then be added. The order of steps to create a grid is as follows: Create the grid object Apply settings Initialize Add data Now we will go over initializing a grid. Initialization choices We can initialize a DHTMLX grid in two ways, similar to the other DHTMLX objects. The first way is to attach it to a DOM element and the second way is to attach it to an existing DHTMLX layout cell or layout. A grid can be constructed by either passing in a JavaScript object with all the settings or built through individual methods. Initialization on a DOM element Let's attach the grid to a DOM element. First we must clear the page and add a div element using JavaScript. Type and run the following code line in the developer tools console: document.body.innerHTML = "<div id='myGridCont'></div>"; We just cleared all of the body tags content and replaced it with a div tag having the id attribute value of myGridCont. Now, create a grid object to the div tag, add some settings and initialize it. Type and run the following code in the developer tools console: var myGrid = new dhtmlXGridObject("myGridCont"); myGrid.setImagePath(config.imagePath); myGrid.setHeader(["Column1", "Column2", "Column3"]); myGrid.init(); You should see the page with showing just the grid header with three columns. Next, we will create a grid on an existing cell object. Initialization on a cell object Refresh the page and add a grid to the appLayout cell. Type and run the following code in the developer tools console: var myGrid = appLayout.cells("a").attachGrid(); myGrid.setImagePath(config.imagePath); myGrid.setHeader(["Column1","Column2","Column3"]); myGrid.init(); You will now see the grid columns just below the toolbar. Grid methods Now let's go over some available grid methods. Then we can add rows and call events on this grid. For these exercises we will be using the global appLayout variable. Refresh the page. attachGrid We will begin by creating a grid to a cell. The attachGrid method creates and attaches a grid object to a cell. This is the first step in creating a grid. Type and run the following code line in the console: var myGrid = appLayout.cells("a").attachGrid(); setImagePath The setImagePath method allows the grid to know where we have the images placed for referencing in the design. We have the application image path set in the config object. Type and run the following code line in the console: myGrid.setImagePath(config.imagePath); setHeader The setHeader method sets the column headers and determines how many headers we will have. The argument is a JavaScript array. Type and run the following code line in the console: myGrid.setHeader(["Column1", "Column2", "Column3"]); setInitWidths The setinitWidths method will set the initial widths of each of the columns. The asterisk mark (*) is used to set the width automatically. Type and run the following code line in the console: myGrid.setInitWidths("125,95,*");   setColAlign The setColAlign method allows us to align the column's content position. Type and run the following code line in the console: myGrid.setColAlign("right,center,left"); init Up until this point, we haven't seen much going on. It was all happening behind the scenes. To see these changes the grid must be initialized. Type and run the following code line in the console: myGrid.init(); Now you see the columns that we provided. addRow Now that we have a grid created let's add a couple rows and start interacting. The addRow method adds a row to the grid. The parameters are ID and columns. Type and run the following code in the console: myGrid.addRow(1,["test1","test2","test3"]); myGrid.addRow(2,["test1","test2","test3"]); We just created two rows inside the grid. setColTypes The setColTypes method sets what types of data a column will contain. The available type options are: ro (readonly) ed (editor) txt (textarea) ch (checkbox) ra (radio button) co (combobox) Currently, the grid allows for inline editing if you were to double-click on grid cell. We do not want this for the application. So, we will set the column types to read-only. Type and run the following code in the console: myGrid.setColTypes("ro,ro,ro"); Now the cells are no longer editable inside the grid. getSelectedRowId The getSelectedRowId method returns the ID of the selected row. If there is nothing selected it will return null. Type and run the following code line in the console: myGrid.getSelectedRowId(); clearSelection The clearSelection method clears all selections in the grid. Type and run the following code line in the console: myGrid.clearSelection(); Now any previous selections are cleared. clearAll The clearAll method removes all the grid rows. Prior to adding more data to the grid we first must clear it. If not we will have duplicated data. Type and run the following code line in the console: myGrid.clearAll(); Now the grid is empty. parse The parse method allows the loading of data to a grid in the format of an XML string, CSV string, XML island, XML object, JSON object, and JavaScript array. We will use the parse method with a JSON object while creating a grid for the application. Here is what the parse method syntax looks like (do not run this in console): myGrid.parse(data, "json"); Grid events The DHTMLX grid component has a vast amount of events. You can view them in their entirety in the documentation. We will cover the onRowDblClicked and onRowSelect events. onRowDblClicked The onRowDblClicked event is triggered when a grid row is double-clicked. The handler receives the argument of the row ID that was double-clicked. Type and run the following code in console: myGrid.attachEvent("onRowDblClicked", function(rowId){ console.log(rowId); }); Double-click one of the rows and the console will log the ID of that row. onRowSelect The onRowSelect event will trigger upon selection of a row. Type and run the following code in console: myGrid.attachEvent("onRowSelect", function(rowId){ console.log(rowId); }); Now, when you select a row the console will log the id of that row. This can be perceived as a single click. Summary In this article, we learned about the DHTMLX grid component. We also added the user grid to the application and tested it with the storage and callbacks methods. Resources for Article: Further resources on this subject: HTML5 Presentations - creating our initial presentation [Article] HTML5: Generic Containers [Article] HTML5 Canvas [Article]
Read more
  • 0
  • 0
  • 13091
article-image-getting-places
Packt
13 Oct 2015
8 min read
Save for later

Getting Places

Packt
13 Oct 2015
8 min read
In this article by Nafiul Islam, the author of Mastering Pycharm, we'll learn all about navigation. It is divided into three parts. The first part is called Omni, which deals with getting to anywhere from any place. The second is called Macro, which deals with navigating to places of significance. The third and final part is about moving within a file and it is called Micro. By the end of this article, you should be able to navigate freely and quickly within PyCharm, and use the right tool for the job to do so. Veteran PyCharm users may not find their favorite navigation tool mentioned or explained. This is because the methods of navigation described throughout this article will lead readers to discover their own tools that they prefer over others. (For more resources related to this topic, see here.) Omni In this section, we will discuss the tools that PyCharm provides for a user to go from anywhere to any place. You could be in your project directory one second, the next, you could be inside the Python standard library or a class in your file. These tools are generally slow or at least slower than more precise tools of navigation provided. Back and Forward The Back and Forward actions allow you to move your cursor back to the place where it was previously for more than a few seconds or where you've made edits. This information persists throughout sessions, so even if you exit the IDE, you can still get back to the positions that you were in before you quit. This falls into the Omni category because these two actions could potentially get you from any place within a file to any place within a file in your directory (that you have been to) to even parts of the standard library that you've looked into as well as your third-party Python packages. The Back and Forward actions are perhaps two of my most used navigation actions, and you can use Keymap. Or, one can simply click on the Navigate menu to see the keyboard shortcuts: Macro The difference between Macro and Omni is subtle. Omni allows you to go to the exact location of a place, even a place of no particular significance (say, the third line of a documentation string) in any file. Macro, on the other hand, allows you to navigate anywhere of significance, such as a function definition, class declaration, or particular class method. Go to definition or navigate to declaration Go to definition is the old name for Navigate to Declaration in PyCharm. This action, like the one previously discussed, could lead you anywhere—a class inside your project or a third party library function. What this action does is allow you to go to the source file declaration of a module, package, class, function, and so on. Keymap is once again useful in finding the shortcut for this particular action. Using this action will move your cursor to the file where the class or function is declared, may it be in your project or elsewhere. Just place your cursor on the function or class and invoke the action. Your cursor will now be directly where the function or class was declared. There is, however, a slight problem with this. If one tries to go to the declaration of a .so object, such as the datetime module or the select module, what one will encounter is a stub file (discussed in detail later). These are helper files that allow PyCharm to give you the code completion that it does. Modules that are .so files are indicated by a terminal icon, as shown here: Search Everywhere The action speaks for itself. You search for classes, files, methods, and even actions. Universally invoked using double Shift (pressing Shift twice in quick succession), this nifty action looks similar to any other search bar. Search Everywhere searches only inside your project, by default; however, one can also use it to search non-project items as well. Not using this option leads to faster search and a lower memory footprint. Search Everywhere is a gateway to other search actions available in PyCharm. In the preceding screenshot, one can see that Search Everywhere has separate parts, such as Recent Files and Classes. Each of these parts has a shortcut next to their section name. If you find yourself using Search Everywhere for Classes all the time, you might start using the Navigate Class action instead which is much faster. The Switcher tool The Switcher tool allows you to quickly navigate through your currently open tabs, recently opened files as well as all of your panels. This tool is essential since you always navigate between tabs. A star to the left indicates open tabs; everything else is a recently opened or edited file. If you just have one file open, Switcher will show more of your recently opened files. It's really handy this way since almost always the files that you want to go to are options in Switcher. The Project panel The Project panel is what I use to see the structure of my project as well as search for files that I can't find with Switcher. This panel is by far the most used panel of all, and for good reason. The Project panel also supports search; just open it up and start typing to find your file. However, the Project panel can give you even more of an understanding of what your code looks similar to if you have Show Members enabled. Once this is enabled, you can see the classes as well as the declared methods inside your files. Note that search works just like before, meaning that your search is limited to only the files/objects that you can see; if you collapse everything, you won't be able to search either your files or the classes and methods in them. Micro Micro deals with getting places within a file. These tools are perhaps what I end up using the most in my development. The Structure panel The Structure panel gives you a bird's eye view of the file that you are currently have your cursor on. This panel is indispensable when trying to understand a project that one is not familiar with. The yellow arrow indicates the option to show inherited fields and methods. The red arrow indicates the option to show field names, meaning if that it is turned off, you will only see properties and methods. The orange arrow indicates the option to scroll to and from the source. If both are turned on (scroll to and scroll from), where your cursor is will be synchronized with what method, field, or property is highlighted in the structure panel. Inherited fields are grayed out in the display. Ace Jump This is my favorite navigation plugin, and was made by John Lindquist who is a developer at JetBrains (creators of PyCharm). Ace Jump is inspired from the Emacs mode with the same name. It allows you to jump from one place to another within the same file. Before one can use Ace Jump, one has to install the plugin for it. Ace Jump is usually invoked using Ctrl or command + ; (semicolon). You can search for Ace Jump in Keymap as well, and is called Ace Jump. Once invoked, you get a small box in which you can input a letter. Choose a letter from the word that you want to navigate to, and you will see letters on that letter pop up immediately. If we were to hit D, the cursor would move to the position indicated by D. This might seem long winded, but it actually leads to really fast navigation. If we wanted to select the word indicated by the letter, then we'd invoke Ace Jump twice before entering a letter. This turns the Ace Jump box red. Upon hitting B, the named parameter rounding will be selected. Often, we don't want to go to a word, but rather the beginning or the end of a line. In order to do this, just hit invoke Ace Jump and then the left arrow for line beginnings or the right arrow for line endings. In this case, we'd just hit V to jump to the beginning of the line that starts with num_type. This is an example, where we hit left arrow instead of the right one, and we get line-ending options. Summary In this article, I discussed some of the best tools for navigation. This is by no means an exhaustive list. However, these tools will serve as a gateway to more precise tools available for navigation in PyCharm. I generally use Ace Jump, Back, Forward, and Switcher the most when I write code. The Project panel is always open for me, with the most used files having their classes and methods expanded for quick search. Resources for Article: Further resources on this subject: Enhancing Your Blog with Advanced Features [article] Adding a developer with Django forms [article] Deployment and Post Deployment [article]
Read more
  • 0
  • 0
  • 13086

article-image-internet-things
Packt
15 Oct 2015
13 min read
Save for later

The Internet of Things

Packt
15 Oct 2015
13 min read
In this article by Charles Hamilton, author of the book BeagleBone Black Cookbook, we will cover location-based recipes and how to hook up GPS. (For more resources related to this topic, see here.) Introduction The Internet of Things is a large basketful of things. In fact, it is so large that no one can see the edges of it yet. It is an evolving and quickly expanding repo of products, concepts, fledgling business ventures, prototypes, middleware, ersatz systems, and hardware. Some define IoT as connecting things that are not normally connected, thus making them a bit more useful than they were as unconnected devices. We will not show you how to turn off the lights in your house using the BBB, or how to auto raise the garage door when you turn onto your street while driving. There are a bunch of tutorials that do that already. Instead, we will take a look at some of the recipes that provide some fundamental elements to build IoT-centric prototypes or use cases. Location-based recipes – hooking up GPS A common question in the IoT realm is where is that darn thing? That Internet of Things thing? Being able to track and pinpoint the location of a device is one of the more typical features of many IoT use cases. So, we will first take a look at a recipe on how to for use everyone's favorite location tech: GPS. Then, we will explore one of the newer innovations spun out of Bluetooth 4.0, a way to capture more precise location-based data rather than GPS using beacons. The UART background In the galaxy of embedded systems, developers use dozens of different serial protocols. On the more common side, consumers are familiar with things such as USB and Ethernet. Then, there are protocols familiar to engineers, such as SPI and I2C, which we have already explored in this book. For this recipe, we will use yet another flavor of serial, UART, an asynchronous or clock-less protocol. This comes in handy in a variety of scenarios to connect IoT-centric devices. Universal asynchronous receiver/transmitter (UART) is a common circuit block used to manage serial data and hardware. As UART does not require a clock signal, it uses fewer wires and pins. In fact, UART uses only two serial wires: RX to receive packets and TX to transmit them. The framework for this recipe comes from AdaFruit's tutorial for the RPi. However, the differences between these two boards are nontrivial, so this recipe needs quite a few more ingredients than the RPi version. Getting ready You will need the following components for this recipe: GPS PCB: You can probably find cheaper versions, but we will use AdaFruit's well regarded and ubiquitous PCB (http://www.adafruit.com/product/746 at around USD $40.00). Antenna: Again, Adafruit's suggested SMA to the uFL adapter antenna is the simplest and cheap at USD $3.00 (https://www.adafruit.com/product/851). 5V power: Powering via the 5V DC in lieu of simply connecting via the mini USB is advisable. The GPS modules consume a good bit of power, a fact apparent to all of us, given how the GPS functionality is a well-known drain on our smartphones. Internet connectivity, either via WiFi or Ethernet Breadboard 4x jumper wires How to do it… For the GPS setup, the steps are as follows: Insert the PCB pins into the breadboard and wire the pins according to the following fritzing diagram: P9_11 (blue wire): This denotes RX on BBB and TX on GPS PCB. At first, it may seem confusing to not wire TX to TX, and so on. However, once you understand the pin's function, the logic is clear: a transmit (TX) pin pairs with a pin that can receive data (RX); a receive pin pairs with a pin that transmits data. P9_13 (green wire): This specifies TX on BBB and RX on GPS PCB. P9_1: This indicates GND. P9_3: This denotes 3.3V. Carefully attach the antenna to the board's uFL connector. Now, power your BBB. Here's where it gets a bit tricky. When your BBB starts up, you will immediately see the Fix button on the GPS board that will begin to flash quickly, around 1x per second. We will come back to check the integrity of the module's satellite connection in a later step. In order to gain access to the UART pins on the BBB, we have to enable them with a Device Tree overlay. Until recently, this was a multistep process. However, now that the BeagleBone universal I/O package comes preloaded on the current versions of the firmware, enabling the pins—in the case, UART4—is a snap. Let's begin by logging in as root with the following command: $ sudo -i Then, run the relevant universal I/O command and check whether it went to the right place, as shown in the following code: # config-pin overlay BB-UART4 # cat /sys/devices/bone_capemgr.*/slots Now, reboot your BBB, then check whether the device is present in the device list using the following command: $ ls -l /dev/ttyO* crw-rw---- 1 root tty 247, 0 Mar 1 20:46 /dev/ttyO0 crw-rw---T 1 root dialout 247, 4 Jul 13 02:12 /dev/ttyO4 Finally, check whether it is loading properly with the following command: $ dmesg This is how the output looks: [ 188.335168] bone-capemgr bone_capemgr.9: part_number 'BB-UART4', version 'N/A' [ 188.335235] bone-capemgr bone_capemgr.9: slot #7: generic override [ 188.335250] bone-capemgr bone_capemgr.9: bone: Using override eeprom data at slot 7 [ 188.335266] bone-capemgr bone_capemgr.9: slot #7: 'Override Board Name,00A0,Override Manuf,BB-UART4' [ 188.335355] bone-capemgr bone_capemgr.9: slot #7: Requesting part number/version based 'BB-UART4-00A0.dtbo [ 188.335370] bone-capemgr bone_capemgr.9: slot #7: Requesting firmware 'BB-UART4-00A0.dtbo' for board-name 'Override Board Name', version '00A0' [ 188.335400] bone-capemgr bone_capemgr.9: slot #7: dtbo 'BB-UART4-00A0.dtbo' loaded; converting to live tree [ 188.335673] bone-capemgr bone_capemgr.9: slot #7: #2 overlays [ 188.343353] 481a8000.serial: ttyO4 at MMIO 0x481a8000 (irq = 45) is a OMAP UART4 [ 188.343792] bone-capemgr bone_capemgr.9: slot #7: Applied #2 overlays. Tips to get a GPS fix Your best method to get the GPS module connected is to take it outdoors. However, as that's not a likely option when you are developing a project, putting it against or even just outside a window will often suffice. If it is cloudy, and if you don't have a reasonably clear sky view from your module's antenna, do not expect a quick connection. Be patient. When a fix is made, the flashing LED will cycle very slowly at about 15-second intervals. Even if the GPS modules do not have a fix, be aware that they will still send data. This can be confusing because you may run some of the following commands and think that your connection is fine, but you just keep getting junk (blank) data. However, to reiterate that the flashing LED has to have slowed down to 15-second intervals to confirm that you have a fix. Although the output is not pretty, the following command is a useful first step in making sure that your devices are hooked up because it will show the raw NMEA data coming out of the GPS: $ cat /dev/ttyO4 The National Marine Electronics Association uses a GPS language protocol standard. Verify that your wiring is correct and that the module is generating the data properly (irrespective of a satellite fix) as follows: $ sudo screen /dev/ttyO4 9600 The output should immediately begin and look something similar to this: $GPGGA,163356.000,4044.0318,N,07400.1854,W,1,5,2.65,4.0,M,-34.2,M,,*67 $GPGSA,A,3,13,06,10,26,02,,,,,,,,2.82,2.65,0.95*04 $GPRMC,163356.000,A,4044.0318,N,07400.1854,W,2.05,68.70,031214,,,A*46 $GPVTG,68.70,T,,M,2.05,N,3.81,K,A*09 $GPGGA,163357.000,4044.0322,N,07400.1853,W,1,5,2.65,3.7,M,-34.2,M,,*68 $GPGSA,A,3,13,06,10,26,02,,,,,,,, Now quit the program using one of the the following methods: Press Ctrl + a, enter or copy and paste :quit with the colon to the highlighted box at the bottom, or press Ctrl + a, k, y. Installing the GPS toolset Perform the following steps to install the GPS toolset: The next set of ingredients in the recipe consists of installing and testing a common toolset to parse GPS on Linux. As always, before installing something new, it is good practice to update your repos with the following command: $ sudo apt-get update Install the tools, including gpsd, a service daemon to monitor your GPS receiver. The package exposes all the data on location, course, and velocity on the TCP port 2947 of your BBB and efficiently parses the NMEA text pouring out of the GPS receiver, as shown in the following command: $ sudo apt-get install gpsd gpsd-clients python-gps For the preceding command, gpsd-clients installs some test clients, and python-gps installs the required Python library so that it can communicate with gpsd via Python scripts. After the installation, you may find it useful to run gpsd and review the package's well-written and informative manual. It provides not only the details around what you just installed, but also the general GPS-related context as well. If the planets or communication satellites are aligned, you can run this command from the newly installed toolset and begin displaying the GPS data: $ sudo gpsmon /dev/ttyO4 You should see a terminal GUI that looks similar to the following screenshot: To quit, press Ctrl + c or type q and then press return (Enter) key. Now, we will test the other principal tool that you just installed with the following command: $ sudo cgps -s The output includes the current date and time in UTC, the latitude and longitude, and the approximate altitude. Troubleshooting: Part 1 You may run into problems here. Commonly, on a first time set up and running, cgps may time out, close by itself, and lead you to believe that there is a problem with your setup. If so, the next steps can lead you back on the path to GPS nirvana: We will begin by stopping all the running instances of GPS, as shown in the following code: $ sudo killall gpsd Now, let's get rid of any sockets that the gpsd commands may have left behind with the following command: $ sudo rm /var/run/gpsd.sock There is a systemd bug that we will typically need to address. Here is how we fix it: Open the systemd GPSD service using the following command: $ sudo nano /lib/systemd/system/gpsd.service Paste it to the window with the following script: [Unit] Description=GPS (Global Positioning System) Daemon Requires=gpsd.socket [Service] ExecStart=/usr/sbin/gpsd -n -N /dev/ttyO4 [Install] Also=gpsd.socket Then, restart the systemd service as follows: $ sudo service gpsd start You should now be able to run either of the following service again: $ sudo gpsmon /dev/ttyO4 Alternatively, you can also run the following command: $ sudo cgps -s Troubleshooting: Part 2 Sometimes, the preceding fixes don't fix it. Here are several more suggestions for troubleshooting purposes: Set up a control socket for GPS with the following command: $ sudo gpsd -N -D3 -F /var/run/gpsd.sock The explanation of the command-line flags or options are as follows: -N: This tells gpsd to immediately post the GPS data. Although, this is useful for testing purposes, it can also be a power consumption, so leave it off if your use case is battery-powered. -F: This creates a control socket for device addition and removal. The option requires a valid pathname on your local filesystem, which is why our command is appended with /var/run/gpsd.sock. We may also need to install a package that lets us examine any port conflict that could be occurring, a shown in the following code: $ sudo apt-get install lsof This installed utility will open and display the system files, including disk files, named pipes, network sockets, and devices opened by all the processes. There are multiple uses for the tool. However, we only want to determine whether the GPS module is speaking correctly to port 2947 and if there are any conflicts. So, we will run the following command: $ sudo lsof -i :2947 This is how the output should look: COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME systemd 1 root 24u IPv4 6907 0t0 TCP localhost:gpsd (LISTEN) gpsd 5960 nobody 4u IPv4 6907 0t0 TCP localhost:gpsd (LISTEN) You may also want to check whether any instances of the GPS are running and then kill them with the following code: $ ps aux |grep gps $ sudo killall gpsd For a final bit of cooking with the GPS board, we want to run a Python script and display the data in tidy, parsed output. The code was originally written for the RPi, but it is useable on the BBB as well. Go, get it using the following command: $ git clone https://github.com/HudsonWerks/gps-tests.git Now, browse to the new directory that we just created and take a look at the file that we will use with the following command: $ cd gps-tests $ sudo nano GPStest1.py Then, open a new Python file and paste the following code to the window: $ sudo nano GPStest1.py The script requires a number of Python libraries, as shown in the following code: import os from gps import * from time import * import time import threading Keep in mind that getting a fix and then obtaining a good GPS data can take several moments before the system settles into a comfortable flow, as shown in the following code: #It may take a second or two to get good data #print gpsd.fix.latitude,', ',gpsd.fix.longitude,' Time: ',gpsd.utc If you find the output overwhelming, you can always modify the print commands to simplify the display as follows: print print ' GPS reading' print '----------------------------------------' print 'latitude ' , gpsd.fix.latitude print 'longitude ' , gpsd.fix.longitude print 'time utc ' , gpsd.utc,' + ', gpsd.fix.time print 'altitude (m)' , gpsd.fix.altitude print 'eps ' , gpsd.fix.eps print 'epx ' , gpsd.fix.epx print 'epv ' , gpsd.fix.epv print 'ept ' , gpsd.fix.ept print 'speed (m/s) ' , gpsd.fix.speed print 'climb ' , gpsd.fix.climb print 'track ' , gpsd.fix.track print 'mode ' , gpsd.fix.mode print print 'sats ' , gpsd.satellites time.sleep(5) Now, close the script and run the following command: $ python GPStest1.py In a few seconds, nicely formatted GPS data should be spitting out in your terminal window. There's more... Sparkfun's tutorial on GPS is definitely worth the read at https://learn.sparkfun.com/tutorials/gps-basics/all For further GPSD troubleshooting, refer to http://www.catb.org/gpsd/troubleshooting.html Summary In this article we discussed how to hook up GPS in detail as one of the major features of IoT. Resources for Article: Further resources on this subject: Learning BeagleBone Python Programming [Article] Learning BeagleBone [Article] Protecting GPG Keys in BeagleBone [Article]
Read more
  • 0
  • 0
  • 13077

article-image-string-encryption-and-decryption
Packt
22 Jun 2017
21 min read
Save for later

String Encryption and Decryption

Packt
22 Jun 2017
21 min read
In this article by Brenton J.W Blawat, author of the book Enterprise PowerShell Scripting Bootcamp, we will learn about string encryption and decryption. Large enterprises often have very strict security standards that are required by industry-specific regulations. When you are creating your Windows server scanning script, you will need to approach the script carefully with certain security concepts in mind. One of the most common situations you may encounter is the need to leverage sensitive data, such as credentials,in your script. While you could prompt for sensitive data during runtime, most enterprises want to automate the full script using zero-touch automation. (For more resources related to this topic, see here.) Zero-touch automation requires that the scripts are self-contained and have all of the required credentials and components to successfully run. The problem with incorporating sensitive data in the script, however, is that data can be obtained in clear text. The usage of clear text passwords in scripts is a bad practice, and violates many regulatory and security standards. As a result, PowerShell scripters need a method to securely store and retrieve sensitive data for use in their scripts. One of the popular methods to secure sensitive data is to encrypt the sensitive strings. This article explores RijndaelManaged symmetric encryption, and how to use it to encrypt and decrypt strings using PowerShell. In this article, we will cover the following topics: Learn about RijndaelManaged symmetric encryption Understand the salt, init, and password for the encryption algorithm Script a method to create randomized salt, init, and password values Encrypt and decrypt strings using RijndaelManaged encryption Create an encoding and data separation security mechanism for encryption passwords The examples in this article build upon each other. You will need to execute the script sequentially to have the final script in this article work properly. RijndaelManaged encryption When you are creating your scripts, it is best practice to leverage some sort of obfuscation, or encryption for sensitive data. There are many different strategies that you can use to secure your data. One is leveraging string and script encoding. Encoding takes your human readable string or script, and scrambles it to make it more difficult for someone to see what the actual code is. The downsides of encoding are that you must decode the script to make changes to it and decoding does not require the use of a password or passphrase. Thus, someone can easily decode your sensitive data using the same method you would use to decode the script. The alternative to encoding is leveraging an encryption algorithm. Encryption algorithms provide multiple mechanisms to secure your scripts and strings. While you can encrypt your entire script, it's most commonly used to encrypt the sensitive data in the scripts themselves, or answer files. One of the most popular encryption algorithms to use with PowerShell is RijndaelManaged. RijndaelManaged is a symmetric block cipher algorithm, which was selected by United States National Institute of Standards and Technology (NIST) for its implementation of Advanced Encryption Standard (AES). When using RijndaelManaged for the standard of AES, it supports 128-bit, 192-bit, and 256-bit encryption. In contrast to encoding, encryption algorithms require additional information to be able to properly encrypt and decrypt the string. When implementing the RijndaelManaged in PowerShell, the algorithm requires salt, a password, and the InitializationVector (IV). The salt is typically a randomized value that changes each time you leverage the encryption algorithm. The purpose of salt in a traditional encryption scenario is to change the encrypted value each time the encryption function is used. This is important in scenarios where you are encrypting multiple passwords or strings with the same value. If two users are using the same password, the encryption value in the database would also be the same. By changing the salt each time, the passwords, though the same value, would have different encrypted values in the database. In this article, we will be leveraging a static salt value. The password typically is a value that is manually entered by a user, or fed into the script using a parameter block. You can also derive the password value from a certificate, active directory attribute values, or a multitude of other sources. In this article, we will be leveraging three sources for the password. The InitializationVector (IV) is a hash generated from the IV string and is used for the EncryptionKey. The IV string is also typically a randomized value that changes each time you leverage the encryption algorithm. The purpose of the IV string is to strengthen the hash created by the encryption algorithm. This was created to thwart a hacker who is leveraging a rainbow attack using precalculated hash tables using no IV strings, or commonly used strings. Since you are setting the IV string, the number of hash combinations exponentially increases and it reduces the effectiveness of a rainbow attack. In this article, we will be using a static initialization vector value. The implementation of randomization of the salt and initialization vector strings become more important in scenarios where you are encrypting a large set of data. An attacker can intercept hundreds of thousands of packets, or strings,which reveals an increasing amount of information about your IV. With this, the attacker can guess the IV and derive the password. The most notable hack of IVs were with WiredEquivalentPrivacy (WEP) wireless protocol that used aweak, or small, initialization vector. After capturing enough packets, anIV hash could be guessed and a hacker could easily obtain the passphrase used on the wireless network. Creating random salt, initialization vector, and passwords As you are creating your scripts, you will want to make sure you use complex random values for the salt, IV string, and password. This is to prevent dictionary attacks where an individual may use common passwords and phrases to guess the salt, IV string, and password. When you create your salt and IVs, make sure they are a minimum of 10 random characters each. It is also recommended that you use a minimum of 30 random characters for the password. To create random passwords in PowerShell, you can do the following: Function create-password { # Declare password variable outside of loop. $password = "" # For numbers between 33 and 126 For ($a=33;$a –le 126;$a++) { # Add the Ascii text for the ascii number referenced. $ascii += ,[char][byte]$a } # Generate a random character form the $ascii character set. # Repeat 30 times, or create 30 random characters. 1..30 | ForEach { $password += $ascii | get-random } # Return the password return $password } # Create four 30 character passwords create-password create-password create-password create-password The output of this command would look like the following: This function will create a string with 30 random characters for use with random password creation. You first start by declaring the create-password function. You then declare the $password variable for use within the function by setting it equal to "". The next step is creating a For command to loop through a set of numbers. These numbers represent ASCII character numbers that you can select from for the password. You then create the For command by writing For ($a=33; $a -le 126;$a++). This means starting at the number 33, increase the value by one ($a++), and continue until the number is less than or equal to 126. You then declare the $ascii variable and construct the variable using the += assignment operator. As the For loop goes through its iterations, it adds a character to the array values. The script then leverages the [char] or character value of the [byte] number contained in $a. After this section, the $ascii array will contain an array of all the ASCII characters with the byte values between 33 and 126. You then continue to the random character generation. You declare the 1..30 command, which means for numbers 1 to 30, repeat the following command. You pipe this to ForEach {, which will designate for each of the 30 iterations. You then call the $ascii array and pipe it to | get-random cmdlet. The get-random cmdlet will randomly select one of the characters in the $ascii array. This value is then joined to the existing values in the $password string using the assignment operator +=. After the 30 iterations, there will be 30 random values in the $password variable. Lastly, you leverage return $password, to return this value to the script. After declaring the function, you call the function four times using create-password. This creates four random passwords for use. To create strings that are less than 30 random characters in length, you can modify the 1..30 to be any value that you want. If you want 15 random character Salt and Initialization Vector, you would use 1..15 instead. Encrypting and decrypting strings To start using RijndaelManaged encryption, you need to import the .NET System.Security Assembly into your script. Much like importing a module to provide additional cmdlets, using .NET assemblies provide an extension to a variety of classes you wouldn't normally have access to in PowerShell. Importing the assembly isn't persistent. This means you will need to import the assembly each time you want to use it in a PowerShell session, or each time you want to run the script. To load the .NET assembly, you can use the Add-Type cmdlet with the -AssemblyName parameter with the System.Security argument. Since the cmdlet doesn't actually output anything to the screen, you may choose to print to the screen successful importing of the assembly. To import the System.Security Assembly with display information, you can do the following: Write-host "Loading the .NET System.Security Assembly For Encryption" Add-Type -AssemblyNameSystem.Security -ErrorActionSilentlyContinue -ErrorVariable err if ($err) { Write-host "Error Importing the .NET System.Security Assembly." PAUSE EXIT } # if err is not set, it was successful. if (!$err) { Write-host "Succesfully loaded the .NET System.Security Assembly For Encryption" } The output from this command looks like the following: In this example, you successfully import the.NET System.SecurityAssembly for use with PowerShell. You first start by writing "Loading the .NET System.Security Assembly for Encryption" to the screen using the Write-host command. You then leverage the Add-Type cmdlet with the -AssemblyName parameter with the System.Security argument, the -ErrorAction parameter with the SilentlyContinue argument, and the -ErrorVariable parameter with the err argument. You then create an if statement to see if $err contains data. If it does, it will use Write-host cmdlet to print"Error Importing the .NET System.Security Assembly." to the screen. It will PAUSE the script so the error can be read. Finally, it will exit the script. If $err is $null, designated by if (!$err) {, it will use the Write-host cmdlet to print "Successfully loaded the .NET System.Security Assembly for Encryption" to the screen. At this point, the script or PowerShell window is ready to leverageSystem.Security Assembly. After you loadSystem.Security Assembly, you can start creating the encryption function. The RijndaelManaged encryption requires a four-step process to encrypt the strings which is represented in the preceeding diagram. The RijndaelManaged encryption process is as follows: The process starts by creating the encryptor. The encryptor is derived from the encryption key (password and salt) and initialization vector. After you define the encryptor, you will need to create a new memory stream using the IO.MemoryStream object. A memory stream is what stores values in memory for use by the encryption assembly. Once the memory stream is open, you define a System.Security.Cryptography.CryptoStream object. The CryptoStream is the mechanism that uses the memory stream and the encryptor to transform the unencrypted data to encrypted data. In order to leverage the CryptoStream, you need to write data to the CryptoStream. The final step is to use the IO.StreamWriter object to write the unencrypted value into the CryptoStream. The output from this transformation is placed into MemoryStream. To access the encrypted value, you read the data in the memory stream. To learn more about the System.Security.Cryptography.RijndaelManaged class, you can view the following MSDN article: https://msdn.microsoft.com/en-us/library/system.security.cryptography.rijndaelmanaged(v=vs.110).aspx. To create a script that encrypts strings using the RijndaelManaged encryption, you would perform the following: Add-Type -AssemblyNameSystem.Security function Encrypt-String { param($String, $Pass, $salt="CreateAUniqueSalt", $init="CreateAUniqueInit") try{ $r = new-Object System.Security.Cryptography.RijndaelManaged $pass = [Text.Encoding]::UTF8.GetBytes($pass) $salt = [Text.Encoding]::UTF8.GetBytes($salt) $init = [Text.Encoding]::UTF8.GetBytes($init) $r.Key = (new-Object Security.Cryptography.PasswordDeriveBytes $pass, $salt, "SHA1", 50000).GetBytes(32) $r.IV = (new-Object Security.Cryptography.SHA1Managed).ComputeHash($init)[0..15] $c = $r.CreateEncryptor() $ms = new-Object IO.MemoryStream $cs = new-Object Security.Cryptography.CryptoStream $ms,$c,"Write" $sw = new-Object IO.StreamWriter $cs $sw.Write($String) $sw.Close() $cs.Close() $ms.Close() $r.Clear() [byte[]]$result = $ms.ToArray() } catch { $err = "Error Occurred Encrypting String: $_" } if($err) { # Report Back Error return $err } else { return [Convert]::ToBase64String($result) } } Encrypt-String "Encrypt This String""A_Complex_Password_With_A_Lot_Of_Characters" The output of this script would look like the following: This function displays how to encrypt a string leveraging the RijndaelManaged encryption algorithm. You first start by importing the System.Security assembly by leveraging Add-Type cmdlet, using the -AssemblyName parameter with the System.Security argument. You then declare the function of Encrypt-String. You include a parameter block to accept and set values into the function. The first value is $string, which is the unencrypted text. The second value is $pass, which is used for the encryption key. The third is a predefined $salt variable set to "CreateAUniqueSalt". You then define the $init variable, which is set to "CreateAUniqueInit". After the parameter block, you declare try { to handle any errors in the .NET assembly. The first step is to declare the encryption class using new-Object cmdlet with the System.Security.Cryptography.RijndaelManaged argument. You place this object inside the $r variable. You then convert the $pass, $salt, and $init values to the character encoding standard of UTF8 and store the character byte values in a variable. This is done specifying [Text.Encoding]::UTF8.GetBytes($pass) for the $pass variable, [Text.Encoding]::UTF8.GetBytes($salt) for the $salt variable, and [Text.Encoding]::UTF8.GetBytes($init) for the $init variable. After setting the proper character encoding, you proceed to create the encryption key for the RijndalManaged encryption algorithm. This is done by setting the RijndaelManaged $r.Key attribute to the object created by (new-Object Security.Cryptography.PasswordDeriveBytes $pass, $salt, "SHA1", 50000).GetBytes(32). This object leverages the Security.Cryptography.PasswordDeriveBytes class and creates a key using the $pass variable, $salt variable, "SHA1" hash name, and iterating the derivative 50000 times. Each iteration of this class generates a different key value, making it more complex to guess the key. You then leverage the .Get-Bytes(32) method to return the 32-byte value of the key. The RijndaelManaged 256-bit encryption is a derivative of the 32 bytes in the key. 32 bytes times 8 bits per byte is 256bits. To create the initialization vector for the algorithm, you set the RijndaelManaged$r.IV attribute to the object created by (new-Object Security.Cryptography.SHA1Managed).ComputeHash($init)[0..15]. This section of the code leverages Security.Cryptography.SHA1Managed and computes the hash based on the $init value. When you invoke the [0..15] range operator, it will obtain the first 16 bytes of the hash and place it into $r.IVattribute. The RijndaelManaged default block size for the initialization vector is 128bits. 16 bytes times 8 bits per byte is 128bits. After setting up the required attributes, you are now ready to start encrypting data. You first start by leveraging the $r RijndaelManaged object with the $r.Key and $r.IV attributes defined. You use the $r.CreateEncryptor() method to generate the encryptor. Once you've generated the encryptor, you have to create a memory stream to do the encryption in memory. This is done by declaring new-Objectcmdlet, set to the IO.MemoryStream class, and placing the memory stream object in the $ms variable. Next, you create CryptoStream. The CryptoStream is used to transform the unencrypted data into the encrypted data. You first declare the new-Object cmdlet with the Security.Cryptopgraphy.CryptoStream argument. You also define the memory stream of $ms, the encryptor of $c, and the operator of "Write" to tell the class to write unencrypted data to the encryption stream in memory. After creating CryptoStream, you are ready to write the unencrypted data into the CryptoStream. This is done using the IO.StreamWriter class. You declare a new-Object cmdlet with the IO.StreamWriter argument, and define CryptoStream of $cs for writing. Last, you take the unencrypted string stored in the $string variable, and pass it into the StreamWriter$sw with $sw.Write($String). The encrypted value is now stored in the memory stream. To stop the writing of data to the CryptoStream and MemoryStream, you close the StreamWriter with $sw.Close(), close the CryptoStream with $cs.Close() and the memory stream with $ms.Close(). For security purposes, you also clear out the encryptor data by declaring $r.Clear(). After the encryption process is done, you will need to export the memory stream to a byte array. This is done calling the $ms.ToArray() method and setting it to the$result variable with the [byte[]] data type. The contents are stored in a byte array in $result. This section of the code is where you declare your catch { statement. If there were any errors in the encryption process, the script will execute this section. You declare the variable of $err with the"Error Occurred Encrypting String: $_" argument. The $_ will be the pipeline error that occurred during the try {} section. You then create an if statement to determine whether there is data in the $err variable. If there is data in $err, it returns the error string to the script. If there were no errors, the script will enter the else { section of the script. It will convert the $result byte array to Base64String by leveraging [Convert]::ToBase64String($result). This converts the byte array to string for use in your scripts. After defining the encryption function, you call the function for use. You first start by calling Encrypt-String followed by "Encrypt This String". You also declare the second argument as the password for the encryptor, which is "A_Complex_Password_With_A_Lot_Of_Characters". After execution, this example receives the encrypted value of hK7GHaDD1FxknHu03TYAPxbFAAZeJ6KTSHlnSCPpJ7c= generated from the function. Your results will vary depending on your salt, init, and password you use for the encryption algorithm. Decrypting strings The decryption of strings is very similar to the process you performed of encrypting strings. Instead of writing data to the memory stream, the function reads the data in the memory stream. Also, instead of using the .CreateEncryptor() method, the decryption process leverages the .CreateDecryptor() method. To create a script that decrypts encrypted strings using the RijndaelManaged encryption, you would perform the following: Add-Type -AssemblyNameSystem.Security function Decrypt-String { param($Encrypted, $pass, $salt="CreateAUniqueSalt", $init="CreateAUniqueInit") if($Encrypted -is [string]){ $Encrypted = [Convert]::FromBase64String($Encrypted) } $r = new-Object System.Security.Cryptography.RijndaelManaged $pass = [System.Text.Encoding]::UTF8.GetBytes($pass) $salt = [System.Text.Encoding]::UTF8.GetBytes($salt) $init = [Text.Encoding]::UTF8.GetBytes($init) $r.Key = (new-Object Security.Cryptography.PasswordDeriveBytes $pass, $salt, "SHA1", 50000).GetBytes(32) $r.IV = (new-Object Security.Cryptography.SHA1Managed).ComputeHash($init)[0..15] $d = $r.CreateDecryptor() $ms = new-Object IO.MemoryStream@(,$Encrypted) $cs = new-Object Security.Cryptography.CryptoStream $ms,$d,"Read" $sr = new-Object IO.StreamReader $cs try { $result = $sr.ReadToEnd() $sr.Close() $cs.Close() $ms.Close() $r.Clear() Return $result } Catch { Write-host "Error Occurred Decrypting String: Wrong String Used In Script." } } Decrypt-String "hK7GHaDD1FxknHu03TYAPxbFAAZeJ6KTSHlnSCPpJ7c=""A_Complex_Password_With_A_Lot_Of_Characters". The output of this script would look like the following: This function displays how to decrypt a string leveraging the RijndaelManaged encryption algorithm. You first start by importing the System.Security assembly by leveraging the Add-Type cmdlet, using the -AssemblyName parameter with the System.Security argument. You then declare the Decrypt-String function. You include a parameter block to accept and set values for the function. The first value is $Encrypted, which is the encrypted text. The second value is the $pass which is used for the encryption key. The third is a predefined $salt variable set to "CreateAUniqueSalt". You then define the $init variable, which is set to "CreateAUniqueInit". After the parameter block, you check to see if the encrypted value is formatted as a string by using if ($Encrypted -is [string]) {. If this evaluates to True, you convert the string to bytes using [Convert]::FromBase64String($Encrypted) and placing the encoded value in the $Encrypted variable. Next, you declare the decryption class using new-Object cmdlet with the System.Security.Cryptography.RijndaelManaged argument. You place this object inside of the $r variable. You then convert the $pass, $salt, and $init values to the character encoding standard of UTF8 and store the character byte values in a variable. This is done specifying [Text.Encoding]::UTF8.GetBytes($pass) for the $pass variable, [Text.Encoding]::UTF8.GetBytes($salt) for the $salt variable, and [Text.Encoding]::UTF8.GetBytes($init) for the $init variable. After setting the proper character encoding, you proceed to create the encryption key for the RijndaelManaged encryption algorithm. This is done by setting the RijndaelManaged $r.Key attribute to the object created by (new-Object Security.Cryptography.PasswordDeriveBytes $pass, $salt, "SHA1", 50000).GetBytes(32). This object leverages the Security.Cryptography.PasswordDeriveBytes class and creates a key using the $pass variable, $salt variable, "SHA1" hash name, and iterating the derivative 50000 times. Each iteration of this class generates a different key value, making it more complex to guess the key. You then leverage the .get-bytes(32) method to return the 32-byte value of the key. To create the initialization vector for the algorithm, you set the RijndaelManaged $r.IV attribute to the object created by (new-Object Security.Cryptography.SHA1Managed).ComputeHash($init)[0..15]. This section of the code leverages the Security.Cryptography.SHA1Managed class and computes the hash based on the $init value. When you invoke the [0..15] range operator, the first 16 bytes of the hash are obtained and placed into $r.IV attribute. After setting up the required attributes, you are now ready to start decrypting data. You first start by leveraging the $r RijndaelManaged object with the $r.key and $r.IV attributes defined. You use the $r.CreateDecryptor() method to generate the decryptor. Once you've generated the decryptor, you have to create a memory stream to do the decryption in memory. This is done by declaring new-Object cmdlet with the IO.MemoryStream class argument. You then reference the $encrypted values to place in the memory stream object with @(,$Encrypted), and store the populated memory stream in the $ms variable. Next, you create CryptoStream, which CryptoStream is used to transform the encrypted data into the decrypted data. You first declare new-Object cmdlet with the Security.Cryptopgraphy.CryptoStream class argument. You also define the memory stream of $ms, the decryptor of $d, and the operator of "Read" to tell the class to read the encrypted data from the encryption stream in memory. After creating CryptoStream, you are ready to read the decrypted datafrom CryptoStream. This is done using the IO.StreamReader class. You declare new-Object with the IO.StreamReader class argument, and define CryptoStream of $cs to read from. At this point, you use try { to catch any error messages that are generated from reading the data in the StreamReader. You call $sr.ReadToEnd(), which calls the StreamReader and reads the complete decrypted value and places the datain the $result variable. To stop the reading of data to CryptoStream and MemoryStream, you close StreamWriter with $sw.Close(), close the CryptoStream with $cs.Close() and the memory stream with $ms.Close(). For security purposes, you also clear out the decryptor data by declaring $r.Clear(). If the decryption is successful, you return the value of $result to the script. After defining the decryption function, you call the function for use. You first start by calling Decrypt-String followed by "hK7GHaDD1FxknHu03TYAPxbFAAZeJ6KTSHlnSCPpJ7c=". You also declare the second argument as the password for the decryptor, which is "A_Complex_Password_With_A_Lot_Of_Characters". After execution, you will receive the decrypted value of "Encrypt This String" generated from the function. Summary In this article, we learned about RijndaelManaged 256-bit encryption. We first started with the basics of the encryption process. Then, we proceeded into learning how to create randomized salt, init, and passwords in scripts. We ended the article with learning how to encrypt and decrypt strings. Resources for Article: Further resources on this subject: WLAN Encryption Flaws [article] Introducing PowerShell Remoting [article] SQL Server with PowerShell [article]
Read more
  • 0
  • 0
  • 13052
article-image-visual-mysql-database-design-mysql-workbench
Packt
21 Oct 2009
3 min read
Save for later

Visual MySQL Database Design in MySQL Workbench

Packt
21 Oct 2009
3 min read
MySQL Workbench is a visual database design tool recently released by MySQL AB. The tool is specifically for designing MySQL database. What you build in MySQL Workbench is called physical data model. A physical data model is a data model for a specific RDBMS product; the model in this article will have some MySQL unique specifications. We can generate (forward-engineer) the database objects from its physical model, which in addition to tables and their columns, can also include other objects such as view. MySQL Workbench has many functions and features; this article by Djoni Darmawikarta shows some of them by way of an example. We’ll build a physical data model for an order system where an order can be a sale order or a purchase order; and then, forward-engineer our model into an MySQL database. The physical model of our example in EER diagram will look like in the following MySQL Workbench screenshot. Creating ORDER Schema Let’s first create a schema where we want to store our order physical model. Click the + button (circled in red). Change the new schema’s default name to ORDER. Notice that when you’re typing in the schema name, its tab name on the Physical Schemata also changes accordingly—a nice feature. The order schema is added to the Catalog (I circled the order schema and its objects in red). Close the schema window. Confirm to rename the schema when prompted. Creating Order Tables We’ll now create three tables that model the order: ORDER table and its two subtype tables: SALES_ORDER and PURCHASE_ORDER, in the ORDER schema. First of all, make sure you select the ORDER schema tab, so that the tables we’ll create will be in this schema. We’ll create our tables as EER diagram (EER = Enhanced Entity Relationship). So, double-click the Add Diagram button. Select (click) the Table icon, and then move your mouse onto the EER Diagram canvas and click on the location you want to place the first table. Repeat for the other two tables. You can move around the tables by dragging and dropping. Next, we’ll work on table1, which we’ll do so using the Workbench’s table editor. We start the table editor by right-clicking the table1 and selecting Edit Table. Next, we’ll work on table1, which we’ll do so using the Workbench’s table editor. We start the table editor by right-clicking the table1 and selecting Edit Table. Rename the table by typing in ORDER over table1. We’ll next add its columns, so select the Columns tab. Replace idORDER column name with ORDER_NO. Select INT as the data type from the drop-down list. We’d like this ORDER_NO column to be valued incrementally by MySQL database, so we specify it as AI column (Auto Increment). AI is a specific feature of MySQL database. You can also specify other physical attributes of the table, such as its Collation; as well as other advanced options, such as its trigger and partioning (the Trigger and Partioning tabs). Notice that on the diagram our table1 has changed to ORDER, and it has its first column, ORDER_NO. In the Catalog you can also see the three tables. The black dots on the right of the tables indicate that they’ve been included in an diagram.  
Read more
  • 0
  • 0
  • 13050

article-image-internet-things-technologies
Packt
30 Jan 2017
9 min read
Save for later

Internet of Things Technologies

Packt
30 Jan 2017
9 min read
In this article by Rubén Oliva Ramos author of the book Internet of Things Programming with JavaScript we will understand different platform for the use of Internet of Things, and how to install Raspbian on micro SD card. (For more resources related to this topic, see here.) Technology has played a huge role in increasing efficiency in the work place, improving living conditions at home, monitoring health and environmental conditions and saving energy and natural resources. This has been made possible through continuous development of sensing and actuation devices. Due to the huge data handling requirements of these devices, the need for a more sophisticated, yet versatile data handling and storage medium, such as the Internet, has arisen. That’s why many developers are adopting the different Internet of Things platforms for prototyping that are available. There are several different prototyping platforms that one can use to add internet connectivity to his or her prototypes. It is important for a developer to understand the capabilities and limitations of the different platforms if he or she wants to make the best choice. So, here is a quick look at each platform. Arduino Arduino is a popular open-source hardware prototyping platform that is used in many devices. It comprises several boards including the UNO, Mega, YUN and DUE among several others. However, out of all the boards, only the Arduino Yun has built-in capability to connect to Wi-Fi and LAN networks. The rest of the boards rely on the shields, such as the Wi-Fi shield and Ethernet shield, to connect to the Internet. The official Arduino Ethernet shield allows you to connect to a network using an RJ45 cable. It has a Wiznet W5100 Ethernet chip that provides a network IP stack that is compatible with UDP and TCP. Using the Ethernet library provided by Arduino you will be able to connect to your network in a few simple steps. If you are thinking of connecting to the Internet wirelessly, you could consider getting the Arduino Wi-Fi shield. It is based on the HDG204 wireless LAN 802.11b/g System in-Package and has an AT32UC3 that provides a network IP stack that is compatible with UDP and TCP. Using this shield and the Wi-Fi library provided by Arduino, you can quickly connect to your prototypes to the web. The Arduino Yun is a hybrid board that has both Ethernet and Wi-Fi connectivity. It is based on the ATmega32u4 and the Atheros AR9331 which supports OpenWrt-Yun. The Atheros processor handles the WiFi and Ethernet interfaces while the ATmega32u4 handles the USB communication. So, if you are looking for a more versatile Arduino board for an Internet of Things project, this is it. There are several advantages to using the Arduino platform for internet of things projects. For starters the Arduino platform is easy to use and has a huge community that you can rely on for technical support. It also is easy to create a prototype using Arduino, since you can design a PCB based on the boards. Moreover, apart from Arduino official shields, the Arduino platform can also work with third party Wi-Fi and Ethernet shields such as the WiFi shield. Therefore, your options are limitless. On the down side, all Arduino boards, apart from Yun, need an external module so as to connect to the internet. So, you have to invest more. In addition to this, there are very many available shields that are compatible with the Arduino. This makes it difficult for you to choose. Also, you still need to choose the right Internet of Things platform for your project, such as Xively or EasyIoT. Raspberry Pi The Raspberry Pi is an open source prototyping platform that features credit-card sized computer boards. The boards have USB ports for a keyboard and a mouse, a HDMI port for display, an Ethernet port for network connection and an SD card to store the operating system. There are several versions of the Raspberry Pi available in the market. They include the Raspberry Pi 1 A, B, A+ and B+ and the Raspberry Pi 2 B. When using a Raspberry Pi board you can connect to the Internet either wirelessly or via an Ethernet cable. Raspberry Pi boards, except version A and A+, have an Ethernet port where you can connect an Ethernet cable. Normally, the boards gain internet connection immediately after you connect the Ethernet cable. However, your router must be configured for Dynamic Host Configuration Protocol (DHCP) for this to happen. Otherwise, you will have to set the IP address of the Raspberry Pi manually and the restart it. To connect your Raspberry Pi to the internet wirelessly, you have to use a WiFi adapter, preferably one that supports the RTL8192cu chipset. This is because Raspbian and Ocidentalis distributions have built-in support for that chip. However, there is no need to be choosy. Almost all Wi-Fi adapters in the market, including the very low cost budget adapters will work without any trouble. Using Raspberry Pi boards for IoT projects is advantageous because you don’t need extra shields or hardware to connect to the internet. Moreover, connecting to your wireless or LAN network happens automatically, so long as the router that has DHCP configured (most routers do). Also, you don’t have to worry if you are a newbie, since there is a huge Raspberry Pi community. You can get help quickly. The disadvantage of using the Raspberry Pi platform to make IoT devices is that it is not easy to use. It would take tremendous time for newbies to learn how to set up everything and code apps. Another demerit is that the Raspberry Pi boards cannot be easily integrated into a product. There are also numerous operating systems that the Pi boards can run on and it is not easy to decide on the best operating system for the device you are creating. Which Platform to Choose? The different features that come with each platform make it ideal for certain applications, but not all. Therefore, if at all you have not yet made up your mind on the platform to use, try selecting them based on usage. The Raspberry Pi is ideal for IoT applications that are server based. This is because it has high storage space and RAM and a powerful processor. Moreover, it supports many programming languages that can create Server-Side apps, such as Node.js. You can also use the Raspberry Pi in instances where you want to access web pages and view data posted on online servers, since you can connect a display to it and view the pages on the web browser. The Raspberry Pi can connect to both LAN and Wi-Fi networks. However, it is not advisable to use the raspberry Pi for projects where you would want to integrate it into a finished product or create a custom made PCB. On the other hand, Arduino comes in handy as a client. It can log data on an online server and retrieve data from the server. It is ideal for logging sensor data and controlling actuators via commands posted on the server by another client. However, there are instances where you can use Arduino boards for server functions such as hosting a simple web page that you can use to control your Arduino from the local network it is connected to. The Arduino platform can connect to both LAN and Wi-Fi networks. The ESP8266 has a very simple hardware architecture and is best used in client applications such as data logging and control of actuators from online server applications. You can use it as a webserver as well, but for applications or web pages that you would want to access from the local network that the module is connected to. The Spark Core platform is ideal for both server and client functions. It can be used to log sensor data onto the Spark.io cloud or receive commands from the cloud. Moreover, you don’t have to worry about getting online server space, since the Spark cloud is available for free. You can also create Graphical User Interfaces based on Node.js to visualize incoming data from sensors connected to the Spark Core and send commands to the Spark Core for activation of actuators. Setting up Raspberry Pi Zero Raspberry Pi is low-cost board dedicated for project purpose this will use a Raspberry Pi Zero board. See the following link https://www.adafruit.com/products/2816 for the Raspberry Pi Zero board and Kit. In order to make the Raspberry Pi work, we need an operating system that acts as a bridge between the hardware and the user, we will be using the Raspbian Jessy that you can download from https://www.raspberrypi.org/downloads/, in this link you will find all the information that you need to download all the software that it’s necessary to use with your Raspberry Pi to deploy Raspbian, we need a micro SD card of at least 4 GB. The kit that I used for testing the Raspberry Pi Zero includes all the necessary items for installing every thing and ready the board. Preparing the SD card The Raspberry Pi Zero only boots from a SD card, and cannot boot from an external drive or USB stick. For now it is recommended to use a Micro SD 4 GB of space. Installing the Raspian Operating System When we have the image file for preparing the SD card that has previously got from the page. Also we insert the micro SD into the adapter, download Win32DiskImager from https://sourceforge.net/projects/win32diskimager/ In the following screenshot you will see the files after downloading the folder It appears the window, open the file image and select the path you have the micro SD card and click on the Write button. After a few seconds we have here the download image and converted files into the micro SD In the next screenshot you can see the progress of the installation Summary In this article we discussed about the different platform for the use of Internet of Things like Ardunio and Raspberry Pi. We also how to setup and prepare Raspberry Pi for further use. Resources for Article: Further resources on this subject: Classes and Instances of Ember Object Model [article] Using JavaScript with HTML [article] Introducing the Ember.JS framework [article]
Read more
  • 0
  • 0
  • 13047
Modal Close icon
Modal Close icon