Search icon CANCEL
Subscription
0
Cart icon
Your Cart (0 item)
Close icon
You have no products in your basket yet
Save more on your purchases! discount-offer-chevron-icon
Savings automatically calculated. No voucher code required.
Arrow left icon
Explore Products
Best Sellers
New Releases
Books
Videos
Audiobooks
Learning Hub
Newsletter Hub
Free Learning
Arrow right icon
timer SALE ENDS IN
0 Days
:
00 Hours
:
00 Minutes
:
00 Seconds

How-To Tutorials

7008 Articles
article-image-stephen-hawking-artificial-intelligence-quotes
Richard Gall
15 Mar 2018
3 min read
Save for later

5 polarizing Quotes from Professor Stephen Hawking on artificial intelligence

Richard Gall
15 Mar 2018
3 min read
Professor Stephen Hawking died today (March 14, 2018) aged 76 at his home in Cambridge, UK. Best known for his theory of cosmology that unified quantum mechanics with Einstein’s General Theory of Relativity, and for his book a Brief History of Time that brought his concepts to a wider general audience, Professor Hawking is quite possibly one of the most important and well-known voices in the scientific world. Among many things, Professor Hawking had a lot to say about artificial intelligence - its dangers, its opportunities and what we should be thinking about, not just as scientists and technologists, but as humans. Over the years, Hawking has remained cautious and consistent in his views on the topic constantly urging AI researchers and machine learning developers to consider the wider implications of their work on society and the human race itself.  The machine learning community is quite divided on all the issues Hawking has raised and will probably continue to be so as the field grows faster than it can be fathomed. Here are 5 widely debated things Stephen Hawking said about AI arranged in chronological order - and if you’re going to listen to anyone, you’ve surely got to listen to him?   On artificial intelligence ending the human race The development of full artificial intelligence could spell the end of the human race….It would take off on its own, and re-design itself at an ever-increasing rate. Humans, who are limited by slow biological evolution, couldn't compete and would be superseded. From an interview with the BBC, December 2014 On the future of AI research The establishment of shared theoretical frameworks, combined with the availability of data and processing power, has yielded remarkable successes in various component tasks such as speech recognition, image classification, autonomous vehicles, machine translation, legged locomotion, and question-answering systems. As capabilities in these areas and others cross the threshold from laboratory research to economically valuable technologies, a virtuous cycle takes hold whereby even small improvements in performance are worth large sums of money, prompting greater investments in research. There is now a broad consensus that AI research is progressing steadily, and that its impact on society is likely to increase.... Because of the great potential of AI, it is important to research how to reap its benefits while avoiding potential pitfalls. From Research Priorities for Robust and Beneficial Artificial Intelligence, an open letter co-signed by Hawking, January 2015 On AI emulating human intelligence I believe there is no deep difference between what can be achieved by a biological brain and what can be achieved by a computer. It, therefore, follows that computers can, in theory, emulate human intelligence — and exceed it From a speech given by Hawking at the opening of the Leverhulme Centre of the Future of Intelligence, Cambridge, U.K., October 2016 On making artificial intelligence benefit humanity Perhaps we should all stop for a moment and focus not only on making our AI better and more successful but also on the benefit of humanity. Taken from a speech given by Hawking at Web Summit in Lisbon, November 2017 On AI replacing humans The genie is out of the bottle. We need to move forward on artificial intelligence development but we also need to be mindful of its very real dangers. I fear that AI may replace humans altogether. If people design computer viruses, someone will design AI that replicates itself. This will be a new form of life that will outperform humans. From an interview with Wired, November 2017
Read more
  • 0
  • 2
  • 72294

article-image-digital-forensics-using-autopsy
Savia Lobo
24 May 2018
10 min read
Save for later

Getting started with Digital forensics using Autopsy

Savia Lobo
24 May 2018
10 min read
Digital forensics involves the preservation, acquisition, documentation, analysis, and interpretation of evidence from various storage media types. It is not only limited to laptops, desktops, tablets, and mobile devices but also extends to data in transit which is transmitted across public or private networks. In this tutorial, we will cover how one can carry out digital forensics with Autopsy. Autopsy is a digital forensics platform and graphical interface to the sleuth kit and other digital forensics tools. This article is an excerpt taken from the book, 'Digital Forensics with Kali Linux', written by Shiva V.N. Parasram. Let's proceed with the analysis using the Autopsy browser by first getting acquainted with the different ways to start Autopsy. Starting Autopsy Autopsy can be started in two ways. The first uses the Applications menu by clicking on Applications | 11 - Forensics | autopsy: Alternatively, we can click on the Show applications icon (last item in the side menu) and type autopsy into the search bar at the top-middle of the screen and then click on the autopsy icon: Once the autopsy icon is clicked, a new terminal is opened showing the program information along with connection details for opening The Autopsy Forensic Browser. In the following screenshot, we can see that the version number is listed as 2.24 with the path to the Evidence Locker folder as /var/lib/autopsy: To open the Autopsy browser, position the mouse over the link in the terminal, then right-click and choose Open Link, as seen in the following screenshot: Creating a new case To create a new case, follow the given steps: When the Autopsy Forensic Browser opens, investigators are presented with three options. Click on NEW CASE: Enter details for the Case Name, Description, and Investigator Names. For the Case Name, I've entered SP-8-dftt, as it closely matches the image name (8-jpeg-search.dd), which we will be using for this investigation. Once all information is entered, click NEW CASE: Several investigator name fields are available, as there may be instances where several investigators may be working together. The locations of the Case directory and Configuration file are displayed and shown as created.  It's important to take note of the case directory location, as seen in the screenshot: Case directory (/var/lib/autopsy/SP-8-dftt/) created. Click ADD HOST to continue: Enter the details for the Host Name (name of the computer being investigated) and the Description of the host. Optional settings: Time zone: Defaults to local settings, if not specified Timeskew Adjustment: Adds a value in seconds to compensate for time differences Path of Alert Hash Database: Specifies the path of a created database of known bad hashes Path of Ignore Hash Database: Specifies the path of a created database of known good hashes similar to the NIST NSRL: Click on the ADD HOST button to continue. Once the host is added and directories are created, we add the forensic image we want to analyze by clicking the ADD IMAGE button: Click on the ADD IMAGE FILE button to add the image file: To import the image for analysis, the full path must be specified. On my machine, I've saved the image file (8-jpeg-search.dd) to the Desktop folder. As such, the location of the file would be /root/Desktop/ 8-jpeg-search.dd. For the Import Method, we choose Symlink. This way the image file can be imported from its current location (Desktop) to the Evidence Locker without the risks associated with moving or copying the image file. If you are presented with the following error message, ensure that the specified image location is correct and that the forward slash (/) is used: Upon clicking Next, the Image File Details are displayed. To verify the integrity of the file, select the radio button for Calculate the hash value for this image, and select the checkbox next to Verify hash after importing? The File System Details section also shows that the image is of a ntfs partition. Click on the ADD button to continue: After clicking the ADD button in the previous screenshot, Autopsy calculates the MD5 hash and links the image into the evidence locker. Press OK to continue: At this point, we're just about ready to analyze the image file. If there are multiple cases listed in the gallery area from any previous investigations you may have worked on, be sure to choose the 8-jpeg-search.dd file and case: Before proceeding, we can click on the IMAGE DETAILS option. This screen gives detail such as the image name, volume ID, file format, file system, and also allows for the extraction of ASCII, Unicode, and unallocated data to enhance and provide faster keyword searches. Click on the back button in the browser to return to the previous menu and continue with the analysis: Before clicking on the ANALYZE button to start our investigation and analysis, we can also verify the integrity of the image by creating an MD5 hash, by clicking on the IMAGE INTEGRITY button: Several other options exist such as FILE ACTIVITY TIMELINES, HASH DATABASES, and so on. We can return to these at any point in the investigation. After clicking on the IMAGE INTEGRITY button, the image name and hash are displayed. Click on the VALIDATE button to validate the MD5 hash: The validation results are displayed in the lower-left corner of the Autopsy browser window: We can see that our validation was successful, with matching MD5 hashes displayed in the results. Click on the CLOSE button to continue. To begin our analysis, we click on the ANALYZE button: Analysis using Autopsy Now that we've created our case, added host information with appropriate directories, and added our acquired image, we get to the analysis stage. After clicking on the ANALYZE button (see the previous screenshot), we're presented with several options in the form of tabs, with which to begin our investigation: Let's look at the details of the image by clicking on the IMAGE DETAILS tab. In the following snippet, we can see the Volume Serial Number and the operating system (Version) listed as Windows XP: Next, we click on the FILE ANALYSIS tab. This mode opens into File Browsing Mode, which allows the examination of directories and files within the image. Directories within the image are listed by default in the main view area: In File Browsing Mode, directories are listed with the Current Directory specified as C:/. For each directory and file, there are fields showing when the item was WRITTEN, ACCESSED, CHANGED, and CREATED, along with its size and META data: WRITTEN: The date and time the file was last written to ACCESSED: The date and time the file was last accessed (only the date is accurate) CHANGED: The date and time the descriptive data of the file was modified CREATED: The data and time the file was created META: Metadata describing the file and information about the file: For integrity purposes, MD5 hashes of all files can be made by clicking on the GENERATE MD5 LIST OF FILES button. Investigators can also make notes about files, times, anomalies, and so on, by clicking on the ADD NOTE button: The left pane contains four main features that we will be using: Directory Seek: Allows for the searching of directories File Name Search: Allows for the searching of files by Perl expressions or filenames ALL DELETED FILES: Searches the image for deleted files EXPAND DIRECTORIES: Expands all directories for easier viewing of contents By clicking on EXPAND DIRECTORIES, all contents are easily viewable and accessible within the left pane and main window. The + next to a directory indicates that it can be further expanded to view subdirectories (++) and their contents: To view deleted files, we click on the ALL DELETED FILES button in the left pane. Deleted files are marked in red and also adhere to the same format of WRITTEN, ACCESSED, CHANGED, and CREATED times. From the following screenshot, we can see that the image contains two deleted files: We can also view more information about this file by clicking on its META entry. By viewing the metadata entries of a file (last column to the right), we can also view the hexadecimal entries for the file, which may give the true file extensions, even if the extension was changed. In the preceding screenshot, the second deleted file (file7.hmm) has a peculiar file extension of .hmm. Click on the META entry (31-128-3) to view the metadata: Under the Attributes section, click on the first cluster labelled 1066 to view header information of the file: We can see that the first entry is .JFIF, which is an abbreviation for JPEG File Interchange Format. This means that the file7.hmm file is an image file but had its extension changed to .hmm. Sorting files Inspecting the metadata of each file may not be practical with large evidence files. For such an instance, the FILE TYPE feature can be used. This feature allows for the examination of existing (allocated), deleted (unallocated), and hidden files. Click on the FILE TYPE tab to continue: Click Sort files into categories by type (leave the default-checked options as they are) and then click OK to begin the sorting process: Once sorting is complete, a results summary is displayed. In the following snippet, we can see that there are five Extension Mismatches: To view the sorted files, we must manually browse to the location of the output folder, as Autopsy 2.4 does not support viewing of sorted files. To reveal this location, click on View Sorted Files in the left pane: The output folder locations will vary depending on the information specified by the user when first creating the case, but can usually be found at /var/lib/autopsy/<case name>/<host name>/output/sorter-vol#/index.html. Once the index.html file has been opened, click on the Extension Mismatch link: The five listed files with mismatched extensions should be further examined by viewing metadata content, with notes added by the investigator. Reopening cases in Autopsy Cases are usually ongoing and can easily be restarted by starting Autopsy and clicking on OPEN CASE: In the CASE GALLERY, be sure to choose the correct case name and, from there, continue your examination: To recap, we looked at forensics using the Autopsy Forensic Browser with The Sleuth Kit. Compared to individual tools, Autopsy has case management features and supports various types of file analysis, searching, and sorting of allocated, unallocated, and hidden files. Autopsy can also perform hashing on a file and directory levels to maintain evidence integrity. If you enjoyed reading this article, do check out, 'Digital Forensics with Kali Linux' to take your forensic abilities and investigations to a professional level, catering to all aspects of a digital forensic investigation from hashing to reporting. What is Digital Forensics? IoT Forensics: Security in an always connected world where things talk Working with Forensic Evidence Container Recipes
Read more
  • 0
  • 0
  • 72212

article-image-introducing-powershell-remoting
Packt
21 Dec 2016
9 min read
Save for later

Introducing PowerShell Remoting

Packt
21 Dec 2016
9 min read
In this article by Sherif Talaat, the author of the book PowerShell 5.0 Advanced Administration Handbook, we will see the PowerShell v2 introduced a powerful new technology, PowerShell remoting, which was refined andexpanded upon for later versions of PowerShell. PowerShell remoting is based primarily upon standardized protocols andtechniques; it is possibly one of the most important aspects of Windows PowerShell. Today, a lot of Microsoft products rely upon it almost entirely for administrative communications across the network. The most important and exciting characteristic of PowerShell is itsremote management capability. PowerShell Remoting can control the target remote computer via the network. It uses Windows Remote Management (WinRM) which is based on Microsoft’s WS-Management protocol. Using PowerShell remoting, the administrator can execute various management operations on dozens of target computers across the network. In this article, we will cover the following topics: PowerShell remoting system requirements Enabling/Disabling remoting Executing remote commands Interactive remoting Sessions Saving remote sessions to disk Understanding session configuration (For more resources related to this topic, see here.) Windows PowerShellremoting It’s very simple: Windows PowerShell remoting is developed to help you ease your administration tasks. The idea is about using the PowerShell console on your local machine to manage and control remote computers in different locations, whether these locations are on a local network, a branch, or even in the cloud. Windows PowerShell remoting relies on Windows Remote Management (WinRM) to connect those computers together even if they’re not physically connected. Sounds cool and exciting, huh?! Windows Remote Management (WinRM) is a Microsoft implementation for the WS-Management protocol. WSMan is a standard Simple Object Access Protocol (SOAP) that allows hardware and operating systems, from different vendors, to interoperate and communicate together in order to access and exchange management information across the entire infrastructure. In order to be able to execute a PowerShell script on remote computers using PowerShell remoting, the user performing this remote execution must meet one of the following conditions: Be a member of the administrators’ group on the remote machine whether as a domain administrator or a local administrator Provide admin privileged credentials at the time of execution, either while establishing the remote session or using a –ComputerName parameter Has access to the PowerShell session configuration on the remote computer Now we understand what PowerShell remoting is, let’s jump to the interesting stuff and start playing with it. Enable/Disable PowerShell Remoting Before using Windows PowerShell remoting, we need to first ensure that it’s already on the computers we want to connect to and manage. You can validate whether PowerShell remoting is enabled on a computer using the Test-WSMan cmdlet. #Verify WinRM service status Test-WSMan –ComputerName Server02 If PowerShell remoting is enabled on the remote computer, (which means that the WinRM service is running), you will get an acknowledgement message similar to the message shown in the following screenshot: However, if the WinRM is not responding, either because it’s not enabled or because the computer is unreachable, you will get an error message similar to the message shown in the following screenshot: Okay, at this stage, we know which computers have remoting enabled and which need to be configured. In order to enable PowerShell remoting on a computer, we use the Enable-PSRemoting cmdlet. The Enable-PSRemoting cmdlet will prompt you with a message to inform you about the changes to be applied on the target computer and ask for your confirmation as shown in the following screenshot: You can skip this prompt message by using the –Forceparameter: #Enable PowerShell Remoting Enable-PSRemoting –Force In client OS versions of Windows, such as Windows 7, 8/8.1, and 10, the network connection type must be set either to domain or private. If it’s set to public, you will get a message as shown in the following: This is the Enable-PSRemoting cmdlet’s default behavior to stop you from enabling PowerShell remoting on a public network which might put your computer in risk. You can skip the network profile check using the –SkipNetworkProfileCheck parameter, or simply change the network profile as shown later in this article: #Enable PowerShell Remoting on Public Network Enable-PSRemoting –Force –SkipNetworkProfileCheck  If, for any reason, you want to temporarily disable a session configuration in order to prevent users from connecting to a local computer using that session configuration, you can use theDisable-PSSessionConfiguration cmdlet along with the -Name parameter to specify which session configuration you want to disable. If we don’t specify a configuration name for the –Name parameter, the default session configurationMicrosoft.PowerShell will be disabled. Later on, if you want to re-enable the session configuration, you can use the Enable-PSSessionConfiguration cmdlet with the -Name parameter to specify which session configuration you need to enable, similarly to the Disable-PSSessionConfiguration cmdlet. Delete a session configuration When you disable a session configuration, PowerShell just denies access to this session configuration by assigning deny all to the defined security descriptors. It doesn’t remove it, which is why you can re-enable it. If you want to permanently remove a session configuration, use theUnregister-PSSessionConfiguration cmdlet. Windows PowerShell Web Access (PSWA) Windows PowerShell Web Access (PSWA) was introduced for the first time as a new feature in Windows PowerShell 3.0.Yes, it is what you are guessing it is! PowerShell Web Access is a web-based version of the PowerShell console that allows you to run and execute PowerShell cmdlets and scripts from any web browser on any desktop, notebook, smartphone, or tablet that meet the following criteria: Allows cookies from the Windows PowerShell Web Access gateway website Capable of opening and reading HTTPS pages Opens and runs websites that use JavaScript PowerShell Web Access allows you to complete your administration tasks smoothly anywhere anytime using any device running a web browser, regardless of whether it is Microsoft or non-Microsoft. Installing and configuring Windows PowerShell Web Access The following are the steps to install and configure Windows PowerShell Web Access: Step 1: Installing the Windows PowerShell Web Access Windows feature In this step we will install the Windows PowerShell Web Access Windows feature. For the purpose of this task, we will use the Install-WindowsFeature cmdlet: #Installing PSWA feature Install-WindowsFeature WindowsPowerShellWebAccess –IncludeAllSubFeature –IncludeManagementTools The following screenshot depicts the output of the cmdlet: Now, we have the PowerShell Web Access feature installed. The next step is to configure it. Step 2: Configuring Windows PowerShell Web Access gateway To configure the PSWA gateway, we use the Install-PswaWebApplication cmdlet which will create an IIS Web Application that runs PowerShell Web Access and configures the SSL certificate. If you don’t have the SSL certificate, then you can use the –UseTestCertificate flag in order to generate and use a self-signed certificate: #Configure PWSA Gateway Install-PswaWebApplication –WebSiteName “Default Web Site“ –WebApplicationName “PSWA“ –UseTestCertificate   Use the–UseTestCertificate for testing purposes in your private lab only. Never use it in a production environment. In your production environments use a certificate from a trusted certificate issued by either your corporate’s Certificate Authority (CA) or a trusted certificate publisher. To verify successful installation and configuration of the gateway. Browse the PSWA URLhttps://<server_name>/PSWA as shown in the following screnshot: The PSWA WebApplication files are located at %windir%WebPowerShellWebAccesswwwroot. Step 3: Configuring PowerShell Web Access authorization rules Now, we have PSWA up and running. However no one will be able to sign-in and use it yet until we create the appropriate authorization rule. Because PSWA could be accessed from anywhere any time –which increases the security risks – PowerShell restricts any access to your network until you create and assign the right access to the right person. The authorization rule is the access control for your PSWA that adds an additional security layer to your PSWA. It is similar to the access list on the firewall and network devices. To create a new access authorization rule, we use the Add-PswaAuthorizationRule cmdlet along with the–UserName parameter to specify the name of the user who will get the access; the–-ComputerName parameter to specify which computer the user will has access to; and the–ConfigurationName parameter to specify the session configuration available to this user: #Adding PSWA Access Authorization RUle Add-PswaAuthorizationRule –UserName PSWAAdministrator –ComputerName PSWA –ConfigurationName Microsoft.PowerShell   The PSWA Authorization Rules files are located at %windir%WebPowerShellWebAccessdataAuthorizationRules.xml There are four different access authorization rules scenarios that we can enable on PowerShell Web Access. These scenarios are: Enable single user access to a single computer: For this scenario we use the –Username parameter to specify the single user, and the–ComputerName parameter to specify the single computer Enable single user access to a group of computers: For this scenario we use the –Username parameter to specify the single user, and the–ComputeGroupNameComputeGroupName parameter to specify the name of the active directory computer group Enable a group of users access to a single computer: For this scenario we use the –UserGroupName parameter to specify the name of active directory users’ group, and the–ComputerName parameter to specify the individual computer Enable groups of users access to a group of computers: For this scenario we use the –UserGroupName parameter to specify the name of active directory users group, and the –ComputeGroupNameComputeGroupName parameter to specify the name of the active directory computer group You can use the Get-PswaAuthorizationRule cmdlet to list all the configured access authorization rules, and use the Remove-PswaAuthorizationRule cmdlet to remove them. Sign-in to PowerShell Web Access Now, let’s verify the installation and start using the PSWA by signing-in to it: Open the Internet browser; you can choose whichever browser you like bearing in mind the browser requirements mentioned earlier. Enter https://<server_name>/PSWA. Enter User Name, Password, Connection Type, and Computer Name. Summary In this article, we learned about one of the most powerful features of PowerShell which is PowerShell remoting, including how to enable, prepare, and configure your environment to use PowerShell remoting. Moreover, we demonstrated some examples of how to use different methods to utilize this remote capability. We learned how to run remote commands on remote computers by using a temporary or persistent connection. Finally, we closed the article with PowerShell Web Access, including how it works and how to configure it. Resources for Article: Further resources on this subject: Installing/upgrading PowerShell [article] DevOps Tools and Technologies [article] Bringing DevOps to Network Operations [article]
Read more
  • 0
  • 0
  • 71804

article-image-statistical-tools-in-wireshark-for-packet-analysis
Vijin Boricha
06 Aug 2018
9 min read
Save for later

Using statistical tools in Wireshark for packet analysis [Tutorial]

Vijin Boricha
06 Aug 2018
9 min read
One of Wireshark's strengths is its statistical tools. When using Wireshark, we have various types of tools, starting from the simple tools for listing end-nodes and conversations, to the more sophisticated tools such as flow and I/O graphs. In this article, we will look at the simple tools in Wireshark that provide us with basic network statistics i.e; who talks to whom over the network, what are the chatty devices, what packet sizes run over the network, and so on. To start statistics tools, start Wireshark, and choose Statistics from the main menu. This article is an excerpt from Network Analysis using Wireshark 2 Cookbook - Second Edition written by Nagendra Kumar Nainar, Yogesh Ramdoss, Yoram Orzach. Using the statistics for capture file properties menu In this recipe, we will learn how to get general information from the data that runs over the network. The capture file properties in Wireshark 2 replaces the summary menu in Wireshark 1. Start Wireshark, click on Statistics. How to do it... From the Statistics menu, choose Capture File Properties: What you will get is the Capture File Properties window (displayed in the following screenshot). As you can see in the following screenshot, we have the following: File: Provides file data, such as filename and path, length, and so on Time: Start time, end time, and duration of capture Capture: Hardware information for the PC that Wireshark is installed on Interfaces: Interface information—the interface registry identifier on the left, if capture filter is turned on, interface type and packet size limit Statistics: General capture statistics, including captured and displayed packets: How it works... This menu simply gives a summary of the filtered data properties and the capture statistics (average packets or bytes per second) when someone wants to learn the capture statistics. Using the statistics for protocol hierarchy menu In this recipe, we will learn how to get protocol hierarchy information of the data that runs over the network. Start Wireshark, click on Statistics. How to do it... From the Statistics menu, choose Protocol Hierarchy: What you will get is data about the protocol distribution in the captured file. You will get the protocol distribution of the captured data. The partial screenshot displayed here depicts the statistics of packets captured on a per-protocol basis: What you will get is the Protocol Hierarchy window: Protocol: The protocol name Percent Packets: The percentage of protocol packets from the total captured packets Packets: The number of protocol packets from the total captured packets Percent Bytes: The percentage of protocol bytes from the total captured packets Bytes: The number of protocol bytes from the total captured packets Bit/s: The bandwidth of this protocol, in relation to the capture time End Packets: The absolute number of packets of this protocol (for the highest protocol in the decode file) End Bytes: The absolute number of bytes of this protocol (for the highest protocol in the decode file) End Bit/s: The bandwidth of this protocol, relative to the capture packets and time (for the highest protocol in the decode file) The end columns counts when the protocol is the last protocol in the packet (that is, when the protocol comes at the end of the frame). These can be TCP packets with no payload (for example, SYN packets) which carry upper layer protocols. That is why you see a zero count for Ethernet, IPv4, and UDP end packets; there are no frames where those protocols are the last protocol in the frame. In this file example, we can see two interesting issues: We can see 1,842 packets of DHCPv6. If IPv6 and DHCPv6 are not required, disable it. We see more than 200,000 checkpoint high availability (CPHA) packets, 74.7% of which are sent over the network we monitored. These are synchronization packets that are sent between two firewalls working in a cluster, updating session tables between the firewalls. Such an amount of packets can severely influence performance. The solution for this problem is to configure a dedicated link between the firewalls so that session tables will not influence the network. How it works... Simply, it calculates statistics over the captured data. Some important things to notice: The percentage always refers to the same layer protocols. For example, in the following screenshot, we see that logical link control has 0.5% of the packets that run over Ethernet, IPv6 has 1.0%, IPv4 has 88.8% of the packets, ARP has 9.6% of the packets and even the old Cisco ISK has 0.1 %—a total of 100 % of the protocols over layer 2 Ethernet. On the other hand, we see that TCP has 75.70% of the data, and inside TCP, only 12.74% of the packets are HTTP, and that is almost it. This is because Wireshark counts only the packets with the HTTP headers. It doesn't count, for example, the ACK packets, data packets, and so on: Using the statistics for conversations menu In this recipe, we will learn how to get conversation information of the data that runs over the network. Start Wireshark, click on Statistics. How to do it... From the Statistics menu, choose Conversations: The following window will come up: You can choose between layer 2 Ethernet statistics, layer 3 IP statistics, or layer 4 TCP or UDP statistics. You can use this statistics tools for: On layer 2 (Ethernet): To find and isolate broadcast storms On layer 3/layer 4 (TCP/IP): To connect in parallel to the internet router port, and check who is loading the line to the ISP If you see that there is a lot of traffic going out to port 80 (HTTP) on a specific IP address on the internet, you just have to copy the address to your browser and find the website that is most popular with your users. If you don't get anything, simply go to a standard DNS resolution website (search Google for DNS lookup) and find out what is loading your internet line. For viewing IP addresses as names, you can check the Name resolution checkbox for name resolution (1 in the previous screenshot). For seeing the name resolution, you will first have to enable it by choosing View | Name Resolution | Enable for Network layer. You can also limit the conversations statistics to a display filter by checking the Limit to display filter checkbox (2). In this way, statistics will be presented on all the packets passing the display filter. A new feature in Wireshark version 2 is the graph feature, marked as (5) in the previous screenshot. When you choose a specific line in the TCP conversations statistics and click Graph..., it brings you to the TCP time/sequence (tcptrace) stream graph. To copy table data, click on the Copy button (3). In TCP or UDP, you can mark a specific line, and then click on the Follow Stream... button (4). This will define a display filter that will show you the specific stream of data. As you can see in the following screenshot, you can also right-click a line and choose to prepare or apply a filter, or to colorize a data stream: We also see that, unlike the previous Wireshark version, in which we saw all types of protocols in the upper tabs, here we can choose which protocols to see when only the identified protocols are presented by default. How it works... A network conversation is the traffic between two specific endpoints. For example, an IP conversation is all the traffic between two IP addresses, and TCP conversations present all TCP connections. Using the statistics for endpoints menu In this recipe, we will learn how to get endpoint statistics information of the captured data. Start Wireshark and click on Statistics. How to do it... To view the endpoint statistics, follow these steps: From the Statistics menu, choose Endpoints: The following window will come up: In this window, you will be able to see layer 2, 3, and 4 endpoints, which is Ethernet, IP, and TCP or UDP. From the left-hand side of the window you can see (here is an example for the TCP tab): Endpoint IP address and port number on this host Total packets sent, and bytes received from and to this host Packets to the host (Packets A → B) and bytes to host (Bytes A → B) Packets to the host (Packets B → A) and bytes to host (Bytes B → A) The Latitude and Longitude columns applicable with the GeoIP configured At the bottom of the window we have the following checkboxes: Name resolution: Provide name resolution in cases where it is configured in the name resolution under the view menu. Limit to display filter: To show statistics only for the display filter configured on the main window. Copy: Copy the list values to the clipboard in CSV or YAML format. Map: In cases where GeoIP is configured, shows the geographic information on the geographical map. How it works... Quite simply, it gives statistics on all the endpoints Wireshark has discovered. It can be any situation, such as the following: Few Ethernet (even on) end nodes (that is, MAC addresses), with many IP end nodes (that is, IP addresses)—this will be the case where, for example, we have a router that sends/receives packets from many remote devices. Few IP end nodes with many TCP end nodes—this will be the case for many TCP connections per host. Can be a regular operation of a server with many connections, and it could also be a kind of attack that comes through the network (SYN attack). We learned about Wireshark's basic statistic tools and how you can leverage those for network analysis. Get over 100 recipes to analyze and troubleshoot network problems using Wireshark 2 from this book Network Analysis using Wireshark 2 Cookbook - Second Edition. What’s new in Wireshark 2.6 ? Wireshark for analyzing issues & malicious emails in POP, IMAP, and SMTP  [Tutorial] Capturing Wireshark Packets
Read more
  • 0
  • 5
  • 71406

article-image-implementing-matrix-operations-using-scipy-numpy
Pravin Dhandre
07 Mar 2018
5 min read
Save for later

Implementing matrix operations using SciPy and NumPy

Pravin Dhandre
07 Mar 2018
5 min read
[box type="note" align="" class="" width=""]This article is an excerpt from a book co-authored by L. Felipe Martins, Ruben Oliva Ramos and V Kishore Ayyadevara titled SciPy Recipes. This book includes hands-on recipes for using different components of the SciPy Stack such as NumPy, SciPy, matplotlib, pandas, etc.[/box] In this article, we will discuss how to leverage the power of SciPy and NumPy to perform numerous matrix operations and solve common challenges faced while proceeding with statistical analysis. Matrix operations and functions on two-dimensional arrays Basic matrix operations form the backbone of quite a few statistical analyses—for example, neural networks. In this section, we will be covering some of the most used operations and functions on 2D arrays: Addition Multiplication by scalar Matrix arithmetic Matrix-matrix multiplication Matrix inversion Matrix transposition In the following sections, we will look into the methods of implementing each of them in Python using SciPy/NumPy. How to do it… Let's look at the different methods. Matrix addition In order to understand how matrix addition is done, we will first initialize two arrays: # Initializing an array x = np.array([[1, 1], [2, 2]]) y = np.array([[10, 10], [20, 20]]) Similar to what we saw in a previous chapter, we initialize a 2 x 2 array by using the np.array function. There are two methods by which we can add two arrays. Method 1 A simple addition of the two arrays x and y can be performed as follows: x+y Note that x evaluates to: [[1 1] [2 2]] y evaluates to: [[10 10] [20 20]] The result of x+y would be equal to: [[1+10 1+10] [2+20 2+20]] Finally, this gets evaluated to: [[11 11] [22 22]] Method 2 The same preceding operation can also be performed by using the add function in the numpy package as follows: np.add(x,y) Multiplication by a scalar Matrix multiplication by a scalar can be performed by multiplying the vector with a number. We will perform the same using the following two steps: Initialize a two-dimensional array. Multiply the two-dimensional array with a scalar. We perform the steps, as follows: To initialize a two-dimensional array: x = np.array([[1, 1], [2, 2]]) To multiply the two-dimensional array with the k scalar: k*x For example, if the scalar value k = 2, then the value of k*x translates to: 2*x array([[2, 2], [4, 4]]) Matrix arithmetic Standard arithmetic operators can be performed on top of NumPy arrays too. The operations used most often are: Addition Subtraction Multiplication Division Exponentials The other major arithmetic operations are similar to the addition operation we performed on two matrices in the Matrix addition section earlier: # subtraction x-y array([[ -9, -9], [-18, -18]]) # multiplication x*y array([[10, 10], [40, 40]]) While performing multiplication here, there is an element to element multiplication between the two matrices and not a matrix multiplication (more on matrix multiplication in the next section): # division x/y array([[ 0.1, 0.1], [ 0.1, 0.1]]) # exponential x**y array([[ 1, 1], [1048576, 1048576]], dtype=int32) Matrix-matrix multiplication Matrix to matrix multiplication works in the following way: We have a set of two matrices with the following shape: Matrix A has n rows and m columns and matrix B has m rows and p columns. The matrix multiplication of A and B is calculated as follows: The matrix operation is performed by using the built-in dot function available in NumPy as follows: Initialize the arrays: x=np.array([[1, 1], [2, 2]]) y=np.array([[10, 10], [20, 20]]) Perform the matrix multiplication using the dot function in the numpy package: np.dot(x,y) array([[30, 30], [60, 60]]) The np.dot function does the multiplication in the following way: array([[1*10 + 1*20, 1*10 + 1*20], [2*10 + 2*20, 2*10 + 2*20]]) Whenever matrix multiplication happens, the number of columns in the first matrix should be equal to the number of rows in the second matrix. Matrix transposition Matrix transposition is performed by using the transpose function available in numpy package. The process to generate the transpose of a matrix is as follows: Initialize a matrix: A = np.array([[1,2],[3,4]]) Calculate the transpose of the matrix: A.transpose() array([[1, 3], [2, 4]]) The transpose of a matrix with m rows and n columns would be a matrix with n rows and m columns Matrix inversion While we performed most of the basic arithmetic operations on top of matrices earlier, we have not performed any specialist functions within scientific computing/analysis—for example, matrix inversion, transposition, ranking of a matrix, and so on. The other functions available within the scipy package shine through (over and above the previously discussed functions) in such a scenario where more data manipulation is required apart from the standard ones. Matrix inversion can be performed by using the function available in scipy.linalg. The process to perform matrix inversion and its implementation in Python is as follows: Import relevant packages and classes/functions within a package: from scipy import linalg Initialize a matrix: A = np.array([[1,2],[3,4]]) Pass the initialized matrix through the inverse function in package: linalg.inv(A) array([[-2. , 1. ], [ 1.5, -0.5]]) We saw how to easily perform implementation of all the basic matrix operations with Python’s scientific library - SciPy. You may check out this book SciPy Recipes to perform advanced computing tasks like Discrete Fourier Transform and K-means with the SciPy stack.
Read more
  • 0
  • 0
  • 71101

article-image-optimizing-graphics-pipelines-with-meshlets-a-guide-to-efficient-geometry-processing
Marco Castorina, Gabriel Sassone
09 Dec 2024
15 min read
Save for later

Optimizing Graphics Pipelines with Meshlets: A Guide to Efficient Geometry Processing

Marco Castorina, Gabriel Sassone
09 Dec 2024
15 min read
This article is an excerpt from the book, "Mastering Graphics Programming with Vulkan", by Marco Castorina, Gabriel Sassone. Mastering Graphics Programming with Vulkan starts by familiarizing you with the foundations of a modern rendering engine. This book will guide you through GPU-driven rendering and show you how to drive culling and rendering from the GPU to minimize CPU overhead. Finally, you’ll explore advanced rendering techniques like temporal anti-aliasing and ray tracing.IntroductionIn modern graphics pipelines, optimizing the geometry stage can have a significant impact on overall rendering performance. This article delves into the concept of meshlets—an approach to breaking down large meshes into smaller, more manageable chunks for efficient GPU processing. By subdividing meshes into meshlets, we can enhance culling techniques, reduce unnecessary shading, and better handle complex geometry. Join us as we explore how meshlets work, their benefits, and practical steps to implement them.Breaking down large meshes into meshletsIn this article, we are going to focus primarily on the geometry stage of the pipeline, the one before the shading stage. Adding some complexity to the geometry stage of the pipeline will pay dividends in later stages as we’ll reduce the number of pixels that need to be shaded.NoteWhen we refer to the geometry stage of the graphics pipeline, we don’t mean geometry shaders. Th e geometry stage of the pipeline refers to input assembly (IA), vertex processing, and primitive assembly (PA). Vertex processing can, in turn, run one or  more of the following shaders: vertex, geometry, tessellation, task, and mesh shaders.Content geometry comes in many shapes, sizes, and complexity. A rendering engine must be able to deal with meshes from small, detailed objects to large terrains. Large meshes (think terrain or buildings) are usually broken down by artists so that the rendering engine can pick out the diff erent levels of details based on the distance from the camera of these objects.Breaking down meshes into smaller chunks can help cull geometry that is not visible, but some of these meshes are still large enough that we need to process them in full, even if only a small portion is visible.Meshlets have been developed to address these problems. Each mesh is subdivided into groups of vertices (usually 64) that can be more easily processed on the GPU.The following image illustrates how meshes can be broken down into meshlets:Figure 6.1 – A meshlet subdivision exampleThese vertices can make up an arbitrary number of triangles, but we usually tune this value according to the hardware we are running on. In Vulkan, the recommended value is 126 (as written in https://developer.nvidia.com/blog/introduction-turing-mesh-shaders/, the number is needed to reserve some memory for writing the primitive count with each meshlet).NoteAt the time of writing, mesh and task shaders are only available on Nvidia hardware through its extension. While some of the APIs described in this chapter are specifi c to this extension, the concepts can be generally applied and implemented using generic compute shaders. A more generic version of this extension is currently being worked on by the Khronos committee so that mesh and task shaders should soon be available from other vendors!Now that we have a much smaller number of triangles, we can use them to have much finer-grained control by culling meshlets that are not visible or are being occluded by other objects.Together with the list of vertices and triangles, we also generate some additional data for each meshlet that will be very useful later on to perform back-face, frustum, and occlusion culling.One additional possibility (that will be added in the future) is to choose the level of detail (LOD) of a mesh and, thus, a different subset of meshlets based on any wanted heuristic.The first of this additional data represents the bounding sphere of a meshlet, as shown in the following screenshot:Figure 6.2 – A meshlet bounding spheres example; some of the larger spheres have been hidden for claritySome of you might ask: why not AABBs? AABBs require at least two vec3 of data: one for the center and one for the half-size vector. Another encoding could be to store the minimum and maximum corners. Instead, spheres can be encoded with a single vec4: a vec3 for the center plus the radius.Given that we might need to process millions of meshlets, each saved byte counts! Spheres can also be more easily tested for frustum and occlusion culling, as we will describe later in the chapter.The next additional piece of data that we’re going to use is the meshlet cone, as shown in the following screenshot:Figure 6.3 – A meshlet cone example; not all cones are displayed for clarityThe cone indicates the direction a meshlet is facing and will be used for back-face culling.Now we have a better understanding of why meshlets are useful and how we can use them to improve the culling of larger meshes, let’s see how we generate them in code!Generating meshletsWe are using an open source library, called MeshOptimizer (https://github.com/zeux/meshoptimizer) to generate the meshlets. An alternative library is meshlete (https:// github.com/JarkkoPFC/meshlete) and we encourage you to try both to find the one that best suits your needs.After we have loaded the data (vertices and indices) for a given mesh, we are going to generate the list of meshlets. First, we determine the maximum number of meshlets that could be generated for our mesh and allocate memory for the vertices and indices arrays that  will describe the meshlets:const sizet max_meshlets = meshopt_buildMeshletsBound( indices_accessor.count, max_vertices, max_triangles ); Array<meshopt_Meshlet> local_meshlets; local_meshlets.init( temp_allocator, max_meshlets, max_meshlets ); Array<u32> meshlet_vertex_indices; meshlet_vertex_indices.init( temp_allocator, max_meshlets * max_vertices, max_meshlets* max_vertices ); Array<u8> meshlet_triangles; meshlet_triangles.init( temp_allocator, max_meshlets * max_triangles * 3, max_meshlets* max_triangles * 3 );Notice the types for the indices and triangle arrays. We are not modifying the original vertex or index buffer, but only generating a list of indices in the original buffers. Another interesting aspect is that we only need 1 byte to store the triangle indices. Again, saving memory is very important to keep meshlet processing efficient!The next step is to generate our meshlets:const sizet max_vertices = 64; const sizet max_triangles = 124; const f32 cone_weight = 0.0f; sizet meshlet_count = meshopt_buildMeshlets( local_meshlets.data, meshlet_vertex_indices.data, meshlet_triangles.data, indices, indices_accessor.count, vertices, position_buffer_accessor.count, sizeof( vec3s ), max_vertices, max_triangles, cone_weight );As mentioned in the preceding step, we need to tell the library the maximum number of vertices and triangles that a meshlet can contain. In our case, we are using the recommended values for the Vulkan API. The other parameters include the original vertex and index buffer, and the arrays we have just created that will contain the data for the meshlets.Let’s have a better look at the data structure of each meshlet:struct meshopt_Meshlet { unsigned int vertex_offset; unsigned int triangle_offset; unsigned int vertex_count; unsigned int triangle_count; };Each meshlet is described by two offsets and two counts, one for the vertex indices and one for the indices of the triangles. Note that these off sets refer to meshlet_vertex_indices and meshlet_ triangles that are populated by the library, not the original vertex and index buff ers of the mesh.Now that we have the meshlet data, we need to upload it to the GPU. To keep the data size to a minimum, we store the positions at full resolution while we compress the normals to 1 byte for each dimension and UV coordinates to half-float for each dimension. In pseudocode, this is as follows:meshlet_vertex_data.normal = ( normal + 1.0 ) * 127.0; meshlet_vertex_data.uv_coords = quantize_half( uv_coords );The next step is to extract the additional data (bounding sphere and cone) for each meshlet:for ( u32 m = 0; m < meshlet_count; ++m ) { meshopt_Meshlet& local_meshlet = local_meshlets[ m ]; meshopt_Bounds meshlet_bounds = meshopt_computeMeshletBounds( meshlet_vertex_indices.data + local_meshlet.vertex_offset, meshlet_triangles.data + local_meshlet.triangle_offset, local_meshlet.triangle_count, vertices, position_buffer_accessor .count, sizeof( vec3s ) ); ... }We loop over all the meshlets and we call the MeshOptimizer API that computes the bounds for each meshlet. Let’s see in more detail the structure of the data that is returned:struct meshopt_Bounds { float center[3]; float radius; float cone_apex[3]; float cone_axis[3]; float cone_cutoff; signed char cone_axis_s8[3]; signed char cone_cutoff_s8; };The first four floats represent the bounding sphere. Next, we have the cone definition, which is comprised of the cone direction (cone_axis) and the cone angle (cone_cutoff). We are not using the cone_apex value as it makes the back-face culling computation more expensive. However, it can lead to better results.Once again, notice that quantized values (cone_axis_s8 and cone_cutoff_s8) help us reduce the size of the data required for each meshlet.Finally, meshlet data is copied into GPU buff ers and it will be used during the execution of task and mesh shaders.For each processed mesh, we will also save an offset and count of meshlets to add a coarse culling based on the parent mesh: if the mesh is visible, then its meshlets will be added.In this article, we have described what meshlets are and why they are useful to improve the culling of geometry on the GPU.ConclusionMeshlets represent a powerful tool for optimizing the rendering of complex geometries. By subdividing meshes into small, efficient chunks and incorporating additional data like bounding spheres and cones, we can achieve finer-grained control over visibility and culling processes. Whether you're leveraging advanced shader technologies or applying these concepts with compute shaders, adopting meshlets can lead to significant performance improvements in your graphics pipeline. With libraries like MeshOptimizer at your disposal, implementing this technique has never been more accessible.Author BioMarco Castorina first became familiar with Vulkan while working as a driver developer at Samsung. Later, he developed a 2D and 3D renderer in Vulkan from scratch for a leading media server company. He recently joined the games graphics performance team at AMD. In his spare time, he keeps up to date with the latest techniques in real-time graphics. He also likes cooking and playing guitar.Gabriel Sassone is a rendering enthusiast currently working as a principal rendering engineer at The Multiplayer Group. Previously working for Avalanche Studios, where he first encountered Vulkan, they developed the Vulkan layer for the proprietary Apex Engine and its Google Stadia port. He previously worked at ReadyAtDawn, Codemasters, FrameStudios, and some other non-gaming tech companies. His spare time is filled with music and rendering, gaming, and outdoor activities.
Read more
  • 0
  • 0
  • 70757
Unlock access to the largest independent learning library in Tech for FREE!
Get unlimited access to 7500+ expert-authored eBooks and video courses covering every tech area you can think of.
Renews at $19.99/month. Cancel anytime
article-image-getting-started-with-react-hooks-by-building-a-counter-with-usestate-and-useeffect
Guest Contributor
12 Feb 2019
7 min read
Save for later

Getting started with React Hooks by building a counter with useState and useEffect

Guest Contributor
12 Feb 2019
7 min read
React 16 added waves of new features, improving the way we build web applications. The most impactful update is the new Hooks feature in version 16.8. Hooks allow us to write functional React components that manage state and side effects, making our code cleaner and providing the ability to easily to share functionality. React is not removing class components, but they cause many problems and are a detriment to upcoming code optimizations. The vision for Hooks is that all new components will be written using the API, resulting in more scalable web applications with better code. This tutorial will walk you through Hooks step-by-step and teach the core hook functionality by building a counter app. An overview of hooks Hooks provide the ability to manage state and side effects in functional components while also providing a simple interface to control the component lifecycle. The 4 built-in hooks provided by React are useState, useEffect, useReducer, and useContext. useState replaces the need for this.state used in class components useEffect manages side effects of the app by controlling the componentDidMount, componentDidUpdate, and componentWillUnmount lifecycle methods. useContext allows us to subscribe to the React context useReducer is similar to useState but allows for more complex state updates. The two main hook functions that you will use are, useState and useEffect, which manage the standard React state and lifecycle. useReducer is used to manage more complex state and useContext is a hook to pass values from the global React context to a component. With the core specification updating frequently, it’s essential to find tutorials to learn React. You can also build your own custom hooks, which can contain the primitive hooks exposed by React. You are able to extract component state into reusable functions that can be accessed by any component. Higher-order components and render props have traditionally been the way to share functionality, but these methods can lead to a bloated component tree with a confusing glob of nested React elements. Hooks offer a straightforward way to DRY out your code by simply importing the custom hook function into your component. Building counter with hooks To build our counter, we will use Create React App to bootstrap the application. You can install the package globally or use npx from the command line: npx create-react-app react-hooks-counter cd react-hooks-counter React Hooks is a brand new feature, so ensure you have v16.8.x installed. Inside your package.json, the version of react and react-dom should look similar to the code snippet below. If not, update them and reinstall using the yarn command. The foundation of hooks is that they are utilized inside functional components. To start, let’s convert the boilerplate file inside src/App.js to a functional component and remove the content. At the top of the file, we can import useState and useEffect from React. import React, { useState, useEffect } from 'react'; The most straightforward hook is useState since its purpose is to maintain a single value, so let’s begin there. The function takes an initial value and returns an array of arguments, with the item at the 0 index containing the state value, and the item at the 1 index containing a function to update the value. We will initialize our count to 0 and name the return variables count and setCount. const [count, setCount] = useState(0); NOTE: The returned value of the useState is an array. To simplify the syntax, we use array destructuring to extract the elements at the 0 and 1 index. Inside our rendered React component, we will display the count and provide a button to increment the count by 1 by using setCount. With a single function, we have eliminated the need to have a class component along with this.state and this.setState to manage our data. Every time you click the increment button, the count will increase by 1. Since we are using a hook, React recognizes this change in state and will re-render the DOM with this updated value. To demonstrate the extensibility of the state updates, we will add buttons increment the count by 2, 5, and 10 as well. We will also DRY out our code by storing these values in an array. We iterate over this array using the .map() function, which will return an array of React components. React will treat this as sibling elements in the DOM. You are now able to increment the count by different values. Now we will integrate the useEffect hook. This hook enables you to manage side effects and handle asynchronous events. The most notable and frequently used side effect is an API call. We will mimic the async nature of an API call using a setTimeout function. We will make a fake API request on the component’s mount that will initialize a random integer 1–10 to our count after waiting 1 second. We will also have an additional useEffect that will update the document title (a side effect) with the current count to show how it responds to a change in state. The useEffect hook takes a function as an argument. useEffect replaces the componentDidMount, componentDidUpdate, and componentWillUnmount class methods. When the state of the component mounts or updates, React will execute the callback function. If your callback function returns a function itself, React will execute this during componentWillUnmount. First, let’s create our effect to update the document title. Inside the body of our function, we declare useEffect which sets document.title = 'Count = ' + count in the callback. When the state count updates, you should see your tab title also updating simultaneously. For the final step, we will create a mock API call that returns an integer to update the state count. We use a setTimeout and a function that returns a Promise because this simulates the time required to wait for an API request to return and the associated return value of a promise, which allows us to handle the response asynchronously. To mock an API, we create a mockApi function above our component. It returns a promise with a resolved random integer between 1 and 10. A common pattern is to make fetch requests in the componentDidMount. To reproduce this is in our functional component, we will add another useState to manage a hasFetched variable: const [hasFetched, setFetch] = useState(false). This is used to prevent the mockApi from being executed on subsequent updates. Our fetch hook will be an async function, so we will use async/await to handle the result. Inside our useEffect function, we will first check if the hasFetched has been executed. If it has not, we call mockApi and setCount with a result to initialize our value and then flip our hasFetched flag to true. Visual indicators are essential for UX and provide feedback for your users of the application status. Since we are waiting for an initial count value, we want to hide our buttons and display “Loading…” text on the screen if the hasFetched is false. This results in the following behavior: The final code Wrapping Up This article introduced hooks and showed how to implement useState and useEffect to simplify your class components into simple functional components. While this is a big win for React developers, the power of hooks is fully realized with the ability to combine them to create custom hooks. This allows you to extract logic and build modular functionality that can seamlessly be shared among React components without the overhead of HOCs or render props. You simply import your custom hook function, and any component can implement it. The only caveat is that all hook functions must follow the rules of hooks. Author Bio Trey Huffine A JavaScript fanatic. He is a software engineer in Silicon Valley building products using React, Node, and Go. Passionate for making the world a better place through code. Reactive programming in Swift with RxSwift and RxCocoa [Tutorial] React 16.8 releases with the stable implementation of Hooks PrimeReact 3.0.0 is now out with Babylon create-react-app template
Read more
  • 0
  • 0
  • 70434

article-image-automating-ocr-and-translation-with-google-cloud-functions-a-step-by-step-guide
Agnieszka Koziorowska, Wojciech Marusiak
05 Nov 2024
15 min read
Save for later

Automating OCR and Translation with Google Cloud Functions: A Step-by-Step Guide

Agnieszka Koziorowska, Wojciech Marusiak
05 Nov 2024
15 min read
This article is an excerpt from the book, "Google Cloud Associate Cloud Engineer Certification and Implementation Guide", by Agnieszka Koziorowska, Wojciech Marusiak. This book serves as a guide for students preparing for ACE certification, offering invaluable practical knowledge and hands-on experience in implementing various Google Cloud Platform services. By actively engaging with the content, you’ll gain the confidence and expertise needed to excel in your certification journey.Introduction In this article, we will walk you through an example of implementing Google Cloud Functions for optical character recognition (OCR) on Google Cloud Platform. This tutorial will demonstrate how to automate the process of extracting text from an image, translating the text, and storing the results using Cloud Functions, Pub/Sub, and Cloud Storage. By leveraging Google Cloud Vision and Translation APIs, we can create a workflow that efficiently handles image processing and text translation. The article provides detailed steps to set up and deploy Cloud Functions using Golang, covering everything from creating storage buckets to deploying and running your function to translate text. Google Cloud Functions Example Now that you’ve learned what Cloud Functions is, I’d like to show you how to implement a sample Cloud Function. We will guide you through optical character recognition (OCR) on Google Cloud Platform with Cloud Functions. Our use case is as follows: 1. An image with text is uploaded to Cloud Storage. 2. A triggered Cloud Function utilizes the Google Cloud Vision API to extract the text and identify the source language. 3. The text is queued for translation by publishing a message to a Pub/Sub topic. 4. A Cloud Function employs the Translation API to translate the text and stores the result in the translation queue. 5. Another Cloud Function saves the translated text from the translation queue to Cloud Storage. 6. The translated results are available in Cloud Storage as individual text files for each translation. We need to download the samples first; we will use Golang as the programming language. Source files can be downloaded from – https://github.com/GoogleCloudPlatform/golangsamples. Before working with the OCR function sample, we recommend enabling the Cloud Translation API and the Cloud Vision API. If they are not enabled, your function will throw errors, and the process will not be completed. Let’s start with deploying the function: 1. We need to create a Cloud Storage bucket.  Create your own bucket with unique name – please refer to documentation on bucket naming under following link: https://cloud.google.com/storage/docs/buckets We will use the following code: gsutil mb gs://wojciech_image_ocr_bucket 2. We also need to create a second bucket to store the results: gsutil mb gs://wojciech_image_ocr_bucket_results 3. We must create a Pub/Sub topic to publish the finished translation results. We can do so with the following code: gcloud pubsub topics create YOUR_TOPIC_NAME. We used the following command to create it: gcloud pubsub topics create wojciech_translate_topic 4. Creating a second Pub/Sub topic to publish translation results is necessary. We can use the following code to do so: gcloud pubsub topics create wojciech_translate_topic_results 5. Next, we will clone the Google Cloud GitHub repository with some Python sample code: git clone https://github.com/GoogleCloudPlatform/golang-samples 6. From the repository, we need to go to the golang-samples/functions/ocr/app/ file to be able to deploy the desired Cloud Function. 7. We recommend reviewing the included go files to review the code and understand it in more detail. Please change the values of your storage buckets and Pub/Sub topic names. 8. We will deploy the first function to process images. We will use the following command: gcloud functions deploy ocr-extract-go --runtime go119 --trigger-bucket wojciech_image_ocr_bucket --entry-point  ProcessImage --set-env-vars "^:^GCP_PROJECT=wmarusiak-book- 351718:TRANSLATE_TOPIC=wojciech_translate_topic:RESULT_ TOPIC=wojciech_translate_topic_results:TO_LANG=es,en,fr,ja" 9. After deploying the first Cloud Function, we must deploy the second one to translate the text.  We can use the following code snippet: gcloud functions deploy ocr-translate-go --runtime go119 --trigger-topic wojciech_translate_topic --entry-point  TranslateText --set-env-vars "GCP_PROJECT=wmarusiak-book- 351718,RESULT_TOPIC=wojciech_translate_topic_results" 10. The last part of the complete solution is a third Cloud Function that saves results to Cloud Storage. We will use the following snippet of code to do so: gcloud functions deploy ocr-save-go --runtime go119 --triggertopic wojciech_translate_topic_results --entry-point SaveResult  --set-env-vars "GCP_PROJECT=wmarusiak-book-351718,RESULT_ BUCKET=wojciech_image_ocr_bucket_results" 11. We are now free to upload any image containing text. It will be processed first, then translated and saved into our Cloud Storage bucket. 12. We uploaded four sample images that we downloaded from the Internet that contain some text. We can see many entries in the ocr-extract-go Cloud Function’s logs. Some Cloud Function log entries show us the detected language in the image and the other extracted text:  Figure 7.22 – Cloud Function logs from the ocr-extract-go function 13. ocr-translate-go translates detected text in the previous function:  Figure 7.23 – Cloud Function logs from the ocr-translate-go function 14. Finally, ocr-save-go saves the translated text into the Cloud Storage bucket:  Figure 7.24 – Cloud Function logs from the ocr-save-go function 15. If we go to the Cloud Storage bucket, we’ll see the saved translated files:  Figure 7.25 – Translated images saved in the Cloud Storage bucket 16. We can view the content directly from the Cloud Storage bucket by clicking Download next to the file, as shown in the following screenshot:  Figure 7.26 – Translated text from Polish to English stored in the Cloud Storage bucket Cloud Functions is a powerful and fast way to code, deploy, and use advanced features. We encourage you to try out and deploy Cloud Functions to understand the process of using them better. At the time of writing, Google Cloud Free Tier offers a generous number of free resources we can use. Cloud Functions offers the following with its free tier: 2 million invocations per month (this includes both background and HTTP invocations) 400,000 GB-seconds, 200,000 GHz-seconds of compute time 5 GB network egress per month Google Cloud has comprehensive tutorials that you can try to deploy. Go to https://cloud.google.com/functions/docs/tutorials to follow one. Conclusion In conclusion, Google Cloud Functions offer a powerful and scalable solution for automating tasks like optical character recognition and translation. Through this example, we have demonstrated how to use Cloud Functions, Pub/Sub, and the Google Cloud Vision and Translation APIs to build an end-to-end OCR and translation pipeline. By following the provided steps and code snippets, you can easily replicate this process for your own use cases. Google Cloud's generous Free Tier resources make it accessible to get started with Cloud Functions. We encourage you to explore more by deploying your own Cloud Functions and leveraging the full potential of Google Cloud Platform for serverless computing. Author BioAgnieszka is an experienced Systems Engineer who has been in the IT industry for 15 years. She is dedicated to supporting enterprise customers in the EMEA region with their transition to the cloud and hybrid cloud infrastructure by designing and architecting solutions that meet both business and technical requirements. Agnieszka is highly skilled in AWS, Google Cloud, and VMware solutions and holds certifications as a specialist in all three platforms. She strongly believes in the importance of knowledge sharing and learning from others to keep up with the ever-changing IT industry.With over 16 years in the IT industry, Wojciech is a seasoned and innovative IT professional with a proven track record of success. Leveraging extensive work experience in large and complex enterprise environments, Wojciech brings valuable knowledge to help customers and businesses achieve their goals with precision, professionalism, and cost-effectiveness. Holding leading certifications from AWS, Alibaba Cloud, Google Cloud, VMware, and Microsoft, Wojciech is dedicated to continuous learning and sharing knowledge, staying abreast of the latest industry trends and developments.
Read more
  • 0
  • 0
  • 70252

article-image-managing-ai-security-risks-with-zero-trust-a-strategic-guide
Mark Simos, Nikhil Kumar
29 Nov 2024
15 min read
Save for later

Managing AI Security Risks with Zero Trust: A Strategic Guide

Mark Simos, Nikhil Kumar
29 Nov 2024
15 min read
This article is an excerpt from the book, "Zero Trust Overview and Playbook Introduction", by Mark Simos, Nikhil Kumar. Get started on Zero Trust with this step-by-step playbook and learn everything you need to know for a successful Zero Trust journey with tailored guidance for every role, covering strategy, operations, architecture, implementation, and measuring success. This book will become an indispensable reference for everyone in your organization.IntroductionIn today’s rapidly evolving technological landscape, artificial intelligence (AI) is both a powerful tool and a significant security risk. Traditional security models focused on static perimeters are no longer sufficient to address AI-driven threats. A Zero Trust approach offers the agility and comprehensive safeguards needed to manage the unique and dynamic security risks associated with AI. This article explores how Zero Trust principles can be applied to mitigate AI risks and outlines the key priorities for effectively integrating AI into organizational security strategies.How can Zero Trust help manage AI security risk?A Zero Trust approach is required to effectively manage security risks related to AI. Classic network perimeter-centric approaches are built on more than 20-year-old assumptions of a static technology environment and are not agile enough to keep up with the rapidly evolving security requirements of AI.The following key elements of Zero Trust security enable you to manage AI risk:Data centricity: AI has dramatically elevated the importance of data security and AI requires a data-centric approach that can secure data throughout its life cycle in any location.Zero Trust provides this data-centric approach and the playbooks in this series guide the roles in your organizations through this implementation.Coordinated management of continuous dynamic risk: Like modern cybersecurity attacks, AI continuously disrupts core assumptions of business, technical, and security processes. This requires coordinated management of a complex and continuously changing security risk.Zero Trust solves this kind of problem using agile security strategies, policies, and architecture to manage the continuous changes to risks, tooling, processes, skills, and more. The playbooks in this series will help you make AI risk mitigation real by providing specific guidance on AI security risks for all impacted roles in the organization. Let’s take a look at which specific elements of Zero Trust are most important to managing AI risk.Zero Trust – the top four priorities for managing AI riskManaging AI risk requires prioritizing a few key areas of Zero Trust to address specific unique aspects of AI. The role of specific guidance in each playbook provides more detail on how each role will incorporate AI considerations into their daily work.These priorities follow the simple themes of learn it, use it, protect against it, and work as a team. This is similar to a rational approach for any major disruptive change to any other type of competition or conflict (a military organization learning about a new weapon, professional sports players learning about a new type of equipment or rule change, and so on).The top four priorities for managing AI risk are as follows:1. Learn it – educate everyone and set realistic expectations: The AI capabilities available today are very powerful, affect everyone, and are very different than what people expect them to be. It’s critical to educate every role in the organization, from board members and CEOs to individual contributors, as they all must understand what AI is, what AI really can and cannot do, as well as the AI usage policy and guidelines. Without this, people’s expectations may be wildly inaccurate and lead to highly impactful mistakes that could have easily been avoided.Education and expectation management is particularly urgent for AI because of these factors:Active use in attacks: Attackers are already using AI to impersonate voices, email writing styles, and more.Active use in business processes: AI is freely available for anyone to use. Job seekers are already submitting AI-generated resumes for your jobs that use your posted job descriptions, people are using public AI services to perform job tasks (and potentially disclosing sensitive information), and much more.Realism: The results are very realistic and convincing, especially if you don’t know how good AI is at creating fake images, videos, and text.How can Zero Trust help manage AI security risk?Confusion: Many people don’t have a good frame of reference for it because of the way AI has been portrayed in popular culture (which is very different from the current reality of AI).2. Use it – integrate AI into security: Immediately begin evaluating and integrating AI into your security tooling and processes to take advantage of their increased effectiveness and efficiency. This will allow you to quickly take advantage of this powerful technology to better manage security risk. AI will impact nearly every part of security, including the following:Security risk discovery, assessment, and management processesThreat detection and incident response processesArchitecture and engineering security defensesIntegrating security into the design and operation of systems…and many more3. Protect against it – update the security strategy, policy, and controls: Organizations must urgently update their strategy, policy, architecture, controls, and processes to account for the use of AI technology (by business units, technology teams, security teams, attackers, and more). This helps enable the organization to take full advantage of AI technology while minimizing security risk.The key focus areas should include the following:Plan for attacker use of AI: One of the first impacts most organizations will experience is rapid adoption by attackers to trick your people. Attackers are using AI to get an advantage on target organizations like yours, so you must update your security strategy, threat models, architectures, user education, and more to defend against attackers using AI or targeting you for your data. This should change the organization’s expectations and assumptions for the following aspects:Attacker techniques: Most attackers will experiment with and integrate AI capabilities into their attacks, such as imitating the voices of your colleagues on phone calls, imitating writing styles in phishing emails, creating convincing fake social media pictures and profiles, creating convincing fake company logos and profiles, and more.Attacker objectives: Attackers will target your data, AI systems, and other related assets because of their high value (directly to the attacker and/or to sell it to others). Your human-generated data is a prized high-value asset for training and grounding AI models and your innovative use of AI may be potentially valuable intellectual property, and more.Secure the organization’s AI usage: The organization must update its security strategy, plans, architecture, processes, and tooling to do the following:Secure usage of external AI: Establish clear policies and supporting processes and technology for using external AI systems safelySecure the organization’s AI and related systems: Protect the organization’s AI and related systems against attackersIn addition to protecting against traditional security attacks, the organization will also need to defend against AI-specific attack techniques that can extract source data, make the model generate unsafe or unintended results, steal the design of the AI model itself, and more. The playbooks include more details for each role to help them manage their part of this risk.Take a holistic approach: It’s important to secure the full life cycle and dependencies of the AI model, including the model itself, the data sources used by the model, the application that uses the model, the infrastructure it’s hosted on, third-party operators such as AI platforms, and other integrated components. This should also take a holistic view of the security life cycle to consider identification, protection, detection, response, recovery, and governance.Update acquisition and approval processes: This must be done quickly to ensure new AI technology (and other technology) meets the security, privacy, and ethical practices of the organization. This helps avoid extremely damaging avoidable problems such as transferring ownership of the organization’s data to vendors and other parties. You don’t want other organizations to grow and capture market share from you by using your data. You also want to avoid expensive privacy incidents and security incidents from attackers using your data against you.This should include supply chain risk considerations to mitigate both direct suppliers and Nth party risk (components of direct suppliers that have been sourced from other organizations). Finding and fixing problems later in the process is much more difficult and expensive than correcting them before or during acquisition, so it is critical to introduce these risk mitigations early.4. Work as a team – establish a coordinated AI approach: Set up an internal collaboration community or a formal Center of Excellence (CoE) team to ensure insights, learning, and best practices are being shared rapidly across teams. AI is a fast-moving space and will drive rapid continuous changes across business, technology, and security teams. You must have mechanisms in place to coordinate and collaborate across these different teams in your organization.How will AI impact Zero Trust?Each playbook describes the specific AI impacts and responsibilities for each affected role.AI shared responsibility model: Most AI technology will be a partnership with AI providers, so managing AI and AI security risk will follow a shared responsibility model between you and your AI providers. Some elements of AI security will be handled by the AI provider and some will be the responsibility of your organization (their customer).This is very similar to how cloud responsibility is managed today (and many AI providers are also cloud providers). This is also similar to a business that outsources some or all of its manufacturing, logistics, sales (for example, channel sales), or other business functions.Now, let’s take a look at how AI impacts Zero Trust.How will AI impact Zero Trust?AI will accelerate many aspects of Zero Trust because it dramatically improves the security tooling and people’s ability to use it. AI promises to reduce the burden and effort for important but tedious security tasks such as the following:Helping security analysts quickly query many data sources (without becoming an expert in query languages or tool interfaces)Helping writing incident response reportsIdentifying common follow-up actions to prevent repeat incidentSimplifying the interface between people and the complex systems they need to use for security will enable people with a broad range of skills to be more productive. Highly skilled people will be able to do more of what they are best at without repetitive and distracting tasks. People earlier in their careers will be able to quickly become more productive in a role, perform tasks at an expert level more quickly, and help them learn by answering questions and providing explanations.AI will NOT replace the need for security experts, nor the need to modernize security. AI will simplify many security processes and will allow fewer security people to do more, but it won’t replace the need for a security mindset or security expertise.Even with AI technology, people and processes will still be required for the following aspects:Ask the right security questions from AI systemsInterpret the results and evaluate their accuracyTake action on the AI results and coordinate across teamsPerform analysis and tasks that AI systems currently can’t cover:Identify, manage, and measure security risk for the organizationBuild, execute, and monitor a strategy and policyBuild and monitor relationships and processes between teamsIntegrate business, technical, and security capabilitiesEvaluate compliance requirements and ensure the organization is meeting them in good faithEvaluate the security of business and technical processesEvaluate the security posture and prioritize mitigation investmentsEvaluate the effectiveness of security processes, tools, and systemsPlan and implement security for technical systemsPlan and implement security for applications and productsRespond to and recover from attacksIn summary, AI will rapidly transform the attacks you face as well as your organization’s ability to manage security risk effectively. AI will require a Zero Trust approach and it will also help your teams do their jobs faster and more efficiently.The guidance in the Zero Trust Playbook Series will accelerate your ability to manage AI risk by guiding everyone through their part. It will help you rapidly align security to business risks and priorities and enable the security agility you need to effectively manage the changes from AI.Some of the questions that naturally come up are where to start and what to do first.ConclusionAs AI reshapes the cybersecurity landscape, adopting a Zero Trust framework is critical to effectively manage the associated risks. From securing data lifecycles to adapting to dynamic attacker strategies, Zero Trust principles provide the foundation for agile and robust AI risk management. By focusing on education, integration, protection, and collaboration, organizations can harness the benefits of AI while mitigating its risks. The Zero Trust Playbook Series offers practical guidance for all roles, ensuring security remains aligned with business priorities and prepared for the challenges AI introduces. Now is the time to embrace this transformative approach and future-proof your security strategies.Author BioMark Simos helps individuals and organizations meet cybersecurity, cloud, and digital transformation goals. Mark is the Lead Cybersecurity Architect for Microsoft where he leads the development of cybersecurity reference architectures, strategies, prescriptive planning roadmaps, best practices, and other security and Zero Trust guidance. Mark also co-chairs the Zero Trust working group at The Open Group and contributes to open standards and other publications like the Zero Trust Commandments. Mark has presented at numerous conferences including Black Hat, RSA Conference, Gartner Security & Risk Management, Microsoft Ignite and BlueHat, and Financial Executives International.Nikhil Kumar is Founder at ApTSi with prior leadership roles at Price Waterhouse and other firms. He has led setup and implementation of Digital Transformation and enterprise security initiatives (such as PCI Compliance) and built out Security Architectures. An Engineer and Computer Scientist with a passion for biology, Nikhil is an expert in Security, Information, and Computer Architecture. Known for communicating to the board and implementing with engineers and architects, he is an MIT mentor, innovator and pioneer. Nikhil has authored numerous books, standards, and articles, and presented at conferences globally. He co-chairs The Zero Trust Working Group, a global standards initiative led by the Open Group.
Read more
  • 5
  • 0
  • 69798

article-image-using-react-router-client-side-redirecting
Antonio Cucciniello
05 Apr 2017
6 min read
Save for later

Using React Router for Client Side Redirecting

Antonio Cucciniello
05 Apr 2017
6 min read
If you are using React in the front end of your web application and you would like to use React Router in order to handle routing, then you have come to the right place. Today, we will learn how to have your client side redirect to another page after you have completed some processing. Let's get started! Installs First we will need to make sure we have a couple of things installed. The first thing here is to make sure you have Node and NPM installed. In order to make this as simple as possible, we are going to use create-react-app to get React fully working in our application. Install this by doing: $ npm install -g create-react-app Now create a directory for this example and enter it; here we will call it client-redirect. $ mkdir client-redirect $ cd client-redirect Once in that directory, initialize create-react-app in the repo. $ create-react-app client Once it is done, test that it is working by running: $ npm start You should see something like this: The last thing you must install is react-router: $ npm install --save react-router Now that you are all set up, let's start with some code. Code First we will need to edit the main JavaScript file, in this case, located at client-redirect/client/src/App.js. App.js App.js is where we handle all the routes for our application and acts as the main js file. Here is what the code looks like for App.js: import React, { Component } from 'react' import { Router, Route, browserHistory } from 'react-router' import './App.css' import Result from './modules/result' import Calculate from './modules/calculate' class App extends Component { render () { return ( <Router history={browserHistory}> <Route path='/' component={Calculate} /> <Route path='/result' component={Result} /> </Router> ) } } export default App If you are familiar with React, you should not be too confused as to what is going on. At the top, we are importing all of the files and modules we need. We are importing React, react-router, our App.css file for styles, and result.js and calculate.js files (do not worry we will show the implementation for these shortly). The next part is where we do something different. We are using react-router to set up our routes. We chose to use a history of type browserHistory. History in react-router listens to the browser's address bar for anything changing and parses the URL from the browser and stores it in a location object so the router can match the specified route and render the different components for that path. We then use <Route> tags in order to specify what path we would like a component to be rendered on. In this case, we are using '/' path for the components in calculate.js and '/result' path for the components in result.js. Let's define what those pages will look like and see how the client can redirect using browserHistory. Calculate.js This page is a basic page with two text boxes and a button. Each text box should receive a number and when the button is clicked, we are going to calculate the sum of the two numbers given. Here is what that looks like: import React, { Component } from 'react' import { browserHistory } from 'react-router' export default class Calculate extends Component { render () { return ( <div className={'Calculate-page'} > <InputBox type='text' name='first number' id='firstNum' /> <InputBox type='text' name='second number' id='secondNum' /> <CalculateButton type='button' value='Calculate' name='Calculate' onClick='result()' /> </div> ) } } var InputBox = React.createClass({ render: function () { return <div className={'input-field'}> <input type={this.props.type} value={this.props.value} name={this.props.name} id={this.props.id} /> </div> } }) var CalculateButton = React.createClass({ result: function () { var firstNum = document.getElementById('firstNum').value var secondNum = document.getElementById('secondNum').value var sum = Number(firstNum) + Number(secondNum) if (sum !== undefined) { const path = '/result' browserHistory.push(path) } window.sessionStorage.setItem('sum', sum) return console.log(sum) }, render: function () { return <div className={'calculate-button'}> <button type={this.props.type} value={this.props.value} name={this.props.name} onClick={this.result} > Calculate </button> </div> } }) The important part we want to focus on is inside the result function of the CalculateButton class. We take the two numbers and sum them. Once we have the sum, we create a path variable to hold the route we would like to go to next. Then browserHistory.push(path) redirects the client to a new path of localhost:3000/result. We then store the sum in sessionStorage in order to retrieve it on the next page. result.js This is simply a page that will display your result from the calculation, but it serves as the page you redirected to with react-router. Here is the code: import React, { Component } from 'react' export default class Result extends Component { render () { return ( <div className={'result-page'} > Result : <DisplayNumber id='result' /> </div> ) } } var DisplayNumber = React.createClass({ componentDidMount () { document.getElementById('result').innerHTML = window.sessionStorage.getItem('sum') }, render: function () { return ( <div className={'display-number'}> <p id={this.props.id} /> </div> ) } }) We simply create a class that wraps a paragraph tag. It also has a componentDidMount() function, which allows us to access the sessionStorage for the sum once the component output has been rendered by the DOM. We update the innerHTML of the paragraph element with the sum's value. Test Let's get back to the client directory of our application. Once we are there, we can run: $ npm start This should open a tab in your web browser at localhost:3000. This is what it should look like when you add numbers in the text boxes (here I added 17 and 17). This should redirect you to another page: Conclusion There you go! You now have a web app that utilizes client-side redirecting! To summarize what we did, here is a quick list of what happened: Installed the prerequisites: node, npm Installed create-react-app Created a create-react-app Installed react-router Added our routing to our App.js Created a module that calculated the sum of two numbers and redirected to a new page Displayed the number on the new redirected page Check out the code for this tutorial on GitHub. Possible Resources Check out my GitHub View my personal blog GitHub pages for: react-router create-react-app About the author Antonio Cucciniello is a software engineer with a background in C, C++, and JavaScript (Node.js). He is from New Jersey, USA. His most recent project called Edit Docs is an Amazon Echo skill that allows users to edit Google Drive files using their voice. He loves building cool things with software, reading books on self-help and improvement, finance, and entrepreneurship. To contact Antonio, e-mail him at Antonio.cucciniello16@gmail.com, follow him on twitter at @antocucciniello, and follow him on GitHub here: https://github.com/acucciniello.
Read more
  • 0
  • 0
  • 69128
article-image-mongodb-is-going-to-acquire-realm-the-mobile-database-management-system-for-39-million
Richard Gall
25 Apr 2019
3 min read
Save for later

MongoDB is going to acquire Realm, the mobile database management system, for $39 million

Richard Gall
25 Apr 2019
3 min read
MongoDB, the open source NoSQL database, is going to acquire mobile database platform Realm. The purchase is certainly one with clear technological and strategic benefits for both companies - and with MongoDB paying $39 million for a company that has up to now raised $40 million since its launch in 2011, it's clear that this is a move that isn't about short term commercial gains. It's important to note that the acquisition is not yet complete. It's expected to close in January 2020 at the end of the second quarter MongoDB's fiscal year. Further details about the acquisition and what it means for both products, will be revealed at MongoDB World in June. Why is MongoDB acquiring Realm? In the materials that announce the launch there's a lot of talk about the alignment between the two projects. "The best thing in the world is when someone just gets you, and you get them" MongoDB CTO Eliot Horowitz wrote in a blog post accompanying the release, "because when you share a vision of the world like that, you can do incredible things together. That’s exactly the case with MongoDB and Realm." At a more fundamental level the acquisition allows MongoDB to do a number of things. It can reach a new community of developers  working primarily in mobile development (according to the press release Realm has 100,000 active users), but it also allows MongoDB to strengthen its capabilities as cloud evolves to become the dominant way that applications are built and hosted. According to Dev Ittycheria, MongoDB President and CEO, Realm "is a natural fit for our global cloud database, MongoDB Atlas, as well as a complement to Stitch, our serverless platform." Serverless might well be a nascent trend at the moment, but the level of conversation and interest around it indicates that it's going to play a big part in application developers lives in the months and years to come. What's in it for Realm? For Realm, the acquisition will give the project access to a new pool of users. With backing from MongoDB, is also provides robust foundations for the project to extend its roadmap and even move faster than it previously would have been able to. Realm CEO David Ratner wrote yesterday (April 24) that: "The combination of MongoDB and Realm will establish the modern standard for mobile application development and data synchronization for a new generation of connected applications and services. MongoDB and Realm are fully committed to investing in the Realm Database and the future of data synchronization, and taking both to the next phase of their evolution. We believe that MongoDB will help accelerate Realm’s product roadmap, go-to-market execution, and support our customers’ use cases at a whole new level of global operational scale." A new chapter for MongoDB? 2019 hasn't been the best year for MongoDB so far. The project withdrew its submission for its controversial Server Side Public License last month following news that Red Hat was dropping it from Enterprise Linux and Fedora. This brought an initiative that the leadership viewed as strategically important in defending MongoDB's interests to a dramatic halt. However, the Realm acquisition sets up a new chapter and could go some way in helping MongoDB bolster itself for a future that it has felt uncertain about.
Read more
  • 0
  • 0
  • 69071

article-image-functional-programming-c
Packt
07 Jul 2016
4 min read
Save for later

Functional Programming in C#

Packt
07 Jul 2016
4 min read
In this article, we are going to explore the following topics: The introductory of functional programming concept Comparison between functional and imperative approach (For more resources related to this topic, see here.) Introduction to functional programming In functional programming, we use mathematic approach to construct our code. The function we've got in the code has similarity with mathematical function we usually use in our daily basis. The variable in the code function represents the value of the function parameter and it similar to the mathematical function. The idea is a programmer defines the functions, which contain the expression, definition, and also parameters which can be expressed by variable in order to solve the problems. After a programmer builds the function and sends computer the function, it's now computer's turn to do its job. In general, the role of computer is to evaluate the expression in the function and return the result. We can imagine that the computer acts like a calculator since it will analyse the expression from the function and yield the result to the user in printed format. Suppose we have the expression 3 + 5 inside a function. The computer will definitely return 8 as the result just after completely evaluates it. However, it is just the trivial example of the acting of computer in evaluating the expression. In fact, the programmer can increase the ability of the computer by making the complex definition and expression inside the function. Not only can the computer evaluate the trivial expression, but it also can evaluate complex calculation and expression. Comparison to imperative programming The main difference between functional and imperative programming is the existence of side effect. In functional programming, since it applies pure function concept, the side effect is avoided. It's different with imperative programming which has to access I/O and modify state outside the function which will produce side effect. In addition, with an imperative approach, the programmer focuses on the way of performing the task and tracking changes in state while a programmer focuses on the kind of desired information and the kind of required transformation in functional approach. The change of states becomes important in imperative programming while no change of states exist in functional programming. The order of execution is also important in imperative function but not really important in functional programming since we need to concern more on constructing the problem as a set of functions to be execute rather than the detail step of the flow. We will continue our discussion about functional and imperative approach by creating some code in the next topics. Summary We have been acquainted with functional approach so far by discussing the introduction of functional programming. We also have compared the functional approach with mathematical concept in function. It's now clear that functional approach uses the mathematical approach to compose functional program. The comparison between functional and imperative programming has also give us the important point to distinguish the two. It's now clear that in functional programming the programmer focuses on the kind of desired information and the kind of required transformation while in imperative approach the programmer focuses on the way of performing the task and tracking changes in state. For more information on C#, visit the following books: C# 5 First Look (https://www.packtpub.com/application-development/c-5-first-look) C# Multithreaded and Parallel Programming (https://www.packtpub.com/application-development/c-multithreaded-and-parallel-programming) C# 6 and .NET Core 1.0: Modern Cross-Platform Development (https://www.packtpub.com/application-development/c-6-and-net-core-10) Resources for Article: Further resources on this subject: Introduction to Object-Oriented Programming using Python, JavaScript, and C#[article] C# Language Support for Asynchrony[article] C# with NGUI[article]
Read more
  • 0
  • 0
  • 68953

article-image-debugging-and-profiling-python-scripts-tutorial
Melisha Dsouza
21 Mar 2019
12 min read
Save for later

Debugging and Profiling Python Scripts [Tutorial]

Melisha Dsouza
21 Mar 2019
12 min read
Debugging and profiling play an important role in Python development. The debugger helps programmers to analyze the complete code. The debugger sets the breakpoints whereas the profilers run our code and give us the details of the execution time. The profilers will identify the bottlenecks in your programs. In this tutorial, we'll learn about the pdb Python debugger, cProfile module, and timeit module to time the execution of Python code. This tutorial is an excerpt from a book written by Ganesh Sanjiv Naik titled Mastering Python Scripting for System Administrators. This book will show you how to leverage Python for tasks ranging from text processing, network administration, building GUI, web-scraping as well as database administration including data analytics & reporting. Python debugging techniques Debugging is a process that resolves the issues that occur in your code and prevent your software from running properly. In Python, debugging is very easy. The Python debugger sets conditional breakpoints and debugs the source code one line at a time. We'll debug our Python scripts using a pdb module that's present in the Python standard library. To better debug a Python program, various techniques are available. We're going to look at four techniques for Python debugging: print() statement: This is the simplest way of knowing what's exactly happening so you can check what has been executed. logging: This is like a print statement but with more contextual information so you can understand it fully. pdb debugger: This is a commonly used debugging technique. The advantage of using pdb is that you can use pdb from the command line, within an interpreter, and within a program. IDE debugger: IDE has an integrated debugger. It allows developers to execute their code and then the developer can inspect while the program executes. Error handling (exception handling) In this section, we're going to learn how Python handles exceptions. An exception is an error that occurs during program execution. Whenever any error occurs, Python generates an exception that will be handled using a try…except block. Some exceptions can't be handled by programs so they result in error messages. Now, we are going to see some exception examples. In your Terminal, start the python3 interactive console and we will see some exception examples: student@ubuntu:~$ python3 Python 3.5.2 (default, Nov 23 2017, 16:37:01) [GCC 5.4.0 20160609] on linux Type "help", "copyright", "credits" or "license" for more information. >>> >>> 50 / 0 Traceback (most recent call last): File "<stdin>", line 1, in <module> ZeroDivisionError: division by zero >>> >>> 6 + abc*5 Traceback (most recent call last): File "<stdin>", line 1, in <module> NameError: name 'abc' is not defined >>> >>> 'abc' + 2 Traceback (most recent call last): File "<stdin>", line 1, in <module> TypeError: Can't convert 'int' object to str implicitly >>> >>> import abcd Traceback (most recent call last): File "<stdin>", line 1, in <module> ImportError: No module named 'abcd' >>> These are some examples of exceptions. Now, we will see how we can handle the exceptions. Whenever errors occur in your Python program, exceptions are raised. We can also forcefully raise an exception using raise keyword. Now we are going to see a try…except block that handles an exception. In the try block, we will write a code that may generate an exception. In the except block, we will write a solution for that exception. The syntax for try…except is as follows: try: statement(s) except: statement(s) A try block can have multiple except statements. We can handle specific exceptions also by entering the exception name after the except keyword. The syntax for handling a specific exception is as follows: try: statement(s) except exception_name: statement(s) We are going to create an exception_example.py script to catch ZeroDivisionError. Write the following code in your script: a = 35 b = 57 try: c = a + b print("The value of c is: ", c) d = b / 0 print("The value of d is: ", d) except: print("Division by zero is not possible") print("Out of try...except block") Run the script as follows and you will get the following output: student@ubuntu:~$ python3 exception_example.py The value of c is: 92 Division by zero is not possible Out of try...except block Debuggers tools There are many debugging tools supported in Python: winpdb pydev pydb pdb gdb pyDebug In this section, we are going to learn about pdb Python debugger. pdb module is a part of Python's standard library and is always available to use. The pdb debugger The pdb module is used to debug Python programs. Python programs use pdb interactive source code debugger to debug the programs. pdb sets breakpoints and inspects the stack frames, and lists the source code. Now we will learn about how we can use the pdb debugger. There are three ways to use this debugger: Within an interpreter From a command line Within a Python script We are going to create a pdb_example.py script and add the following content in that script: class Student: def __init__(self, std): self.count = std def print_std(self): for i in range(self.count): print(i) return if __name__ == '__main__': Student(5).print_std() Using this script as an example to learn Python debugging, we will see how we can start the debugger in detail. Within an interpreter To start the debugger from the Python interactive console, we are using run() or runeval(). Start your python3 interactive console. Run the following command to start the console: $ python3 Import our pdb_example script name and the pdb module. Now, we are going to use run() and we are passing a string expression as an argument to run() that will be evaluated by the Python interpreter itself: student@ubuntu:~$ python3 Python 3.5.2 (default, Nov 23 2017, 16:37:01) [GCC 5.4.0 20160609] on linux Type "help", "copyright", "credits" or "license" for more information. >>> >>> import pdb_example >>> import pdb >>> pdb.run('pdb_example.Student(5).print_std()') > <string>(1)<module>() (Pdb) To continue debugging, enter continue after the (Pdb) prompt and press Enter. If you want to know the options we can use in this, then after the (Pdb) prompt press the Tab key twice. Now, after entering continue, we will get the output as follows: student@ubuntu:~$ python3 Python 3.5.2 (default, Nov 23 2017, 16:37:01) [GCC 5.4.0 20160609] on linux Type "help", "copyright", "credits" or "license" for more information. >>> >>> import pdb_example >>> import pdb >>> pdb.run('pdb_example.Student(5).print_std()') > <string>(1)<module>() (Pdb) continue 0 1 2 3 4 >>> From a command line The simplest and most straightforward way to run a debugger is from a command line. Our program will act as input to the debugger. You can use the debugger from command line as follows: $ python3 -m pdb pdb_example.py When you run the debugger from the command line, source code will be loaded and it will stop the execution on the first line it finds. Enter continue to continue the debugging. Here's the output: student@ubuntu:~$ python3 -m pdb pdb_example.py > /home/student/pdb_example.py(1)<module>() -> class Student: (Pdb) continue 0 1 2 3 4 The program finished and will be restarted > /home/student/pdb_example.py(1)<module>() -> class Student: (Pdb) Within a Python script The previous two techniques will start the debugger at the beginning of a Python program. But this third technique is best for long-running processes. To start the debugger within a script, use set_trace(). Now, modify your pdb_example.py file as follows: import pdb class Student: def __init__(self, std): self.count = std def print_std(self): for i in range(self.count): pdb.set_trace() print(i) return if __name__ == '__main__': Student(5).print_std() Now, run the program as follows: student@ubuntu:~$ python3 pdb_example.py > /home/student/pdb_example.py(10)print_std() -> print(i) (Pdb) continue 0 > /home/student/pdb_example.py(9)print_std() -> pdb.set_trace() (Pdb) set_trace() is a Python function, therefore you can call it at any point in your program. So, these are the three ways by which you can start a debugger. Debugging basic program crashes In this section, we are going to see the trace module. The trace module helps in tracing the program execution. So, whenever your Python program crashes, we can understand where it crashes. We can use trace module by importing it into your script as well as from the command line. Now, we will create a script named trace_example.py and write the following content in the script: class Student: def __init__(self, std): self.count = std def go(self): for i in range(self.count): print(i) return if __name__ == '__main__': Student(5).go() The output will be as follows: student@ubuntu:~$ python3 -m trace --trace trace_example.py --- modulename: trace_example, funcname: <module> trace_example.py(1): class Student: --- modulename: trace_example, funcname: Student trace_example.py(1): class Student: trace_example.py(2): def __init__(self, std): trace_example.py(5): def go(self): trace_example.py(10): if __name__ == '__main__': trace_example.py(11): Student(5).go() --- modulename: trace_example, funcname: init trace_example.py(3): self.count = std --- modulename: trace_example, funcname: go trace_example.py(6): for i in range(self.count): trace_example.py(7): print(i) 0 trace_example.py(6): for i in range(self.count): trace_example.py(7): print(i) 1 trace_example.py(6): for i in range(self.count): trace_example.py(7): print(i) 2 trace_example.py(6): for i in range(self.count): trace_example.py(7): print(i) 3 trace_example.py(6): for i in range(self.count): trace_example.py(7): print(i) 4 So, by using trace --trace at the command line, the developer can trace the program line-by-line. So, whenever the program crashes, the developer will know the instance where it crashes. Profiling and timing programs Profiling a Python program means measuring an execution time of a program. It measures the time spent in each function. Python's cProfile module is used for profiling a Python program. The cProfile module As discussed previously, profiling means measuring the execution time of a program. We are going to use the cProfile Python module for profiling a program. Now, we will write a cprof_example.py script and write the following code in it: mul_value = 0 def mul_numbers( num1, num2 ): mul_value = num1 * num2; print ("Local Value: ", mul_value) return mul_value mul_numbers( 58, 77 ) print ("Global Value: ", mul_value) Run the program and you will see the output as follows: student@ubuntu:~$ python3 -m cProfile cprof_example.py Local Value: 4466 Global Value: 0 6 function calls in 0.000 seconds Ordered by: standard name ncalls tottime percall cumtime percall filename:lineno(function) 1 0.000 0.000 0.000 0.000 cprof_example.py:1(<module>) 1 0.000 0.000 0.000 0.000 cprof_example.py:2(mul_numbers) 1 0.000 0.000 0.000 0.000 {built-in method builtins.exec} 2 0.000 0.000 0.000 0.000 {built-in method builtins.print} 1 0.000 0.000 0.000 0.000 {method 'disable' of '_lsprof.Profiler' objects} So, using cProfile, all functions that are called will get printed with the time spent on each function. Now, we will see what these column headings mean: ncalls: Number of calls tottime: Total time spent in the given function percall: Quotient of tottime divided by ncalls cumtime: Cumulative time spent in this and all subfunctions percall: Quotient of cumtime divided by primitive calls filename:lineno(function): Provides the respective data of each function timeit timeit is a Python module used to time small parts of your Python script. You can call timeit from the command line as well as import the timeit module into your script. We are going to write a script to time a piece of code. Create a timeit_example.py script and write the following content into it: import timeit prg_setup = "from math import sqrt" prg_code = ''' def timeit_example(): list1 = [] for x in range(50): list1.append(sqrt(x)) ''' # timeit statement print(timeit.timeit(setup = prg_setup, stmt = prg_code, number = 10000)) Using timeit, we can decide what piece of code we want to measure the performance of. So, we can easily define the setup code as well as the code snippet on which we want to perform the test separately. The main code runs 1 million times, which is the default time, whereas the setup code runs only once. Making programs run faster There are various ways to make your Python programs run faster, such as the following: Profile your code so you can identify the bottlenecks Use built-in functions and libraries so the interpreter doesn't need to execute loops Avoid using globals as Python is very slow in accessing global variables Use existing packages Summary In this tutorial, we learned about the importance of debugging and profiling programs. We learned what the different techniques available for debugging are. We learned about the pdb Python debugger and how to handle exceptions and how to use the cProfile and timeit modules of Python while profiling and timing our scripts. We also learned how to make your scripts run faster. To learn how to to use the latest features of Python and be able to build powerful tools that will solve challenging, real-world tasks, check out our book Mastering Python Scripting for System Administrators. 5 blog posts that could make you a better Python programmer Using Python Automation to interact with network devices [Tutorial] 4 tips for learning Data Visualization with Python
Read more
  • 0
  • 0
  • 68721
article-image-brute-forcing-http-applications-and-web-applications-using-nmap-tutorial
Savia Lobo
11 Nov 2018
6 min read
Save for later

Brute forcing HTTP applications and web applications using Nmap [Tutorial]

Savia Lobo
11 Nov 2018
6 min read
Many home routers, IP webcams, and web applications still rely on HTTP authentication these days, and we, as system administrators or penetration testers, need to make sure that the system or user accounts are not using weak credentials. Now, thanks to the NSE script http-brute, we can perform robust dictionary attacks against HTTP basic, digest, and NTLM authentication. This article is an excerpt taken from the book Nmap: Network Exploration and Security Auditing Cookbook - Second Edition, written by Paulino Calderon. This book includes the basic usage of Nmap and related tools like Ncat, Ncrack, Ndiff, and Zenmap and much more. In this article, we will learn how to perform brute force password auditing against web servers that are using HTTP authentication and also against popular and custom web applications with Nmap. Brute forcing HTTP applications How to do it... Use the following Nmap command to perform brute force password auditing against a resource protected by HTTP's basic authentication: $ nmap -p80 --script http-brute <target> The results will return all the valid accounts that were found (if any): PORT STATE SERVICE REASON 80/tcp open http syn-ack | http-brute: | Accounts | admin:secret => Valid credentials | Statistics |_ Perfomed 603 guesses in 7 seconds, average tps: 86 How it works... The Nmap options -p80 --script http-brute tells Nmap to launch the http-brute script against the web server running on port 80. This script was originally committed by Patrik Karlsson, and it was created to launch dictionary attacks against URIs protected by HTTP authentication. The http-brute script uses, by default, the database files usernames.lst and passwords.lst located at /nselib/data/ to try each password, for every user, to hopefully find a valid account. There's more... The script http-brute depends on the NSE libraries unpwdb and brute. Read the Appendix B, Brute Force Password Auditing Options, for more information. To use different username and password lists, set the arguments userdb and passdb: $ nmap -p80 --script http-brute --script-args userdb=/var/usernames.txt,passdb=/var/passwords.txt <target> To quit after finding one valid account, use the argument brute.firstOnly: $ nmap -p80 --script http-brute --script-args brute.firstOnly <target> By default, http-brute uses Nmap's timing template to set the following timeout limits: -T3,T2,T1: 10 minutes -T4: 5 minutes -T5: 3 minutes For setting a different timeout limit, use the argument unpwd.timelimit. To run it indefinitely, set it to 0: $ nmap -p80 --script http-brute --script-argsunpwdb.timelimit=0 <target> $ nmap -p80 --script http-brute --script-args unpwdb.timelimit=60m <target> Brute modes The brute library supports different modes that alter the combinations used in the attack. The available modes are: user: In this mode, for each user listed in userdb, every password in passdb will be tried: $ nmap --script http-brute --script-args brute.mode=user <target> pass: In this mode, for each password listed in passdb, every user in userdb will be tried: $ nmap --script http-brute --script-args brute.mode=pass <target> creds: This mode requires the additional argument brute.credfile: $ nmap --script http-brute --script-args brute.mode=creds,brute.credfile=./creds.txt <target> Brute forcing web applications Performing brute force password auditing against web applications is an essential step to evaluate the password strength of system accounts. There are powerful tools such as THC Hydra, but Nmap offers great flexibility as it is fully configurable and contains a database of popular web applications, such as WordPress, Joomla!, Django, Drupal, MediaWiki, and WebSphere. How to do it... Use the following Nmap command to perform brute force password auditing against web applications using forms: $ nmap --script http-form-brute -p 80 <target> If credentials are found, they will be shown in the results: PORT STATE SERVICE REASON 80/tcp open http syn-ack | http-form-brute: | Accounts | user:secret - Valid credentials | Statistics |_ Perfomed 60023 guesses in 467 seconds, average tps: 138   How it works... The Nmap options -p80 --script http-form-brute tells Nmap to launch the http-form-brute script against the web server running on port 80. This script was originally committed by Patrik Karlsson, and it was created to launch dictionary attacks against authentication systems based on web forms. The script automatically attempts to detect the form fields required to authenticate, and it uses internally a database of popular web applications to help during the form detection phase. There's more... The script http-form-brute depends on the correct detection of the form fields. Often you will be required to manually set via script arguments the name of the fields holding the username and password variables. If the script argument http-form-brute.passvar is set, form detection will not be performed: $ nmap -p80 --script http-form-brute --script-args http-form-brute.passvar=contrasenia,http-form-brute.uservar=usuario <target> In a similar way, often you will need to set the script arguments http-form-brute.onsuccess or http-form-brute.onfailure to set the success/error messages returned when attempting to authenticate: $nmap -p80 --script http-form-brute --script-args http-form-brute.onsuccess=Exito <target> Brute forcing WordPress installations If you are targeting a popular application, remember to check whether there are any NSE scripts specialized on attacking them. For example, WordPress installations can be audited with the script http-wordpress-brute: $ nmap -p80 --script http-wordpress-brute <target> To set the number of threads, use the script argument http-wordpress-brute.threads: $ nmap -p80 --script http-wordpress-brute --script-args http-wordpress-brute.threads=5 <target>   If the server has virtual hosting, set the host field using the argument http-wordpress-brute.hostname: $ nmap -p80 --script http-wordpress-brute --script-args http-wordpress-brute.hostname="ahostname.wordpress.com" <target> To set a different login URI, use the argument http-wordpress-brute.uri: $ nmap -p80 --script http-wordpress-brute --script-args http-wordpress-brute.uri="/hidden-wp-login.php" <target> To change the name of the POST variable that stores the usernames and passwords, set the arguments http-wordpress-brute.uservar and http-wordpress-brute.passvar: $ nmap -p80 --script http-wordpress-brute --script-args http-wordpress-brute.uservar=usuario,http-wordpress-brute.passvar=pasguord <target> Brute forcing WordPress installations Another good example of a specialized NSE brute force script is http-joomla-brute. This script is designed to perform brute force password auditing against Joomla! installations. By default, our generic brute force script for HTTP will fail against Joomla! CMS since the application generates dynamically a security token, but this NSE script will automatically fetch it and include it in the login requests. Use the following Nmap command to launch the script: $ nmap -p80 --script http-joomla-brute <target> To set the number of threads, use the script argument http-joomla-brute.threads: $ nmap -p80 --script http-joomla-brute --script-args http-joomla-brute.threads=5 <target> To change the name of the POST variable that stores the login information, set the arguments http-joomla-brute.uservar and http-joomla-brute.passvar: $ nmap -p80 --script http-joomla-brute --script-args http-joomla-brute.uservar=usuario,http-joomla-brute.passvar=pasguord <target> To summarize, we learned how to brute force password auditing against web servers custom web applications with Nmap. If you've enjoyed reading this post, do check out our book, Nmap: Network Exploration and Security Auditing Cookbook - Second Edition to know more to learn about Lua programming and NSE script development which will allow you to further extend the power of Nmap. Discovering network hosts with ‘TCP SYN’ and ‘TCP ACK’ ping scans in Nmap [Tutorial] Introduction to the Nmap Scripting Engine Exploring the Nmap Scripting Engine API and Libraries
Read more
  • 0
  • 0
  • 68240

article-image-conditional-statements-functions-and-lists
Packt
05 Apr 2017
14 min read
Save for later

Conditional Statements, Functions, and Lists

Packt
05 Apr 2017
14 min read
In this article, by Sai Yamanoor and Srihari Yamanoor, author of the book Python Programming with Raspberry Pi Zero, you will learn about conditional statements and how to make use of logical operators to check conditions using conditional statements. Next, youwill learn to write simple functions in Python and discuss interfacing inputs to the Raspberry Pi's GPIO header using a tactile switch (momentary push button). We will also discuss motor control (this is a run-up to the final project) using the Raspberry Pi Zero and control the motors using the switch inputs. Let's get to it! In this article, we will discuss the following topics: Conditional statements in Python Using conditional inputs to take actions based on GPIO pin states Breaking out of loops using conditional statement Functions in Python GPIO callback functions Motor control in Python (For more resources related to this topic, see here.) Conditional statements In Python, conditional statements are used to determine if a specific condition is met by testing whether a condition is True or False. Conditional statements are used to determine how a program is executed. For example,conditional statements could be used to determine whether it is time to turn on the lights. The syntax is as follows: if condition_is_true: do_something() The condition is usually tested using a logical operator, and the set of tasks under the indented block is executed. Let's consider the example,check_address_if_statement.py where the user input to a program needs to be verified using a yesor no question: check_address = input("Is your address correct(yes/no)? ") if check_address == "yes": print("Thanks. Your address has been saved") if check_address == "no": del(address) print("Your address has been deleted. Try again")check_address = input("Is your address correct(yes/no)? ") In this example, the program expects a yes or no input. If the user provides the input yes, the condition if check_address == "yes"is true, the message Your address has been savedis printed on the screen. Likewise, if the user input is no, the program executes the indented code block under the logical test condition if check_address == "no" and deletes the variable address. An if-else statement In the precedingexample, we used an ifstatement to test each condition. In Python, there is an alternative option named the if-else statement. The if-else statement enables testing an alternative condition if the main condition is not true: check_address = input("Is your address correct(yes/no)? ") if check_address == "yes": print("Thanks. Your address has been saved") else: del(address) print("Your address has been deleted. Try again") In this example, if the user input is yes, the indented code block under if is executed. Otherwise, the code block under else is executed. An if-elif-else statement In the precedingexample, the program executes any piece of code under the else block for any user input other than yesthat is if the user pressed the return key without providing any input or provided random characters instead of no, the if-elif-else statement works as follows: check_address = input("Is your address correct(yes/no)? ") if check_address == "yes": print("Thanks. Your address has been saved") elifcheck_address == "no": del(address) print("Your address has been deleted. Try again") else: print("Invalid input. Try again") If the user input is yes, the indented code block under the ifstatement is executed. If the user input is no, the indented code block under elif (else-if) is executed. If the user input is something else, the program prints the message: Invalid input. Try again. It is important to note that the code block indentation determines the block of code that needs to be executed when a specific condition is met. We recommend modifying the indentation of the conditional statement block and find out what happens to the program execution. This will help understand the importance of indentation in python. In the three examples that we discussed so far, it could be noted that an if-statement does not need to be complemented by an else statement. The else and elif statements need to have a preceding if statement or the program execution would result in an error. Breaking out of loops Conditional statements can be used to break out of a loop execution (for loop and while loop). When a specific condition is met, an if statement can be used to break out of a loop: i = 0 while True: print("The value of i is ", i) i += 1 if i > 100: break In the precedingexample, the while loop is executed in an infinite loop. The value of i is incremented and printed onthe screen. The program breaks out of the while loop when the value of i is greater than 100 and the value of i is printed from 1 to 100. The applications of conditional statements: executing tasks using GPIO Let's discuss an example where a simple push button is pressed. A button press is detected by reading the GPIO pin state. We are going to make use of conditional statements to execute a task based on the GPIO pin state. Let us connect a button to the Raspberry Pi's GPIO. All you need to get started are a button, pull-up resistor, and a few jumper wires. The figure given latershows an illustration on connecting the push button to the Raspberry Pi Zero. One of the push button's terminals is connected to the ground pin of the Raspberry Pi Zero's GPIO pin. The schematic of the button's interface is shown here: Raspberry Pi GPIO schematic The other terminal of the push button is pulled up to 3.3V using a 10K resistor.The junction of the push button terminal and the 10K resistor is connected to the GPIO pin 2. Interfacing the push button to the Raspberry Pi Zero's GPIO—an image generated using Fritzing Let's review the code required to review the button state. We make use of loops and conditional statements to read the button inputs using the Raspberry Pi Zero. We will be making use of the gpiozero. For now, let’s briefly discuss the concept of classes for this example. A class in Python is a blueprint that contains all the attributes that define an object. For example, the Button class of the gpiozero library contains all attributes required to interface a button to the Raspberry Pi Zero’s GPIO interface.These attributes include button states and functions required to check the button states and so on. In order to interface a button and read its states, we need to make use of this blueprint. The process of creating a copy of this blueprint is called instantiation. Let's get started with importing the gpiozero library and instantiate the Button class of the gpiozero. The button is interfaced to GPIO pin 2. We need to pass the pin number as an argument during instantiation: from gpiozero import Button #button is interfaced to GPIO 2 button = Button(2) The gpiozero library's documentation is available athttp://gpiozero.readthedocs.io/en/v1.2.0/api_input.html.According to the documentation, there is a variable named is_pressed in the Button class that could be tested using a conditional statement to determine if the button is pressed: if button.is_pressed: print("Button pressed") Whenever the button is pressed, the message Button pressed is printed on the screen. Let's stick this code snippet inside an infinite loop: from gpiozero import Button #button is interfaced to GPIO 2 button = Button(2) while True: if button.is_pressed: print("Button pressed") In an infinite while loop, the program constantly checks for a button press and prints the message as long as the button is being pressed. Once the button is released, it goes back to checking whether the button is pressed. Breaking out a loop by counting button presses Let's review another example where we would like to count the number of button presses and break out of the infinite loop when the button has received a predetermined number of presses: i = 0 while True: if button.is_pressed: button.wait_for_release() i += 1 print("Button pressed") if i >= 10: break In this example, the program checks for the state of the is_pressed variable. On receiving a button press, the program can be paused until the button is released using the method wait_for_release.When the button is released, the variable used to store the number of presses is incremented by 1. The program breaks out of the infinite loop, when the button has received 10 presses. A red momentary push button interfaced to Raspberry Pi Zero GPIO pin 2 Functions in Python We briefly discussed functions in Python. Functions execute a predefined set of task. print is one example of a function in Python. It enables printing something to the screen. Let's discuss writing our own functions in Python. A function can be declared in Python using the def keyword. A function could be defined as follows: defmy_func(): print("This is a simple function") In this function my_func, the print statement is written under an indented code block. Any block of code that is indented under the function definition is executed when the function is called during the code execution. The function could be executed as my_func(). Passing arguments to a function: A function is always defined with parentheses. The parentheses are used to pass any requisite arguments to a function. Arguments are parameters required to execute a function. In the earlierexample, there are no arguments passed to the function. Let's review an example where we pass an argument to a function: defadd_function(a, b): c = a + b print("The sum of a and b is ", c) In this example, a and b are arguments to the function. The function adds a and b and prints the sum onthe screen. When the function add_function is called by passing the arguments 3 and 2 as add_function(3,2) where a=3 and b=2, respectively. Hence, the arguments a and b are required to execute function, or calling the function without the arguments would result in an error. Errors related to missing arguments could be avoided by setting default values to the arguments: defadd_function(a=0, b=0): c = a + b print("The sum of a and b is ", c) The preceding function expects two arguments. If we pass only one argument to thisfunction, the other defaults to zero. For example,add_function(a=3), b defaults to 0, or add_function(b=2), a defaults to 0. When an argument is not furnished while calling a function, it defaults to zero (declared in the function). Similarly,the print function prints any variable passed as an argument. If the print function is called without any arguments, a blank line is printed. Returning values from a function Functions can perform a set of defined operations and finally return a value at the end. Let's consider the following example: def square(a): return a**2 In this example, the function returns a square of the argument. In Python, the return keyword is used to return a value requested upon completion of execution. The scope of variables in a function There are two types of variables in a Python program:local and global variables. Local variables are local to a function,that is, it is a variable declared within a function is accessible within that function.Theexample is as follows: defadd_function(): a = 3 b = 2 c = a + b print("The sum of a and b is ", c) In this example, the variables a and b are local to the function add_function. Let's consider an example of a global variable: a = 3 b = 2 defadd_function(): c = a + b print("The sum of a and b is ", c) add_function() In this case, the variables a and b are declared in the main body of the Python script. They are accessible across the entire program. Now, let's consider this example: a = 3 defmy_function(): a = 5 print("The value of a is ", a) my_function() print("The value of a is ", a) In this case, when my_function is called, the value of a is5 and the value of a is 3 in the print statement of the main body of the script. In Python, it is not possible to explicitly modify the value of global variables inside functions. In order to modify the value of a global variable, we need to make use of the global keyword: a = 3 defmy_function(): global a a = 5 print("The value of a is ", a) my_function() print("The value of a is ", a) In general, it is not recommended to modify variables inside functions as it is not a very safe practice of modifying variables. The best practice would be passing variables as arguments and returning the modified value. Consider the following example: a = 3 defmy_function(a): a = 5 print("The value of a is ", a) return a a = my_function(a) print("The value of a is ", a) In the preceding program, the value of a is 3. It is passed as an argument to my_function. The function returns 5, which is saved to a. We were able to safely modify the value of a. GPIO callback functions Let's review some uses of functions with the GPIO example. Functions can be used in order tohandle specific events related to the GPIO pins of the Raspberry Pi. For example,the gpiozero library provides the capability of calling a function either when a button is pressed or released: from gpiozero import Button defbutton_pressed(): print("button pressed") defbutton_released(): print("button released") #button is interfaced to GPIO 2 button = Button(2) button.when_pressed = button_pressed button.when_released = button_released while True: pass In this example, we make use of the attributeswhen_pressed and when_releasedof the library's GPIO class. When the button is pressed, the function button_pressed is executed. Likewise, when the button is released, the function button_released is executed. We make use of the while loop to avoid exiting the program and keep listening for button events. The pass keyword is used to avoid an error and nothing happens when a pass keyword is executed. This capability of being able to execute different functions for different events is useful in applications like Home Automation. For example, it could be used to turn on lights when it is dark and vice versa. DC motor control in Python In this section, we will discuss motor control using the Raspberry Pi Zero. Why discuss motor control? In order to control a motor, we need an H Bridge motor driver (Discussing H bridge is beyond our scope. There are several resources for H bridge motor drivers: http://www.mcmanis.com/chuck/robotics/tutorial/h-bridge/). There are several motor driver kits designed for the Raspberry Pi. In this section, we will make use of the following kit: https://www.pololu.com/product/2753. The Pololu product page also provides instructions on how to connect the motor. Let's get to writing some Python code tooperate the motor: from gpiozero import Motor from gpiozero import OutputDevice import time motor_1_direction = OutputDevice(13) motor_2_direction = OutputDevice(12) motor = Motor(5, 6) motor_1_direction.on() motor_2_direction.on() motor.forward() time.sleep(10) motor.stop() motor_1_direction.off() motor_2_direction.off() Raspberry Pi based motor control In order to control the motor, let's declare the pins, the motor's speed pins and direction pins. As per the motor driver's documentation, the motors are controlled by GPIO pins 12,13 and 5,6, respectively. from gpiozero import Motor from gpiozero import OutputDevice import time motor_1_direction = OutputDevice(13) motor_2_direction = OutputDevice(12) motor = Motor(5, 6) Controlling the motor is as simple as turning on the motor using the on() method and moving the motor in the forward direction using the forward() method: motor.forward() Similarly, reversing the motor direction could be done by calling the method reverse(). Stopping the motor could be done by: motor.stop() Some mini-project challenges for the reader: In this article, we discussed interfacing inputs for the Raspberry Pi and controlling motors. Think about a project where we could drive a mobile robot that reads inputs from whisker switches and operate a mobile robot. Is it possible to build a wall following robot in combination with the limit switches and motors? We discussed controlling a DC motor in this article. How do we control a stepper motor using a Raspberry Pi? How can we interface a motion sensor to control the lights at home using a Raspberry Pi Zero. Summary In this article, we discussed conditional statements and the applications of conditional statements in Python. We also discussed functions in Python, passing arguments to a function, returning values from a function and scope of variables in a Python program. We discussed callback functions and motor control in Python. Resources for Article: Further resources on this subject: Sending Notifications using Raspberry Pi Zero [article] Raspberry Pi Gaming Operating Systems [article] Raspberry Pi LED Blueprints [article]
Read more
  • 0
  • 0
  • 67979
Modal Close icon
Modal Close icon