Search icon CANCEL
Subscription
0
Cart icon
Your Cart (0 item)
Close icon
You have no products in your basket yet
Arrow left icon
Explore Products
Best Sellers
New Releases
Books
Events
Videos
Audiobooks
Packt Hub
Free Learning
Arrow right icon
timer SALE ENDS IN
0 Days
:
00 Hours
:
00 Minutes
:
00 Seconds

How-To Tutorials

7019 Articles
article-image-visualize
Packt
19 Feb 2015
17 min read
Save for later

Visualize This!

Packt
19 Feb 2015
17 min read
This article is written by Michael Phillips, the author of the book TIBCO Spotfire: A Comprehensive Primer, discusses that human beings are fundamentally visual in the way they process information. The invention of writing was as much about visually representing our thoughts to others as it was about record keeping and accountancy. In the modern world, we are bombarded with formalized visual representations of information, from the ubiquitous opinion poll pie chart to clever and sophisticated infographics. The website http://data-art.net/resources/history_of_vis.php provides an informative and entertaining quick history of data visualization. If you want a truly breathtaking demonstration of the power of data visualization, seek out Hans Rosling's The best stats you've ever seen at http://ted.com. (For more resources related to this topic, see here.) We will spend time getting to know some of Spotfire's data capabilities. It's important that you continue to think about data; how it's structured, how it's related, and where it comes from. Building good visualizations requires visual imagination, but it also requires data literacy. This article is all about getting you to think about the visualization of information and empowering you to use Spotfire to do so. Apart from learning the basic features and properties of the various Spotfire visualization types, there is much more to learn about the seamless interactivity that Spotfire allows you to build in to your analyses. We will be taking a close look at 7 of the 16 visualization types provided by Spotfire, but these 7 visualization types are the most commonly used. We will cover the following topics: Displaying information quickly in tabular form Enriching your visualizations with color categorization Visualizing categorical information using bar charts Dividing a visualization across a trellis grid Key Spotfire concept—marking Visualizing trends using line charts Visualizing proportions using pie charts Visualizing relationships using scatter plots Visualizing hierarchical relationships using treemaps Key Spotfire concept—filters Enhancing tabular presentations using graphical tables Now let's have some fun! Displaying information quickly in tabular form While working through the data examples, we used the Spotfire Table visualization, but now we're going to take a closer look. People will nearly always want to see the "underlying data", the details behind any visualization you create. The Table visualization meets this need. It's very important not to confuse a table in the general data sense with the Spotfire Table visualization; the underlying data table remains immutable and complete in the background. The Table visualization is a highly manipulatable view of the underlying data table and should be treated as a visualization, not a data table. The data used here is BaseballPlayerData.xls There is always more than one way to do the same thing in Spotfire, and this is particularly true for the manipulation of visualizations. Let's start with some very quick manipulations: First, insert a table visualization by going to the Insert menu, selecting New Visualization, and then Table. To move a column, left-click on the column name, hold, and drag it. To sort by a column, left-click on the column name. To sort by more than one column, left-click on the first column name and then press Shift + left-click on the subsequent columns in order of sort precedence. To widen or narrow a column, hover the mouse over the right-hand edge of the column title until you see the cursor change to a two-way arrow, and then click and drag it. These and other properties of the Table visualization are also accessed via visualization properties. As you work through the various Spotfire visualizations, you'll notice that some types have more options than others, but there are common trends and an overall consistency in conventions. Visualization properties can be opened in a number of ways: By right-clicking on the visualization, a table in this case, and selecting Properties. By going to the Edit menu and selecting Visualization Properties. By clicking on the Visualization Properties icon, as shown in the following screenshot, in the icon tray below the main menu bar. It's beyond the scope of this book to explore every property and option. The context-sensitive help provided by Spotfire is excellent and explains all the options in glorious detail. I'd like to highlight four important properties of the Table visualization: The General property allows you to change the table visualization title, not the name of the underlying data table. It also allows you to hide the title altogether. The Data property allows you to switch the underlying data table, if you have more than one table loaded into your analysis. The Columns property allows you to hide columns and order the columns you do want to show. The Show/Hide Items property allows you to limit what is shown by a rule you define, such as top five hitters. After clicking on the Add button, you select the relevant column from a dropdown list, choose Rule type (Top), and finally, choose Value for the rule (5). The resulting visualization will only show the rows of data that meet the rule you defined. Enriching your visualizations with color categorization Color is a strong feature in Spotfire and an important visualization tool, often underestimated by report creators. It can be seen as merely a nice-to-have customization, but paying attention to color can be the difference between creating a stimulating and intuitive data visualization rather than an uninspiring and even confusing corporate report. Take some pride and care in the visual aesthetics of your analytics creations! Let's take a look at the color properties of the Table visualization. Open the Table visualization properties again, select Colors, and then Add the column Runs. Now, you can play with a color gradient, adding points by clicking on the Add Point button and customizing the colors. It's as easy as left-clicking on any color box and then selecting from a prebuilt palette or going into a full RGB selection dialog by choosing More Colors…. The result is a heatmap type effect for runs scored, with yellow representing low run totals, transitioning to green as the run total approaches the average value in the data, and becoming blue for the highest run totals. Visualizing categorical information using bar charts We saw how the Table visualization is perfect for showing and ordering detailed information. It's quite similar to a spreadsheet. The Bar Chart visualization is very good for visualizing categorical information, that is, where you have categories with supporting hard numbers—sales by region, for example. The region is the category, whereas the sales is the hard number or fact. Bar charts are typically used to show a distribution. Depending on your data or your analytic requirement, the bars can be ordered by value, placed side by side, stacked on top of each other, or arranged vertically or horizontally. There is a special case of the category and value combination and that is where you want to plot the frequencies of a set of numerical values. This type of bar chart is referred to as a histogram, and although it is number against number, it is still, in essence, a distribution plot. It is very common in fact to transform the continuous number range in such cases into a set of discrete bins or categories for the plot. For example, you could take some demographic data and plot age as the category and the number of people at that age as the value (the frequency) on a bar chart. The result, for a general population, would approach a bell-shaped curve. Let's create a bar chart using the baseball data. The data we will use is BaseballPlayerData.xls, which you can download from http://www.insidespotfire.com. Create a new page by right-clicking on any page tab and selecting New Page. You can also select New Page from the Insert menu or click on the new page icon in the icon bar below the main menu. Create a Bar Chart visualization by left-clicking on the bar chart icon or by selecting New Visualization and then Bar Chart from the Insert menu. Spotfire will automatically create a default chart, that is, rarely exactly what you want, so the next step is to configure the chart. Two distributions might be interesting to look at: the distribution of home runs across all the teams and the distribution of player salaries across all the teams. The axes are easy to change; simply use the axes selectors.   If the bars are vertical, it means that the category—Team, in our case—should be on the horizontal axis, with the value—Home Runs or Salary—on the vertical axis, representing the height of the bars.   We're going to pick Home Runs from the vertical axis selector and then an appropriate aggregation dropdown, which is highlighted in red in the screenshot. Sum would be a valid option, but let's go with Avg (Average). Similarly, select Team from the horizontal axis dropdown selector. The vertical, or value, axis must be an aggregation because there is more than one home run value for each category. You must decide if you want a sum, an average, a minimum, and so on. You can modify the visualization properties just as you did for the Table visualization. Some of the options are the same; some are specific to the bar chart. We're going to select the Sort bars by value option in the Appearance property. This will order the bars in descending order of value. We're also going to check the option Vertically under Scale labels | Show labels for the Category Axis property. There are two more actions to perform: create an identical bar chart except with average salary as the value axis, and give each bar chart an appropriate title (Visualization Properties|General|Title:). To copy an existing visualization, simply right-click on it and select Duplicate Visualization. We can now compare the distribution of home run average and salary average across all the baseball teams, but there's a better way to do this in a single visualization using color. Close the salary distribution bar chart by left-clicking on X in the upper right-hand corner of the visualization (X appears when you hover the mouse) or right-clicking on the visualization and selecting Close. Now, open the home run bar chart visualization properties, go to the Colors property, and color by Avg(Salary). Select a Gradient color mode, and add a median point by clicking on the Add Point button and selecting Median from the dropdown list of options on the added point. Finally, choose a suitable heat map range of colors; something like blue (min) through pale yellow (median) through red (max). You will still see the distribution of home runs across the baseball teams, but now you will have a superimposed salary heat map. Texas and Cleveland appear to be getting much more bang for their buck than the NY Yankees. Dividing a visualization across a trellis grid Trellising, whereby you divide a series of visualizations into individual panels, is a useful technique when you want to subdivide your analysis. In the example we've been working with, we might, for instance, want to split the visualization by league. Open the visualization properties for the home runs distribution bar chart colored by salary and select the Trellis property. Go to Panels and split by League (use the dropdown column selector). Spotfire allows you to build layers of information with even basic visualizations such as the bar chart. In one chart, we see the home run distribution by team, salary distribution by team, and breakdown by league. Key Spotfire concept – marking It's time to introduce one of the most important Spotfire concepts, called marking, which is central to the interactivity that makes Spotfire such a powerful analysis tool. Marking refers to the action of selecting data in a visualization. Every element you see is selectable, or markable, that is, a single row or multiple rows in a table, a single bar or multiple bars in a bar chart. You need to understand two aspects to marking. First, there is the visual effect, or color(s) you see, when you mark (select) visualization elements. Second, there is the behavior that follows marking: what happens to data and the display of data when you mark something. How to change the marking color From Spotfire v5.5 onward, you can choose, on a visualization-by-visualization basis, two distinct visual effects for marking: Use a separate color for marked items: all marked items are uniformly colored with the marking color, and all unmarked items retain their existing color. Keep existing color attributes and fade out unmarked items: all marked items keep their existing color, and all unmarked items also keep their existing color but with a high degree of color fade applied, leaving the marked items strongly highlighted. The second option is not available in versions older than v5.5 but is the default option in Versions 5.5 onward. The setting is made in the visualization's Appearance property by checking or unchecking the option Use separate color for marked items. The default color when using a separate color for marked items is dark green, but this can be changed by going to Edit|Document Properties|Markings|Edit. The new option has the advantage of retaining any underlying coloring you defined, but you might not like how the rest of the chart is washed out. Which approach you choose depends on what information you think is critical for your particular situation. When you create a new analysis, a default marking is created and applied to every visualization you create by default. You can change the color of the marking in Document Properties, which is found in the Edit menu. Just open Document Properties, click on the Markings tab, select the marking, click on the Edit button, and change the color. You can also create as many markings as you need, giving them convenient names for reference purposes, but we'll just focus on using one for now. How to set the marking behavior of a visualization Marking behavior depends fundamentally on data relationships. The data within a single data table is intrinsically related; the data in separate data tables must be explicitly related before you configure marking behavior for visualizations based on separate datasets. When you mark something in a visualization, five things can happen depending on the data involved and how you configured your visualizations: Conditions Behavior Two visualizations with the same underlying data table (they can be on different pages in the analysis file) and the same marking scheme applied. Marking data on one visualization will automatically mark the same data on the other. Two visualizations with related underlying data tables and the same marking scheme applied. The same as the previous condition's behavior, but subject to differences in data granularity. For example, marking a baseball team in one visualization will mark all the team's players in another visualization that is based on a more detailed table related by team. Two visualizations with the same or related data tables where one has been configured with data dependency on the marking in the other. Nothing will display in the marking-dependent visualization other than what is marked in the reference visualization. Visualizations with unrelated underlying data tables. No marking interaction will occur, and the visualizations will mark completely independently of one another. Two visualizations with the same underlying data table or related data tables and with different marking schemes applied. Marking data on one visualization will not show on the other because the marking schemes are different. Here's how we set these behaviors: Open the visualization properties of the bar chart we have been working with and navigate to the Data property.   You'll notice that two settings refer to marking: Marking and Limit data using markings. Use the dropdown under Marking to select the marking to be used for the visualization. Having no marking is an option. Visualizations with the same marking will display synchronous selection, subject to the data relation conditions described earlier. The options under Limit data using markings determine how the visualization will be limited to marking elsewhere in the analysis. The default here is no dependency. If you select a marking, then the visualization will only display data selected elsewhere with that marking. It's not good to have the same marking for Marking and Limit data using markings. If you are using the limit data setting, select no marking, or create a second marking and select it under Marking. You're possibly a bit confused by now. Fortunately, marking is much harder to describe than to use! Let's build a tangible example. We'll start a new analysis, so close any analysis you have open and create a new one, loading the player-level baseball data (BaseballPlayerData.xls). Add two bar charts and a table. You can rearrange the layout by left-clicking on the title bar of a visualization, holding, and dragging it. Position the visualizations any way you wish, but you can place the two bar charts side by side, with the table below them spanning both. Save your analysis file at this point and at regular intervals. It's good behavior to save regularly as you build an analysis. It will save you a lot of grief if your PC fails in any way. There is no autosave function in Spotfire. For the first bar chart, set the following visualization properties: Property Value General | Title Home Runs Data | Marking Marking Data | Limit data using markings Nothing checked Appearance | Orientation Vertical bars Appearance | Sort bars by value Check Category Axis | Columns Team Value Axis | Columns Avg(Home Runs) Colors | Columns Avg(Salary) Colors | Color mode Gradient Add Point for median Max = strong red; Median = pale yellow; Min = strong blue Labels | Show labels for Marked Rows Labels | Types of labels | Complete bar Check For the second bar chart, set the following visualization properties: Property Value General | Title Roster Data | Marking Marking Data | Limit data using markings Nothing checked Appearance | Orientation Horizontal bars Appearance | Sort bars by value Check Category Axis | Columns Team Value Axis | Columns Count(Player Name) Colors | Columns Position Colors | Color mode Categorical For the table, set the following visualization properties: Property Value General | Title Details Data | Marking (None) Data | Limit data using markings Check Marking   Columns Team, Player Name, Games Played, Home Runs, Salary, Position Now start selecting visualization elements with your mouse. You can click on elements such as bars or segments of bars, or you can click and drag a rectangular block around multiple elements. When you select a bar on the Home Runs bar chart, the corresponding team bar automatically selects the Roster bar chart, and details for all the players in that team display in the Details table. When you select a bar segment on the Roster bar chart, the corresponding team bar automatically selects on the Home Runs bar chart and only players in the selected position for the team selected appear in the details. There are some very useful additional functions associated with marking, and you can access these by right-clicking on a marked item. They are Unmark, Invert, Delete, Filter To, and Filer Out. You can also unmark by left-clicking on any blank space in the visualization. Play with this analysis file until you are comfortable with the marking concept and functionality. Summary This article is a small taste of the book TIBCO Spotfire: A comprehensive primer. You've seen how the Table visualization is an easy and traditional way to display detailed information in tabular form and how the Bar Chart visualization is excellent for visualizing categorical information, such as distributions. You've learned how to enrich visualizations with color categorization and how to divide a visualization across a trellis grid. You've also been introduced to the key Spotfire concept of marking. Apart from gaining a functional understanding of these Spotfire concepts and techniques, you should have gained some insight into the science and art of data visualization. Resources for Article: Further resources on this subject: The Spotfire Architecture Overview [article] Interacting with Data for Dashboards [article] Setting Up and Managing E-mails and Batch Processing [article]
Read more
  • 0
  • 0
  • 6769

article-image-null-16
Packt
30 Jan 2013
4 min read
Save for later

Getting Started with HTML5 Modernizr

Packt
30 Jan 2013
4 min read
Detect and design with features, not User Agents (browsers) What if you could build your website based on features instead of for the individual browser idiosyncrasies by manufacturer and version, making your website not just backward compatible but also forward compatible? You could quite potentially build a fully backward and forward compatible experience using a single code base across the entire UA spectrum. What I mean by this is instead of baking in an MSIE 7.0 version, an MSIE 8.0 version , a Firefox version, and so on of your website, and then using JavaScript to listen for, or sniff out, the browser version in play, it would be much simpler to instead build a single version of your website that supports all of the older generation, latest generation, and in many cases even future generation technologies, such as a video API , box-shadow, first-of-type , and more. Think of your website as a full-fledged cable television network broadcasting over 130 channels, and your users as customers that sign up for only the most basic package available, of only 15 channels. Any time that they upgrade their cable (browser) package to one offering additional channels (features), they can begin enjoying them immediately because you have already been broadcasting to each one of those channels the entire time. What happens now is that a proverbial line is drawn in the sand, and the site is built on the assumption that a particular set of features will exist and are thus supported. If not, fallbacks are in place to allow a smooth degradation of the experience as usual, but more importantly the site is built to adopt features that the browser will eventually have. Modernizr can detect CSS features, such as @font-face , box-shadow , and CSS gradients. It can also detect HTML5 elements, such as canvas, localstorage, and application cache. In all it can detect over 40 features for you, the developer. Another term commonly used to describe this technique is progressive enhancement. When the time finally comes that the user decides to upgrade their browser, the new features that the more recent browser version brings with it, for example text-shadow, will automatically be detected and picked up by your website, to be leveraged by your site with no extra work or code from you when they do. Without any additional work on your part, any text that is assigned text-shadow attributes will turn on at the flick of a switch so that user's experience will smoothly, and progressively be enhanced. What is Modernizr? More importantly, why should you use it? At its foundation, Modernizr is a feature-detection library powered by none other than JavaScript. Here is an example of conditionally adding CSS classes based on the browser, also known as the User Agent. When the browser parses this HTML document and finds a match, that class will be conditionally added to the page. code 1 Now that the browser version has been found, the developer can use CSS to alter the page based on the version of the browser that is used to parse the page. In the following example, IE 7, IE 8, and IE 9 all use a different method for a drop shadow attribute on an anchor element: code 2 The problem with the conditional method of applying styles is that not only does it require more code, but it also leaves a burden on the developer to know what browser version is capable of a given feature, in this case box-shadow . Here is the same example using Modernizr. Note how Modernizr has done the heavy lifting for you, irrespective of whether or not the box-shadow feature is supported by the browser: code 3
Read more
  • 0
  • 0
  • 6768

article-image-modifying-existing-theme-drupal-6-part-1
Packt
20 Oct 2009
10 min read
Save for later

Modifying an Existing Theme in Drupal 6: Part 1

Packt
20 Oct 2009
10 min read
Setting up the workspace There are several software tools that can make your work modifying themes more efficient. Though no specific tools are required to work with Drupal themes, there are a couple of applications that you might want to consider adding to your tool kit. I work with Firefox as my primary browser, principally due to the fact that I can add into Firefox various extensions that make my life easier. The Web Developer extension, for example, is hugely helpful when dealing with CSS and related issues. I recommend the combination of Firefox and the Web Developer extension to anyone working with Drupal themes. Another extension popular with many developers is Firebug, which is very similar to the Web Developer extension, and indeed more powerful in several regards. Pick up Web Developer, Firebug, and other popular Firefox add-ons at https://addons.mozilla.org/en-US/firefox/ When it comes to working with PHP files and the various theme files, you will need an editor. The most popular application is probably Dreamweaver, from Adobe, although any editor that has syntax highlighting would work well too. I use Dreamweaver as it helps me manage multiple projects and provides a number of features that make working with code easier (particularly for designers). If you choose to use Dreamweaver, you will want to tailor the program a little bit to make it easier to work with Drupal theme files. Specifically, you should configure the application preferences to open and edit the various types of files common to PHPTemplate themes. To set this up, open Dreamweaver, then: Go to the Preferences dialogue. Open file types/editors. Add the following list of file types to Dreamweaver's open in code view field: .engine.info.module.install.theme Save the changes and exit. With these changes, your Dreamweaver application should be able to open and edit all the various PHPTemplate theme files. Previewing your work Note that, as a practical matter, previewing Drupal themes requires the use of a server. Themes are really difficult to preview (with any accuracy) without a server environment. A quick solution to this problem is the XAMPP package. XAMPP provides a one step installer containing everything you need to set up a server environment on your local machine (Apache, MySQL, PHP, phpMyAdmin, and more). Visit http://www.ApacheFriends.org to download XAMPP and you can have your own Dev Server quickly and easily. Another tool that should be on the top of your list is the Theme developer extension for the popular Drupal Devel module. Theme developer can save you untold hours of digging around trying to find the right function or template. When the module is active, all you need to do is click on an element and the Theme developer pop-up window will show you what is generating the element, along with other useful information. In the example later in this article, we will also use another feature of the Devel module, that is, the ability to automatically generate sample content for your site. You can download Theme developer as part of the Devel project at Drupal.org: http://drupal.org/project/devel Note that Theme developer only works on Drupal 6 and due to the way it functions, is only suitable for use in a development environment—you don't want this installed on a client's public site! Visit http://drupal.org/node/209561 for more information on the Theme developer aspects of the Devel module. The article includes links to a screencast showing the module in action—a good quick start and a solid help in grasping what this useful tool can do. Planning the modifications We're going to base our work on the popular Zen theme. We'll take Zen, create a new subtheme, and then modify the subtheme until we reach our final goal. Let's call our new theme "Tao". The Zen theme was chosen for this exercise because it has a great deal of flexibility. It is a good solid place to start if you wish to build a CSS-based theme. The present version of Zen even comes with a generic subtheme (named "STARTERKIT") designed specifically for themers who wish to take a basic theme and customize it. We'll use the Starterkit subtheme as the way forward in the steps that follow. The Zen theme is one of the most active theme development projects. Updated versions of the theme are released regularly. We used version 6.x-1.0-beta2 for the examples in this article. Though that version was current at the time this text was prepared, it is unlikely to be current at the time you read this. To avoid difficulties, we have placed a copy of the files used in this article in the software archive that is provided on the Packt website. Download the files used in this article at http://www.packtpub.com/files/code/5661_Code.zip. You can download the current version of Zen at http://drupal.org/project/zen. Any time you set off down the path of transforming an existing theme into something new, you need to spend some time planning. The principle here is the same as in many other areas of life: A little time spent planning at the front end of a project can pay off big in savings later. A proper dissertation on site planning and usability is beyond the scope of this article; so for our purposes let us focus on defining some loose goals and then work towards satisfying a specific wish list for the final site functionality. Our goal is to create a two-column blog-type theme with solid usability and good branding. Our hypothetical client for this project needs space for advertising and a top banner. The theme must also integrate a forum and a user comments functionality. Specific changes we want to implement include: Main navigation menu in the right column Secondary navigation mirrored at the top and bottom of each page A top banner space below top nav but above the branding area Color scheme and fonts to match brand identity Enable and integrate the Drupal blog, forum, and comments modules In order to make the example easier to follow and to avoid the need to install a variety of third-party extensions, the modifications we will make in this article will be done using only the default components—excepting only the theme itself, Zen. Arguably, were you building a site like this for deployment in the real world (rather than simply for skills development) you might wish to consider implementing one or more specialized third-party extensions to handle certain tasks. Creating a new subtheme Install the Zen theme if you have not done so before now; once that is done we're ready to create a new subtheme. First, make a copy of the directory named STARTERKIT and place the copied files into the directory sites/all/themes. Rename the directory "tao". Note that in Drupal 5.x, subthemes were kept in the same directory as the parent theme, but for Drupal 6.x this is no longer the case. Subthemes should now be placed in their own directory inside the sites/all/themes/directory. Note that the authors of Zen have chosen to vary from the default stylesheet naming. Most themes use a file named style.css for their primary CSS. In Zen, however, the file is named zen.css. We need to grab that file and incorporate it into Tao. Copy the Zen CSS (zen/zen/zen.css) file. Rename it tao.css and place it in the Tao directory (tao/tao.css). When you look in the zen/zen directory, in addition to the key zen.css file, you will note the presence of a number of other CSS files. We need not concern ourselves with the other CSS files. The styles contained in those stylesheets will remain available to us (we inherit them as Zen is our base theme) and if we need to alter them, we can override the selectors as needed via our new tao.css file. In addition to renaming the theme directory, we also need to rename any other theme-name-specific files or functions. Do the following: Rename the STARTERKIT.info file to tao.info. Edit the tao.info file to replace all occurrences of STARTERKIT with tao. Open the tao.info file and find this copy: The name and description of the theme used on the admin/build/themes page. name = Zen Themer's StarterKit description = Read the <a href="http://drupal.org/node/226507">online docs</a> on how to create a sub-theme. Replace that text with this copy: The name and description of the theme used on the admin/build/themes page. name = Tao description = A 2-column fixed-width sub-theme based on Zen. Make sure the name= and description = content is not commented out, else it will not register. Edit the template.php file to replace all occurrences of STARTERKIT with tao. Edit the theme-settings.php file to replace all occurrences of STARTERKIT with tao. Copy the file zen/layout-fixed.css and place it in the tao directory, creating tao/layout-fixed.css. Include the new layout-fixed.css by modifying the tao.info file. Change style sheets[all][] = layout.css to style sheets[all][] = layout-fixed.css. The .info file functions similar to a .ini file: It provides configuration information, in this case, for your theme. A good discussion of the options available within the .info file can be found on the Drupal.org site at: http://drupal.org/node/171205 Making the transition from Zen to Tao The process of transforming an existing theme into something new consists of a set of tasks that can categorized into three groups: Configuring the Theme Adapting the CSS Adapting the Templates & Themable Functions Configuring the theme As stated previously, the goal of this redesign is to create a blog theme with solid usability and a clean look and feel. The resulting site will need to support forums and comments and will need advertising space. Let's start by enabling the functionality we need and then we can drop in some sample contents. Technically speaking, adding sample content is not 100% necessary, but practically speaking, it is extremely useful; let's see the impact of our work with the CSS, the templates, and the themable functions. Before we begin, enable your new theme, if you have not done so already. Log in as the administrator, then go to the themes manager (Administer | Site building | Themes), and enable the theme Tao. Set it to be the default theme and save the changes. Now we're set to begin customizing this theme, first through the Drupal system's default configuration options, and then through our custom styling. Enabling Modules To meet the client's functional requirements, we need to activate several features of Drupal which, although contained in the default distro, are not by default activated. Accordingly, we need to identify the necessary modules and enable them. Let's do that now. Access the module manager screen (Administer | Site building | Modules), and enable the following modules: Blog (enables blog-type presentation of content) Contact (enables the site contact forms) Forum (enables the threaded discussion forum) Search (enables users to search the site) Save your changes and let's move on to the next step in the configuration process.
Read more
  • 0
  • 0
  • 6765

article-image-installing-and-running-lxc
Packt
19 Jan 2017
22 min read
Save for later

Installing and running LXC

Packt
19 Jan 2017
22 min read
In this article by Konstantin Ivanov, the author of the book Containerization with LXC, we will see how to install and run LXC. LXC takes advantage of the kernel namespaces and cgroups to create process isolation we often refer to as containers. As such LXC is not a separate software component in the Linux kernel, but rather a set of userspace tools, the liblxc library and various language bindings. In this article, we are going to cover the following topics: Installing LXC on Ubuntu Building and starting containers using the provided templates and configuration files Showcase the various LXC operations (For more resources related to this topic, see here.) Installing LXC At the time of writing there are two long-term support versions of LXC – 1.0 and 2.0. The userspace tools that they provide have some minor differences in command line flags and deprecations that I'll point out as we use them. Installing LXC on Ubuntu with apt Let's start by installing LXC 1.0 on Ubuntu 14.04 Trusty: Install the main LXC package, tooling and dependencies: root@ubuntu:~# lsb_release -dc Description: Ubuntu 14.04.5 LTS Codename: trusty root@ubuntu:~# apt-get –y install -y lxc bridge-utils debootstrap libcap-dev cgroup-bin libpam-systemd bridge-utils root@ubuntu:~# The package versions that Trusty provides at this time is 1.0.8: root@ubuntu:~# dpkg --list | grep lxc | awk '{print $2,$3}' liblxc1 1.0.8-0ubuntu0.3 lxc 1.0.8-0ubuntu0.3 lxc-templates 1.0.8-0ubuntu0.3 python3-lxc 1.0.8-0ubuntu0.3 root@ubuntu:~# To install LXC 2.0 we'll need the backports repository: Add the following two lines in the apt sources file: root@ubuntu:~# vim /etc/apt/sources.list deb http://archive.ubuntu.com/ubuntu trusty-backports main restricted universe multiverse deb-src http://archive.ubuntu.com/ubuntu trusty-backports main restricted universe multiverse Resynchronize the package index files from their sources: root@ubuntu:~# apt-get update Install the main LXC package, tooling and dependencies: root@ubuntu:~# apt-get –y install -y lxc=2.0.3-0ubuntu1~ubuntu14.04.1 lxc1=2.0.3-0ubuntu1~ubuntu14.04.1 liblxc1=2.0.3-0ubuntu1~ubuntu14.04.1 python3-lxc=2.0.3-0ubuntu1~ubuntu14.04.1 cgroup-lite=1.11~ubuntu14.04.2 lxc-templates=2.0.3-0ubuntu1~ubuntu14.04.1 bridge-utils root@ubuntu:~# Ensure the package versions are on the 2.x branch, in this case 2.0.3: root@ubuntu:~# dpkg --list | grep lxc | awk '{print $2,$3}' liblxc1 2.0.3-0ubuntu1~ubuntu14.04.1 lxc 2.0.3-0ubuntu1~ubuntu14.04.1 lxc-common 2.0.3-0ubuntu1~ubuntu14.04.1 lxc-templates 2.0.3-0ubuntu1~ubuntu14.04.1 lxc1 2.0.3-0ubuntu1~ubuntu14.04.1 lxcfs 2.0.2-0ubuntu1~ubuntu14.04.1 python3-lxc 2.0.3-0ubuntu1~ubuntu14.04.1 root@ubuntu:~# LXC directory installation layout The following table shows the directory layout of LXC that is created after package and source installation. The directories vary depending on distribution and installation method. Ubuntu package CentOS package Source installation Description /usr/share/lxc /usr/share/lxc /usr/local/share/lxc LXC base directory /usr/share/lxc/config /usr/share/lxc/config /usr/local/share/lxc/config Collection of distribution based LXC configuration files /usr/share/lxc/templates /usr/share/lxc/templates /usr/local/share/lxc/templates Collection of container template scripts /usr/bin /usr/bin /usr/local/bin Location for most LXC binaries /usr/lib/x86_64-linux-gnu /usr/lib64 /usr/local/lib Location of liblxc libraries /etc/lxc /etc/lxc /usr/local/etc/lxc Location of default LXC config files /var/lib/lxc/ /var/lib/lxc/ /usr/local/var/lib/lxc/ Location of the root filesystem and config for created container /var/log/lxc /var/log/lxc /usr/local/var/log/lxc LXC log files  We will explore most of the directories during building, starting and terminating of LXC containers. Building and manipulating LXC containers Managing the container life cycle with the provided userspace tools is quite convenient compared to manually creating namespaces and applying resource limits with cgroups. In essence this is exactly what the LXC tools do, creation and manipulation of the namespaces through calls to the liblxc API and cgroups. LXC comes packaged with various templates for building root file systems for different Linux distributions. We can use them to create a variety of container flavors. Building our first container We can create our first container by using a template. The lxc-download file, like the rest of the templates in the templates directory, is a script written in bash: root@ubuntu:~# ls -la /usr/share/lxc/templates/ drwxr-xr-x 2 root root 4096 Aug 29 20:03 . drwxr-xr-x 6 root root 4096 Aug 29 19:58 .. -rwxr-xr-x 1 root root 10557 Nov 18 2015 lxc-alpine -rwxr-xr-x 1 root root 13534 Nov 18 2015 lxc-altlinux -rwxr-xr-x 1 root root 10556 Nov 18 2015 lxc-archlinux -rwxr-xr-x 1 root root 9878 Nov 18 2015 lxc-busybox -rwxr-xr-x 1 root root 29149 Nov 18 2015 lxc-centos -rwxr-xr-x 1 root root 10486 Nov 18 2015 lxc-cirros -rwxr-xr-x 1 root root 17354 Nov 18 2015 lxc-debian -rwxr-xr-x 1 root root 17757 Nov 18 2015 lxc-download -rwxr-xr-x 1 root root 49319 Nov 18 2015 lxc-fedora -rwxr-xr-x 1 root root 28253 Nov 18 2015 lxc-gentoo -rwxr-xr-x 1 root root 13962 Nov 18 2015 lxc-openmandriva -rwxr-xr-x 1 root root 14046 Nov 18 2015 lxc-opensuse -rwxr-xr-x 1 root root 35540 Nov 18 2015 lxc-oracle -rwxr-xr-x 1 root root 11868 Nov 18 2015 lxc-plamo -rwxr-xr-x 1 root root 6851 Nov 18 2015 lxc-sshd -rwxr-xr-x 1 root root 23494 Nov 18 2015 lxc-ubuntu -rwxr-xr-x 1 root root 11349 Nov 18 2015 lxc-ubuntu-cloud root@ubuntu:~# If you examine the scripts closely you'll notice that most of them create chroot environments, where packages and various configuration files are then installed to create the root filesystem for the selected distribution. Let's start by building a container using the lxc-download template, which will ask for the distribution, release and architecture, then use the appropriate template to create the file system and configuration for us: root@ubuntu:~# lxc-create -t download -n c1 Setting up the GPG keyring Downloading the image index --- DIST RELEASE ARCH VARIANT BUILD --- centos 6 amd64 default 20160831_02:16 centos 6 i386 default 20160831_02:16 centos 7 amd64 default 20160831_02:16 debian jessie amd64 default 20160830_22:42 debian jessie arm64 default 20160824_22:42 debian jessie armel default 20160830_22:42 ... ubuntu trusty amd64 default 20160831_03:49 ubuntu trusty arm64 default 20160831_07:50 ubuntu yakkety s390x default 20160831_03:49 --- Distribution: ubuntu Release: trusty Architecture: amd64 Unpacking the rootfs --- You just created an Ubuntu container (release=trusty, arch=amd64, variant=default) To enable sshd, run: apt-get install openssh-server For security reason, container images ship without user accounts and without a root password. Use lxc-attach or chroot directly into the rootfs to set a root password or create user accounts. root@ubuntu:~# Let's list all containers: root@ubuntu:~# lxc-ls -f NAME STATE IPV4 IPV6 AUTOSTART ---------------------------------------------------- c1 STOPPED - - NO root@nova-perf:~# Depending on the version of LXC some of the command options might be different, read the man page for each of the tools if you encounter errors Our container is currently not running, let's start it in the background and increase the log level to DEBUG: root@ubuntu:~# lxc-start -n c1 -d -l DEBUG On some distributions LXC does not create the host bridge when building the first container, which results in an error. If this happens you can create it by running: brctl addbr virbr0 root@ubuntu:~# lxc-ls -f NAME STATE IPV4 IPV6 AUTOSTART ---------------------------------------------------------- c1 RUNNING 10.0.3.190 - NO root@ubuntu:~# To obtain more information about the container run: root@ubuntu:~# lxc-info -n c1 Name: c1 State: RUNNING PID: 29364 IP: 10.0.3.190 CPU use: 1.46 seconds BlkIO use: 112.00 KiB Memory use: 6.34 MiB KMem use: 0 bytes Link: vethVRD8T2 TX bytes: 4.28 KiB RX bytes: 4.43 KiB Total bytes: 8.70 KiB root@ubuntu:~# The new container is now connected to the host bridge lxcbr0: root@ubuntu:~# brctl show bridge name bridge id STP enabled interfaces lxcbr0 8000.fea50feb48ac no vethVRD8T2 root@ubuntu:~# ip a s lxcbr0 4: lxcbr0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default link/ether fe:a5:0f:eb:48:ac brd ff:ff:ff:ff:ff:ff inet 10.0.3.1/24 brd 10.0.3.255 scope global lxcbr0 valid_lft forever preferred_lft forever inet6 fe80::465:64ff:fe49:5fb5/64 scope link valid_lft forever preferred_lft forever root@ubuntu:~# ip a s vethVRD8T2 8: vethVRD8T2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master lxcbr0 state UP group default qlen 1000 link/ether fe:a5:0f:eb:48:ac brd ff:ff:ff:ff:ff:ff inet6 fe80::fca5:fff:feeb:48ac/64 scope link valid_lft forever preferred_lft forever root@ubuntu:~# By using the download template and not specifying any network settings, the container obtains its IP address from a dnsmasq server that runs on a private network, 10.0.3.0/24 in this case. The host allows the container to connect to the rest of the network and Internet by using NAT rules in iptables: root@ubuntu:~# iptables -L -n -t nat Chain PREROUTING (policy ACCEPT) target prot opt source destination Chain INPUT (policy ACCEPT) target prot opt source destination Chain OUTPUT (policy ACCEPT) target prot opt source destination Chain POSTROUTING (policy ACCEPT) target prot opt source destination MASQUERADE all -- 10.0.3.0/24 !10.0.3.0/24 root@ubuntu:~# Other containers connected to the bridge will have access to each other and to the host, as long as they are all connected to the same bridge and are not tagged with different VLAN IDs. Let's see how the process tree looks like after starting the container: root@ubuntu:~# ps axfww … 1552 ? S 0:00 dnsmasq -u lxc-dnsmasq --strict-order --bind-interfaces --pid-file=/run/lxc/dnsmasq.pid --conf-file= --listen-address 10.0.3.1 --dhcp-range 10.0.3.2,10.0.3.254 --dhcp-lease-max=253 --dhcp-no-override --except-interface=lo --interface=lxcbr0 --dhcp-leasefile=/var/lib/misc/dnsmasq.lxcbr0.leases --dhcp-authoritative 29356 ? Ss 0:00 lxc-start -n c1 -d -l DEBUG 29364 ? Ss 0:00 _ /sbin/init 29588 ? S 0:00 _ upstart-udev-bridge --daemon 29597 ? Ss 0:00 _ /lib/systemd/systemd-udevd --daemon 29667 ? Ssl 0:00 _ rsyslogd 29688 ? S 0:00 _ upstart-file-bridge --daemon 29690 ? S 0:00 _ upstart-socket-bridge --daemon 29705 ? Ss 0:00 _ dhclient -1 -v -pf /run/dhclient.eth0.pid -lf /var/lib/dhcp/dhclient.eth0.leases eth0 29775 pts/6 Ss+ 0:00 _ /sbin/getty -8 38400 tty4 29777 pts/1 Ss+ 0:00 _ /sbin/getty -8 38400 tty2 29778 pts/5 Ss+ 0:00 _ /sbin/getty -8 38400 tty3 29787 ? Ss 0:00 _ cron 29827 pts/7 Ss+ 0:00 _ /sbin/getty -8 38400 console 29829 pts/0 Ss+ 0:00 _ /sbin/getty -8 38400 tty1 root@ubuntu:~# Notice the new init child process that was cloned from the lxc-start command. This is PID 1 in the actual container. Next, let's attach to the container, list all processes, network interfaces and check connectivity: root@ubuntu:~# lxc-attach -n c1 root@c1:~# ps axfw PID TTY STAT TIME COMMAND 1 ? Ss 0:00 /sbin/init 176 ? S 0:00 upstart-udev-bridge --daemon 185 ? Ss 0:00 /lib/systemd/systemd-udevd --daemon 255 ? Ssl 0:00 rsyslogd 276 ? S 0:00 upstart-file-bridge --daemon 278 ? S 0:00 upstart-socket-bridge --daemon 293 ? Ss 0:00 dhclient -1 -v -pf /run/dhclient.eth0.pid -lf /var/lib/dhcp/dhclient.eth0.leases eth0 363 lxc/tty4 Ss+ 0:00 /sbin/getty -8 38400 tty4 365 lxc/tty2 Ss+ 0:00 /sbin/getty -8 38400 tty2 366 lxc/tty3 Ss+ 0:00 /sbin/getty -8 38400 tty3 375 ? Ss 0:00 cron 415 lxc/console Ss+ 0:00 /sbin/getty -8 38400 console 417 lxc/tty1 Ss+ 0:00 /sbin/getty -8 38400 tty1 458 ? S 0:00 /bin/bash 468 ? R+ 0:00 ps ax root@c1:~# ip a s 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo valid_lft forever preferred_lft forever inet6 ::1/128 scope host valid_lft forever preferred_lft forever 7: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000 link/ether 00:16:3e:b2:34:8a brd ff:ff:ff:ff:ff:ff inet 10.0.3.190/24 brd 10.0.3.255 scope global eth0 valid_lft forever preferred_lft forever inet6 fe80::216:3eff:feb2:348a/64 scope link valid_lft forever preferred_lft forever root@c1:~# ping -c 3 google.com PING google.com (216.58.192.238) 56(84) bytes of data. 64 bytes from ord30s26-in-f14.1e100.net (216.58.192.238): icmp_seq=1 ttl=52 time=1.77 ms 64 bytes from ord30s26-in-f14.1e100.net (216.58.192.238): icmp_seq=2 ttl=52 time=1.58 ms 64 bytes from ord30s26-in-f14.1e100.net (216.58.192.238): icmp_seq=3 ttl=52 time=1.75 ms --- google.com ping statistics --- 3 packets transmitted, 3 received, 0% packet loss, time 2003ms rtt min/avg/max/mdev = 1.584/1.705/1.779/0.092 ms root@c1:~# exit exit root@ubuntu:~# On some distributions like CentOS, or if installed from source, the dnsmasq server is not configured and started by default. You can either install it and configure it manually, or configure the container with an IP address and a default gateway instead, as I demonstrate later in this article. Notice how the hostname changed on the terminal once we attached to the container. This is an example of how LXC uses the UTS namespaces. Let's examine the directory that was created after building the c1 container: root@ubuntu:~# ls -la /var/lib/lxc/c1/ total 16 drwxrwx--- 3 root root 4096 Aug 31 20:40 . drwx------ 3 root root 4096 Aug 31 21:01 .. -rw-r--r-- 1 root root 516 Aug 31 20:40 config drwxr-xr-x 21 root root 4096 Aug 31 21:00 rootfs root@ubuntu:~# The rootfs directory looks like a regular Linux filesystem. You can manipulate the container directly by making changes to the files there, or by using chroot. To demonstrate this, let's change the root password of the c1 container not by attaching to it, but by using chroot rootfs: root@ubuntu:~# cd /var/lib/lxc/c1/ root@ubuntu:/var/lib/lxc/c1# chroot rootfs root@ubuntu:/# ls -al total 84 drwxr-xr-x 21 root root 4096 Aug 31 21:00 . drwxr-xr-x 21 root root 4096 Aug 31 21:00 .. drwxr-xr-x 2 root root 4096 Aug 29 07:33 bin drwxr-xr-x 2 root root 4096 Apr 10 2014 boot drwxr-xr-x 4 root root 4096 Aug 31 21:00 dev drwxr-xr-x 68 root root 4096 Aug 31 22:12 etc drwxr-xr-x 3 root root 4096 Aug 29 07:33 home drwxr-xr-x 12 root root 4096 Aug 29 07:33 lib drwxr-xr-x 2 root root 4096 Aug 29 07:32 lib64 drwxr-xr-x 2 root root 4096 Aug 29 07:31 media drwxr-xr-x 2 root root 4096 Apr 10 2014 mnt drwxr-xr-x 2 root root 4096 Aug 29 07:31 opt drwxr-xr-x 2 root root 4096 Apr 10 2014 proc drwx------ 2 root root 4096 Aug 31 22:12 root drwxr-xr-x 8 root root 4096 Aug 31 20:54 run drwxr-xr-x 2 root root 4096 Aug 29 07:33 sbin drwxr-xr-x 2 root root 4096 Aug 29 07:31 srv drwxr-xr-x 2 root root 4096 Mar 13 2014 sys drwxrwxrwt 2 root root 4096 Aug 31 22:12 tmp drwxr-xr-x 10 root root 4096 Aug 29 07:31 usr drwxr-xr-x 11 root root 4096 Aug 29 07:31 var root@ubuntu:/# passwd Enter new UNIX password: Retype new UNIX password: passwd: password updated successfully root@ubuntu:/# exit exit root@ubuntu:/var/lib/lxc/c1# Notice how the path changed on the console when we used chroot and after exiting the jailed environment. To test the root password, let's install SSH server in the container by first attaching to it and then using ssh to connect: root@ubuntu:~# lxc-attach -n c1 root@c1:~# apt-get update && apt-get install –y openssh-server root@c1:~# sed -i 's/without-password/yes/g' /etc/ssh/sshd_config root@c1:~# service ssh restart root@c1:/# exit exit root@ubuntu:/var/lib/lxc/c1# ssh 10.0.3.190 root@10.0.3.190's password: Welcome to Ubuntu 14.04.5 LTS (GNU/Linux 3.13.0-91-generic x86_64) * Documentation: https://help.ubuntu.com/ Last login: Wed Aug 31 22:25:39 2016 from 10.0.3.1 root@c1:~# exit logout Connection to 10.0.3.190 closed. root@ubuntu:/var/lib/lxc/c1# We were able to ssh to the container and use the root password that was manually set earlier. Autostarting LXC containers By default LXC containers do not start after a server reboot. To change that, we can use the lxc-autostart tool and the containers configuration file. To demonstrate this, let's create a new container first: root@ubuntu:~# lxc-create --name autostart_container --template ubuntu root@ubuntu:~# lxc-ls -f NAME STATE AUTOSTART GROUPS IPV4 IPV6 autostart_container STOPPED 0 - - - root@ubuntu:~# Next, add the lxc.start.auto stanza to its config file: root@ubuntu:~# echo "lxc.start.auto = 1" >> /var/lib/lxc/autostart_container/config root@ubuntu:~# List all containers that are configured to start automatically: root@ubuntu:~# lxc-autostart --list autostart_container root@ubuntu:~# Now we can use the lxc-autostart command again to start all containers configured to autostart, in this case just one: root@ubuntu:~# lxc-autostart --all root@ubuntu:~# lxc-ls -f NAME STATE AUTOSTART GROUPS IPV4 IPV6 autostart_container RUNNING 1 - 10.0.3.98 – root@ubuntu:~# Two other useful autostart configuration parameters are adding a delay to the start and defining a group in which multiple containers can start as a single unit. Stop the container and add the following to configuration options: root@ubuntu:~# lxc-stop --name autostart_container root@ubuntu:~# echo "lxc.start.delay = 5" >> /var/lib/lxc/autostart_container/config root@ubuntu:~# echo "lxc.group = high_priority" >> /var/lib/lxc/autostart_container/config root@ubuntu:~# Next, lets list the containers configured to autostart again: root@ubuntu:~# lxc-autostart --list root@ubuntu:~# Notice that no containers showed from the preceding output. This is because our container now belongs to an autostart group. Let's specify it: root@ubuntu:~# lxc-autostart --list --group high_priority autostart_container 5 root@ubuntu:~# Similarly to start all containers belong to a given autostart group: root@ubuntu:~# lxc-autostart --group high_priority root@ubuntu:~# lxc-ls -f NAME STATE AUTOSTART GROUPS IPV4 IPV6 autostart_container RUNNING 1 high_priority 10.0.3.98 - root@ubuntu:~# In order for lxc-autostart to automatically start containers after a server reboot, it first needs to be started. This can be achieved by either adding the preceding command in crontab, or creating an init script. Finally in order to clean up run: root@ubuntu:~# lxc-destroy --name autostart_container Destroyed container autostart_container root@ubuntu:~# lxc-ls -f root@ubuntu:~# LXC container hooks LXC provides a convenient way to execute programs during the life cycle of containers. The following table summarizes the various configuration options available to allow for this feature: Option Description lxc.hook.pre-start A hook to be run in the host namespace before the container ttys, consoles, or mounts are loaded. lxc.hook.pre-mount A hook to be run in the container's filesystem namespace, but before the rootfs has been set up. lxc.hook.mount A hook to be run in the container after mounting has been done, but before the pivot_root. lxc.hook.autodev A hook to be run in the container after mounting has been done and after any mount hooks have run, but before the pivot_root. lxc.hook.start A hook to be run in the container right before executing the container's init. lxc.hook.stop A hook to be run in the host's namespace after the container has been shut down. lxc.hook.post-stop A hook to be run in the host's namespace after the container has been shut down. lxc.hook.clone A hook to be run when the container is cloned. lxc.hook.destroy A hook to be run when the container is destroyed.  To demonstrate this, let's create a new container and write a simple script that will output the values of four LXC variables to a file, during container start. First, create the container and add the lxc.hook.pre-start option to its configuration file: root@ubuntu:~# lxc-create --name hooks_container --template ubuntu root@ubuntu:~# echo "lxc.hook.pre-start = /var/lib/lxc/hooks_container/pre_start.sh" >> /var/lib/lxc/hooks_container/config root@ubuntu:~# Next, create a simple bash script and make it executable: root@ubuntu:~# cat /var/lib/lxc/hooks_container/pre_start.sh #!/bin/bash LOG_FILE=/tmp/container.log echo "Container name: $LXC_NAME" | tee -a $LOG_FILE echo "Container mounted rootfs: $LXC_ROOTFS_MOUNT" | tee -a $LOG_FILE echo "Container config file $LXC_CONFIG_FILE" | tee -a $LOG_FILE echo "Container rootfs: $LXC_ROOTFS_PATH" | tee -a $LOG_FILE root@ubuntu:~# root@ubuntu:~# chmod u+x /var/lib/lxc/hooks_container/pre_start.sh root@ubuntu:~# Start the container and check the contents of the file that the bash script should have written to, ensuring the script got triggered: root@ubuntu:~# lxc-start --name hooks_container root@ubuntu:~# lxc-ls -f NAME STATE AUTOSTART GROUPS IPV4 IPV6 hooks_container RUNNING 0 - 10.0.3.237 - root@ubuntu:~# cat /tmp/container.log Container name: hooks_container Container mounted rootfs: /usr/lib/x86_64-linux-gnu/lxc Container config file /var/lib/lxc/hooks_container/config Container rootfs: /var/lib/lxc/hooks_container/rootfs root@ubuntu:~# From the preceding output we can see that the script got triggered when we started the container and the value of the LXC variables got written to the temp file. Attaching directories from the host OS and exploring the running filesystem of a container The root filesystem of LXC containers is visible from the host OS as a regular directory tree. We can directly manipulate files in a running container by just making changes in that directory. LXC also allows for attaching directories from the host OS inside of the container using bind mount. A bind mount is a different view of the directory tree. It achieves this by replicating the existing directory tree under a different mount point. To demonstrate this, let's create a new container, directory and a file on the host: root@ubuntu:~# mkdir /tmp/export_to_container root@ubuntu:~# hostname -f > /tmp/export_to_container/file root@ubuntu:~# lxc-create --name mount_container --template ubuntu root@ubuntu:~# Next, we are going to use the lxc.mount.entry option in the configuration file of the container, telling LXC what directory to bind mount from the host and the mount point inside the container to bind to: root@ubuntu:~# echo "lxc.mount.entry = /tmp/export_to_container/ /var/lib/lxc/mount_container/rootfs/mnt none ro,bind 0 0" >> /var/lib/lxc/mount_container/config root@ubuntu:~# Once the container is started we can see that the /mnt inside of it now contains the file that we created in /tmp/export_to_container directory on the host OS earlier: root@ubuntu:~# lxc-start --name mount_container root@ubuntu:~# lxc-attach --name mount_container root@mount_container:~# cat /mnt/file ubuntu root@mount_containerr:~# exit exit root@ubuntu:~# When an LXC container is in a running state some files are only visible from /proc on the host OS. To examine the running directory of a container, first grab its PID: root@ubuntu:~# lxc-info --name mount_container Name: mount_container State: RUNNING PID: 8594 IP: 10.0.3.237 CPU use: 1.96 seconds BlkIO use: 212.00 KiB Memory use: 8.50 MiB KMem use: 0 bytes Link: vethBXR2HO TX bytes: 4.74 KiB RX bytes: 4.73 KiB Total bytes: 9.46 KiB root@ubuntu:~# With the PID in hand we can examine the running directory of the container: root@ubuntu:~# ls -la /proc/8594/root/run/ total 44 drwxr-xr-x 10 root root 420 Sep 14 23:28 . drwxr-xr-x 21 root root 4096 Sep 14 23:28 .. -rw-r--r-- 1 root root 4 Sep 14 23:28 container_type -rw-r--r-- 1 root root 5 Sep 14 23:28 crond.pid ---------- 1 root root 0 Sep 14 23:28 crond.reboot -rw-r--r-- 1 root root 5 Sep 14 23:28 dhclient.eth0.pid drwxrwxrwt 2 root root 40 Sep 14 23:28 lock -rw-r--r-- 1 root root 112 Sep 14 23:28 motd.dynamic drwxr-xr-x 3 root root 180 Sep 14 23:28 network drwxr-xr-x 3 root root 100 Sep 14 23:28 resolvconf -rw-r--r-- 1 root root 5 Sep 14 23:28 rsyslogd.pid drwxr-xr-x 2 root root 40 Sep 14 23:28 sendsigs.omit.d drwxrwxrwt 2 root root 40 Sep 14 23:28 shm drwxr-xr-x 2 root root 40 Sep 14 23:28 sshd -rw-r--r-- 1 root root 5 Sep 14 23:28 sshd.pid drwxr-xr-x 2 root root 80 Sep 14 23:28 udev -rw-r--r-- 1 root root 5 Sep 14 23:28 upstart-file-bridge.pid -rw-r--r-- 1 root root 4 Sep 14 23:28 upstart-socket-bridge.pid -rw-r--r-- 1 root root 5 Sep 14 23:28 upstart-udev-bridge.pid drwxr-xr-x 2 root root 40 Sep 14 23:28 user -rw-rw-r-- 1 root utmp 2688 Sep 14 23:28 utmp root@ubuntu:~# Make sure you replace the PID with the output of lxc-info from your host, as it will differ from the above example. In order to make persistent changes in the root filesystem of a container, modify the files in /var/lib/lxc/mount_container/rootfs/ instead. Freezing a running container LXC takes advantage of the freezer cgroup to freeze all the processes running inside of a container. The processes will be in a blocked state until thawed. Freezing a container can be useful in cases where the system load is high and you want to free some resources without actually stopping the container and preserve its running state. Ensure you have a running container and check its state from the freezer cgroup: root@ubuntu:~# lxc-ls -f NAME STATE AUTOSTART GROUPS IPV4 IPV6 hooks_container RUNNING 0 - 10.0.3 root@ubuntu:~# cat /sys/fs/cgroup/freezer/lxc/hooks_container/freezer.state THAWED root@ubuntu:~# Notice how a currently running container shows as thawed. Let's freeze it: root@ubuntu:~# lxc-freeze -n hooks_container root@ubuntu:~# lxc-ls -f NAME STATE AUTOSTART GROUPS IPV4 IPV6 hooks_container FROZEN 0 - 10.0.3.237 – root@ubuntu:~# The container state shows as frozen, let's check the cgroup file: root@ubuntu:~# cat /sys/fs/cgroup/freezer/lxc/hooks_container/freezer.state FROZEN root@ubuntu:~# To unfreeze it run: root@ubuntu:~# lxc-unfreeze --name hooks_container root@ubuntu:~# lxc-ls -f NAME STATE AUTOSTART GROUPS IPV4 IPV6 hooks_container RUNNING 0 - 10.0.3.237 - root@ubuntu:~# cat /sys/fs/cgroup/freezer/lxc/hooks_container/freezer.state THAWED root@ubuntu:~# We can monitor the state change by running the lxc-monitor command on a separate console while freezing and unfreezing a container. The change of the container's state will show as the following: root@ubuntu:~# lxc-monitor --name hooks_container 'hooks_container' changed state to [FREEZING] 'hooks_container' changed state to [FROZEN] 'hooks_container' changed state to [THAWED] Limiting container resource usage LXC comes with tools that are just as straightforward and easy to use. Lets start by setting up the available memory to a container to 512MB: root@ubuntu:~# lxc-cgroup -n hooks_container memory.limit_in_bytes 536870912 root@ubuntu:~# We can verify that the new setting has been applying by directly inspecting the memory cgroup for the container: root@ubuntu:~# cat /sys/fs/cgroup/memory/lxc/hooks_container/memory.limit_in_bytes 536870912 root@ubuntu:~# Changing the value only requires running the same command again. Let's change the available memory to 256 MB and inspect the container by attaching to it and running the free utility: root@ubuntu:~# lxc-cgroup -n hooks_container memory.limit_in_bytes 268435456 root@ubuntu:~# cat /sys/fs/cgroup/memory/lxc/hooks_container/memory.limit_in_bytes 268435456 root@ubuntu:~# lxc-attach --name hooks_container root@hooks_container:~# free -m total used free shared buffers cached Mem: 256 63 192 0 0 54 -/+ buffers/cache: 9 246 Swap: 0 0 0 root@hooks_container:~# exit root@ubuntu:~# As the preceding output shows the container only sees 256 MB of total available memory. Similarly we can pin a CPU core to container. In the next example our test server has two cores. Let's allow the container to only run on core 0: root@ubuntu:~# cat /proc/cpuinfo | grep processor processor : 0 processor : 1 root@ubuntu:~# root@ubuntu:~# lxc-cgroup -n hooks_container cpuset.cpus 0 root@ubuntu:~# cat /sys/fs/cgroup/cpuset/lxc/hooks_container/cpuset.cpus 0 root@ubuntu:~# lxc-attach --name hooks_container root@hooks_container:~# cat /proc/cpuinfo | grep processor processor : 0 root@hooks_container:~# exit exit root@ubuntu:~# By attaching to the container and checking the available CPUs we see that only one is presented, as expected. To make changes persist server reboots we need to add them to the configuration file of the container: root@ubuntu:~# echo "lxc.cgroup.memory.limit_in_bytes = 536870912" >> /var/lib/lxc/hooks_container/config root@ubuntu:~# Setting various other cgroup parameters is done in a similar way. For example let's see the CPU shares and the block IO on a container: root@ubuntu:~# lxc-cgroup -n hooks_container cpu.shares 512 root@ubuntu:~# lxc-cgroup -n hooks_container blkio.weight 500 root@ubuntu:~# lxc-cgroup -n hooks_container blkio.weight 500 root@ubuntu:~# Summary In this article we demonstrated how to install LXC, build containers using the provided templates and showed some basic operations to manage the instances. Resources for Article: Further resources on this subject: Remote Authentication [article] Wireless Attacks in Kali Linux [article] Revisiting Linux Network Basics [article]
Read more
  • 0
  • 0
  • 6760

article-image-moodle-authentication-methods
Packt
11 Mar 2011
6 min read
Save for later

Moodle: Authentication Methods

Packt
11 Mar 2011
6 min read
Moodle Security Learn how to install and configure Moodle in the most secure way possible Basics of authentication Authentication is the process of confirming that something or someone is really who they claim to be. The ways in which someone may be authenticated fall into three categories, based on what are known as the factors of authentication: Knowledge (something you know): password, PIN code, etc. Ownership (something you have): security token, phone, etc. Inherence (something you are): fingerprint, signature, various biometric identifiers Following the path of most computer systems, Moodle offers basic authentication based on a knowledge factor. This means that in order to operate in Moodle any person must have a user account. A user account consists of a username, password, and other personal information. Both username and password are used to authenticate a person who wishes to access the platform. Based on the outcome of an authentication, a user will be given or declined access to the platform. The authentication is performed (usually) by comparing provided data from the person trying to access the platform with the data located in the Authoritative Data Source (of user identity). Moodle supports 13 different types of authentication and this actually means that it has support for consulting 13 different types of Authoritative Data Sources. An Authoritative Data Source is a recognized or official data production source with a designated mission statement or source/product to publish reliable and accurate data for subsequent use by users or by other computer programs. Logon procedure Logon in Moodle is implemented using a HTML form that submits supplied data over HTTP or HTTPS to the server where it is being processed. Hypertext Transfer Protocol (HTTP) is a networking protocol used for transferring and rendering content on the World Wide Web. HTTP Secure (HTTPS) is a combination of a HTTP protocol and SSL/TLS (Security Socket Layer/ Transport Layer Security) protocol that offers encrypted and thus secures communication and identification between two computers on the Internet. HTTPS connections are often used for payments transactions and other sensitive information's transfer. The user enters his assigned credentials into the supplied fields on the login form and presses Login. That sends data to Moodle for processing. Common authentication attacks Any type of security attack is directed toward potential weak spots in the system that is under attack. The most common weaknesses related to the authentication and ways of protecting from them are as follows: Weak passwords A password that is easily guessed and does not provide an effective defense against unauthorized access to a resource is considered weak. Such passwords are usually: Short Set to dictionary word or name Set to be the same as username Set to some predefined value When we have a platform with weak passwords it can be attacked using brute force login technique (also known as dictionary attack). Dictionary attack is a technique for defeating authentication mechanism by trying to determine its pass-phrase by searching likely possibilities. In practice this means that a bot (automated script) constantly tries to log on by sending various usernames and passwords from a predefined list of words (usually a dictionary list of words—hence the name dictionary attack). Enforcing a good password policy In order to prevent this attack, make sure you have enabled the password policy. Visit Administration | Security | Site policies and locate the Password Policy checkbox. You should arrive at the following screenshot: Password policy is enabled by default starting from Moodle 1.9.7. This applies to both new installs and upgrades. Protecting user login By default, Moodle is configured to use unencrypted HTTP as the main communication protocol between client and server. This is fine for general usage of the platform but it also exposes credential information to the potential eavesdropper who can intercept and read it. This is a common case known as man-in-the-middle attack. The perpetrator makes a separate connection with the client (user's computer) and server (Moodle), forcing all communication to go over his connection. That permits him to look at the entire communication and even inject his own version of messages and responses. Closing the security breach We need to make sure that credential transmission is performed using secure HTTP (HTTPS) because that prevents (or makes it really hard) for anybody to hook into a protected conversation. Here are the steps: Firstly, you should install and configure a valid SSL (Secure Sockets Layer) certificate on your web-server. It is important to do this properly before doing anything else in Moodle; otherwise you might block yourself from accessing the platform. The procedure for installing an SSL certificate is beyond the scope of this book since it involves too many different factors that depend on your server configuration, OS type, and the way you manage it. Please refer to the manual for your particular web server and/or particular procedure of your hosting provider. Valid SSL certificates can be obtained only from certified root authorities—companies with a license for issuing certificates. VeriSign, Thawte, and Comodo are one of the several certificate providers. You need to specify which web server you are using since some of them prefer particular formats. Secondly, you should activate HTTPS log-in in your Moodle. You can do that by going to Administration | Security | HTTP security page and checking Use HTTPS for logins. If everything is configured properly you should see a login page that shows a valid certificate box (see following screenshot) in your browser. This means that a certificate is issued by a valid root authority and that communication between your browser and Moodle is secure which is what we wanted to accomplish in the first place. Every time a user tries to login in Moodle they will be redirected to the secure version of the login page which effectively prevents the interception of user credentials. Password change By default, all newly created users in Moodle (excluding admin) are assigned the Authenticated user role. The authenticated user role by default has permission to change their own password. This feature can be utilized by accessing user profile page. Recover a forgotten password Forgetting a username and/or password is a common situation in which many users find themselves. Moodle offers a procedure for getting a username and resetting the password. The user will be presented with a form where he can enter his username or his e-mail. If the username or email exists in the database, a mail with a reset link will be sent to that user. By clicking on that link, the user is offered a chance to enter a new password. If not configured properly, this feature can be used for determining valid user emails or user-names. See the following screenshot: An attacker would be able to tailor a script that could probe for usernames and, based on the response, can determine valid users.  
Read more
  • 0
  • 0
  • 6760

article-image-galera-cluster-basics
Packt
23 Sep 2014
5 min read
Save for later

Galera Cluster Basics

Packt
23 Sep 2014
5 min read
This article written by Pierre MAVRO, the author of MariaDB High Performance, provides a brief introduction to Galera Cluster. (For more resources related to this topic, see here.) Galera Cluster is a synchronous multimaster solution created by Codership. It's a patch for MySQL and MariaDB with its own commands and configuration. On MariaDB, it has been officially promoted as the MariaDB Cluster. Galera Cluster provides certification-based replication. This means that each node certifies the replicated write set against other write sets. You don't have to worry about data integrity, as it manages it automatically and very well. Galera Cluster is a young product, but is ready for production. If you have already heard of MySQL Cluster, don't be confused; this is not the same thing at all. MySQL Cluster is a solution that has not been ported to MariaDB due to its complexity, code, and other reasons. MySQL Cluster provides availability and partitioning, while Galera Cluster provides consistency and availability. Galera Cluster is a simple yet powerful solution. How Galera Cluster works The following are some advantages of Galera Cluster: True multimaster: It can read and write to any node at any time Synchronous replication: There is no slave lag and no data is lost at node crash Consistent data: All nodes have the same state (same data exists between nodes at a point in time) Multithreaded slave: This enables better performance with any workload No need of an HA Cluster for management: There are no master-slave failover operations (such as Pacemaker, PCR, and so on) Hot standby: There is no downtime during failover Transparent to applications: No specific drivers or application changes are required No read and write splitting needed: There is no need to split the read and write requests WAN: Galera Cluster supports WAN replication Galera Cluster needs at least three nodes to work properly (because of the notion of quorum, election, and so on). You can also work with a two-node cluster, but you will need an arbiter (hence three nodes). The arbiter could be used on another machine available in the same LAN of your Galera Cluster, if possible. The multimaster replication has been designed for InnoDB/XtraDB. It doesn't mean you can't perform a replication with other storage engines! If you want to use other storage engines, you will be limited by the following: They can only write on a single node at a time to maintain consistency. Replication with other nodes may not be fully supported. Conflict management won't be supported. Applications that connect to Galera will only be able to write on a single node (IP/DNS) at the same time. As you can see in the preceding diagram, HTTP and App servers speak directly to their respective DBMS servers without wondering which node of the Galera Cluster they are targeting. Usually, without Galera Cluster, you can use a cluster software such as Pacemaker/Corosync to get a VIP on a master node that can switch over in case a problem occurs. No need to get PCR in that case; a simple VIP with a custom script will be sufficient to check whether the server is in sync with others is enough. Galera Cluster uses the following advanced mechanisms for replication: Transaction reordering: Transactions are reordered before commitment to other nodes. This increases the number of successful transaction certification pass tests. Write sets: This reduces the number of operations between nodes by writing sets in a single write set to avoid too much node coordination. Database state machine: Read-only transactions are processed on the local node. Write transactions are executed locally on shadow copies and then broadcasted as a read set to the other nodes for certification and commit. Group communication: High-level abstraction for communication between nodes to guarantee consistency (gcomm or spread). To get consistency and similar IDs between nodes, Galera Cluster uses GTID, similar to MariaDB 10 replication. However, it doesn't use the MariaDB GTID replication mechanism at all, as it has its own implementation for its own usage. Galera Cluster limitations Galera Cluster has limitations that prevent it from working correctly. Do not go live in production if you haven't checked that your application is in compliance with the limitations listed. The following are the limitations: Galera Cluster only fully supports InnoDB tables. TokuDB is planned but not yet available and MyISAM is partially supported. Galera Cluster uses primary keys on all your tables (mandatory) to avoid different query execution orders between all your nodes. If you do not do it on your own, Galera will create one. The delete operation is not supported on the tables without primary keys. Locking/unlocking tables and lock functions are not supported. They will be ignored if you try to use them. Galera Cluster disables query cache. XA transactions (global transactions) are not supported. Query logs can't be directed to a table, but can be directed to a file instead. Other less common limitations exist (please refer to the full list if you want to get them all: http://galeracluster.com/documentation-webpages/limitations.html) but in most cases, you shouldn't be annoyed with those ones. Summary This article introduced the benefits and drawbacks of Galera Cluster. It also discussed the features of Galera Cluster that makes it a good solution for write replications. Resources for Article: Further resources on this subject: Building a Web Application with PHP and MariaDB – Introduction to caching [Article] Using SHOW EXPLAIN with running queries [Article] Installing MariaDB on Windows and Mac OS X [Article]
Read more
  • 0
  • 0
  • 6759
Unlock access to the largest independent learning library in Tech for FREE!
Get unlimited access to 7500+ expert-authored eBooks and video courses covering every tech area you can think of.
Renews at ₹800/month. Cancel anytime
article-image-asterisk-gateway-interface-scripting-php
Packt
16 Oct 2009
4 min read
Save for later

Asterisk Gateway Interface Scripting with PHP

Packt
16 Oct 2009
4 min read
PHP-CLI vs PHP-CGI Most Linux distributions include both versions of PHP when installed, especially if you are using a modern distribution such as CentOS or Mandriva. When writing AGI scripts with PHP, it is imperative that you use PHP-CLI, and not PHP-CGI. Why is this so important? The main issue is that PHP-CLI and PHP-CGI handle their STDIN (standard input) slightly differently, which makes the reading of channel variables via PHP-CGI slightly more problematic. The php.ini configuration file The PHP interpreter includes a configuration file that defines a set of defaults for the interpreter. For your scripts to work in an efficient manner, the following must be set—either via the php.ini file, or by your PHP script: ob_implicit_flush(false); set_time_limit(5); error_log = filename;error_reporting(0); The above code snippet performs the following: Directive Description ob_implicit_flush(false); Sets your PHP output buffering to false, in order to make sure that output from your AGI script to Asterisk is not buffered, and takes longer to execute set_time_limit(5); Sets a time limit on your AGI scripts to verify that they don't extend beyond a reasonable time of execution; there is no rule of thumb relating to the actual value; it is highly dependant on your implementation Depending on your system and applications, your maximum time limit may be set to any value; however, we suggest that you verify your scripts, and are able to work with a maximum limit of 30 seconds. error_log=filename; Excellent for debugging purposes; always creates a log file error_reporting(E_NONE); Does not report errors to the error_log; changes the value to enable different logging parameters; check the PHP website for additional information about this AGI script permissions All AGI scripts must be located in the directory /var/lib/asterisk/agi-bin, which is Asterisk's default directory for AGI scripts. All AGI scripts should have the execute permission, and should be owned by the user running Asterisk. If you are unfamiliar with these, consult with your system administrator for additional information. The structure of a PHP based AGI script Every PHP based AGI script takes the following form: #!/usr/bin/php -q <? $stdin = fopen(‘php://stdin’, ‘r’); $stdout = fopen(‘php://stdout’, ‘w’); $stdlog = fopen(‘my_agi.log’, ‘w’); /* Operational Code starts here */ .. .. ..?> Upon execution, Asterisk transmits a set of information to our AGI script via STDIN. Handling of that input is best performed in the following manner: #!/usr/bin/php -q <? $stdin = fopen(‘php://stdin’, ‘r’); $stdout = fopen(‘php://stdout’, ‘w’); $stdlog = fopen(‘my_agi.log’, ‘w’); /* Handling execution input from Asterisk */ while (!feof($stdin)) { $temp = fgets($stdin); $temp = str_replace("n","",$temp); $s = explode(":",$temp); $agivar[$s[0]] = trim($s[1]); if $temp == "") { break; } } /* Operational Code starts here */ .. .. ..?> Once we have handled our inbound information from the Asterisk server, we can start our actual operational flow. Communication between Asterisk and AGI The communication between Asterisk and an AGI script is performed via STDIN and STDOUT (standard output). Let's examine the following diagram: In the above diagram, ASC refers to our AGI script, while AST refers to Asterisk itself. As you can see from the diagram above, the entire flow is fairly simple. It is just a set of simple I/O queries and responses that are carried through the STDIN/STDOUT data streams. Let's now examine a slightly more complicated example: The above figure shows an example that includes two new elements in our AGI logic—access to a database, and to information provided via a web service. For example, the above image illustrates something that may be used as a connection between the telephony world and a dating service. This leads to an immediate conclusion that just as AGI is capable of connecting to almost any type of information source, depending solely on the implementation of the AGI script and not on Asterisk, Asterisk is capable of interfacing with almost any type of information source via out-of-band facilities. Enough of talking! Let's write our first AGI script.
Read more
  • 0
  • 0
  • 6738

article-image-first-steps-r
Packt
30 Jul 2013
6 min read
Save for later

First steps with R

Packt
30 Jul 2013
6 min read
(For more resources related to this topic, see here.) Obtaining and installing R The way to obtain R is downloading it from the CRAN website (http://www.r-project.org/). The Comprehensive R Archive Network (CRAN) is a network of FTP and web servers around the world that stores identical, up-to-date versions of code and documentation for R. The CRAN is directly accessible from the R website and on such website it is also possible to find information about R, some technical manuals, the R journal, and details about the packages developed for R and stored on the CRAN repositories. The functionalities of the R environment can then also be expanded thanks to software libraries which can be installed and recalled if needed. These libraries or packages are a collection of source code and other additional files that, when installed in R, allow the user to load them in the workspace via a call to the library() function. An example of code to load the package lattice may be found as follows: > library(lattice) An R installation contains one or more libraries of packages. Some of these packages are part of the basic installation and are loaded automatically as soon as the session is started. Other can be installed from the CRAN, the official R repository, or downloaded and installed manually. Interacting with the console As soon as you will start R, you will see that a workspace is open; you can see a screenshot of the R Console window in the image below. The workspace is the environment in which you are working, where you will load your data, and create your variables. The screen prompt > is the R prompt that waits for commands. On the starting screen, you can either type any function, command, or you can use R to perform basic calculation. R uses the usual symbols for addition (+), subtraction (-), multiplication (*), division (/), and exponentiation (^). Parentheses ( ) can be used to specify the order of operations. R also provides %% for taking the modulus and %/% for integer division. Comments in R are defined by the character #, so everything after such character up to the end of the line will be ignored by R. R has a number of built-in functions, for example, sin(x), cos(x), tan(x), (all in radians), exp(x), log(x), and sqrt(x). Some special constants such as pi are also pre-defined. You can see an example of the use of such function in the following code: > exp(2.5)[1] 12.18249 Understanding R objects In every computer language, variables provide a means of accessing the data stored in memory. R does not provide direct access to the computer’s memory but rather provides a number of specialized data structures called objects. These objects are referred to through symbols or variables. Vectors The basic object in R is the vector; even scalars are vectors of length one. Vectors can be thought of as a series of data of the same class. There are six basic vector type (called atomic vectors): logical, integer, real, complex, string (or character), and raw. Integer and real represent numeric objects; logicals are Boolean data type with possible value TRUE or FALSE. Among such atomic vectors, the more common ones are logical, string, and numeric (integer and real). There are several ways to create vectors. For instance the operator : (colon) is a sequence-generating operator, it creates sequences by incrementing or decrementing by one. > 1:10 [1] 1 2 3 4 5 6 7 8 9 10> 5:-6 [1] 5 4 3 2 1 0 -1 -2 -3 -4 -5 -6 If the interval between the numbers is not one, you can use the seq() function. Here an example > seq(from=2, to=2.5, by=0.1)[1] 2.0 2.1 2.2 2.3 2.4 2.5 One of the more important features of R is the possibility to use entire vector as arguments of functions, thus avoiding the use of cyclic loops. Most of the functions in R allow the use of vector as argument, as example the use of some of these functions is reported as follows > x <- c(12,10,4,6,9)> max(x)[1] 12> min(x)[1] 4> mean(x)[1] 8.2 Matrices and arrays In R, the matrix notation is extended to elements of any kind, so in example it is possible to have a matrix of character strings. Matrices and arrays are basically vectors with a dimension attribute. The function matrix() may be used to create matrices. By default, such function creates the matrix by column; as alternative it is possible to specify to the function to build the matrix by row: > matrix(1:9,nrow=3,byrow=TRUE) [,1] [,2] [,3][1,] 1 2 3[2,] 4 5 6[3,] 7 8 9 Lists A list in R is a collection of different objects. One of the main advantages of lists is that the object contained within a list may be of different type, for example, numeric and character values. In order to define a list, you simply will need to provide the object that you want to include as argument of the function list(). Data frame A data frame corresponds to a data set; it is basically a special list in which the elements have the same length. Elements may be different type in different columns, but within the same column all the elements are of the same type. You can easily create data frames using the function data.frame(), and a specific column can be recall using the operator $. Top features you’ll want to know about In addition to the basic object creation and manipulation, many more complex tasks can be performed with R, spanning from data manipulation, programming, statistical analysis and the realization of very high quality graphs. Some of the most useful features are Data input and output Flow control (for, if…else, while) Create your own functions Debugging functions and handling exceptions Plotting data Summary In this article we saw what is R, how to obtain and install R, and how to interacting with the console. We also saw at few R objects and also looked at the top features you would want to know about Resources for Article: Further resources on this subject: Organizing, Clarifying and Communicating the R Data Analyses [Article] Customizing Graphics and Creating a Bar Chart and Scatterplot in R [Article] Graphical Capabilities of R [Article]
Read more
  • 0
  • 0
  • 6724

article-image-professional-environment-react-native-part-1
Pierre Monge
09 Jan 2017
5 min read
Save for later

A Professional Environment for React Native, Part 1

Pierre Monge
09 Jan 2017
5 min read
React Native, a new framework, allows you to build mobile apps using JavaScript. It uses the same design as React.js, letting you compose a rich mobile UI from declarative components. Although many developers are talking about this technology, React Native is not yet approved by most professionals for several reasons: React Native isn’t fully stable yet. At the time of writing this, we are at version 0.40 It can be scary to use a web technology in a mobile application It’s hard to find good React Native developers because knowing the React.js stack is not enough to maintain a mobile React Native app from A to Z! To confront all these prejudices, this series will act as a guide, detailing how we see things in my development team. We will cover the entire React Native environment as well as discuss how to maintain a React Native application. This series may be of interest to companies who want to implement a React Native solution and also of interest to anyone who is looking for the perfect tools to maintain a mobile application in React Native. Let’s start here in part 1 by exploring the React Native environment. The environment The React Native environment is pretty consistent. To manage all of the parts of such an application, you will need a native stack, a JavaScript stack, and specific components from React Native. Let's examine all the aspects of the React Native environment: The Native part consists of two important pieces of software: Android Studio (Android) and Xcode (iOS). Both the pieces of software are provided with their emulators, so there is no need for a physical device! The negative point of Android Studio, however, is that you need to download the SDK, and you will have to find the good versions and download them all. In addition, these two programs take up a lot of room on your hard disk! The JavaScript part naturally consists of Node.js, but we must add Watchman to that to check the changes in a file in real time. The React Native CLI will automate the linking of all software. You only have to run react-native init helloworld to create a project and react-native run-ios --scheme 'Dev' to launch the project on an iOS simulator in debug mode. The supplied react-native controls will load almost everything! You have, no doubt, come to our first conclusion. React Native has a lot of prerequisites, and although it makes sense to have as much dependency as possible, you will have to master them all, which can take some time. And also a lot of space on your hard drive! Try this as your starting point if you want more information on getting started with React Native. Atom, linter, and flow A developer never starts coding without his text editor and his little tricks, just as a woodcutter never goes out into the forest without his ax! More than 80% of people around me use Atom as a text editor. And they are not wrong! React Native is 100% OpenSource, and Atom is also open source. And it is full of plug-ins and themes of all kinds. I personally use a lot of plug-ins, such as color-picker, file-icons, indent-guide-improved, minimap, etc., but there are some plug-ins that should be essential for every JavaScript coder, especially for your React Native application. linter-eslint To work alone or in a group, you must have a common syntax for all your files. To do this, we use linter-eslint with the fbjs configurations. This plugin provides the following: Good indentation Good syntax on how to define variables, objects, classes, etc. Indications on non-existent, unused variables and functions And many other great benefits. Flow One question you may be thinking is what is the biggest problem with using JavaScript? One issue with using JavaScript has always been that it's a language that has no type. In fact, there are types, such as String, Number, Boolean, Function, etc., but that's just not enough. There is no static type. To deal with this, we use Flow, which allows you to perform type check before run-time. This is, of course, useful for predicting bugs! There is even a plug-in version for Atom: linter-flow. Conclusion At this point, you should have everything you need to create your first React Native mobile applications. Here are some great examples of apps that are out there already. Check out part 2 in this series where I cover the tools that can help you maintain your React Native apps. About the author Pierre Monge (liroo.pierre@gmail.com) is a 21 year old student. He is a developer in C, JavaScript, and all things related to web development, and he has recently been creating mobile applications. He is currently working as an intern at a company named Azendoo, where he is developing a 100% React Native application.
Read more
  • 0
  • 0
  • 6723

article-image-testing-tools-and-techniques-python
Packt
20 Dec 2010
10 min read
Save for later

Testing Tools and Techniques in Python

Packt
20 Dec 2010
10 min read
  Python Testing: Beginner's Guide An easy and convenient approach to testing your powerful Python projects Covers everything you need to test your code in Python Easiest and enjoyable approach to learn Python testing Write, execute, and understand the result of tests in the unit test framework Packed with step-by-step examples and clear explanations So let's get on with it! Code coverage Tests tell you when the code you're testing doesn't work the way you thought it would, but they don't tell you a thing about the code that you're not testing. They don't even tell you that the code you're not testing isn't being tested. Code coverage is a technique, which can be used to address that shortcoming. A code coverage tool watches while your tests are running, and keeps track of which lines of code are (and aren't) executed. After the tests have run, the tool will give you a report describing how well your tests cover the whole body of code. It's desirable to have the coverage approach 100%, as you probably figured out already. Be careful not to focus on the coverage number too intensely though, it can be a bit misleading. Even if your tests execute every line of code in the program, they can easily not test everything that needs to be tested. That means you can't take 100% coverage as certain proof that your tests are complete. On the other hand, there are times when some code really, truly doesn't need to be covered by the tests—some debugging support code, for example—and so less than 100% coverage can be completely acceptable. Code coverage is a tool to give you insight into what your tests are doing, and what they may be overlooking. It's not the definition of a good test suite. coverage.py We're going to be working with a module called coverage.py, which is—unsurprisingly—a code coverage tool for Python. Since coverage.py isn't built in to Python, we'll need to download and install it. You can download the latest version from the Python Package Index at http://pypi.python.org/pypi/coverage. As before, users of Python 2.6 or later can install the package by unpacking the archive, changing to the directory, and typing: $ python setup.py install --user Users of older versions of Python need write permission to the system-wide site-packages directory, which is part of the Python installation. Anybody who has such permission can install coverage by typing: $ python setup.py install We'll walk through the steps of using coverage.py here, but if you want more information you can find it on the coverage.py home page at http://nedbatchelder.com/code/coverage/. Time for action – using coverage.py We'll create a little toy code module with tests, and then apply coverage.py to find out how much of the code the tests actually use. Place the following test code into test_toy.py. There are several problems with these tests, which we'll discuss later, but they ought to run. from unittest import TestCase import toy class test_global_function(TestCase): def test_positive(self): self.assertEqual(toy.global_function(3), 4) def test_negative(self): self.assertEqual(toy.global_function(-3), -2) def test_large(self): self.assertEqual(toy.global_function(2**13), 2**13 + 1) class test_example_class(TestCase): def test_timestwo(self): example = toy.example_class(5) self.assertEqual(example.timestwo(), 10) def test_repr(self): example = toy.example_class(7) self.assertEqual(repr(example), '<example param="7">') Put the following code into toy.py. Notice the if __name__ == '__main__' clause at the bottom. We haven't dealt with one of those in a while, so I'll remind you that the code inside that block runs doctest if we were to run the module with python toy.py. python toy.py. def global_function(x): r""" >>> global_function(5) 6 """ return x + 1 class example_class: def __init__(self, param): self.param = param def timestwo(self): return self.param * 2 def __repr__(self): return '<example param="%s">' % self.param if __name__ == '__main__': import doctest doctest.testmod() Go ahead and run Nose. It should find them, run them, and report that all is well. The problem is, some of the code isn't ever tested. Let's run it again, only this time we'll tell Nose to use coverage.py to measure coverage while it's running the tests. $ nosetests --with-coverage --cover-erase What just happened? In step 1, we have a couple of TestCase classes with some very basic tests in them. These tests wouldn't be much use in a real world situation, but all we need them for is to illustrate how the code coverage tool works. In step 2, we have the code that satisfies the tests from step 1. Like the tests themselves, this code wouldn't be much use, but it serves as an illustration. In step 4, we passed --with-coverage and --cover-erase as command line parameters when we ran Nose. What did they do? Well, --with-coverage is pretty straightforward: it told Nose to look for coverage.py and to use it while the tests execute. That's just what we wanted. The second parameter, --cover-erase, tells Nose to forget about any coverage information that was acquired during previous runs. By default, coverage information is aggregated across all of the uses of coverage.py. This allows you to run a set of tests using different testing frameworks or mechanisms, and then check the cumulative coverage. You still want to erase the data from previous test runs at the beginning of that process, though, and the --cover-erase command line is how you tell Nose to tell coverage.py that you're starting anew. What the coverage report tells us is that 9/12 (in other words, 75%) of the executable statements in the toy module were executed during our tests, and that the missing lines were line 16 and a lines 19 through 20. Looking back at our code, we see that line 16 is the __repr__ method. We really should have tested that, so the coverage check has revealed a hole in our tests that we should fix. Lines 19 and 20 are just code to run doctest, though. They're not something that we ought to be using under normal circumstances, so we can just ignore that coverage hole. Code coverage can't detect problems with the tests themselves, in most cases. In the above test code, the test for the timestwo method violates the isolation of units and invokes two different methods of example_class. Since one of the methods is the constructor, this may be acceptable, but the coverage checker isn't in a position to even see that there might be a problem. All it saw was more lines of code being covered. That's not a problem— it's how a coverage checker ought to work— but it's something to keep in mind. Coverage is useful, but high coverage doesn't equal good tests. Version control hooks Most version control systems have the ability to run a program that you've written in response to various events, as a way of customizing the version control system's behavior. These programs are commonly called hooks. Version control systems are programs for keeping track of changes to a source code tree, even when those changes are made by different people. In a sense, they provide an universal undo history and change log for the whole project, going all the way back to the moment you started using the version control system. They also make it much easier to combine work done by different people into a single, unified entity, and to keep track of different editions of the same project. You can do all kinds of things by installing the right hook programs, but we'll only focus on one use. We can make the version control program automatically run our tests, when we commit a new version of the code to the version control repository. This is a fairly nifty trick, because it makes it difficult for test-breaking bugs to get into the repository unnoticed. Somewhat like code coverage, though there's potential for trouble if it becomes a matter of policy rather than simply being a tool to make your life easier. In most systems, you can write the hooks such that it's impossible to commit code that breaks tests. That may sound like a good idea at first, but it's really not. One reason for this is that one of the major purposes of a version control system is communication between developers, and interfering with that tends to be unproductive in the long run. Another reason is that it prevents anybody from committing partial solutions to problems, which means that things tend to get dumped into the repository in big chunks. Big commits are a problem because they make it hard to keep track of what changed, which adds to the confusion. There are better ways to make sure you always have a working codebase socked away somewhere, such as version control branches. Bazaar Bazaar is a distributed version control system, which means that it is capable of operating without a central server or master copy of the source code. One consequence of the distributed nature of Bazaar is that each user has their own set of hooks, which can be added, modified, or removed without involving anyone else. Bazaar is available on the Internet at http://bazaar-vcs.org/. If you don't have Bazaar already installed, and don't plan on using it, you can skip this section. Time for action – installing Nose as a Bazaar post-commit hook Bazaar hooks go in your plugins directory. On Unix-like systems, that's ~/.bazaar/plugins/, while on Windows it's C:Documents and Settings<username>Application DataBazaar<version>plugins. In either case, you may have to create the plugins subdirectory, if it doesn't already exist. Place the following code into a file called run_nose.py in the plugins directory. Bazaar hooks are written in Python: from bzrlib import branch from os.path import join, sep from os import chdir from subprocess import call def run_nose(local, master, old_num, old_id, new_num, new_id): try: base = local.base except AttributeError: base = master.base if not base.startswith('file://'): return try: chdir(join(sep, *base[7:].split('/'))) except OSError: return call(['nosetests']) branch.Branch.hooks.install_named_hook('post_commit', run_nose, 'Runs Nose after each commit') Make a new directory in your working files, and put the following code into it in a file called test_simple.py. These simple (and silly) tests are just to give Nose something to do, so that we can see that the hook is working. from unittest import TestCase class test_simple(TestCase): def test_one(self): self.assertNotEqual("Testing", "Hooks") def test_two(self): self.assertEqual("Same", "Same") Still in the same directory as test_simple.py, run the following commands to create a new repository and commit the tests to it. The output you see might differ in details, but it should be quite similar overall. $ bzr init $ bzr add $ bzr commit Notice that there's a Nose test report after the commit notification. From now on, any time you commit to a Bazaar repository, Nose will search for and run whatever tests it can find within that repository. What just happened? Bazaar hooks are written in Python, so we've written our hook as a function called run_nose. Our run_nose function checks to make sure that the repository which we're working on is local, and then it changes directories into the repository and runs nose. We registered run_nose as a hook by calling branch.Branch.hooks.install_named_hook.
Read more
  • 0
  • 0
  • 6722
article-image-lets-make-particles
Packt
04 Sep 2013
15 min read
Save for later

Let's Make Particles

Packt
04 Sep 2013
15 min read
(For more resources related to this topic, see here.) The particle systems in Motion 5 are a powerful engine by which we can take nearly any object, image, layer, or group and animate it using the parameters available to us in the HUD and Inspector. A particle system consists of two items—the emitter and the cell. The cell is referenced by the emitter and the emitter creates the animation over a lifespan specified by you. Let's say we had a PNG layer of an orange. If we created particles out of it, that orange would be put into a cell that is referenced by the emitter. You could use the emitter's parameters to duplicate that orange multiple times per second and have it animate in a particular direction until you decide it should end or die. On top of the ability to turn nearly anything your heart desires into particles, in Motion's library there are pre-animated particle emitters available to incorporate in all of your animations. Making particles and changing values in the HUD Let's take a look at how we can create a particle system using a shape from Motion's library and tweak a few of its parameters using the HUD. Getting ready From the exercise files of this article, double-click the 07_01 project. There is the Shape layer in the Layers tab, whose scale has been animated to repeat for the duration of the project. Our goal is to place the heart in a particle system so that we can have hundreds rain down onto the Canvas. How to do it... The following steps will take you through creating your first particle system: Make sure your playhead is at the beginning of your project. Select the Pink Heart layer and press E , or from the toolbar choose Create a particle emitter (the icon with the three bubbles rising up, as shown in the following screenshot): Play back the project. Several things just happened after you pressed that button. In the Layers tab, notice that the Pink Heart layer has been turned off. A cell and emitter have been created just above it. The cell holds information about the heart while the emitter is creating all the duplicate copies of the heart that are shooting out in a 360-degree circle across the screen at the same speed. You can see a screenshot of this next. Press F7 to bring up the HUD and see some of these parameters in more detail: Right now, thirty hearts are being born every second and live for a duration of five seconds where they pop off the screen. All the duplicate hearts also hold the original scale of the heart being referenced. Change the Birth Rate value to 5 and Life to 10 . Bring down the Scale parameter to 50 , as shown in the following screenshot: Instead of having the hearts come from the center of the screen, let's have them rain down from the top of the Canvas. Decrease the size of your Canvas by clicking on it and pressing Command + - a few times. Select the particle emitter and drag it up and offscreen. Use the Shift key to constrain the movement. Change the Emission Range slider from 360 to 180 and make sure the arrows point down. Play back the animation and tweak the Birth Rate and Scale sliders as desired. See the following screenshot for reference: There's more… If you're finding the HUD limited in terms of the options available to you for the emitter, don't worry, the Inspector has several additional parameters, including the option to add random values to your emitter that will add more realism to it! Take a peak at the following screenshot: Tweaking particle parameters in the Inspector In the previous recipe, we switched a few parameters in the HUD for our particle system of hearts, but if we're looking to fine-tune our animation, we need to go to the Inspector. Let's take a look at the several additional parameters available. Getting ready From the exercise files of this article, double-click the 07_02 project. Play back the project. There is a particle system in the center of the Canvas being emitted in the form of a rectangle. Currently, thirty hearts are being born every second and live for five seconds. We're going to tweak our particle system to change the flow of the animation so that our hearts emit out in a circle and away from the screen. How to do it... By following these steps, we'll gain a better understanding of how we can control the particle system: Select the Emitter in the Layers tab. Press F4 to open the Emitter tab of the Inspector. Change the Shape menu's option from rectangle to circle and the Arrangement option to Random Fill . Set the radius to 500. Increase Birth Rate to 200 and set Speed in Cell Controls to 0. If you play back the project, you'll see the hearts trying to form the shape of a circle. Since there is no speed value, the hearts stay put in the shape they originate from, as shown in this screenshot: Set Angle Value to 15 and Spin to 30. Rather than having the particles pop onscreen when they're born and pop offscreen when they die, we're going to add some tags to the Opacity Over Life parameter. The trick is to click the white line where we want to add a tag above it. Click the line once close to the beginning and twice close to the end. Use the following screenshot for reference: Click on the first tag and under the line, drag the Opacity slider to 0 (indicated in this screenshot). Repeat this step for the last slider. Now the hearts fade in at birth and fade out at death: As a final step, we're going to keyframe the speed of the particle system to start at 100 percent and eventually go down to 0 to reveal the shape of the circle we created. Go to the beginning of the Timeline and click the diamond icon next to Speed to add a keyframe. Change its value to 100. Move to three seconds and add another keyframe by clicking the diamond icon next to Speed again. Move to three seconds and set Speed to 0. Play back the animation and compare it to the following screenshot: There's more... You can create some amazing animations by changing the type of shape used to emit particles. For instance, if you change the shape of your particles to Image and use text as the source, the particles will spell out the word as long there is a substantial number of hearts being born and the Arrangement option is set to Random Fill . We'll see an example of this in a later exercise, but the following screenshot gives you a sneak peak on the end results you can get: Adding randomness values To give your particle systems just a little more, we're going to add a little randomness to some of the parameters. This randomness will help give your animations a more organic feel. Getting ready From the exercise files of this article, double-click the 07_03 project. Play back the project. The particle system consists of little demograms that rain down from a line. The Angle and Spin values cause the demograms to rotate while moving, and the Opacity Over Life parameter allows each particle to fade in and out during its lifespan. The particles also change color over their life as reflected in the Color Mode menu and the gradient outlining the cycle. We're going to add randomness values to tweak the animation. Whenever we add a randomness value, it's going to look at the original value above it and add and subtract to it randomly every second based on the number you specify. For example, say the Scale value is set to 50. If you add a Scale randomness of 10, every second the demograms will be somewhere between a scale of 40 and 60. How to do it... Let's start to randomize different values in this recipe: Select the emitter in the Layers tab and press F4 . In the Emitter tab of the Inspector, change the Birth Rate Randomness slider's value to 10. Now, every second, somewhere between 10 and 30 objects are born. Now, set Life Randomness to 3, Angle Randomness to 40, Spin Randomness to 50, and Scale Randomness to 80. Play back the project. Now we have our demograms growing at different sizes, being born at different angles and spinning at different rates! The following screenshot shows the Inspector with all of the mentioned changes and also a Color Over Life change (see the previous recipe to learn how to do it). The following is a screenshot showing a frame of the particle animation in the Canvas with random values applied: While playing back, click the Generate button next to Random Seed . You'll notice that your animation changes and a new number is created next to Generate . Random Seed takes all the random parameters and creates the animation based on this number. Change this number and change the randomness of the animation, as shown here: Working with particle behaviors While there are already a ton of parameters you can animate by keyframing their values in the particle and cell emitters, Motion offers you a few particle behaviors worth taking a look at from the library as well. Getting ready From the exercise files of this article, double-click the 07_04 project. Play back the project. The animation consists of a bouncing alarm clock from Motion's library in the foreground and a particle system referencing that alarm clock in the background. Unlike the other particle systems we've worked with in the previous recipes, the source of the particle system has been turned back on to preserve it. Our goal is to add two particle behaviors to have the clocks scale down and spin over its life cycle. By adjusting the speed and direction of our emitter, we will also have the clocks look like they're being pulled off into the distance. Also note, the anchor point of the clock was adjusted in advance in order to have the particles spin around the center of the clock. How to do it... Open the Particle group and select Emitter . Press Command + 2 to go to the Library tab. Under Behaviors , choose Particles | Spin Over Life . Click Apply . Press F7 to open the HUD. Set Increment Type to Birth and Death Values . Set Birth to 360 and Death to 0. If you're having trouble, try changing the values in the Behaviors tab of the Inspector. Now, play back the animation. Now, let's have the clocks in the particle system scale over life. Select the Emitter again. Press Command + 2 to go to the Library . Under Behaviors , choose Particles | Scale Over Life .Click Apply. In the HUD, set Increment Type to Birth and Death Values . Set Scale at Birth to 100 and Scale at Death to 0. If you're having trouble, try changing the values in the Behaviors tab of the Inspector. Play back the animation. The clocks now scale down gradually until they die, as shown in the following screenshot: Press F4 to go to the Emitter tab of the Inspector. Change the Emission Longitude value to 180 and Speed to 2000 , as shown in the following screenshot. Change the other parameters as desired. There's more... When you start building complex particle animations, keep in mind that you can save them to the Library as well as any of the behaviors you tweak. All particle presets can be found in the Library under Particle Emitters in a dedicated theme folder you create. The more time you spend automating your work, the more time you can dedicate to the creative process! Working with particle presets So far, we've created our own particle systems by selecting objects to place into cells referenced by the emitter. Motion ships with over 200 presets! One of the best ways to learn Motion is to dissect how some of the particle presets were created. Let's take a look at what it has to offer! Getting ready From the exercise files of this article, double-click on the 07_05 project. There are two particle presets in this project but one has been turned off. Play back the project and familiarize yourself with the Magic Dust preset. Stop the playback. Turn off the Magic Dust group and turn on the Buskerit group. Play back the project again. In this recipe, we'll take a look at each of these presets and change a few parameters to look at how the animations were created. How to do it... Let's now tweak these parameters of our presets in more detail: Reveal the content of the Buskerit group by clicking the disclosure triangle next to it. It contains a group called busker and seven still images that have been turned off. The images are being referenced by the emitter. Click the disclosure triangle next to the Busker group. It contains the content of the Busker emitter and the seven cells that hold information from those layers that have been shot off. The Emitter also has a Scale Over Life behavior in it. While the object continually changes scale over the animation, the Scale Over Life behavior was used to animate the "pop-up" intro of the instruments, whereas keyframes were used to change the scale over the project. Select the Scale Over Life behavior and press F2 to go to the Behaviors tab of the Inspector. Notice the Increment Type parameter set to Custom . Click the disclosure triangle next to Custom Scale to see the graph, as shown in the following screenshot: Select the Emitter in the Layers tab and press Command + 8 to open the Keyframe Editor . Notice that both the scale and rotation of the emitter are keyframed periodically throughout the entire animation, as shown in the following screenshot. In the Keyframe Editor , press Shift + K to move forward between keyframes. Change the Rotation and Scale values by double-clicking the values while on an existing keyframe. You'll know you're on one because the diamond shape will appear highlighted! Click the disclosure triangle to close the Buskerit group, turn it off, and turn on the Magic Dust group. Click the disclosure triangle to open it. A still image of a spark was used as the source for the Emitter. Several behaviors were used to animate the sparks. With the emitter selected, press F4 to go to the Emitter tab of the Inspector. Set Emission Range to 45 , Birth Rate to 200 , and Speed to 1000 . Play back the project to see the effect it has and compare it to the following: Select the Magic Dust group and go to the gear icon under the mini-Timeline. Choose Basic Motion | Motion Path . Tweak the path as desired. Play back the animation. There's more... Learning how particle presets work is a great way to get used to Motion. Particle presets are the best way to learn Motion! Looking at the library and seeing how something was created is the best way to get under the hood of the application and start creating your own animations. Don't be afraid to explore. Working with particle presets in 3D Particle presets are already powerful on their own but turn them into 3D and you'll find that a few of them will actually look like they have been extruded. Let's take a look! Getting ready From the exercise files of this article, double-click on the 07_06 project. Play back the project. There is a particle system of thin water being projected across the screen. Go to the Properties tab in the Inspector and click the rotation disclosure triangle to see the object rotate 90 degrees on its y axis. The water becomes invisible because it's flat the minute you change perspective. If we promote a particle system to 3D, we can get rid of this problem and have a few additional options available to us in terms of the way we emit the water. How to do it... Let's get rid of the flatness of our particle system by making it 3D: Select the Particle Emitter category and press F4 to go to the Inspector. Click on the 3D checkbox. As soon as you press the button, more options become available to you in the Emitter Controls section, including Emission Latitude and Emission Longitude , as seen here: Two additional selections—Box and Sphere —also became available under the Shape drop-down. Click on the word Rectangle and choose Box . Press F1 to go to the Properties tab of the Inspector. Click the disclosure triangle next to Scale and increase Scale Z to 200 . Rotate the water 90 degrees on the Y axis and notice it's no longer flat. Use the following screenshot of the Properties tab for reference: Go to the beginning of the Timeline and bring back Rotation Y to 0. Click the diamond shape next to Rotation Y to add a keyframe. Move to the end of the Timeline; and change the parameter to 720. Play back the project and see that, no matter what angle it is, the particle system never goes flat, as shown here:
Read more
  • 0
  • 0
  • 6722

article-image-deploying-your-applications-websphere-application-server-70
Packt
01 Oct 2009
9 min read
Save for later

Deploying Your Applications on WebSphere Application Server 7.0

Packt
01 Oct 2009
9 min read
Data access applications We have just deployed an application that did not require database connectivity. Often, applications in the business world require access to a RDBMS to fulfill their business objective. If an application requires the ability to retrieve from, or store information in, a database, then you will need to create a data source which will allow the application to connect and use the database (DB). Looking at the figure below, we can see the logical flow of the sample data access application that we are going to install. The basic idea of the application is to display a list of tables that exist in a database schema. Since the application requires a database connection, we need to configure WebSphere before we can deploy the application. We will now cover the preparation work before we install our application. Data sources Each data source is associated with a JDBC provider that is configured for access to a specific database type. The data source provides connectivity which allows an application to communicate with the database. Preparing our sample database Before you create a data source, you need to ensure that the appropriate client database driver software is installed. For our demonstration, we are going to use Oracle Express Edition (Oracle XE) for Linux which is the free version of Oracle. We are using version Oracle XE 10g for Linux and the download size is about 210MB, so it will take time to download. We installed Oracle XE using the default install option for installing an RPM. The administration process is fully documented on Oracle's web site and in the documentation which is installed with the product. We could have chosen to use many open source/free databases, however their explanations and configurations would detract from the point. We have chosen to use Oracle's free RDBMS called Oracle XE, and JDBC with Oracle XE is quite easy to configure. By following these steps, you will be able to apply the same logic to any of the major vendors' full RDMS products, that is, DB/2, Oracle, SQL Server, and so on. Another reason why we chose Oracle XE is that it is an enterprise-ready DB and is administered by a simple web interface and comes with sample databases. We need to test that we can connect to our database without WebSphere so that we can evaluate the DB design. To do this, we will need to install Oracle XE. We will now cover the following steps one by one. Download Oracle XE from Oracle's web site using the following URL:http://www.oracle.com/technology/products/database/xe/index.html. Transfer the oracle-xe-10.2.0.1-1.0.i386.rpm file to an appropriate directory on your Linux server using WinSCP (Secure Copy) or your chosen Secure FTP client. Since the XE installer uses X Windows, ensure that you have Xming running. Then install Oracle XE by using the rpm command, as shown here: rpm -ivh oracle-xe-10.2.0.1-1.0.i386.rpm Follow the installer steps as prompted: HTTP port = 8080 Listener port = 1521 SYS & SYSTEM / password = oracle Autostart = y Oracle XE requires 1024 minimum swap space and requires 1.5 GB of disk space to install. Ensure that Oracle XE is running. You can now access the web interface via a browser from the local machine; by default, XE will only accept a connection locally. As shown in the following figure, we have a screenshot of using Firefox to connect to OracleXE using the URL http://localhost:8080/apex. The reason we use Firefox on Linux is that this is the most commonly installed default browser on the newer Linux distributions. When the administration application loads, you will be presented with a login screen as seen in the following screenshot. You can log in using the username SYSTEM and password oracle as set by your installation process. Oracle XE comes with a pre-created user called HR which is granted ownership to the HR Schema. However, the account is locked by default for security reasons and so we need to unlock the HR user account. To unlock an account, we need to navigate to the Database Users | Manage Users screen, as demonstrated in the following screenshot: You will notice that the icon for the HR user is locked. You will see a small padlock on the HR icon, as seen in this figure: Click on the HR user icon and unlock the account as shown in the following figure. You need to reset the password and change Account Status to Unlocked, and then click Alter User to set the new password. The following figure shows that the HR account is unlocked: The HR account is now unlocked as seen above. Log out and log back into the administration interface using the HR user to ensure that the account is now unlocked. Another good test to perform to ensure connectivity to Oracle is to use an Oracle admin tool called sqlplus. Sqlplus is a command line tool which database administrators can use to administer Oracle. We are going to use sqlplus to do a simple query to list the tables in the HR schema. To run sqlplus, we need to set up an environment variable called $ORACLE_HOME which is required to run sqlplus. To set $ORACLE_HOME, type the following command in a Linux shell: export ORACLE_HOME=/usr/lib/oracle/xe/app/oracle/product/10.2.0/server If you have installed Oracle XE in a non-default location, then you may have to use a different path. To run sqlplus, type the following command: <oracle_home>/bin/sqlplus The result will be a login screen as shown below: You will be prompted for a username. Type the following command: hr@xe<enter> For the password, type the following command: hr<enter> When you have successfully logged in, you can type the following commands in the SQL prompt: SELECT TABLE_NAME FROM user_tables<enter> /<enter> The / command means execute the command buffer. The result will be a list of tables in the HR schema, as shown in the following screenshot: We have now successfully verified that Oracle works from a command line, and thus it is very likely that WebSphere will also be able to communicate with Oracle. Next, we will cover how to configure WebSphere to communicate with Oracle. JDBC providers Deployed applications use JDBC providers to communicate with RDBMS. The JDBC provider object provides the actual JDBC driver implementation class for access to a specific database type, that is, Oracle, SQL Server, DB/2, and so on. You associate a data source with a JDBC provider. A data source provides the connection to the RDBMS. The JDBC provider and the data source provide connectivity to a database. Creating a JDBC provider Before creating a JDBC provider, you will need to understand the application's resource requirements, that is, the data sources that the application references. You should know the answer to the following questions: Does your application require a data source? Not all applications use a database. The security credentials required to connect to the database. Often databases are secured and you will need a username and password to access a secure database. Are there any web components (Servlets, JSP, and so on) or EJBs which need to access a database. Answering these questions will determine the amount of configuration required for your database connectivity configurations. To create a JDBC provider, log into the administration console and click on the JDBC Provider link in the JDBC category of the Resources section located in the left-hand panel of the administration console as shown below. We need to choose an appropriate scope from the Scope drop-down pick list. Scope determines how the provider will be seen by applications. We will talk more about scope in the JNDI section. For now, please choose the Cell scope as seen below. Click New and the new JDBC provider wizard is displayed. Select the Database type as Oracle, Provider type as Oracle JDBC Driver, Implementation type as Connection pool data source, and Name for the new JDBC provider. We are going to enter MyJDBCDriver as the provider name as seen in the previous screenshot. We also have to choose an Implementation type. There are two implementation types for Oracle JDBC Drivers. The table below explains the two different types. Implementation Type Description Connection pool data source Use Connection Pool datasource if your application does not require connection that supports two-phase commit transactions... XA Datasource Use XA Datasource if your application requires two-phase commit transactions. Click Next to go to the database classpath screen. As shown in the following screenshot, enter the database class path information for the JDBC provider. As long as you have installed Oracle XE using the default paths, you will be able to use the following path in the Directory location field: /usr/lib/oracle/xe/oracle/product/10.2.0/server/jdbc/lib. Click Next to proceed to the next step, where you will be presented with a summary as shown in the following screenshot. Review the JDBC provider information that you have entered and click Finish. You will now be prompted to save the JDBC provider configuration. Click Save, as shown in the following screenshot. Saving this will persist the configuration to disk the resources to resources.xml. Before we finish, we need to update the JDBC Provider with the correct JAR file as the default one is not the one that we wish to use as it was assuming a later Oracle driver which we are not using. To change the driver, we must first select the driver that we created earlier called MyJDBCDriver as shown in the following screenshot: In the screen presented, we are going to change the Classpath field from: ${ORACLE_JDBC_DRIVER_PATH}/ojdbc6.jar to ${ORACLE_JDBC_DRIVER_PATH}/ojdbc14.jar Since WAS 7.0 is the latest version of WebSphere, the wizard already knows about the new version of the oracle 11g JDBC Driver. We are connecting to Oracle XE 10g and the driver for this is ojdbc14.jar.The classpath file can contain a list of paths or JAR file names which together form the location for the resource provider classes. Class path entries are separated by using the ENTER key and must not contain path separator characters (such as ; or :). Class paths can contain variable (symbolic) names that can be substituted using a variable map. Check your driver installation notes for specific JAR file names that are required. Click Apply and save the configuration.
Read more
  • 0
  • 0
  • 6721

article-image-analyzing-complex-dataset
Packt
16 Apr 2014
6 min read
Save for later

Analyzing a Complex Dataset

Packt
16 Apr 2014
6 min read
(For more resources related to this topic, see here.) We may need to analyze volumes of data that are too large for a simple spreadsheet. A document with more than a few hundred rows can rapidly become bewildering. Handling thousands of rows can be very challenging, indeed. How can we do this more sophisticated analysis? The answer for many such analytic problems is Python. It handles larger sets of data with ease. We can very easily write sophisticated sorting, filtering and calculating rules. We're not limited by the row-and-column structure, either. We'll look at some data that—on the surface—is boring, prosaic budget information. We've chosen budgets because they're relatively simple to understand with a lot of problem domain background. Specifically, we'll download the budget for the city of Chicago, Illinois. Why Chicago? Chicago has made their data available to the general public, so we can use this real-world data for an example that's not contrived or simplified. A number of other municipalities have also made similar data available. How do we proceed? Clearly, step one is to get the data so we can see what we're working with. For more information, we can start with this URL: https://data.cityofchicago.org/ The city's data portal offers a number of formats: CSV, JSON, PDF, RDF, RSS, XLS, XLSX, and XML. Of these, the JSON is perhaps the easiest to work with. We can acquire the appropriation information with the following URL: https://data.cityofchicago.org/api/views/v9er-fp6q/rows.json?accessType=DOWNLOAD This will yield a JSON-format document. We can gather this data and cache it locally with a small Python script. import urllib.request budget_url= "https://data.cityofchicago.org/api/views/v9er-fp6q/rows.json?accessType=DOWNLOAD" with open( "budget_appropriations.json", "w") as target: with urllib.request.urlopen(budget_url) as document: target.write( document.read() ) We can use a similar script to gather the salary data that goes with the budget appropriation. The salary information will use this URL: https://data.cityofchicago.org/api/views/etzw-ycze/rows.json?accessType=DOWNLOAD Clearly, we can create a more general script to download from these two slightly different URL's to gather two very different JSON files. We'll focus on the appropriations for this article because the data organization turns out to be simpler. We could download this data manually using a browser. It rapidly becomes difficult to automate data collection and analysis when we introduce a manual step involving a person pointing and clicking using a browser. A similar comment applies to trying to use a spreadsheet to analyze the data: merely putting it on a computer doesn't really automate or formalize a process. The results of step one, then, are two files: budget_appropriations.json and budget_salaries.json. Parsing the JSON document Since the data is encoded in JSON, we can simply open the files in our development environment to see what they look like. Informally, it appears that two data sets have some common columns and some distinct columns. We'll need to create a more useful side-by-side comparison of the two files. We'll import the JSON module. We almost always want to pretty-print during exploration, so we'll import the pprint() function just in case we need it. Here are the first two imports: import json from pprint import pprint One thing we may have noted when looking at the JSON is that are two important-looking keys: a 'meta' key and a 'data' key. The 'meta' key is associated with a sequence of column definitions. The 'data' object is associated with a sequence of rows of actual data. We can use a quick script like the following to discover the details of the metadata: def column_extract( ): for filename in "budget_appropriations.json", "budget_salaries.json": with open(filename) as source: print( filename ) dataset= json.load( source ) for col in dataset['meta']['view']['columns']: if col['dataTypeName'] != "meta_data": print( col['fieldName'], col['dataTypeName'] ) print() We've opened each of our source files, and loaded the JSON document into an internal mapping named 'dataset'. The metadata can be found by navigating through the dictionaries that are part of this document. The path is dataset['meta']['view']['columns']. This leads to a sequence of column definitions. For each column definition, we can print out two relevant attributes using the keys 'fieldName' and 'dataTypeName'. This will reveal items that are dimensions and items that are facts within this big pile of data. This small function can be used in a short script file to see the various columns involved. We can write a short script like this: if __name__ == "__main__": column_extract() We can see that we have columns which are any type number, text, and money. The number and text columns can be termed "dimensions", they describe the facts. The money columns are the essential facts that we'd like to analyze. Designing around a user story Now that we have the data, it makes sense to see where we're going with it. Our goal is to support queries against salaries and appropriations. Our users want to see various kinds of subtotals and some correlations between the two datasets. The terms "various kinds" reveal that the final analysis details are open-ended. It's most important for us to build a useful model of the data rather than solve some very specific problem. Once we have the data in a usable model, we can solve a number of specific problems. A good approach is to follow Star Schema or Facts and Dimensions design pattern that supports data warehouse design. We'll decompose each table into facts—measurements that have defined units—and dimensions that describe those facts. We might also call the dimensions attributes of Business Entities that are measured by the facts. The facts in a budget analysis context are almost always money. Almost everything else will be some kind of dimension: time, geography, legal organization, government service, or financial structure. In the long run, we might like to load a relational database using Python objects. This allows a variety of tools to access the data. It would lead to more complex technology stack. For example, we'd need an Object-Relational Mapping (ORM) layer in addition to a star schema layer. For now, we'll populate a pure python model. We'll show how this model can be extended to support SQLAlchemy as an ORM layer.
Read more
  • 0
  • 0
  • 6717
article-image-miscellaneous-tips
Packt
29 Oct 2013
24 min read
Save for later

Miscellaneous Tips

Packt
29 Oct 2013
24 min read
(For more resources related to this topic, see here.) Mission Briefing The topics covered here include: Tracing Tkinter variables Widget traversal Validating user input Formatting widget data More on fonts Working with Unicode characters Tkinter class hierarchy Custom-made mixins Tips for code cleanup and program optimization Distributing the Tkinter application Limitations of Tkinter Tkinter alternatives Getting interactive help Tkinter in Python 3. x Tracing Tkinter variables When you specify a Tkinter variable as a textvariable for a widget (textvariable = myvar), the widget automatically gets updated whenever the value of the variable changes. However, there might be times when, in addition to updating the widget, you need to do some extra processing at the time of reading or writing (or modifying) the variable. Tkinter provides a method to attach a callback method that would be triggered every time the value of a variable is accessed. Thus, the callback acts as a variable observer. The callback method is named trace_variable(self, mode, callback), or simply trace(self, mode, callback). The mode argument can take any one of 'r', 'w', 'u' values, which stand for read, write, or undefined. Depending upon the mode specifications, the callback method is triggered if the variable is read or written. The callback method gets three arguments by default. The arguments in order of their position are: Name of the Tkinter variable The index of the variable, if the Tkinter variable is an array, else an empty string The access modes ('w', 'r', or 'u') Note that the triggered callback function may also modify the value of the variable. This modification does not, however, trigger any additional callbacks. Let's see a small example of variable tracing in Tkinter, where writing into the Tkinter variable into an entry widget triggers a callback function (refer to the 8.01 trace variable.py Python file available in the code bundle): from Tkinter import * root = Tk() myvar = StringVar() def trace_when_myvar_written(var,indx,mode): print"Traced variable %s"%myvar.get() myvar.trace_variable("w", trace_when_myvar_written) Label(root, textvariable=myvar).pack(padx=5, pady=5) Entry(root, textvariable=myvar).pack(padx=5, pady=5) root.mainloop() The description of the preceding code is as follows: This code creates a trace variable on the Tkinter variable myvar in the write ("w") mode The trace variable is attached to a callback method named trace_when_myvar_written (this means that every time the value of myvar is changed, the callback method will be triggered) Now, every time you write into the entry widget, it modifies the value of myvar. Because we have set a trace on myvar, it triggers the callback method, which in our example, simply prints the new value into the console. The code creates a GUI window similar to the one shown here: It also produces a console output in IDLE, which shows like the following once you start typing in the GUI window: Traced variable T Traced variable Tr Traced variable Tra Traced variable Trac Traced variable Traci Traced variable Tracin Traced variable Tracing The trace on a variable is active until it is explicitly deleted. You can delete a trace using: trace_vdelete(self, mode, callbacktobedeleted) The trace method returns the name of the callback method. This can be used to get the name of the callback method that is to be deleted. Widget traversal When a GUI has more than one widget, a given widget can come under focus by an explicit mouse-click on the widget. Alternatively, the focus can be shifted to another given widget by pressing the Tab key on the keyboard in the order the widgets were created in the program. It is therefore vital to create widgets in the order we want the user to traverse through them, or else the user will have a tough time navigating between the widgets using the keyboard. Different widgets are designed to behave differently to different keyboard strokes. Let's therefore spend some time trying to understand the rules of traversing through widgets using the keyboard. Let's look at the code of the 8.02 widget traversal.py Python file to understand the keyboard traversal behavior for different widgets. Once you run the mentioned .py file, it shows a window something like the following: The code is simple. It adds an entry widget, a few buttons, a few radio buttons, a text widget, and a scale widget. However, it also demonstrates some of the most important keyboard traversal behaviors for these widgets. Here are some important points to note (refer to 8.02 widget traversal.py): The Tab key can be used to traverse forward, and Shift + Tab can be used to traverse backwards. The text widget cannot be traversed using the Tab key. This is because the text widget can contain tab characters as its content. Instead, the text widget can be traversed using Ctrl + Tab. Buttons on the widget can be pressed using the spacebar. Similarly, check buttons and radio buttons can also be toggled using the spacebar. You can go up and down the items in a Listbox widget using the up and down arrows. The Scale widget responds to both the left and right keys or up and down keys. Similarly, the Scrollbar widget responds to both the left/right or up/down keys, depending on their orientation. Most of the widgets (except Frame, Label, and Menus) get an outline by default when they have the focus set on them. This outline normally displays as a thin black border around the widget. You can even set the Frame and Label widgets to show this outline by specifying the highlightthickness option to a non-zero Integer value for these widgets. We change the color of the outline using highlightcolor= 'red' in our code. Frame, Label, and Menu are not included in the tab navigation path. However, they can be included in the navigation path by using the takefocus = 1 option. You can explicitly exclude a widget from the tab navigation path by setting the takefocus= 0 option. The Tab key traverses widgets in the order they were created. It visits a parent widget first (unless it is excluded using takefocus = 0) followed by all its children widgets. You can use widget.focus_force() to force the input focus to the widget. Validating user input Let's now discuss input data validation. Most of the applications we have developed in this article are point and click-based (drum machine, chess, drawing application), where validation of user input is not required. However, data validation is a must in programs like our phonebook application, where the user enters some data, and we store it in a database. Ignoring the user input validation can be dangerous in such applications because input data can be misused for SQL injection. In general, any application where an user can enter textual data, is a good candidate for validating user input. In fact, it is almost considered a maxim not to trust user inputs. A wrong user input may be intentional or accidental. In either case, if you fail to validate or sanitize the data, you may cause unexpected error in your program. In worst cases, user input can be used to inject harmful code that may be capable of crashing a program or wiping out an entire database. Widgets such as Listbox, Combobox, and Radiobuttons allow limited input options, and hence, cannot normally be misused to input wrong data. On the other hand, widgets such as Entry widget, Spinbox widget, and Text widget allow a large possibility of user inputs, and hence, need to be validated for correctness. To enable validation on a widget, you need to specify an additional option of the form validate = 'validationmode' to the widget. For example, if you want to enable validation on an entry widget, you begin by specifying the validate option as follows: Entry( root, validate="all", validatecommand=vcmd) The validation can occur in one of the following validation modes: Validation Mode Explanation none This is the default mode. No validation occurs if validate is set to "none" focus When validate is set to "focus", the validate command is called twice; once when the widget receives focus and once when the focus is lost focusin The validate command is called when the widget receives focus focusout The validate command is called when the widget loses focus key The validate command is called when the entry is edited all The validate command is called in all the above cases The code of the 8.03 validation mode demo.py file demonstrates all these validation modes by attaching them to a single validation method. Note the different ways different Entry widgets respond to different events. Some Entry widgets call the validation method on focus events while others call the validation method at the time of entering key strokes into the widget, while still others use a combination of focus and key events. Although we did set the validation mode to trigger the validate method, we need some sort of data to validate against our rules. This is passed to the validate method using percent substitution. For instance, we passed the mode as an argument to our validate method by performing a percent substitution on the validate command, as shown in the following: vcmd = (self.root.register(self.validate), '%V') We followed by passing the value of v as an argument to our validate method: def validate(self, v) In addition to %V, Tkinter recognizes the following percent substitutions: Percent substitutions Explanation %d Type of action that occurred on the widget-1 for insert, 0 for delete, and -1 for focus, forced, or textvariable validation. %i Index of char string inserted or deleted, if any, else it will be -1. %P The value of the entry if the edit is allowed. If you are configuring the Entry widget to have a new textvariable, this will be the value of that textvariable. %s The current value of entry, prior to editing. %S The text string being inserted/deleted, if any, {} otherwise. %v The type of validation currently set. %V The type of validation that triggered the callback method (key, focusin,  focusout, and forced). %W The name of the Entry widget. These validations provide us with the necessary data we can use to validate the input. Let's now pass all these data and just print them through a dummy validate method just to see the kind of data we can expect to get for carrying out our validations (refer to the code of 8.04 percent substitutions demo.py): Take particular note of data returned by %P and %s, because they pertain to the actual data entered by the user in the Entry widget. In most cases, you will be checking either of these two data against your validation rules. Now that we have a background of rules of data validation, let's see two practical examples that demonstrate input validation. Key Validation Let's assume that we have a form that asks for a user's name. We want the user to input only alphabets or space characters in the name. Thus, any number or special character is not to be allowed, as shown in the following screenshot of the widget: This is clearly a case of 'key' validation mode, because we want to check if an entry is valid after every key press. The percent substitution that we need to check is %S, because it yields the text string being inserted or deleted in the Entry widget. Accordingly, the code that validates the entry widget is as follows (refer to 8.05 key validation.py): import Tkinter as tk class KeyValidationDemo(): def __init__(self): root = tk.Tk() tk.Label(root, text='Enter your name').pack() vcmd = (root.register(self.validate_data), '%S') invcmd = (root.register(self.invalid_name), '%S') tk.Entry(root, validate="key", validatecommand=vcmd,invalidcommand=invcmd).pack(pady=5, padx=5) self.errmsg = tk.Label(root, text= '', fg='red') self.errmsg.pack() root.mainloop() def validate_data (self, S): self.errmsg.config(text='') return (S.isalpha() or S =='') # always return True or False def invalid_name (self, S): self.errmsg.config(text='Invalid characters n name canonly have alphabets'%S) app= KeyValidationDemo() The description of the preceding code is as follows: We first register two options validatecommand (vcmd) and invalidcommand (invcmd). In our example, validatecommand is registered to call the validate_data method, and the invalidcommand option is registered to call another method named invalid_name. The validatecommand option specifies a method to be evaluated which would validate the input. The validation method must return a Boolean value, where a True signifies that the data entered is valid, and a False return value signifies that data is invalid. If the validate method returns False (invalid data), no data is added to the Entry widget and the script registered for invalidcommand is evaluated. In our case, a False validation would call the invalid_name method. The invalidcommand method is generally responsible for displaying error messages or setting back the focus to the Entry widget. Let's look at the code register(self, func, subst=None, needcleanup=1). The register method returns a newly created Tcl function. If this function is called, the Python function func is executed. If an optional function subst is provided it is executed before func. Focus Out Validation The previous example demonstrated validation in 'key' mode. This means that the validation method was called after every key press to check if the entry was valid. However, there are situations when you might want to check the entire string entered into the widget, rather than checking individual key stroke entries. For example, if an Entry widget accepts a valid e-mail address, we would ideally like to check the validity after the user has entered the entire e-mail address, and not after every key stroke entry. This would qualify as validation in 'focusout' mode. Check out the code of 8.06 focus out validation.py for a demonstration on e-mail validation in the focusout mode: import Tkinter as tk import re class FocusOutValidationDemo(): def __init__(self): self.master = tk.Tk() self.errormsg = tk.Label(text='', fg='red') self.errormsg.pack() tk.Label(text='Enter Email Address').pack() vcmd = (self.master. register(self.validate_email), '%P' ) invcmd = (self.master. register(self.invalid_email), '%P' ) self.emailentry = tk.Entry(self.master, validate ="focusout", validatecommand=vcmd , invalidcommand=invcmd ) self.emailentry.pack() tk.Button(self.master, text="Login").pack() tk.mainloop() def validate_email(self, P): self.errormsg.config(text='') x = re.match(r"[^@]+@[^@]+.[^@]+", P) return (x != None)# True(valid email)/False(invalid email) def invalid_email(self, P): self.errormsg.config(text='Invalid Email Address') self.emailentry.focus_set() app = FocusOutValidationDemo() The description of the preceding code is as follows: The code has a lot of similarities to the previous validation example. However, note the following differences: The validate mode is set to 'focusout' in contrast to the 'key' mode in the previous example. This means that the validation would be done only when the Entry widget loses focus. This program uses data provided by the %P percentage substitution, in contrast to %S, as used in the previous example. This is understandable as %P provides the value entered in the Entry widget, but %S provides the value of the last key stroke. This program uses regular expressions to check if the entered value corresponds to a valid e-mail format. Validation usually relies on regular expressions and a whole lot of explanation to cover this topic, but it is out of the scope of this project and the article. For more information on regular expression modules, visit the following link: http://docs.python.org/2/library/re.html This concludes our discussion on input validation in Tkinter. Hopefully, you should now be able to implement input validation to suit your custom needs. Formatting widget data Several input data such as date, time, phone number, credit card number, website URL, IP number, and so on have an associated display format. For instance, date is better represented in a MM/DD/YYYY format. Fortunately, it is easy to format the data in the required format as the user enters them in the widget (refer to 8.07 formatting entry widget to display date.py). The mentioned Python file formats the user input automatically to insert forward slashes at the required places to display user-entered date in the MM/DD/YYYY format. from Tkinter import * class FormatEntryWidgetDemo: def __init__(self, root): Label(root, text='Date(MM/DD/YYYY)').pack() self.entereddata = StringVar() self.dateentrywidget =Entry(textvariable=self.entereddata) self.dateentrywidget.pack(padx=5, pady=5) self.dateentrywidget.focus_set() self.slashpositions = [2, 5] root.bind('<Key>', self.format_date_entry_widget) def format_date_entry_widget(self, event): entrylist = [c for c in self.entereddata.get() if c != '/'] for pos in self.slashpositions: if len(entrylist) > pos: entrylist.insert(pos, '/') self.entereddata.set(''.join(entrylist)) # Controlling cursor cursorpos = self.dateentrywidget.index(INSERT) for pos in self.slashpositions: if cursorpos == (pos + 1): # if cursor is on slash cursorpos += 1 if event.keysym not in ['BackSpace', 'Right', 'Left','Up', 'Down']: self.dateentrywidget.icursor(cursorpos) root = Tk() FormatEntryWidgetDemo(root) root.mainloop() The description of the preceding code is as follows: The Entry widget is bound to the key press event, where every new key press calls the related callback format_date_entry_widget method. First, the format_date_entry_widget method breaks down the entered text into an equivalent list by the name entrylist, also ignoring any slash '/' symbol if entered by the user. It then iterates through the self.slashpositions list and inserts the slash symbol at all required positions in the entrylist argument. The net result of this is a list that has slash inserted at all the right places. The next line converts this list into an equivalent string using join(), and then sets the value of our Entry widget to this string. This ensures that the Entry widget text is formatted into the aforementioned date format. The remaining pieces of code simply control the cursor to ensure that the cursor advances by one position whenever it encounters a slash symbol. It also ensures that key presses, such as 'BackSpace', 'Right', 'Left', 'Up', and 'Down' are handled properly. Note that this method does not validate the date value and the user may add any invalid date. The method defined here will simply format it by adding forward slash at third and sixth positions. Adding date validation to this example is left as an exercise for you to complete. This concludes our brief discussion on formatting data within widgets. Hopefully, you should now be able to create formatted widgets for a wide variety of input data that can be displayed better in a given format. More on fonts Many Tkinter widgets let you specify custom font specifications either at the time of widget creation or later using the configure() option. For most cases, default fonts provide a standard look and feel. However, should you want to change font specifications, Tkinter lets you do so. There is one caveat though. When you specify your own font, you need to make sure it looks good on all platforms where the program is intended to be deployed. This is because a font might look good and match well on a particular platform, but may look awful on another. Unless you know what you are doing, it is always advisable to stick to Tkinter's default fonts. Most platforms have their own set of standard fonts that are used by the platform's native widgets. So, rather than trying to reinvent the wheel on what looks good on a given platform or what would be available for a given platform, Tkinter assigns these standard platform-specific fonts into its widget, thus providing a native look and feel on every platform. Tkinter assigns nine fonts to nine different names, which you can therefore use in your programs. The font names are as follows: TkDefaultFont TkTextFont TkFixedFont TkMenuFont TkHeadingFont TkCaptionFont TkSmallCaptionFont TkIconFont TkTooltipFont Accordingly, you can use them in your programs in the following way: Label(text="Sale Up to 50% Off !", font="TkHeadingFont 20") Label(text="**Conditions Apply", font="TkSmallCaptionFont 8") Using these kinds of fonts mark up, you can be assured that your font will look native across all platforms. Finer Control over Font In addition to the above method on handling fonts, Tkinter provides a separate Font class implementation. The source code of this class is located at the following link: <Python27_installtion_dir>Liblib-tktkfont.py. To use this module, you need to import tkFont into your namespace.(refer to 8.08 tkfont demo.py): from Tkinter import Tk, Label, Pack import tkFont root=Tk() label = Label(root, text="Humpty Dumpty was pushed") label.pack() currentfont = tkFont.Font(font=label['font']) print'Actual :' + str(currentfont. actual ()) print'Family :' + currentfont. cget ("family") print'Weight :' + currentfont.cget("weight") print'Text width of Dumpty : %d' %currentfont. measure ("Dumpty") print'Metrics:' + str(currentfont. metrics ()) currentfont.config(size=14) label.config (font=currentfont) print'New Actual :' + str(currentfont. actual ()) root.mainloop() The console output of this program is as follows: Actual :{'family': 'Segoe UI', 'weight': 'normal', 'slant': 'roman', 'overstrike': 0, 'underline': 0, 'size': 9} Family : Segoe UI Weight : normal Text width of Dumpty : 43 Metrics:{'fixed': 0, 'ascent': 12, 'descent': 3, 'linespace': 15} As you can see, the tkfont module provides a much better fine-grained control over various aspects of fonts, which are otherwise inaccessible. Font Selector Now that we have seen the basic features available in the tkfont module, let's use it to implement a font selector. The font selector would look like the one shown here: The code for the font selector is as follows (refer to 8.09 font selector.py): from Tkinter import * import ttk import tkFont class FontSelectorDemo (): def __init__(self): self.currentfont = tkFont.Font(font=('Times New Roman',12)) self.family = StringVar(value='Times New Roman') self.fontsize = StringVar(value='12') self.fontweight =StringVar(value=tkFont.NORMAL) self.slant = StringVar(value=tkFont.ROMAN) self.underlinevalue = BooleanVar(value=False) self.overstrikevalue= BooleanVar(value=False) self. gui_creator () The description of the preceding code is as follows: We import Tkinter (for all widgets), ttk (for the Combobox widget), and tkfont for handling font-related aspects of the program We create a class named FontSelectorDemo and use its __init_ method to initialize al attributes that we intend to track in our program. Finally, the __init__ method calls another method named gui_creator(), which is be responsible for creating all the GUI elements of the program Creating the GUI The code represented here is a highly abridged version of the actual code (refer to 8.09 font selector.py). Here, we removed all the code that creates basic widgets, such as Label and Checkbuttons, in order to show only the font-related code: def gui_creator(self): # create the top labels – code removed fontList = ttk.Combobox(textvariable=self.family) fontList.bind('<<ComboboxSelected>>', self.on_value_change) allfonts = list(tkFont.families()) allfonts.sort() fontList['values'] = allfonts # Font Sizes sizeList = ttk.Combobox(textvariable=self.fontsize) sizeList.bind('<<ComboboxSelected>>', self.on_value_change) allfontsizes = range(6,70) sizeList['values'] = allfontsizes # add four checkbuttons to provide choice for font style # all checkbuttons command attached to self.on_value_change #create text widget sampletext ='The quick brown fox jumps over the lazy dog' self.text.insert(INSERT,'%sn%s'% (sampletext,sampletext.upper()), 'fontspecs' ) self.text.config(state=DISABLED) The description of the preceding code is as follows: We have highlighted the code that creates two Combobox widgets; one for the Font Family, and the other for the Font Size selection. We use tkfont.families() to fetch the list of all the fonts installed on a computer. This is converted into a list format and sorted before it is inserted into the fontList Combobox widget. Similarly, we add a font size range of values from 6 to 70 in the Font Size combobox. We also add four Checkbutton widgets to keep track of font styles bold, italics, underline , and overstrike. The code for this has not been shown previously, because we have created similar check buttons in some of our previous programs. We then add a Text widget and insert a sample text into it. More importantly, we add a tag to the text named fontspec. Finally, all our widgets have a command callback method connecting back to a common method named on_value_change. This method will be responsible for updating the display of the sample text at the time of changes in the values of any of the widgets. Updating Sample Text def on_value_change(self, event=None): try: self.currentfont.config(family=self.family.get(), size=self.fontsize.get(), weight=self.fontweight.get(), slant=self.slant.get(), underline=self.underlinevalue.get(), overstrike=self.overstrikevalue.get()) self.text.tag_config('fontspecs', font=self.currentfont) except ValueError: pass ### invalid entry - ignored for now. You can use a tkMessageBox dialog to show an error The description of the preceding code is as follows: This method is called at the time of a state change for any of the widgets This method simply fetches all font data and configures our currentfont attribute with the updated font values Finally, it updates the text content tagged as fontspec with the values of the current font Working with Unicode characters Computers only understand binary numbers. Therefore, all that you see on your computer, for example, texts, images, audio, video, and so on need to be expressed in terms of binary numbers. This is where encoding comes into play. An encoding is a set of standard rules that assign unique numeral values to each text character. Python 2.x default encoding is ASCII (American Standard Code for Information Interchange). The ASCII character encoding is a 7-bit encoding that can encode 2 ^7 (128) characters. Because ASCII encoding was developed in America, it encodes characters from the English alphabet, namely, the numbers 0-9, the letters a-z and A-Z, some common punctuation symbols, some teletype machine control codes, and a blank space. It is here that Unicode encoding comes to our rescue. The following are the key features of Unicode encoding: It is a way to represent text without bytes It provides unique code point for each character of every language It defines more than a million code points, representing characters of all major scripts on the earth Within Unicode, there are several Unicode Transformation Formats (UTF) UTF-8 is one of the most commonly used encodings, where 8 means that 8-bit numbers are used in the encoding Python also supports UTF-16 encoding, but it's less frequently used, and UTF-32 is not supported by Python 2. x Say you want to display a Hindi character on a Tkinter Label widget. You would intuitively try to run a code like the following: from Tkinter import * root = Tk() Label( root, text = " भारतमेंआपकास्वागतहै " ).pack() root.mainloop() If you try to run the previous code, you will get an error message as follows: SyntaxError: Non-ASCII character 'xe0' in file 8.07.py on line 4, but no encoding declared; see http://www.Python.org/peps/pep-0263.html for details. This means that Python 2.x, by default, cannot handle non-ASCII characters. Python standard library supports over 100 encodings, but if you are trying to use anything other than ASCII encoding you have to explicitly declare the encoding. Fortunately, handling other encodings is very simple in Python. There are two ways in which you can deal with non-ASCII characters. Declaring line encoding The first way is to mark a string containing Unicode characters with the prefix u explicitly, as shown in the following code snippet (refer to 8.10 line encoding.py): from Tkinter import * root = Tk() Label(root, text = u"भारतमेंआपकास्वागतहै").pack() root.mainloop() When you try to run this program from IDLE, you get a warning message similar to the following one: Simply click on Ok to save this file as UTF-8 and run this program to display the Unicode label. Summary In this article, we discussed some vital aspects of GUI programming form a common theme in many GUI programs. Resources for Article: Further resources on this subject: Getting Started with Spring Python [Article] Python Testing: Installing the Robot Framework [Article] Getting Up and Running with MySQL for Python [Article]
Read more
  • 0
  • 0
  • 6716

article-image-setting-most-popular-journal-articles-your-personalized-community-liferay-portal
Packt
21 Oct 2009
6 min read
Save for later

Setting up the most Popular Journal Articles in your Personalized Community in Liferay Portal

Packt
21 Oct 2009
6 min read
Personal community is a dynamic feature of Liferay portal. By default, the personal community is a portal-wide setting that will affect all of the users. It would be nice to have more features in the personal community such as showing the most popular journal articles. This article by Jonas Yuan will address how to set up the most popular journal articles in you personalized community and view the counter for other assets. In a web site, we will have a lot of journal articles (that is, web content) for a given article type. For example, for the article type Article Content, we will have articles talking about product family. We may want to know how many times the end users read each article. Meanwhile, it would be nice if we could show the most popular articles (for example, TOP 10 articles) for this given article type. As shown in the following screenshot, a journal article My EDI Product I is shown via a portlet Ext Web Content Display. Rating and comments on this article are also exhibited. At the same time, the medium-size image, polls, and related content of this article are listed, too. A view counter of this article is especially displayed under the ratings. Moreover, the most popular articles are exhibited with article title and number of views under related content. All these articles belong to the article type article-content. That is, the article in the current portlet Ext Web Content Display has the most popular articles only for the article type article-content. Of course, you can customize the portlet Web Content Display directly through changing JSP files. For demo purposes, we will implement the view counter in the portlet Ext Web Content Display. Meanwhile, we will implement the mostly popular articles via VM services and article templates. In addition, we will analyze the view counter for other assets such as Image Gallery images, Document Library documents, Wiki articles, Blog entries, Message Boards threads, and so on. Adding a view counter in the Web Content Display portlet First of all, let's add a view counter in the Ext Web Content Display portlet. As the function of view counter for assets (including journal articles) is provided in the model TagsAssetModel of the com.liferay.portlet.tags.model package in the /portal/portal-service/src folder, we could use this feature in this portlet directly. To do so, use the following steps: Create a folder journal_content in the folder /ext/ext-web/docroot/html/portlet/. Copy the JSP file view.jsp in the folder /portal/portal-web/docroot/html/portlet/ to the folder /ext/ext-web/docroot/html/portlet/journal_content and open it. Add the line <%@ page import="com.liferay.portlet.tags.model.TagsAsset" %> after the line <%@ include file="/html/portlet/journal_content/init.jsp" %>, and check the following lines: JournalArticleDisplay articleDisplay = (JournalArticleDisplay) request.getAttribute( WebKeys.JOURNAL_ARTICLE_DISPLAY); if (articleDisplay != null) { TagsAssetLocalServiceUtil.incrementViewCounter( JournalArticle.class.getName(), articleDisplay.getResourcePrimKey());} Then add the following lines after the line <c:if test="<%=enableComments %>"> and save it: <span class="view-count"> <% TagsAsset asset = TagsAssetLocalServiceUtil.getAsset (JournalArticle.class.getName(), articleDisplay.getResourcePrimKey());%> <c:choose> <c:when test="<%= asset.getViewCount() == 1 %>"> <%= asset.getViewCount() %> <liferay-ui:message key="view" />, </c:when> <c:when test="<%= asset.getViewCount() > 1 %>"> <%= asset.getViewCount() %> <liferay-ui:message key="views" />, </c:when> </c:choose></span> The code above shows a way to increase the view counter via the TagsAssetLocalServiceUtil.incrementViewCounter method. This method takes two parameters className and classPK as inputs. For the current journal article, the two parameters are JournalArticle.class.getName() and articleDisplay.getResourcePrimKey(). Then, this code shows a way to display view counted through the TagsAssetLocalServiceUtil.getAsset method. Similarly, this method also takes two parameters, className and classPK, as inputs. This approach would be useful for other assets, as the className parameter could be Image Gallery, Document Library, Wiki, Blogs, Message Boards, Bookmark, and so on. Setting up VM service We can set up the VM service to exhibit the most popular articles. We can also add the getMostPopularArticles method in the custom velocity tool ExtVelocityToolUtil. To do so, first add the following method in the ExtVelocityToolService interface: public List<TagsAsset> getMostPopularArticles(String companyId, String groupId, String type, int limit); And then add an implementation of the getMostPopularArticles method in the ExtVelocityToolServiceImpl class as follows: public List<TagsAsset> getMostPopularArticles(String companyId, String groupId, String type, int limit) { List<TagsAsset> results = Collections.synchronizedList(new ArrayList<TagsAsset>()); DynamicQuery dq0 = DynamicQueryFactoryUtil.forClass( JournalArticle.class, "journalarticle"). setProjection(ProjectionFactoryUtil.property ("resourcePrimKey")).add(PropertyFactoryUtil. forName("journalarticle.companyId"). eqProperty("tagsasset.companyId")). add(PropertyFactoryUtil.forName( "journalarticle.groupId").eqProperty( "tagsasset.groupId")).add(PropertyFactoryUtil. forName("journalarticle.type").eq( "article-content")); DynamicQuery query = DynamicQueryFactoryUtil.forClass( TagsAsset.class, "tagsasset") .add(PropertyFactoryUtil.forName( "tagsasset.classPK").in(dq0)) .addOrder(OrderFactoryUtil.desc( "tagsasset.viewCount")); try{ List<Object> assets = TagsAssetLocalServiceUtil. dynamicQuery(query); int index = 0; for (Object obj: assets) { TagsAsset asset = (TagsAsset) obj; results.add(asset); index ++; if(index == limit) break; } } catch (Exception e){ return results; } return results; } The preceding code shows a way to get the most popular articles by company ID, group ID, article type, and limited articles to be returned. DynamicQuery API allows us to leverage the existing mapping definitions through access to the Hibernate session. For example, DynamicQuery dq0 selects the journal articles by companyID, groupId, and type; DynamicQuery query selects tagsassets by classPK, which exists in DynamicQuery dq0; and tagsassets are ordered by viewCount as well. Finally, add the following method to register the above method in ExtVelocityToolUtil: public List<TagsAsset> getRelatedArticles(String companyId, String groupId, String articleId, int limit){ return _extVelocityToolService.getRelatedArticles(companyId, groupId, articleId, limit);} The code above shows a generic approach to get TOP 10 articles for any article types. Of course, you can extend this approach to find TOP 10 assets. This can include Image Gallery images, Document Library documents, Wiki articles, Blog entries, Message Boards threads, Bookmark entries, slideshow, videos, games, video queue, video list, playlist, and so on. You may practice these TOP 10 assets feature. Building article template for the most popular journal articles We have added view counter on journal articles. We have already built VM service for the most popular articles too. Now let's build an article template for them. Setting up the default article type As mentioned earlier, there is a set of types of journal articles, for example, announcements, blogs, general, news, press-release, updates, article-tout, article-content, and so on. In real case, only some of these types will require view counter, for example article-content. Let's configure the default article type for mostly popular articles. We can add the following line at the end of portal-ext.properties. ext.most_popular_articles.article_type=article-content The code above shows that the default article type for most_popular_articles is article-content.
Read more
  • 0
  • 0
  • 6716
Modal Close icon
Modal Close icon