How-To Tutorials

The term "social network" means different things to different people. However, the starting point of any network is the individuals within it. A user profile provides a place for site members to describe themselves, and for other site members to find out about them. In this article, we will examine how to create a user profile that is aligned with the goals of your site. Identifying the Goals of Your Profile User profiles can be used for a range of purposes. On one end of the spectrum, a profile can be used to store basic information about the user. On the other end of the spectrum, a user profile can be a place for a user to craft and share an online identity. As you create the functionality behind your user profile page, you should know the type of profile you want to create for your users. Drupal ships with a core Profile module. This module is a great starting point, and for many sites will provide all of the functionality needed. If, however, you want a more detailed profile, you will probably need to take the next step: building a node-based profile. This involves creating a content type that stores profile information. Node-based profiles offer several practical advantages; these nodes can be extended using CCK fields, and they can be categorized using a taxonomy. In Drupal 6, user profiles become nodes through using the Content Profile module. The most suitable approach to user profiles will be determined by the goals of your site. Using Drupal's core Profile module provides some simple options that will be easy to set up and use. Extending profiles via the Content Profile module allows for a more detailed profile, but requires more time to set up. In this article, we will begin by describing how to set up profiles using the core Profile module. Then we will look at how to use the Content Profile module. Using the Core Profile Module To use the core profile module, click on the Administer | Site building | Modules link, or navigate to admin/build/modules. In the Core – optional section, enable the Profile module. Click the Save configuration button to submit the form and save the settings. Once the Profile module has been enabled, you can see a user's profile information by navigating to http://example.com/user/UID, where UID is the user's ID number on the site. To see your own user profile, navigate to http://example.com/user when logged in, or click the My Account link. The default user profile page exposes some useful functionality. First, it shows the user's profile, and secondly, it provides the Edit tab that allows a user to edit their profile. The Edit tab will only be visible to the owner of the profile, or to administrative users with elevated permissions. Other modules can add tabs to the core Profile page. As shown in the preceding screenshot by Item 1, the core Tracker module adds a Track tab; this tab gives an overview of all of the posts to which this user has participated. As shown in the preceding screenshot by item, the Contact tab has been added by the core Contact module. The Contact module allows users to contact one another via the site. Customizing the Core Profile The first step in customizing the user profile requires us to plan what we want the profile to show. By default, Drupal only requires users to create a username and provide an email address. From a user privacy perspective, this is great. However, for a teacher trying to track multiple students across multiple classes, this can be less than useful. For this sample profile, we will add two fields using the core Profile module: a last name, and a birthday. The admin features for the core profile module are accessible via the Administer | User Management | Profiles link, or you can navigate to admin/user/profile. As seen in the preceding screenshot, the core profile module offers the following possibilities for customization: single-line textfield—adds a single line of text; useful for names or other types of brief information. multi-line text field—adds a larger textarea field; useful for narrative-type profile information. checkbox—adds a checkbox; useful for Yes/No options. list selection—allows the site admin to create a set of options; the user can then select from these pre-defined options. Functionally, this is similar to a controlled vocabulary created using the core Taxonomy module. freeform list—adds a field where the user can enter a comma-separated list. Functionally, this is similar to a tag-based vocabulary created using the core Taxonomy module. URL—allows users to enter a URL; this is useful for allowing users to add a link to their personal blog. date—adds a date field. In our example profile—adding a last name and a birthday—our last name will be a single-line textfield; our birthday will be a date field.

Knowledge requirements A candidate appearing for the CISSP exam should have knowledge in the following areas that relate to access control: Control access by applying concepts, methodologies, and techniques Identify, evaluate, and respond to access control attacks such as Brute force attack, dictionary, spoofing, denial of service, etc. Design, coordinate, and evaluate penetration test(s) Design, coordinate, and evaluate vulnerability test(s) The approach In accordance with the knowledge expected in the CISSP exam, this domain is broadly grouped under five sections as shown in the following diagram: Section 1: The Access Control domain consists of many concepts, methodologies, and some specific techniques that are used as best practices. This section coverssome of the basic concepts, access control models, and a few examples of access control techniques. Section 2: Authentication processes are critical for controlling access to facilities and systems. This section looks into important concepts that establish the relationship between access control mechanisms and authentication processes. Section 3: A system or facility becomes compromised primarily through unauthorized access either through the front door or the back door. We'll see some of the common and popular attacks on access control mechanisms, and also learn about the prevalent countermeasures to such attacks. Section 4: An IT system consists of an operating system software, applications, and embedded software in the devices to name a few. Vulnerabilities in such software are nothing but holes or errors. In this section we see some of the common vulnerabilities in IT systems, vulnerability assessment techniques, and vulnerability management principles. Section 5: Vulnerabilities are exploitable, in the sense that the IT systems can be compromised and unauthorized access can be gained by exploiting the vulnerabilities. Penetration testing or ethical hacking is an activity that tests the exploitability of vulnerabilities for gaining unauthorized access to an IT system. Today, we'll quickly review some of the important concepts in the Sections 1, 2,and 3. Access control concepts, methodologies, and techniques Controlling access to the information systems and the information processing facilities by means of administrative, physical, and technical safeguards is the primary goal of access control domain. Following topics provide insight into someof the important access control related concepts, methodologies, and techniques. Basic concepts One of the primary concepts in access control is to understand the subject and the object. A subject may be a person, a process, or a technology component that either seeks access or controls the access. For example, an employee trying to access his business email account is a subject. Similarly, the system that verifies the credentials such as username and password is also termed as a subject. An object can be a file, data, physical equipment, or premises which need controlled access. For example, the email stored in the mailbox is an object that a subject is trying to access. Controlling access to an object by a subject is the core requirement of an access control process and its associated mechanisms. In a nutshell, a subject either seeks or controls access to an object. An access control mechanism can be classified broadly into the following two types: If access to an object is controlled based on certain contextual parameters, such as location, time, sequence of responses, access history, and so on, then it is known as a context-dependent access control. In this type of control, the value of the asset being accessed is not a primary consideration. Providing the username and password combination followed by a challenge and response mechanism such as CAPTCHA, filtering the access based on MAC adresses in wireless connections, or a firewall filtering the data based on packet analysis are all examples of context-dependent access control mechanisms. Completely Automated Public Turing test to tell Computers and Humans Apart (CAPTCHA) is a challenge-response test to ensure that the input to an access control system is supplied by humans and not by machines. This mechanism is predominantly used by web sites to prevent Web Robots(WebBots) to access the controlled section of the web site by brute force methods The following is an example of CAPTCHA: If the access is provided based on the attributes or content of an object,then it is known as a content-dependent access control. In this type of control, the value and attributes of the content that is being accessed determines the control requirements. For example, hiding or showing menus in an application, views in databases, and access to confidential information are all content-dependent.

Add Devices Though we might have, auto-discovered devices on our networks, but sometimes we don't want to add all the available devices on the network to the inventory or it may be that all our devices may not be found. To compensate for both these scenarios, Zenoss allows us to add one device at a time to the device inventory. To add a single device, select Add Device from the navigation panel. The Add Device page is divided into multiple sections for general device information, Attributes, and Relations as shown in the following screenshot. We can be as detailed as we want to be when we add the device manually. However, at a minimum, we should enter a Device Name, Device Class Path,and Discovery Protocol. The Device Name identifies the IP address or resolvable hostname, while the device class sets the monitoring properties we want our device to inherit by default. If the device is not SNMP-enabled, select None, otherwise Zenoss will not add the device.We'll continue We'll continue monitoring our Mill Race location by adding a new Linux server with the following configuration: Device Name: 192.168.1.110 Device Class: /Server/Linux Discovery Protocol: None OS Manufacturer: Ubuntu Location: /Mill Race/Second Floor System: /Development Group: /Developers/Software Testers The Add Device Options table lists the available configuration information we can set when adding a device manually Add Device Options Field Name Description IP Address Enter either an IP address or resolvable host name to identify the device. Device Class Path Select the appropriate device classifications: Ex: /Server/Linux. Discovery Protocol Choose either SNMP or None depending on whether or not you monitor the device with SNMP or not. SNMP Community Enter the community string of the device. The most common default is public.     Attributes SNMP Port The default port for SNMP communication is 161. Tag Number If the device has a tag number, such as a service tag number, enter the value. Serial Number Record the manufacturer's serial number. Production State Select the current state of the device: Ex. Production, maintenance, decommissioned. Priority Highest, high, normal, low, lowest, trivial. Rack Slot Record the physical rack location of the device. Comments Use the comments to enter device specific information, including a description, device users, or who is responsible for the device.     Relations HW Manufacturer Select a manufacturer name from the list. Ex: Cisco or Linksys. HW Product Select a product from the list. The HW Product lists gets populated based on the HW Manufacturer selection. OS Manufacturer Select a manufacturer name from the list. Ex: Microsoft or Fedora Core. OS Product Select from a product from the list. The OS Product lists gets populated based on the HW Manufacturer selection. Location Path Select the location of the device. Create a new location by typing the name in the New Location field and clicking Add. Systems Select a system organizer. Create a new system by typing the name in the New System field and clicking Add. Groups Select a group organizer. Create a new group by typing the name in the New Device Group field and clicking Add. Status Monitor Select a status monitor to define how often the device availability is monitored. The default is localhost. Create a new status monitor by typing the name in the New Status Monitor field and clicking Add. Refer to chapter 06 for configuration information. Performance Monitor Select a performance monitor to define how often device performance data is collected. The default is localhost. Create a new performance monitor by typing the name in the New Performance Monitor field. Refer to chapter 06 for configuration information. The Add Device Status page provides a hyperlink at the bottom of the page that says, "Navigate to device 192.168.1.110." If we click on the device name, the Device Status page is displayed. Device Status The Device status page displays an overview of our device and contains the same information we encountered on the Add Device page. As we look at the Device Status table for 192.168.1.110 as shown in the following screenshot, we can determine several important monitoring statistics in one glance. In our example, the device name and IP address are the same, but they do not need to be the same. If the host has multiple CNAMEs or interfaces, we can specify a name other than the name we used to find the device, via DNS resolution. We may find that we want to implement a custom naming scheme for devices. Regardless of what we name the device, Zenoss uses the IP address to monitor, not the name The Device Status table lists the number of events by severity and color code. The Device Severities table lists Zenoss's severity: Device Severities Color Severity Red Critical Orange Error Yellow Warning Blue Information Grey Debug The Device Status page also lists important statistics of the device. The Availability and Uptime values are automatically calculated, and the Production State and Priority values can be changed via the device's Edit page. We can lock the device to prevent Zenoss from removing or updating the device configuration. The Last Change, Last Collection, and the First Seen values provide a quick way to verify the modeling history of the device by listing the last time Zenoss detected a change with the device configuration and the last time the device was modeled. In the Device Status page, we also see a list of Component Types and the Status of each monitored component. As we build our monitoring solution, the components we monitor will change per device, but common components include SNMP, ipServices, Windows event logs, and syslogs If we look closely at the previous screen shot that shows the status of 192.168.1.110, we notice that the SNMP component displays an error condition. This indicates that our device does not have SNMP installed or is not configured correctly.

Hibernate allows transparent persistence, which means the application is absolutely isolated from the underlying database storage format. Three players in the Hibernate scene implement this feature: Hibernate dialect, Hibernate types, and HQL. The Hibernate dialect allows us to use a range of different databases, supporting different, proprietary variants of SQL and column types. In addition, HQL allows us to query persisted objects, regardless of their relational persisted form in the database. Hibernate types are a representation of databases SQL types, provide an abstraction of the underlying database types, and prevent the application from getting involved with the actual database column types. They allow us to develop the application without worrying about the target database and the column types that the database supports. Instead, we get involved with mapping Java types to Hibernate types. The database dialect, as part of Hibernate, is responsible for transforming Java types to SQL types, based on the target database. This gives us the flexibility to change the database to one that may support different column types or SQL without changing the application code. Built-in types Hibernate includes a rich and powerful range of built-in types. These types satisfy most needs of a typical application, providing a bridge between basic Java types and common SQL types. Java types mapped with these types range from basic, simple types, such as long and int, to large and complex types, such as Blob and Clob. The following table categorizes Hibernate built-in types with corresponding Java and SQL types: Java Type Hibernate Type Name SQL Type Primitives Boolean or boolean boolean BIT true_false CHAR(1)('T'or'F') yes_no CHAR(1)('Y'or'N') Byte or byte byte TINYINT char or Character character CHAR double or Double double DOUBLE float or float float FLOAT int or Integer integer INTEGER long or Long long BIGINT short or Short short SMALLINT String java.lang.String string VARCHAR character CHAR(1) text CLOB Arbitrary Precision Numeric java.math.BigDecimal big_decimal NUMERIC Byte Array byte[] or Byte[] binary VARBINARY   Time and Date java.util.Date date DATE time TIME timestamp TIMESTAMP java.util.Calendar calendar TIMESTAMP calendar_date DATE java.sql.Date date DATE java.sql.Time time TIME java.sql.Timestamp timestamp TIMESTAMP Localization java.util.Locale locale VARCHAR java.util.TimeZone timezone java.util.Currency currency Class Names java.lang.Class class VARCHAR Any Serializable Object java.io.Serializable Serializable VARBINARY JDBC Large Objects java.sql.Blob blob BLOB java.sql.Clob clob CLOB

Model Devices When we talk about Zenoss, two related but different words often come up, monitoring and modeling. Monitoring refers to the availability of the device and answers the question, "Is the device accessible?" Modeling defines a relationship between devices and identifies the components available on a device, such as services, interfaces, and file systems. Zenoss models devices via SNMP, SSH, port scan, and telnet and gathers information via collector plug-ins. Each class has a default set of collector plug-ins that tells Zenoss how to model the devices assigned to the class. We can add or remove collector plug-ins at the device level for individual changes or at the class level for all the devices in the class. The collector plug-in names reflect the monitoring protocol they are used for. All the SNMP collectors contain "snmp" in the name. The SSH and telnet plug-in names contain "cmd," and the port scan plug-in contains "portscan" in the name. We'll step through modeling examples for SNMP, SSH, and port scan; however, we'll skip telnet because it's similar to SSH. SNMP Zenoss defaults to SNMP monitoring, and the monitored device needs to have SNMP installed and configured to work properly. Zenoss supports SNMP v1, v2c, and v3. The example commands used in this section to troubleshoot SNMP specify v1. Test SNMP If we're unsure of our SNMP setup, we can test it by running the snmpwalk command to retrieve the values of the MIB tree on the monitored device. We'll demonstrate both working and broken SNMP configurations on the MillRace network. From the Device List, select the device named Coyote. From the Device Status page menu, choose Run Commands > snmpwalk. A new window opens and we see the results of the snmpwalk command as shown in the following screenshot. Now we select the device Bobcat from the Device List view. Run the snmpwalk command from the Device Status page. This time, we receive a Timeout error, which indicates that we have a problem with SNMP on the device Bobcat as shown in the following screenshot. Assuming that SNMP is properly configured on the device and that the monitored device accepts traffic on port 161, we may need to update the device's community string. To update the community: Select the device from the Device List view. From the Device Status page menu, select More > zProperties. Find the Community field and enter the correct value. Save the changes. After updating the SNMP community string in the zProperties, we run the snmpwalk command again to see if we have fixed the problem. If we continue to encounter problems getting Zenoss to model a device with SNMP, we can try to narrow down the problem by running the following snmpwalk command from the monitored devices shell prompt: snmpwalk -v1 -c public localhost system Replace public with the correct community string. If the command is successful when using localhost, edit the snmpd configuration file. As root, edit /etc/default/snmpd or /etc/default/snmp and remove 127.0.0.1 from the following line: SNMPDPORTS = '-Lsd -Lf /dev/null -u snmp -I -smux -p /var/run/snmpd.pid 127.0.0.1' After editing the /etc/default/snmpd file, restart the snmpd service as root. For example: /etc/init.d/snmpd stop/etc/init.d/snmpd start Retest the snmpwalk command to confirm that SNMP is working correctly.Windows users can run wbemtest from the command line to test SNMP. Windows Considerations In order to collect information from WMI,regarding the Windows SNMP installation we need to configure the zProperties for the Windows device. Navigate to the Windows device and open the zProperties page by selecting the More > zProperties from the page menu. Scroll to the bottom of the page and make the following changes: Set zWinEventLog to true. Enter the Windows user's password in the zWinPassword field. Enter the user name with administrative rights in the zWinUser field in the following formats: .user for local user accounts DOMAINuser for domain user accounts Set zWMIMonitorIgnore to false. Save the changes, and Zenoss is ready to model the information on the Windows device. We can force a model by selecting Manage > Model Device from thepage menu. SNMP Collector Plug-ins The Collector Plug-ins assigned to the device determines how Zenoss models the device. Let's take a look at our example device Coyote and see what collectors are currently assigned. From the Device Status page for Coyote, select More > Collector Plug-ins from the page menu. A page showing the assigned collector plugi-ns displays in the left column of the page with an Add Fields link on the right. When we click on the Add Fields link, a column of unassigned plug-ins appears and the link name changes to Hide Fields as shown in the following screenshot. The plug-in names are intuitive in that the name suggests the type of information we expect to be modeling. For example, zenoss.snmp.IpServiceMap returns a list of active IP services on the device, such as HTTP. The Dell specif?i c plugi-ns retrieve more detailed information from Dell devices using OpenManage, and the HPplugi-ns provide more information about devices using Insight Management agents. To remove a plug-in from the assigned plug-in list, click on the "x" next to the plug-in name. To assign a plug-in, drag the plug-in name from available list to the assigned list. To see how our devices are affected, let's remove the zenoss.snmp.IpServiceMap and add zenoss.cmd.df. After we make the changes to the plug-ins for Coyote, scroll to the bottom of the page and click Save. Model Device Zenoss automatically models each device in our inventory every six hours, but we can manually force Zenoss to model the device. From the Device Status page, select Manage > Model Device from the page menu. Zenoss displays the results of the zenmodeler command in the window as shown in the following screenshot. Zenoss first determines which plugi-ns are available and then collects information based on those plug-ins. Notice that no cmd plug-ins are found, which means that the zenoss.cmd.df plug-in we added to Coyote will not be collected. After Zenoss models the device, we can review the device overview page to see what component types Zenoss discovered. IpService should not be listed. If we go back to the Collector Plug-ins page for Coyote, we can add the zenoss.snmp.IpService plug-in and then model the device again. Now, IpServices is displayed in the Component Type list as shown in the following screenshot. The Component Type list gets updated as part of the modeling process and so does the OS fields in the Device Information table (the greyed-out fields in the screen shot). If we enter values in these fields during the Add Device step, the values would be overwritten with the SNMP values. Our example made changes to the device level, which means that if we view the collector plug-ins for the /Server/Linux device class, the original plug-ins are specified. To view the plug-ins for the class: Select Devices from the navigation panel. Select Server from the sub-devices list. Select Linux from the sub-devices list. From the /Devices/Server/Linux page menu, select More > CollectorPlug-ins. Devices automatically inherit any changes we make to the class collector plug-ins the next time Zenoss models the devices. SSH Modeling If the monitored device does not support SNMP, or if we need to monitor a device behind a firewall, SSH provides an alternative to SNMP. Unlike SNMP, SSH needs the Zenoss Plug-ins installed on each monitored device and platform support is limited to Linux, Darwin, and FreeBSD. We also need to make sure that the monitored device has an SSH server installed so that the Zenoss system can log in and retrieve information. OpenSSH from openssh.com offers a good cross-platform SSH solution. The level of modeling provided by the Zenoss Plug-ins varies between platforms. For this reason, we may not achieve the same level of detail as we do with SNMP, but SSH modeling provides more detail than a port scan. To help us setup our SSH monitoring, Zenoss provides the /Server/Cmd class which is already configured with the command plug-ins we need to monitor via SSH. SSH Collector Plug-ins From the navigation panel, select Devices. Navigate to the /Server/Cmd class and click on the zProperties tab. Find the zCollectorPlug-ins field and click on the Edit link. A list of the assigned collector plug-ins is displayed as shown in the following screenshot. The important thing to note with the command collector plug-ins is the new level of specificity in the name. The zenoss.cmd.uname and zenoss.cmd.df plug-ins are common to all architectures, while the plug-ins with "linux" in the name work with Linux systems. Mac OS X platforms use the plug-ins with "darwin" in the name. If we did not have any any OS X systems to monitor, then we could remove allthe Darwin-based plug-ins from the /Server/Cmd class or, if we know that wedon't want to monitor the memory usage for any of our devices, we can removethat plug-in. We'll leave the collector plug-ins as they are for the /Server/Cmd class and change our test device Coyote to use SSH instead of SNMP. Zenoss Plug-ins Zenoss will monitor and retrieve some data using the SSH modeler even if we do not install the Zenoss plug-ins, but the device model will be incomplete. For example, file systems will be detected along with the size of each drive., but the usage statistics will not be reported. Zenoss also generates warning events if it cannot find the zenplugin.py command on the monitored system. The monitored system needs a Python environment installed. This can beinstalled using your distribution's package manager. If you have setuptoolsinstalled, you can install the Zenoss-Plug-ins package from the Cheese Shop (http://pypi.python.org/pypi/) with the following command as root: easy_install Zenoss-Plugins We can also build the Zenoss Plug-ins package from source: Download the Zenoss Plug-ins package from http://www.zenoss.com/download/. Extract the plug-in file. From the plug-in source directory, run the following commands as root: python setup.py build python setup.py install The setuptools procedure installs zenplugin.py to /usr/bin, which is important because we need to configure the device zProperties to look for the plug-ins in the correct location. To ensure that the plug-in file is working correctly, run the following command on the monitored device, which is Coyote in our example: zenplugin.py –list-plugins The command outputs the detected platform and the supported plug-ins as shown in the following screenshot.

The examples are: Plotting data from a database Plotting data from a web page Plotting the data extracted by parsing an Apache log file Plotting the data read from a comma-separated values (CSV) file Plotting extrapolated data using curve fitting Third-party tools using Matplotlib (NetworkX and mpmath) Let's begin Plotting data from a database Databases often tend to collect much more information than we can simply extract and watch in a tabular format (let's call it the "Excel sheet" report style). Databases not only use efficient techniques to store and retrieve data, but they are also very good at aggregating it. One suggestion we can give is to let the database do the work. For example, if we need to sum up a column, let's make the database sum the data, and not sum it up in the code. In this way, the whole process is much more efficient because: There is a smaller memory footprint for the Python code, since only the aggregate value is returned, not the whole result set to generate it The database has to read all the rows in any case. However, if it's smart enough, then it can sum values up as they are read The database can efficiently perform such an operation on more than one column at a time The data source we're going to query is from an open source project: the Debian distribution. Debian has an interesting project called UDD , Ultimate Debian Database, which is a relational database where a lot of information (either historical or actual) about the distribution is collected and can be analyzed. On the project website http://udd.debian.org/, we can fi nd a full dump of the database (quite big, honestly) that can be downloaded and imported into a local PostgreSQL instance (refer to http://wiki.debian.org/UltimateDebianDatabase/CreateLocalReplica for import instructions Now that we have a local replica of UDD, we can start querying it: # module to access PostgreSQL databasesimport psycopg2# matplotlib pyplot moduleimport matplotlib.pyplot as plt Since UDD is stored in a PostgreSQL database, we need psycopg2 to access it. psycopg2 is a third-party module available at http://initd.org/projects/psycopg # connect to UDD databaseconn = psycopg2.connect(database="udd")# prepare a cursorcur = conn.cursor() We will now connect to the database server to access the udd database instance, and then open a cursor on the connection just created. # this is the query we'll be makingquery = """select to_char(date AT TIME ZONE 'UTC', 'HH24'), count(*) from upload_history where to_char(date, 'YYYY') = '2008' group by 1 order by 1""" We have prepared the select statement to be executed on UDD. What we wish to do here is extract the number of packages uploaded to the Debian archive (per hour) in the whole year of 2008. date AT TIME ZONE 'UTC': As date field is of the type timestamp with time zone, it also contains time zone information, while we want something independent from the local time. This is the way to get a date in UTC time zone. group by 1: This is what we have encouraged earlier, that is, let the database do the work. We let the query return the already aggregated data, instead of coding it into the program. # execute the querycur.execute(query)# retrieve the whole result setdata = cur.fetchall() We execute the query and fetch the whole result set from it. # close cursor and connectioncur.close()conn.close() Remember to always close the resources that we've acquired in order to avoid memory or resource leakage and reduce the load on the server (removing connections that aren't needed anymore). # unpack data in hours (first column) and# uploads (second column)hours, uploads = zip(*data) The query result is a list of tuples, (in this case, hour and number of uploads), but we need two separate lists—one for the hours and another with the corresponding number of uploads. zip() solves this with *data, we unpack the list, returning the sublists as separate arguments to zip(), which in return, aggregates the elements in the same position in the parameters into separated lists. Consider the following example: In [1]: zip(['a1', 'a2'], ['b1', 'b2'])Out[1]: [('a1', 'b1'), ('a2', 'b2')] To complete the code: # graph codeplt.plot(hours, uploads)# the the x limits to the 'hours' limitplt.xlim(0, 23)# set the X ticks every 2 hoursplt.xticks(range(0, 23, 2))# draw a gridplt.grid()# set title, X/Y labelsplt.title("Debian packages uploads per hour in 2008")plt.xlabel("Hour (in UTC)")plt.ylabel("No. of uploads") The previous code snippet is the standard plotting code, which results in the following screenshot: From this graph we can see that in 2008, the main part of Debian packages uploads came from European contributors. In fact, uploads were made mainly in the evening hours (European time), after the working days are over (as we can expect from a voluntary project). Plotting data from the Web Often, the information we need is not distributed in an easy-to-use format such as XML or a database export but for example only on web sites. More and more often we find interesting data on a web page, and in that case we have to parse it to extract that information: this is called web scraping . In this example, we will parse a Wikipedia article to extracts some data to plot. The article is at http://it.wikipedia.org/wiki/Demografia_d'Italia and contains lots of information about Italian demography (it's in Italian because the English version lacks a lot of data); in particular, we are interested in the population evolution over the years. Probably the best known Python module for web scraping is BeautifulSoup ( http://www.crummy.com/software/BeautifulSoup/). It's a really nice library that gets the job done quickly, but there are situations (in particular with JavaScript embedded in the web page, such as for Wikipedia) that prevent it from working. As an alternative, we find lxml quite productive (http://codespeak.net/lxml/). It's a library mainly used to work with XML (as the name suggests), but it can also be used with HTML (given their quite similar structures), and it is powerful and easy–to-use. Let's dig into the code now: # to get the web pagesimport urllib2# lxml submodule for html parsingfrom lxml.html import parse# regular expression moduleimport re# Matplotlib moduleimport matplotlib.pyplot as plt Along with the Matplotlib module, we need the following modules: urllib2: This is the module (from the standard library) that is used to access resources through URL (we will download the webpage with this). lxml: This is the parsing library. re: Regular expressions are needed to parse the returned data to extract the information we need. re is a module from the standard library, so we don't need to install a third-party module to use it. # general urllib2 configuser_agent = 'Mozilla/5.0 (compatible; MSIE 5.5; Windows NT)'headers = { 'User-Agent' : user_agent }url = "http://it.wikipedia.org/wiki/Demografia_d'Italia" Here, we prepare some configuration for urllib2, in particular, the user_agent header is used to access Wikipedia and the URL of the page. # prepare the request and open the urlreq = urllib2.Request(url, headers=headers)response = urllib2.urlopen(req) Then we make a request for the URL and get the HTML back. # we parse the webpage, getroot() return the document rootdoc = parse(response).getroot() We parse the HTML using the parse() function of lxml.html and then we get the root element. XML can be seen as a tree, with a root element (the node at the top of the tree from where every other node descends), and a hierarchical structure of elements. # find the data table, using css elementstable = doc.cssselect('table.wikitable')[0] We leverage the structure of HTML accessing the first element of type table of class wikitable because that's the table we're interested in. # prepare data structures, will contain actual datayears = []people = [] Preparing the lists that will contain the parsed data. # iterate over the rows of the table, except first and last onesfor row in table.cssselect('tr')[1:-1]: We can start parsing the table. Since there is a header and a footer in the table, we skip the first and the last line from the lines (selected by the tr tag) to loop over. # get the row cell (we will use only the first two)data = row.cssselect('td') We get the element with the td tag that stands for table data: those are the cells in an HTML table. # the first cell is the yeartmp_years = data[0].text_content()# cleanup for cases like 'YYYY[N]' (date + footnote link)tmp_years = re.sub('[.]', '', tmp_years) We take the first cell that contains the year, but we need to remove the additional characters (used by Wikipedia to link to footnotes). # the second cell is the population counttmp_people = data[1].text_content()# cleanup from '.', used as separatortmp_people = tmp_people.replace('.', '') We also take the second cell that contains the population for a given year. It's quite common in Italy to separate thousands in number with a '.' character: we have to remove them to have an appropriate value. # append current data to data lists, converting to integersyears.append(int(tmp_years))people.append(int(tmp_people)) We append the parsed values to the data lists, explicitly converting them to integer values. # plot dataplt.plot(years,people)# ticks every 10 yearsplt.xticks(range(min(years), max(years), 10))plt.grid()# add a note for 2001 Censusplt.annotate("2001 Census", xy=(2001, people[years.index(2001)]), xytext=(1986, 54.5*10**6), arrowprops=dict(arrowstyle='fancy')) Running the example results in the following screenshot that clearly shows why the annotation is needed: In 2001, we had a national census in Italy, and that's the reason for the drop in that year: the values released from the National Institute for Statistics (and reported in the Wikipedia article) are just an estimation of the population. However, with a census, we have a precise count of the people living in Italy. Plotting data by parsing an Apache log file Plotting data from a log file can be seen as the art of extracting information from it. Every service has a log format different from the others. There are some exceptions of similar or same format (for example, for services that come from the same development teams) but then they may be customized and we're back at the beginning. The main differences in log files are: Fields orders: Some have time information at the beginning, others in the middle of the line, and so on Fields types: We can find several different data types such as integers, strings, and so on Fields meanings: For example, log levels can have very different meanings From all the data contained in the log file, we need to extract the information we are interested in from the surrounding data that we don't need (and hence we skip). In our example, we're going to analyze the log file of one of the most common services: Apache. In particular, we will parse the access.log file to extract the total number of hits and amount of data transferred per day. Apache is highly configurable, and so is the log format. Our Apache configuration, contained in the httpd.conf file, has this log format: "%h %l %u %t "%r" %>s %b "%{Referer}i" "%{User-Agent}i"" This is in LogFormat specification where Log directive Description %h The host making the request %l Identity of the client (which is usually not available) %u User making the request (usually not available) %t The time the request was received %r The request %>s The status code %b The size (in bytes) of the response sent to the client (excluding the headers) %{Referer}i The page from where the requests originated (for example, the HTML page where a PNG image is requested) %{User-Agent}i The user agent used to make the request

Embedding a progress bar in a status bar This topic explains how to embed a progress bar in a panel's status bar, a scenario found in countless user interfaces: How to do it Create a click handler that will simulate a long-running activity and update the progress bar: Ext.onReady(function() { var loadFn = function(btn, statusBar) { btn = Ext.getCmp(btn); btn.disable(); Ext.fly('statusTxt').update('Saving...'); pBar.wait({ interval: 200, duration: 5000, increment: 15, fn: function() { btn.enable(); Ext.fly('statusTxt').update('Done'); } });}; Create an instance of the progress bar: var pBar = new Ext.ProgressBar({ id: 'pBar', width: 100}); Create a host panel and embed the progress bar in the bbar of the panel. Also, add a button that will start the progress bar updates: var pnl = new Ext.Panel({ title: 'Status bar with progress bar', renderTo: 'pnl1', width: 400, height: 200, bodyStyle: 'padding:10px;', items: [{ xtype: 'button', id: 'btn', text: 'Save', width:'75', handler: loadFn.createCallback('btn', 'sBar') }], bbar: { id: 'sBar', items: [{ xtype: 'tbtext', text: '',id:'statusTxt' },'->', pBar] }}); How it works The first step consists of creating loadFn, a function that simulates a long-running operation, so that we can see the progress bar animation when the button is clicked. The heart of loadFn is a call to ProgressBar.wait(…), which initiates the progress bar in an auto-update mode. And this is how the status bar is embedded in the bbar of the panel: bbar: { id: 'sBar', items: [{ xtype: 'tbtext', text: '',id:'statusTxt' },'->', pBar] Observe how the progress bar is sent to the rightmost location in the status bar with the help of a Toolbar.Fill instance, declared with '->'. Creating a custom look for the status bar items Customizing the look of toolbar items is relatively simple. In this recipe, you will learn how to create toolbar items with a sunken look that can be found in many desktop applications: How to do it Create the styles that will provide the custom look of the status bar text items: .custom-status-text-panel{ border-top:1px solid #99BBE8; border-right:1px solid #fff; border-bottom:1px solid #fff; border-left:1px solid #99BBE8; padding:1px 2px 2px 1px;} Create a host panel: Ext.onReady(function() { var pnl = new Ext.Panel({ title: 'Status bar with sunken text items', renderTo: 'pnl1', width: 400, height: 200, bodyStyle: 'padding:10px;', Define the panel's bbar with the text items: bbar: { id: 'sBar', items: [ { id: 'cachedCount', xtype:'tbtext', text: 'Cached: 15' }, ' ', { id: 'uploadedCount', xtype: 'tbtext', text: 'Uploaded: 7' }, ' ', { id: 'invalidCount', xtype: 'tbtext', text: 'Invalid: 2' } ]}, Now, add a handler for the afterrender event and use it to modify the styles of the text items: listeners: { 'afterrender': { fn: function() { Ext.fly(Ext.getCmp('cachedCount').getEl()).parent(). addClass('custom-status-text-panel'); Ext.fly(Ext.getCmp('uploadedCount').getEl()).parent(). addClass('custom-status-text-panel'); Ext.fly(Ext.getCmp('invalidCount').getEl()).parent(). addClass('custom-status-text-panel'); }, delay:500 }} How it works The actual look of the items is defined by the style in the custom-status-text-panel CSS class. After the host panel and toolbar are created and rendered, the look of the items is changed by applying the style to each of the TD elements that contain the items. For example: Ext.fly(Ext.getCmp('uploadedCount').getEl()).parent(). addClass('custom-status-text-panel'); See also... The previous recipe, Embedding a progress bar in a status bar, explains how a progress bar can be embedded in a panel's status bar Using a progress bar to indicate that your application is busy In this topic, you will learn how to use a progress bar to indicate that your application is busy performing an operation. The next screenshot shows a progress bar built using this recipe: How to do it Define the progress bar: Ext.onReady(function() { Ext.QuickTips.init(); var pBar = new Ext.ProgressBar({ id: 'pBar', width: 300, renderTo: 'pBarDiv'}); Add a handler for the update event and use it to update the wait message: pBar.on('update', function(val) { //Handle this event if you need to // execute code at each progress interval. Ext.fly('pBarText').dom.innerHTML += '.';}); Create a click handler for the button that will simulate a long-running activity: var btn = Ext.get('btn');btn.on('click', function() { Ext.fly('pBarText').update('Please wait'); btn.dom.disabled = true; pBar.wait({ interval: 200, duration: 5000, increment: 15, fn: function() { btn.dom.disabled = false; Ext.fly('pBarText').update('Done'); } });}); Add the button to the page: <button id="btn">Start long-running operation</button> How it works After creating the progress bar, the handler for its update event is created. While I use this handler simply to update the text message, you can use it to execute some other code every time that a progress interval occurs. The click handler for the button calls the progress bar's wait(…) function, which causes the progress bar to auto-update at the configured interval and reset itself after the configured duration: pBar.wait({ interval: 200, duration: 5000, increment: 15, fn: function() { btn.dom.disabled = false; Ext.fly('pBarText').update('Done'); }}); There's more The progress bar can also be configured to run indefinitely by not passing the duration config option. Clearing the progress bar in this scenario requires a call to the reset() function. See also... The next recipe, Using a progress bar to report progress updates, illustrates how a progress bar can be set up to notify the user that progress is being made in the execution of an operation The Changing the look of a progress bar recipe (covered later in this article) shows you how easy it is to change the look of the progress bar using custom styles Summary This article consisted recipes that examined the commonly-used menu items, as well as the different ways of setting up toolbars and progress bars in your applications. [ 1 | 2 ] If you have read this article you may be interested to view :   Making Progress with Menus and Toolbars using Ext JS 3.0: Part 1 Load, Validate, and Submit Forms using Ext JS 3.0: Part 1 Load, Validate, and Submit Forms using Ext JS 3.0: Part 2 Load, Validate, and Submit Forms using Ext JS 3.0: Part 3

Plotting data from a CSV file A common format to export and distribute datasets is the Comma-Separated Values (CSV) format. For example, spreadsheet applications allow us to export a CSV from a working sheet, and some databases also allow for CSV data export. Additionally, it's a common format to distribute datasets on the Web. In this example, we'll be plotting the evolution of the world's population divided by continents, between 1950 and 2050 (of course they are predictions), using a new type of graph: bars stacked. Using the data available at http://www.xist.org/earth/pop_continent.aspx (that fetches data from the official UN data at http://esa.un.org/unpp/index.asp), we have prepared the following CSV file: Continent,1950,1975,2000,2010,2025,2050Africa,227270,418765,819462,1033043,1400184,1998466Asia,1402887,2379374,3698296,4166741,4772523,5231485Europe,547460,676207,726568,732759,729264,691048Latin America,167307,323323,521228,588649,669533,729184Northern America,171615,242360,318654,351659,397522,448464Oceania,12807,21286,31160,35838,42507,51338 In the first line, we can find the header with a description of what the data in the columns represent. The other lines contain the continent's name and its population (in thousands) for the given years. In the first line, we can find the header with a description of what the data in the columns represent. The other lines contain the continent's name and its population (in thousands) for the given years. There are several ways to parse a CSV file, for example: NumPy's loadtxt() (what we are going to use here) Matplotlib's mlab.csv2rec() The csv module (in the standard library) but we decided to go with loadtxt() because it's very powerful (and it's what Matplotlib is standardizing on). Let's look at how we can plot it then: # for file opening made easierfrom __future__ import with_statement We need this because we will use the with statement to read the file. # numpyimport numpy as np NumPy is used to load the CSV and for its useful array data type. # matplotlib plotting moduleimport matplotlib.pyplot as plt# matplotlib colormap moduleimport matplotlib.cm as cm# needed for formatting Y axisfrom matplotlib.ticker import FuncFormatter# Matplotlib font managerimport matplotlib.font_manager as font_manager In addition to the classic pyplot module, we need other Matplotlib submodules: cm (color map): Considering the way we're going to prepare the plot, we need to specify the color map of the graphical elements FuncFormatter: We will use this to change the way the Y-axis labels are displayed font_manager: We want to have a legend with a smaller font, and font_manager allows us to do that def billions(x, pos): """Formatter for Y axis, values are in billions""" return '%1.fbn' % (x*1e-6) This is the function that we will use to format the Y-axis labels. Our data is in thousands. Therefore, by dividing it by one million, we obtain values in the order of billions. The function is called at every label to draw, passing the label value and the position. # bar widthwidth = .8 As said earlier, we will plot bars, and here we defi ne their width. The following is the parsing code. We know that it's a bit hard to follow (the data preparation code is usually the hardest one) but we will show how powerful it is. # open CSV filewith open('population.csv') as f: The function we're going to use, NumPy loadtxt(), is able to receive either a filename or a file descriptor, as in this case. We have to open the file here because we have to strip the header line from the rest of the file and set up the data parsing structures. # read the first line, splitting the yearsyears = map(int, f.readline().split(',')[1:]) Here we read the first line, the header, and extract the years. We do that by calling the split() function and then mapping the int() function to the resulting list, from the second element onwards (as the first one is a string). # we prepare the dtype for exacting data; it's made of:# <1 string field> <len(years) integers fields>dtype = [('continents', 'S16')] + [('', np.int32)]*len(years) NumPy is flexible enough to allow us to define new data types. Here, we are creating one ad hoc for our data lines: a string (of maximum 16 characters) and as many integers as the length of years list. Also note how the fi rst element has a name, continents, while the last integers have none: we will need this in a bit. # we load the file, setting the delimiter and the dtype abovey = np.loadtxt(f, delimiter=',', dtype=dtype) With the new data type, we can actually call loadtxt(). Here is the description of the parameters: f: This is the file descriptor. Please note that it now contains all the lines except the first one (we've read above) which contains the headers, so no data is lost. delimiter: By default, loadtxt() expects the delimiter to be spaces, but since we are parsing a CSV file, the separator is comma. dtype: This is the data type that is used to apply to the text we read. By default, loadtxt() tries to match against float values # "map" the resulting structure to be easily accessible:# the first column (made of string) is called 'continents'# the remaining values are added to 'data' sub-matrix# where the real data arey = y.view(np.dtype([('continents', 'S16'), ('data', np.int32, len(years))])) Here we're using a trick: we view the resulting data structure as made up of two parts, continents and data. It's similar to the dtype that we defined earlier, but with an important difference. Now, the integer's values are mapped to a field name, data. This results in the column continents with all the continents names,and the matrix data that contains the year's values for each row of the file. data = y['data']continents = y['continents'] We can separate the data and the continents part into two variables for easier usage in the code. # prepare the bottom arraybottom = np.zeros(len(years)) We prepare an array of zeros of the same length as years. As said earlier, we plot stacked bars, so each dataset is plot over the previous ones, thus we need to know where the bars below finish. The bottom array keeps track of this, containing the height of bars already plotted. # for each line in datafor i in range(len(data)): Now that we have our information in data, we can loop over it. # create the bars for each element, on top of the previous barsbt = plt.bar(range(len(data[i])), data[i], width=width, color=cm.hsv(32*i), label=continents[i], bottom=bottom) and create the stacked bars. Some important notes: We select the the i-th row of data, and plot a bar according to its element's size (data[i]) with the chosen width. As the bars are generated in different loops, their colors would be all the same. To avoid this, we use a color map (in this case hsv), selecting a different color at each iteration, so the sub-bars will have different colors. We label each bar set with the relative continent's name (useful for the legend) As we have said, they are stacked bars. In fact, every iteration adds a piece of the global bars. To do so, we need to know where to start drawing the bar from (the lower limit) and bottom does this. It contains the value where to start drowing the current bar. # update the bottom arraybottom += data[i] We update the bottom array. By adding the current data line, we know what the bottom line will be to plot the next bars on top of it. # label the X ticks with yearsplt.xticks(np.arange(len(years))+width/2, [int(year) for year in years]) We then add the tick's labels, the years elements, right in the middle of the bar. # some information on the plotplt.xlabel('Years')plt.ylabel('Population (in billions)')plt.title('World Population: 1950 - 2050 (predictions)') Add some information to the graph. # draw a legend, with a smaller fontplt.legend(loc='upper left', prop=font_manager.FontProperties(size=7)) We now draw a legend in the upper-left position with a small font (to better fit the empty space). # apply the custom function as Y axis formatterplt.gca().yaxis.set_major_formatter(FuncFormatter(billions) Finally, we change the Y-axis label formatter, to use the custom formatting function that we defined earlier. The result is the next screenshot where we can see the composition of the world population divided by continents: In the preceding screenshot, the whole bar represents the total world population, and the sections in each bar tell us about how much a continent contributes to it. Also observe how the custom color map works: from bottom to top, we have represented Africa in red, Asia in orange, Europe in light green, Latin America in green, Northern America in light blue, and Oceania in blue (barely visible as the top of the bars). Plotting extrapolated data using curve fitting While plotting the CSV values, we have seen that there were some columns representing predictions of the world population in the coming years. We'd like to show how to obtain such predictions using the mathematical process of extrapolation with the help of curve fitting. Curve fitting is the process of constructing a curve (a mathematical function) that better fits to a series of data points. This process is related to other two concepts: interpolation: A method of constructing new data points within the range of a known set of points extrapolation: A method of constructing new data points outside a known set of points The results of extrapolation are subject to a greater degree of uncertainty and are influenced a lot by the fitting function that is used. So it works this way: First, a known set of measures is passed to the curve fitting procedure that computes a function to approximate these values With this function, we can compute additional values that are not present in the original dataset Let's first approach curve fitting with a simple example: # Numpy and Matplotlibimport numpy as npimport matplotlib.pyplot as plt These are the classic imports. # the known points setdata = [[2,2],[5,0],[9,5],[11,4],[12,7],[13,11],[17,12]] This is the data we will use for curve fitting. They are the points on a plane (so each has a X and a Y component) # we extract the X and Y components from previous pointsx, y = zip(*data) We aggregate the X and Y components in two distinct lists. # plot the data points with a black crossplt.plot(x, y, 'kx') Then plot the original dataset as a black cross on the Matplotlib image. # we want a bit more data and more fine grained for# the fitting functionsx2 = np.arange(min(x)-1, max(x)+1, .01) We prepare a new array for the X values because we wish to have a wider set of values (one unit on the right and one on to the left of the original list) and a fine grain to plot the fitting function nicely. # lines styles for the polynomialsstyles = [':', '-.', '--'] To differentiate better between the polynomial lines, we now define their styles list. # getting style and count one at timefor d, style in enumerate(styles): Then we loop over that list by also considering the item count. # degree of the polynomialdeg = d + 1 We define the actual polynomial degree. # calculate the coefficients of the fitting polynomialc = np.polyfit(x, y, deg) Then compute the coefficients of the fitting polynomial whose general format is: c[0]*x**deg + c[1]*x**(deg – 1) + ... + c[deg]# we evaluate the fitting function against x2y2 = np.polyval(c, x2) Here, we generate the new values by evaluating the fitting polynomial against the x2 array. # and then we plot itplt.plot(x2, y2, label="deg=%d" % deg, linestyle=style) Then we plot the resulting function, adding a label that indicates the degree of the polynomial and using a different style for each line. # show the legendplt.legend(loc='upper left') We then show the legend, and the final result is shown in the next screenshot: Here, the polynomial with degree=1 is drawn as a dotted blue line, the one with degree=2 is a dash-dot green line, and the one with degree=3 is a dashed red line. We can see that the higher the degree, the better is the fit of the function against the data. Let's now revert to our main intention, trying to provide an extrapolation for population data. First a note: we take the values for 2010 as real data and not predictions (well, we are quite near to that year) else we have very few values to create a realistic extrapolation. Let's see the code: # for file opening made easierfrom __future__ import with_statement# numpyimport numpy as np# matplotlib plotting moduleimport matplotlib.pyplot as plt# matplotlib colormap moduleimport matplotlib.cm as cm# Matplotlib font managerimport matplotlib.font_manager as font_manager# bar widthwidth = .8# open CSV filewith open('population.csv') as f: # read the first line, splitting the years years = map(int, f.readline().split(',')[1:]) # we prepare the dtype for exacting data; it's made of: # <1 string field> <6 integers fields> dtype = [('continents', 'S16')] + [('', np.int32)]*len(years) # we load the file, setting the delimiter and the dtype above y = np.loadtxt(f, delimiter=',', dtype=dtype) # "map" the resulting structure to be easily accessible: # the first column (made of string) is called 'continents' # the remaining values are added to 'data' sub-matrix # where the real data are y = y.view(np.dtype([('continents', 'S16'), ('data', np.int32, len(years))]))# extract fieldsdata = y['data']continents = y['continents'] This is the same code that is used for the CSV example (reported here for completeness). x = years[:-2]x2 = years[-2:] We are dividing the years into two groups: before and after 2010. This translates to split the last two elements of the years list. What we are going to do here is prepare the plot in two phases: First, we plot the data we consider certain values After this, we plot the data from the UN predictions next to our extrapolations # prepare the bottom arrayb1 = np.zeros(len(years)-2) We prepare the array (made of zeros) for the bottom argument of bar(). # for each line in datafor i in range(len(data)): # select all the data except the last 2 values d = data[i][:-2] For each data line, we extract the information we need, so we remove the last two values. # create bars for each element, on top of the previous barsbt = plt.bar(range(len(d)), d, width=width, color=cm.hsv(32*(i)), label=continents[i], bottom=b1)# update the bottom arrayb1 += d Then we plot the bar, and update the bottom array. # prepare the bottom arrayb2_1, b2_2 = np.zeros(2), np.zeros(2) We need two arrays because we will display two bars for the same year—one from the CSV and the other from our fitting function. # for each line in datafor i in range(len(data)): # extract the last 2 values d = data[i][-2:] Again, for each line in the data matrix, we extract the last two values that are needed to plot the bar for CSV. # select the data to compute the fitting functiony = data[i][:-2] Along with the other values needed to compute the fitting polynomial. # use a polynomial of degree 3c = np.polyfit(x, y, 3) Here, we set up a polynomial of degree 3; there is no need for higher degrees. # create a function out of those coefficientsp = np.poly1d(c) This method constructs a polynomial starting from the coefficients that we pass as parameter. # compute p on x2 values (we need integers, so the map)y2 = map(int, p(x2)) We use the polynomial that was defined earlier to compute its values for x2. We also map the resulting values to integer, as the bar() function expects them for height. # create bars for each element, on top of the previous barsbt = plt.bar(len(b1)+np.arange(len(d)), d, width=width/2, color=cm.hsv(32*(i)), bottom=b2_1) We draw a bar for the data from the CSV. Note how the width is half of that of the other bars. This is because in the same width we will draw the two sets of bars for a better visual comparison. # create the bars for the extrapolated valuesbt = plt.bar(len(b1)+np.arange(len(d))+width/2, y2, width=width/2, color=cm.bone(32*(i+2)), bottom=b2_2) Here, we plot the bars for the extrapolated values, using a dark color map so that we have an even better separation for the two datasets. # update the bottom arrayb2_1 += db2_2 += y2 We update both the bottom arrays. # label the X ticks with yearsplt.xticks(np.arange(len(years))+width/2, [int(year) for year in years]) We add the years as ticks for the X-axis. # draw a legend, with a smaller fontplt.legend(loc='upper left', prop=font_manager.FontProperties(size=7)) To avoid a very big legend, we used only the labels for the data from the CSV, skipping the interpolated values. We believe it's pretty clear what they're referring to. Here is the screenshot that is displayed on executing this example: The conclusion we can draw from this is that the United Nations uses a different function to prepare the predictions, especially because they have a continuous set of information, and they can also take into account other environmental circumstances while preparing such predictions. Tools using Matplotlib Given that it's has an easy and powerful API, Matplotlib is also used inside other programs and tools when plotting is needed. We are about to present a couple of these tools: NetworkX Mpmath

What do people mean by "sticky"? If you have ever ran a blog or web site before, you may have noticed that it's fairly easy to get a spike in traffic by submitting a good story to a few social bookmarking sites or by being lucky enough to get a link to one of your posts from a much larger site. The problem is that after a day or so, when the submissions fall off the front page, it's likely your traffic will die down to its usual levels again. Some site owners fall into the trap of chasing after the next traffic spike, using "linkbait" articles with intentionally controversial titles and content, when they should really be focusing on quality content, improving the site, and working towards sustained growth. Many bloggers submit their site to StumbleUpon.com. StumbleUpon is a web service where users can enter their interests, and be sent to a random site that will match those interests. Those users can then either give a "thumbs up" to the site they are sent to indicating that they like the site or a "thumbs down" if they don't like it. Those votes are used to improve future suggestions and increase the chances of the next site that they "Stumble Upon" being one that they are interested in. Other popular sites for increasing traffic include Technorati (a site that measures the "authority" of a blog based on how many other bloggers are linking to it), and the news/story-related sites Reddit (a general interest site with everything from politics to gadgets-related news), and Digg (a site with a focus on tech and gaming news). A sticky blog is one that doesn't just attract new visitors, it keeps them. Instead of having a visitor click through from a link on Technorati or visit by using the Stumble! feature of StumbleUpon, skim the page they land on and then leave, a sticky blog would make that visitor stay around a little longer. Ideally, visitors would read the article they were interested in and then find themselves intrigued enough to read more articles. They may comment on some articles and then keep returning to read answers to their comments. Or, they may decide to subscribe to the blog so that they can read future posts. A sticky site encourages readers to become engaged with the community, resulting in long-term increases in traffic. When new readers arrive at the site for the first time, they get involved themselves and keep coming back. They may also tell their friends or link to the site from their own sites, giving you free promotion. Letting readers and authors communicate Interesting content is vital, but one of the best ways to get people coming back to your blog network is to give them a chance to interact with the site's authors and with each other. This not only makes the readers feel valued, it also opens up a dialogue that encourages repeat visitors. Contact forms Providing visitors with a way to contact you privately is useful for several reasons. The visitor may want to discuss advertising opportunities, submit some news you may be interested in, or ask for help with a problem they have accessing part of the site. You could post your email address on the site, but this makes you vulnerable to spam attacks. A contact form is a safer way to allow your visitors to contact you. Time for action – setting up contact forms Let's set up a contact form: Download Contact Form 7 from http://wordpress.org/extend/plugins/contact-form-7/. To install, upload the contents of the archive file to /wp-content/plugins. Activate the plugin and go to the settings page (Tools | Contact Form 7). You can also access the page by clicking Settings under the plugin name, which appears on the Manage Plugins page. You can add new fields using the Generate Tag drop-down menu. Further down the admin page you will see options to set error messages (such as the message users will see if they miss out a required field, or if they try to upload a file that is too big). Once you have created the form, make a note of the tag at the top of the screen (in our case this was [contact-form 1 "Contact form 1"] ). Create a new page (Pages | Add New) called Contact Us, add a short message to the page, and then paste the contact form tag into the page. Depending on the theme you are using, you may need to add the Pages widget to your sidebar so that visitors can find the new page. Your page should look something like this: What just happened? Contact Form 7 is a powerful contact form tool that supports CAPTCHAs (via the Really Simple CAPTCHA  plugin), file uploads, drop-down menus, and more. You can define multiple contact forms and have each one submit to a different email address. This could be useful if you wish to have different people contacted for, say, advertising queries, news submissions, and tech support. You can also have a contact form submit to multiple email addresses. So, as well as having the relevant person receive a copy of each message, the site administrator could ensure they receive a copy of all messages too. You can set a prefix for each message, in addition to the subject line the visitor sets. For example, if you set the prefix to [Slayer-Form1], all emails from that contact form will have a subject line that begins with that text. You can use this to set up filters in your email application, making it easy to prioritize emails from different contact forms. Improved comments The basic WordPress MU comment feature allows readers to post their thoughts about a blog post, but it is not very good for encouraging discussion. One useful service for bloggers is IntenseDebate. This service allows for threaded discussion in comments, subscription to comments by RSS and email, and the ability to tie blog commenting in with other social networking sites and follow comments made by other blog readers. Time for action – IntenseDebate Comments Download the IntenseDebate Comments plugin from http://wordpress.org/extend/plugins/intensedebate/. You will need to sign up for an account at http://intensedebate.com/. Activate the plugin. Go to Settings | IntenseDebate. You will be presented with a login screen. Enter the account details for the account you created in step 2. Once you have logged in, click Start Importing Comments. The import process can take a very long time, even if you don't have many comments to import. Once the import process is complete, you can tweak the settings to suit your blog—although I found the default ones were a good starting point. The IntenseDebate Comments plugin has its own Comments caption, so you may want to remove the Comments header from the index.php file in your theme folder. The new comment box should look something like this: You can moderate comments using the already familiar WordPress MU interface or the dashboard on the IntenseDebate site. What just happened? IntenseDebate is a commenting system that sits on top of WordPress and WordPress MU. It is ideal for all blogs, whether they are part of a blog network, or a standalone blog. It does not replace the existing WordPress comment system; it only complements it. This means you can use IntenseDebate in conjunction with other plugins that rely on the WordPress MU comment system. Readers can comment on your blog using the IntenseDebate comment system. If they have JavaScript turned off, they will be presented with the normal WordPress comment system instead. IntenseDebate has lots of useful features that will make your users feel a greater sense of engagement with your site's authors. Those features are described below: Threaded discussions: IntenseDebate supports threaded comments. This makes it easy for readers to follow the discussions going on in the comments section. Readers can reply to the blog post itself, or reply to a specific comment, and IntenseDebate will break related comments into threads so that the discussion is easy to keep track of. Track comments or comment anonymously: Readers can comment anonymously, or, if they have an IntenseDebate profile they can log in to it and comment using it. Any comments made will be stored in the WordPress comments database and also be sent to IntenseDebate. Subscribe to comments: Readers can subscribe to comments on a particular post by email or through their favorite RSS reader. If they have an IntenseDebate account, they also have the option to send a Twitter message or "Tweet" to alert their friends that they have commented on a particular post. Reputation and voting: Another useful feature is the reputation system. Visitors can vote on comments, and comments that get a lot of negative votes will be hidden from view unless a user requests to see them. This is a handy form of "self moderation" for the community. The reputation system applies to only logged in users and gives each user an overall rating based on the quality of their comments on sites all over the Internet. Activating IntenseDebate on your users' blogs One important thing to remember is that even if you set IntenseDebate to automatically activate for your users, it won't do anything unless they set it up. Your users will still have the original WordPress MU comment system. They will be alerted to the fact that the plugin is not working for them by a message that will appear at the top of every page in their admin panel. Have a go hero – tweaking IntenseDebate IntenseDebate has so many features that there is not enough room to cover them all here. Take a look at the Extras (http://intensedebate.com/extras) page for some widgets that you may want to add to your blog. Also, check the Settings page for your blog in IntenseDebate. You can edit the moderation settings on that page. The default settings include a list of spam words that will cause comments to be flagged for moderation. Comments will also be flagged for moderation if they contain more than two URLs. You can tweak the commenting system's settings to filter by IP address, email address, key words, and profanity. You can also alter how the comments are displayed, the text displayed when people report comments, the layout, and the location of the blog's RSS feed. You may want to change that to use the FeedBurner version of the RSS feed. >> continue Reading: Sticky Features for your Blog Network with WordPress MU 2.8: Part 2

Creating validation functions for URLs, email addresses, and other types of data Ext JS has an extensive library of validation functions. This is how it can be used to validate URLs, email addresses, and other types of data. The following screenshot shows email address validation in action: This screenshot displays URL validation in action: How to do it... Initialize the QuickTips singleton: Ext.QuickTips.init(); Create a form with fields that accept specific data formats: Ext.onReady(function() { var commentForm = new Ext.FormPanel({ frame: true, title: 'Send your comments', bodyStyle: 'padding:5px', width: 550, layout: 'form', defaults: { msgTarget: 'side' }, items: [ { xtype: 'textfield', fieldLabel: 'Name', name: 'name', anchor: '95%', allowBlank: false }, { xtype: 'textfield', fieldLabel: 'Email', name: 'email', anchor: '95%', vtype: 'email' }, { xtype: 'textfield', fieldLabel: 'Web page', name: 'webPage', vtype: 'url', anchor: '95%' }, { xtype: 'textarea', fieldLabel: 'Comments', name: 'comments', anchor: '95%', height: 150, allowBlank: false }], buttons: [{ text: 'Send' }, { text: 'Cancel' }] }); commentForm.render(document.body);}); How it works... The vtype configuration option specifies which validation function will be applied to the field. There's more... Validation types in Ext JS include alphanumeric, numeric, URL, and email formats. You can extend this feature with custom validation functions, and virtually, any format can be validated. For example, the following code shows how you can add a validation type for JPG and PNG files: Ext.apply(Ext.form.VTypes, { Picture: function(v) { return /^.*.(jpg|JPG|png|PNG)$/.test(v); }, PictureText: 'Must be a JPG or PNG file';}); If you need to replace the default error text provided by the validation type, you can do so by using the vtypeText configuration option: { xtype: 'textfield', fieldLabel: 'Web page', name: 'webPage', vtype: 'url', vtypeText: 'I am afraid that you did not enter a URL', anchor: '95%'} See also... The Specifying the required fields in a form recipe, covered earlier in this article, explains how to make some form fields required The Setting the minimum and maximum length allowed for a field's value recipe, covered earlier in this article, explains how to restrict the number of characters entered in a field The Changing the location where validation errors are displayed recipe, covered earlier in this article, shows how to relocate a field's error icon Refer to the previous recipe, Deferring field validation until form submission, to know how to validate all fields at once upon form submission, instead of using the default automatic field validation The next recipe, Confirming passwords and validating dates using relational field validation, explains how to perform validation when the value of one field depends on the value of another field The Rounding up your validation strategy with server-side validation of form fields recipe (covered later in this article) explains how to perform server-side validation Confirming passwords and validating dates using relational field validation Frequently, you face scenarios where the values of two fields need to match, or the value of one field depends on the value of another field. Let's examine how to build a registration form that requires the user to confirm his or her password when signing up. How to do it… Initialize the QuickTips singleton: Ext.QuickTips.init(); Create a custom vtype to handle the relational validation of the password: Ext.apply(Ext.form.VTypes, { password: function(val, field) { if (field.initialPassField) { var pwd = Ext.getCmp(field.initialPassField); return (val == pwd.getValue()); } return true; }, passwordText: 'What are you doing?<br/>The passwords entered do not match!'}); Create the signup form: var signupForm = { xtype: 'form', id: 'register-form', labelWidth: 125, bodyStyle: 'padding:15px;background:transparent', border: false, url: 'signup.php', items: [ { xtype: 'box', autoEl: { tag: 'div', html: '<div class="app-msg"><img src="img/businessman add.png" class="app-img" /> Register for The Magic Forum</div>' } }, { xtype: 'textfield', id: 'email', fieldLabel: 'Email', allowBlank: false, minLength: 3, maxLength: 64,anchor:'90%', vtype:'email' }, { xtype: 'textfield', id: 'pwd', fieldLabel: 'Password', inputType: 'password',allowBlank: false, minLength: 6, maxLength: 32,anchor:'90%', minLengthText: 'Password must be at least 6 characters long.' }, { xtype: 'textfield', id: 'pwd-confirm', fieldLabel: 'Confirm Password', inputType: 'password', allowBlank: false, minLength: 6, maxLength: 32,anchor:'90%', minLengthText: 'Password must be at least 6 characters long.', vtype: 'password', initialPassField: 'pwd' }],buttons: [{ text: 'Register', handler: function() { Ext.getCmp('register-form').getForm().submit(); }},{ text: 'Cancel', handler: function() { win.hide(); } }]} Create the window that will host the signup form: Ext.onReady(function() { win = new Ext.Window({ layout: 'form', width: 340, autoHeight: true, closeAction: 'hide', items: [signupForm] }); win.show();

Deleting events After creating, moving, and editing events, we might come across a case where the edit form is different from the create form. There is no reason why you would want to delete an event that has not yet been created, so there is no reason to add a delete button to the "Create Event" form. We have a choice—add the delete button to the section at the bottom of the modal dialog, next to Save and Cancel, or add it to the body of the form itself. I always try to add delete buttons and links where I think they cannot be hit by accident. Therefore in this case, I chose not to add it to the row of buttons at the bottom. Instead, I placed it in the form itself where there's little chance it will be clicked by accident while saving or closing the form. And even then, if the link is clicked, there is always a secondary "Are you sure?" confirmation box. Client-side code In the calendar_edit_entry function in calender.js, change the beginning of the $.getJSON call to this: $.getJSON('./calendar.php?action=get_event&id='+calEvent.id, function(eventdata){ var controls='<a href="javascript:calendar_delete_entry' +'('+eventdata.id+');">[delete]</a>'; $('<div id="calendar_edit_entry_form" title="Edit Calendar Entry">' +'<div style="float:right;text-align:right">'+controls+'</div>' +'event name<br />' +'<input id="calendar_edit_entry_form_title" value="'+eventdata.title+'" /><br />' +'body text<br />' +'<textarea style="width:400px;height:200px" id="calendar_edit_entry_form_body">' '+eventdata.body+'</textarea></div>' ).appendTo($('body')); $("#calendar_edit_entry_form").dialog({ The only real change is to add the controls variable, which lets us create more buttons if necessary, and add that variable's contained HTML to the form. The only control there at the moment is a delete link, which calls the calendar_delete_entry function when clicked. Add this function now: function calendar_delete_entry(id){ if(confirm('are you sure you want to delete this entry?')){ $('#calendar_edit_entry_form').remove(); $.getJSON('./calendar.php?action=delete_event&id='+id, function(ret){ $('#calendar_wrapper').weekCalendar('refresh'); } ); } } Server-side code On the server side, we add a case to handle deletes: case 'delete_event': // { $id=(int)$_REQUEST['id']; unset($_SESSION['calendar'][$id]); echo 1; exit; // } All it needs to do is to unset the session variable. With that completed, you now have the finished basics of a calendar, where you can create events, move them around, edit them, and delete them. Walk-through of the calendar so far We've built the basics of a weekly calendar, and before we go on to discuss recurring events, let's take the time to walk through the calendar so far with a simple example. Let's say you have an appointment on Tuesday at 2 pm with a business partner. You add that by clicking on that time, as follows: You think that the meeting will go on for about two hours, so you resize it: Now Bob calls up early on Tuesday to say that he's not going to be able to make it, and suggests moving it to Wednesday at 1 pm. You drag the event over: He also says that he won't be able to make it, but Sally would be there. So, you click on the event and edit the form accordingly: Wednesday comes, and of course, something has come up on your end. You call Sally and explain that you won't be able to make it, and delete the event by clicking on the event, and then clicking on the delete link. Simple and quick. What more would you want? Let's do some recurring events. Recurring events Sometimes you will want to have the same event automatically populated in the calendar on a recurrent basis. For example, you go to lunch every day at 1 pm, or there might be a weekly office meeting every Monday morning. In this case, we need to come up with a way of having events recur. This can be simple or very complex. The simplest method is what we'll demonstrate in this article. The simple method involves entering a frequency (daily, monthly, and so on) and a final date, where the events stop recurring. On the server side, when it is asked to create that recurring event, the server actually iterates over the entire requested period and adds each individual event. This is not extremely efficient, but it's simple to write, and it's not likely that anyone would be placing years-long recurrent events on a very regular basis, so it's justifiable. The more complex method is to only create events that are actually visible in the week you are viewing, and whenever you change the week, you check to see if there are any events that are supposed to recur that week but are not visible. This is arguably even less efficient than the simple method, but it would allow us to be a little more flexible—for example, to leave out the final date so that the events just keep recurring. Anyway, given that there are no major drawbacks to either method, we will choose the simpler method. Client-side code On the client side, recurrences are created at the same time as the recurrence of the first event. So, we edit the "Create Event" form. In calendar.js , adapt the calendar_new_entry function by replacing the form-creation line with this: var recurbox='<select id="calendar_new_entry_form_recurring">' +'<option value="0">non-recurring</option>' +'<option value="1">Daily</option>' +'<option value="7">Weekly</option>' +'</select>';$('<div id="calendar_new_entry_form" title="New Calendar Entry">event name<br />' +'<input value="new event" id="calendar_new_entry_form_title" /><br />' +'body text<br />' +'<textarea style="width:400px;height:200px" id="calendar_new_entry_form_body">' event description</textarea>' +recurbox+'</div>') .appendTo($('body')); $('#calendar_new_entry_form_recurring') .change(calendar_new_get_recurring_end); This adds a select box below the body text area, requesting the user to choose a recurring frequency (defaulting to non-recurring). When the select box is changed, the function calendar_new_get_recurring_end is called. This function is used to request the final recurring date. We could use a plain old text field, but jQuery UI includes a really cool date widget, which allows us to request the date and have it stored in our own choice of format. I've chosen yyyy-mm-dd format, as it is easy to manipulate. Add this to calendar.js: function calendar_new_get_recurring_end(){ if(document.getElementById('calendar_new_recurring_end')) return; var year = new Date().getFullYear(); var month = new Date().getMonth(); var day = new Date().getDate(); $('<span> until <input id="calendar_new_recurring_end" value="'+year+'-'+(month+1)+'-'+day+'" style="font-size:14px" class="date" />' +' </span>' ).insertAfter('#calendar_new_entry_form_recurring'); $('.date').datepicker({ 'dateFormat':'yy-mm-dd', 'yearRange':'-10:+50', 'changeYear':true }); } That creates an input field after the dropdown box, and when it is clicked, a calendar pops up: Whoops! What's happened here is that the date pop up's z-index is lower than the modal dialog. That can be corrected by adding this CSS line to the  < head > section of calendar.html: <style type="text/css"> #ui-datepicker-div{ z-index: 2000; }</style> When reloaded, the calendar now looks correct: Great! Now, we just need to send the data to the server. To do that, change the Save button's $.getJSON parameters in the calendar_new_entry function in calendar.js to these (new parameters are highlighted): 'body':$('#calendar_new_entry_form_body').val(), 'title':$('#calendar_new_entry_form_title').val(), 'recurring':$('#calendar_new_entry_form_recurring').val(), 'recurring_end':$('#calendar_new_recurring_end').val() And we're done on the client side. Server-side code On the server side, the save switch case is going to change considerably, so I'll provide the entire section: case 'save': // { $start_date=(int)$_REQUEST['start']; $data=array( 'title'=>$_REQUEST['title'], 'body' =>$_REQUEST['body'], 'start'=>date('c',$start_date), 'end' =>date('c',(int)$_REQUEST['end']) ); $id=(int)$_REQUEST['id']; if($id && isset($_SESSION['calendar'][$id])){ if(isset($_SESSION['calendar'][$id]['recurring'])) $data['recurring']=$_SESSION['calendar'][$id]['recurring']; $_SESSION['calendar'][$id]=$data; } else{ $id=++$_SESSION['calendar']['ids']; $rec=(int)$_REQUEST['recurring']; if($rec) $data['recurring']=$id; $_SESSION['calendar'][$id]=$data; if($rec && $rec==1 || $rec==7){ list($y,$m,$d)=explode('-',$_REQUEST['recurring_end']); $length=(int)$_REQUEST['end']-$start_date; $end_date=mktime(23,59,59,$m,$d,$y); $step=3600*24*$rec; for($j=1,$i=$start_date+$step;$i<$end_date;$j++,$i+=$step){ $data['start']=date('c',$i); $data['end']=date('c',$i+$length); $nextid=++$_SESSION['calendar']['ids']; $_SESSION['calendar'][$nextid]=$data; } } } echo 1; exit; // } OK. From the data point of view, we've added a single field, recurring, which records the first event in the series. This is needed when deleting recurring events that are not needed anymore. When editing an existing event, all that's changed is that the recurring field (if it exists) is copied from the original before the event is overwritten with fresh data (shown highlighted). The real action happens when creating a new event. If a recurring period is required, then the event is copied and pasted at the requested frequency from the event's first creation until the expiry date. This is figured out by counting the seconds, and incrementing as needed. We can immediately see that recurring events work. Here's an example of a week's lunch hours created from the new recurring method: You can shift individual events around, and even delete them, without affecting the rest.

Placing buttons in a toolbar You can embed different types of components in a toolbar. This topic teaches you how to build a toolbar that contains image-only, text-only and image/text buttons, a toggle button, and a combo box. How to do it Create the styles for the toolbar items: #tbar{ width:600px;}.icon-data{ background:url(img/data.png) 0 no-repeat !important;}.icon-chart{ background:url(img/pie-chart.png) 0 no-repeat !important;}.icon-table{ background:url(img/table.png) 0 no-repeat !important;} Define a data store for the combo box: Ext.onReady(function() { Ext.QuickTips.init(); var makesStore = new Ext.data.ArrayStore({ fields: ['make'], data: makes // from cars.js }); Create a toolbar and define the buttons and combo box inline: var tb = new Ext.Toolbar({ renderTo: 'tbar', items: [{ iconCls: 'icon-data', tooltip: 'Icon only button', handler:clickHandler }, '-', { text: 'Text Button' }, '-', { text: 'Image/Text Button', iconCls: 'icon-chart' }, '-', { text: 'Toggle Button', iconCls: 'icon-table', enableToggle: true, toggleHandler: toggleHandler, pressed: true }, '->', 'Make: ', { xtype: 'combo', store: makesStore, displayField: 'make', typeAhead: true, mode: 'local', triggerAction: 'all', emptyText: 'Select a make...', selectOnFocus: true, width: 135 }]}); Finally, create handlers for the push button and the toggle button: function clickHandler(btn) { Ext.Msg.alert('clickHandler', 'button pressed');}function toggleHandler(item, pressed) { Ext.Msg.alert('toggleHandler', 'toggle pressed');} How it works The buttons and the combo box are declared inline. While the standard button uses a click handler through the handler config option, the toggle button requires the toggleHandler config option.The button icons are set with the iconCls option, using the classes declared in the first step of the topic. As an example, note the use of the Toolbar.Separator instances in this fragment: }, '-', { text: 'Text Button'}, '-', { text: 'Image/Text Button', iconCls: 'icon-chart'}, '-', { Using '-' to declare a Toolbar.Separator is equivalent to using xtype: 'tbseparator'. Similarly, using '->' to declare Toolbar.Fill is equivalent to using xtype:'tbfill'. See also... The next recipe, Working with the new ButtonGroup component, explains how to use the ButtonGroup class to organize a series of related buttons Working with the new ButtonGroup component A welcome addition to Ext JS is the ability to organize buttons in groups. Here's how to create a panel with a toolbar that contains two button groups: How to do it Create the styles for the buttons: #tbar{ width:600px;}.icon-data{ background:url(img/data.png) 0 no-repeat !important;}.icon-chart{ background:url(img/pie-chart.png) 0 no-repeat !important;}.icon-table{ background:url(img/table.png) 0 no-repeat !important;}.icon-sort-asc{ background:url(img/sort-asc.png) 0 no-repeat !important;}.icon-sort-desc{ background:url(img/sort-desc.png) 0 no-repeat !important;}.icon-filter{ background:url(img/funnel.png) 0 no-repeat !important;} Define a panel that will host the toolbar: Ext.onReady(function() { var pnl = new Ext.Panel({ title: 'My Application', renderTo:'pnl-div', height: 300, width: 500, bodyStyle: 'padding:10px', autoScroll: true, Define a toolbar inline and create two button groups: tbar: [{ xtype: 'buttongroup', title: 'Data Connections', columns: 1, defaults: { scale: 'small' }, items: [{ xtype:'button', text: 'Data Sources', iconCls:'icon-data' }, { xtype: 'button', text: 'Tables', iconCls: 'icon-table' }, { xtype: 'button', text: 'Reports', iconCls: 'icon-chart' }]}, { xtype: 'buttongroup', title: 'Sort & Filter', columns: 1, defaults: { scale: 'small' }, items: [{ xtype: 'button', text: 'Sort Ascending', iconCls: 'icon-sort-asc' }, { xtype: 'button', text: 'Sort Descending', iconCls: 'icon-sort-desc' }, { xtype: 'button', text: 'Filter', iconCls: 'icon-filter' }]}] How it works Using a button group consists of adding a step to the process of adding buttons, or other items, to a toolbar. Instead of adding the items directly to the toolbar, you need to firstly define the group and then add the items to the group: tbar: [{ xtype: 'buttongroup', title: 'Data Connections', columns: 1, defaults: { scale: 'small' }, items: [{ xtype:'button', text: 'Data Sources', iconCls:'icon-data' }, { xtype: 'button', text: 'Tables', iconCls: 'icon-table' }, { xtype: 'button', text: 'Reports', iconCls: 'icon-chart' }]} See also... The next recipe, Placing buttons in a toolbar, illustrates how you can embed different types of components in a toolbar Placing menus in a toolbar In this topic, you will see how simple it is to use menus inside a toolbar. The panel's toolbar that we will build, contains a standard button and a split button, both with menus: How to do it Create the styles for the buttons: #tbar{ width:600px;}.icon-data{ background:url(img/data.png) 0 no-repeat !important;}.icon-chart{ background:url(img/pie-chart.png) 0 no-repeat !important;}.icon-table{ background:url(img/table.png) 0 no-repeat !important;} Create a click handler for the menus: Ext.onReady(function() { Ext.QuickTips.init(); var clickHandler = function(action) { alert('Menu clicked: "' + action + '"');}; Create a window to host the toolbar: var wnd = new Ext.Window({ title: 'Toolbar with menus', closable: false, height: 300, width: 500, bodyStyle: 'padding:10px', autoScroll: true, Define the window's toolbar inline, and add the buttons and their respective menus: tbar: [{ text: 'Button with menu', iconCls: 'icon-table', menu: [ { text: 'Menu 1', handler:clickHandler.createCallback('Menu 1'), iconCls: 'icon-data' }, { text: 'Menu 1', handler: clickHandler.createCallback('Menu 2'), iconCls: 'icon-data'}]}, '-',{ xtype: 'splitbutton', text: 'Split button with menu', iconCls: 'icon-chart', handler: clickHandler.createCallback('Split button with menu'), menu: [ { text: 'Menu 3', handler: clickHandler.createCallback('Menu 3'), iconCls: 'icon-data' }, { text: 'Menu 4', handler: clickHandler.createCallback('Menu 4'), iconCls: 'icon-data'}] }]}); Finally, show the window: wnd.show(); How it works This is a simple procedure. Note how the split button is declared with the xtype: 'splitbutton' config option. Also, observe how the createCallback() function is used to invoke the clickHandler() function with the correct arguments for each button. See also... The next recipe, Commonly used menu items, shows the different items that can be used in a menu Commonly used menu items To show you the different items that can be used in a menu, we will build a menu that contains radio items, a checkbox menu, a date menu, and a color menu.This is how the radio options and checkbox menu will look: The Pick a Date menu item will display a date picker, as shown in the next screenshot: The Pick a Color menu item displays a color picker, as seen here: How to do it Create a handler for the checkbox menu: Ext.onReady(function() { Ext.QuickTips.init(); var onCheckHandler = function(item, checked) { Ext.Msg.alert('Menu checked', item.text + ', checked: ' + (checked ? 'checked' : 'unchecked')); }; Define a date menu: var dateMenu = new Ext.menu.DateMenu({ handler: function(dp, date) { Ext.Msg.alert('Date picker', date); }}); Define a color menu: var colorMenu = new Ext.menu.ColorMenu({ handler: function(cm, color) { Ext.Msg.alert('Color picker', String.format('You picked {0}.', color)); }}); Create a main menu. Now add the date and color menus, as well as a few inline menus: var menu = new Ext.menu.Menu({ id: 'mainMenu', items: [{ text: 'Radio Options', menu: { items: [ '<b>Choose a Theme</b>', { text: 'Aero Glass', checked: true, group: 'theme', checkHandler: onCheckHandler }, { text: 'Vista Black', checked: false, group: 'theme', checkHandler: onCheckHandler }, { text: 'Gray Theme', checked: false, group: 'theme', checkHandler: onCheckHandler }, { text: 'Default Theme', checked: false, group: 'theme', checkHandler: onCheckHandler } ] } }, { text: 'Pick a Date', iconCls: 'calendar', menu: dateMenu }, { text: 'Pick a Color', menu: colorMenu }, { text: 'The last menu', checked: true, checkHandler: onCheckHandler }]}); Create a toolbar and add the main menu: var tb = new Ext.Toolbar({ renderTo: 'tbar', items: [{ text: 'Menu Items', menu: menu }]}); How it works After defining the date and color pickers, the main menu is built. This menu contains the pickers, as well as a few more items that are defined inline. To display checked items (see the checked: true config option) with a radio button instead of a checkbox, the menu items need to be defined using the group config option. This is how the theme selector menu is built: menu: { items: [ '<b>Choose a Theme</b>', { text: 'Aero Glass', checked: true, group: 'theme', checkHandler: onCheckHandler }, { text: 'Vista Black', checked: false, group: 'theme', checkHandler: onCheckHandler See also... The Placing buttons in a toolbar recipe (covered earlier in this article) illustrates how you can embed different types of components in a toolbar >> Continue Reading Making Progress with Menus and Toolbars using Ext JS 3.0: Part 2 [ 1 | 2 ]   If you have read this article you may be interested to view : Making Progress with Menus and Toolbars using Ext JS 3.0: Part 2 Load, Validate, and Submit Forms using Ext JS 3.0: Part 1 Load, Validate, and Submit Forms using Ext JS 3.0: Part 2 Load, Validate, and Submit Forms using Ext JS 3.0: Part 3

eZ Publish templating In the first part of this article, we will introduce the basics of the eZ Publish templating system, which will help us to better understand the rest of the article Templating eZ Publish has its own templating system based on the decoupling of layout and content. This will help us to assign a custom layout to any content object in different sections. Moreover, just as other templating platforms, such as Smarty (http://www.smarty.net), eZ Publish has its own markup to help developers with control structure operations, subtemplating, and on-the-fly content editing. It also exposes a particular function to fetch and filter content from a database. The official eZ Publish website has a constant, up-to-date reference with the entire templating markup. We suggest you to use the following link every time that you need to know more details about the available arguments:http://ez.no/doc/ez_publish/technical_manual/4_0/templates/ The templating markup All of the eZ Publish templating code should be placed between curly brackets. When the CMS will parse our template file and find the curly brackets, it will start executing the related code. Escaping the curly bracketsIf we need to use curly brackets, for example to write a javascript function inside our template, we need to use the {literal} operator. {literal}<script type="text/javascript">function alertMe() { window.alert('Harkonen approaching!');}</script>{/literal} Control structure operators We can divide these function into two main families: Conditional (IF-THEN-ELSE) Looping (FOR-FOREACH-WHILE) Whereas the first one should be used to change the template behavior according to some predefined condition, the other one will help us to seek and manage array and content structures. Conditional control Conditional control is sometimes useful for changing the output when some data is received by the system. For example, we would need a different CSS class for a particular value, or to change the <div> class, if the current month is the same as the one displayed, as shown below: {def $current_month=currentdate()|datetime(custom, '%F')}{if $node.name|eq($current_month) }<span class="this-month">{else}<span class="default-month">{/if}{undef $current_month} In the first line, we define a $current_month variable that has a value of the name of the month (for example, October), retrieved by the datetime() operator. Then we use the IF conditional control to choose the correct class. In the last line, we delete the variable previously created, by releasing it from system memory. Loop control As stated above, the loop control structure can be used to iterate through an array. We can, for example, create an unordered list (<ul>) from an array of items. <ul>{foreach $items as $item} <li>{node_view_gui content_node=$item view=line}</li>{/foreach}</ul> This will be rendered as: <ul> <li>1st item</li> <li>2nd item</li> <li>3rd item</li> ...</ul> As you can see, the FOREACH structure is similar to the PHP structure. In this example, the most interesting line is the definition of the list object. This we can literally read as: render the content node (node_view_gui) from a specific node (content_node=$issue) using the line view template (view=line). Fetch functions With the fetch functions, we can retrieve all of the information about a content object for a module. The fetch functions can also be used to create custom queries to retrieve only the information we need, and not everything. eZ Publish exposes many fetch functions, which can be read about on the documentation site at http://ez.no/doc/ez_publish/technical_manual/4_0/reference/template_fetch_functions The most important, and most used, fetch functions are those regarding the content, sections, and user modules. For example, we can fetch the root content object by using the following code in our template: {$object = fetch('content', 'object', hash('id', '1'))} We can then use the $object variable to display the object inside the HTML code. Generic template functions and operators The CMS gives us a lot of functions and operators, all of them described in the reference manual of the eZ System documentation site. As a thumb rule, we should remember that to execute a particular function, we have to use the following syntax: {function_name parameter1=value1 ... parameterN=valueN } All parameters are separated by spaces and can be specified in no particular order. If we want to manage the operators, we have to remember that they accept the parameters passed in a specific order, separated by a comma. Moreover, an operator should handle a parameter passed to it with a pipe (|). {$piped_parameter|my_operator( parameter1, ..., parameterN ) } Every time we see a pipe after a variable, we have to remember that we are passing a value to an operator. We used the datetime() operator in the previous example for the conditional control functionality. As a reference to API functions and operators, you can use the official variable documentation that is constantly updated on the eZ System site:http://ez.no/doc/ez_publish/technical_manual/4_0/reference/template_operatorshttp://ez.no/doc/ez_publish/technical_manual/4_0/reference/template_functions Layout variables By default, the page layout template can access some of the variables passed by the CMS. These variables, named Layout variables, can be used to render system and user information, or to change the output. These variables are automatically configured by eZ Publish when it analyzes and executes the code related to a view. One of the most important variables is $module_result, which contains the results generated by the module and the view that is being executed. A module is an HTTP interface that interacts with eZ Publish. A module consists of a set of views that contain the code to be executed. For example, if we call the following URL, the system executes the login view code of the user module:http://www.example.com/index.php/user/login. As an API reference, you can use the official variable documentation that is constantly updated on the eZ System site:http://ez.no/doc/ez_publish/technical_manual/4_0/templates/the_pagelayout/variables_in_pagelayout Overriding a template eZ Publish offers a set of standard templates that are useful, but they cannot cover all the possible design needs. To solve this issue, the CMF provides a fallback system that allows us to load different templates based on specific rules. This system is usually referred to as overriding, and allows us to change the template for each module's view by overriding the default template when the user is in a particular context. Embedding HTML inside the WYSIWYG XML editor, pt.2 We have to override a standard behavior of eZ Publish to create a generic HTML block inside the WYSIWYG XML editor. We use a content style named html for the online editor and we will work on it for the frontend to render it correctly. First, we have to create a file named literal.tpl and place it in the design folder of our extension. The following code will do exactly what we need: # mkdir -p /var/www/packtmediaproject/extension/packtmedia/design/magazine/templates/datatype/view/ezxmltags/# cd /var/www/packtmediaproject/extension/packtmedia/design/magazine/templates/datatype/view/ezxmltags/# touch literal.tpl Next, we will open the literal.tpl file in our preferred IDE. Now we will add the code that will, by default, render everything surrounded by a <pre> tag and the raw HTML code, if the class is html: {if ne( $classification, 'html' )} <pre {if ne( $classification|trim, '' )} class="{$classification|wash}"{/if}>{$content|wash( xhtml )}</pre>{else} {$content}{/if} This code will check to see if the $classification variable is different from the "html" string in order to add the <pre> tag and then, again, it will add a class attribute to the <pre> tag if the $classification variable is not null. To use it, we only need to reset the cache from the shell prompt by using the following command: cd /var/www/packmediaproject/php bin/php/ezcache.php --clear-all --purge The ezcache.php file is a PHP shell script that can be used to clear and manage the eZ Publish cache. This file has many parameters, which can be viewed by using the --help parameter. Creating a new design Before starting work on the eZ Webin template code, we need to create a wireframe in order to decide on the layout structure. We will use this structure to override the standard layout files. A wireframe is a basic visual guide that is used in web design to suggest the structure of a website and the relationships between its pages. Wireframe editorsThere are a lot of commercial and free wireframe editors. To create our site's wireframes, we will use the Firefox plugin called Pencil(http://www.evolus.vn/Pencil/).We have chosen Pencil because it is open source and works on every platform that runs the Firefox browser.If you need something more complete, you should take a look at Balsamiq (http://www.balsamiq.com/) or at OmniGraffle (http://www.omnigroup.com/applications/OmniGraffle/) if you have an Apple computer. Our site will have at least six different page layouts: The homepage The issue page, where we will display the cover and the articles list The issue archive page, by month and by years The staff profile page, where we will display the latest articles that the editor has written, along with his profile The article and the forum pages, with the default layout based on the eZ Webin design Now we will illustrate the first four layouts because we will work on them, overriding their standard eZ Webin layout. The homepage Starting from the homepage, we can see that the site will have, in the top-left corner, a logo for the magazine and a place-holder for a banner. Under these, we will have the main navigation menu and the main content area. We have chosen a three-column layout in order to easily manage the content that we want to show. In the homepage, the first column will show the latest news and the middle column will show the information and cover of the latest issue. The last column will have two boxes—one with the most important article from the latest issue and the other with the forum thread. Issue page The issue page will show some information of a specific magazine issue. In this page, the middle box of the homepage will shift towards the left, and in the right column there will be the highlighted article for the issue. At the bottom of the page, we will find all of the other articles. The issue archive We have to remember that our magazine is released monthly, so we need an archive page where we can collect all of the past issues. The issue archive page, which can be reached by clicking on the main navigation menu, will again show some information from the latest issue. (We need to sell our articles!) The rightmost column of the template will show all of the covers for the current or selected year. At the bottom of the page, we will create a box with links to the past issues grouped by years and months. The staff profile page The staff profile page will display information from a staff profile, such as his avatar, biography, and the latest articles that he has written. The staff profile page will have three columns. The first column will show information regarding the editor's profile, the middle column will show all of the articles the editor has written (paged five by five) and the third will be used for banners or other images. >> Continue Reading: Creating a Design with eZ Publish 4 Templating System: Part 2

These instructions are Windows based but Linux users should have no difficulty adjusting them for their environment. Checking your installation If you already have SOA Suite and JDeveloper installed, confirm that you have the correct version and configuration by following the steps in the section below called Testing your installation. In addition, you may want to complete the items in the section called Additional actions. Finally, you must complete the section called Configuration to run a tutorial. What you will need and where to get it This installation requires 3 GB or more available memory. If you have less memory, try separating the installation of the database, the servers, and JDeveloper onto different machines. Memory and Disk Space requirements This installation requires 3 GB or more available memory. If you have less memory, try separating the installation of the database, the servers, and JDeveloper onto different machines. The installation process requires about 12 GB of disk space. After installation, you can delete the files used by installation to save about 4 GB. As you can see, you are installing a lot of software with a large memory and disk footprint. Running your disk defragmentation program now, before you start downloading and installing, can significantly improve install time as well as performance and disk space usage later on. Downloading files Download all the software to get started. In the following steps, save all downloaded files to c:stageSOA. This document assumes that path. If you save them somewhere else then make sure there are no spaces in your path and adjust accordingly when c:stageSOA is referenced in this document. Go to: http://www.oracle.com/technology/products/soa/soasuite/index.html, and download the following from SOA Suite 11g Release 1 (11.1.1.1.0) to c:stageSOA: WebLogic Server:wls1031_win32.exe Repository Creation Utility:ofm_rcu_win32_11.1.1.1.0_disk1_1of1.zip SOA Suite:ofm_soa_generic_11.1.1.1.0_disk1_1of1.zip JDeveloper Studio, base install:jdevstudio11111install.jar Unzip the SOA Suite ZIP file to c:stageSOA. Unzip the RCU ZIP file to c:stageSOA. Additional Files needed: Tutorial Files: In Chapter 3, you were directed to download the files needed for this tutorial. Do that now as some are used during installation. You can download the files from here:http://www.oracle.com/technology/products/soa/soasuite/11gthebook.html. Unzip the tutorial ZIP file to c:stageSOA. SOA Extension for JDeveloper: You will get this later using the JDeveloper update option. Oracle Service Bus: When you are ready to do the Oracle Service Bus (OSB) lab, you will download the install file to install OSB. Checking your database Having your database up and running is the most important pre-requisite for installing SOA Suite. Read the following bulleted requirements carefully to be sure you are ready to begin the SOA Suite installation: You need one of: Oracle XE Universal database version 10.2.0.1 Oracle 10g database version 10.2.0.4+ Oracle 11g database version 11.1.0.7+ You cannot use any other database version in 11gR1 (certification of additional databases is on the roadmap). Specifically, you cannot use XE Standard, it must be Universal. We have seen problems with installing XE when a full 10g database is already installed in the environment. The Windows registry sometimes gets the database file location confused. It is recommended to pick one or the other to avoid such issues. If you need to uninstall XE, make sure that you follow the instructions in Oracle Database Express Edition Installation Guide 10g  Release 2 (10.2) for Microsoft Windows Part Number B25143-03, Section 7, Deinstalling Oracle Database XE (http://download.oracle.com/docs/cd/B25329_01/doc/install.102/b25143/toc.htm). If you need to uninstall 10.2, be sure to follow the instructions in Oracle Database Installation Guide 10g Release 2 (10.2) for Microsoft Windows (32-Bit) Part Number B14316-04, Section 6, Removing Oracle Database Software(http://download.oracle.com/docs/cd/B19306_01/install.102/b14316/ deinstall.htm). Optional: Install OracleXEUniv.exe—recommended for a small footprint database. Make sure that you read step 1 above before installing. You can get XE from here: http://www.oracle.com/technology/products/database/ xe/index.html. When you are using XE, you will see a warning when you install the database schema that this database version is too old. You can safely ignore this warning as it applies only to production environments. If needed, configure Oracle XE Universal. When you are using Oracle XE, you must update database parameters if you have never done this for your database installation. You only have to do this once after installing. Set the processes parameter to >=200 using the following instructions. C:OracleMiddlewarehome_11gR1user_projectsdomains domain1binsetSOADomainEnv.cmdCODE 1sqlplus sys/welcome1@XE as sysdbaSQL> show parameter sessionSQL> show parameter processesSQL> alter system reset sessions scope=spfile sid='*';SQL> alter system set processes=200 scope=spfile;SQL> shutdown immediateSQL> startupSQL> show parameter sessionSQL> show parameter processes The shutdown command can take a few minutes and sometimes the shutdown/startup command fails. In that case, simply restart the XE service in the Control Panel | Administrative Tools | Services dialog after setting up your parameters. Checking your browser Oracle SOA Suite 11gR1 has specific browser version requirements. Enterprise Manager requires Firefox 3 or IE 7. Firefox 3—get a portable version, such as the one available from http://portableapps.com, if you want it to co-exist peacefully with your Firefox 2 installation. Firefox 2 and IE 6 are not supported and will not work. BAM requires IE 7. Beware of certain IE 7 plugins that can create conflicts (a few search plugins have proved to be incompatible with BAM). IE 8 is not supported with 11gR1 (but is on the roadmap). IE 6 has a few issues and Firefox will not work with BAM Studio. Checking your JDK If you are going to install WebLogic server and JDeveloper on the same machine, you will use the JDK from WebLogic for JDeveloper too. However, if you are going to install on two machines, you need Java 1.6 update 11 JDK for JDeveloper. JDK 1.6 update 11—from the Sun downloads page: http://java.sun.com/products/archive/ You must use Java 1.6 update 11. Update 12 does not work.

Using video files Video files are generally large due to the amount of content they contain and their length. It's beyond the scope of our article to describe them in detail, but in basic terms, they are a linear sequence of still images placed together to create a sequence of movement, usually accompanied by an audio track. Original video files are compressed using a codec to produce a compressed video file. The various codecs produce different results for file size, quality, and export. Video files play in the browser by downloading the data through the Internet, progressively streaming it so the movie begins to play before the whole file has downloaded. Audio files work in a similar way, but are often not as large. The final quality of a video also depends on the method used to capture it and how it's stored. The better the quality of the camera, the better the result. If you want to learn more about video, Wikipedia has a page at http://en.wikipedia.org/wiki/Video_formats. Just like anything else, there are pros and cons of adding videos to your website. YouTube alone has proven there is a strong market for a more visual medium. However, there are still many people who prefer text-based content as well. Consider whether adding a video to your site will enhance your user's experience. Is the material promotional or instructional? Is the content better demonstrated than explained? Video material can broaden your target audience. Many people prefer watching a video online to reading lengthy bodies of text. Videos aren't that great for search engine optimization. Consider adding a transcript to the page as well, in order to increase the ability to search. Choosing the best video file format Video played through the Internet requires a media player, which acts as an interface between the video file and the browser. These days most Internet users have one embedded within their browser. Popular versions include: QuickTime, a player created by Apple Windows Media Player WINAMP Real Player, developed by Real Networks Adobe Flash Player The following are some of the video file types that can be played through your website using third-party media players: .wmv files are a popular format developed by Microsoft and which come bundled within the Internet Explorer software package and are, therefore, preinstalled on Windows PCs. This is a format good for movies with movement within them. This format works with Windows Media Player, RealPlayer, and another called VLC Player. This format isn't very compatible with Mac or Linux computer users. .mov files are a QuickTime video platform extension that also plays back on the Windows operating system. The Apple QuickTime movie player software can be easily downloaded from Apple at http://www.apple.com/quicktime/download/. While not many browsers have the QuickTime media player installed, this format does provide very high quality video. You can always provide a link to the URL to download the software in order to play the video. .avi files are often the format of videos with smaller dimensions, played back through a website. They are a container for audio and video files (hence the name!). They can sometimes be quite large in file size, depending on the codec used to compress the video footage. They are a mainstream format. .swf and .flv videos are excellent for web video streaming and can also include interactive features. Most Mac and PCs have the Flash Shockwave Player installed; however, it can be downloaded from http://get.adobe.com/flashplayer/. Take note of the requirements for your individual operating system and browser preferences. Keep the following in mind when considering a video for your website: Ensure the video is succinct and the file size as small as possible. Even with a high speed download, time is still required to fully download the complete file. Keeping the video between one to three minutes long and the file size under five megabytes The more the movement in a video, the larger the file size. Consider whether the video really enhances the message. Viewers are only interested in material that is useful to them and will resent consuming their download resources on a video that holds no value for them. The larger the file size, the longer it takes to upload. Consider your audience's data rate. Do they have high speed downloads or are there some with dial-up connections? Video files are generally large due to the amount of content they contain. They stream (streaming is the way the Internet transfers multimedia information) through the data so the video will begin playing before it has fully downloaded itself into the browser, allowing it to be played back as quickly as possible. Audio files work in a similar way, but are not usually big files. There are various video file formats available and most website users have a player to see them already contained within the browser. Many users have QuickTime, a player created by Apple (that also runs on PCs) and Real Player, developed by Real Networks. Videos require a special plugin to play them through an article on your site, once you have uploaded it. Alternatively, you can embed a link from the popular YouTube site (http://www.youtube.com/). We'll look at how to do both in relation to the Party People website. Uploading a video We'll upload a new video, much the same way we would upload an image, to a new subfolder called videos within the Party People website. The steps are as follows: Navigate to the Media Manager. Select the stories folder and type videos into the Files input box. Click Create Folder. Select the new videos folder icon; then click the Browse button to choose the video from our desktop computer. Click Start Upload. Now we have a video file ready to be inserted into an article. The Party People website has the popular AllVideos plugin installed to do this. Updating videos—AllVideos plugin This is another neat plugin that works in much the same way as the Simple Image Gallery, a stablemate from this team of developers. If you don't have it installed and you would like to present videos on your site, ask your developer to install it for you or refer to the developers website http://www.joomlaworks.gr/content/view/35/41/ for instructions. Our Party People website has a .mov video on the Products and Services page, which we will update. To update the video display: Navigate to the Article Manager through the top menu and open theOur Services Include... article. Change the name of the video file between the { } and {/} tags within the text editor to the new filename. Depending on the format of the video being presented, the code should look like this: {mov}promoVideo{/mov} This code displays a QuickTime movie within the article. Save the changes. The following screenshot shows us how the video will look in context. Note that you do not need to include the format extension at the end of the filename, as the tag surrounding the name addresses this. Changing to a different video file and format The AllVideos plugin supports a number of video file formats and the developer's website lists them all at http://www.joomlaworks.gr/content/view/35/41/. We'll change the video we just linked to a different one which is in the .wmv format. The steps are as follows: Navigate to the Article containing the video presentation. Change the tag between the { } braces to reflect the new file type, taking care not to delete any of the symbols. For example: {wmv}updatedServicesVideo{/wmv} Save the changes to your article. You should take care to avoid rearranging any of the formatting within the code, as this will prevent the movie from playing. That is, don't add any extra spaces, colons, commas, and so on.