Search icon CANCEL
Subscription
0
Cart icon
Your Cart (0 item)
Close icon
You have no products in your basket yet
Arrow left icon
Explore Products
Best Sellers
New Releases
Books
Events
Videos
Audiobooks
Packt Hub
Free Learning
Arrow right icon
timer SALE ENDS IN
0 Days
:
00 Hours
:
00 Minutes
:
00 Seconds

How-To Tutorials

7019 Articles
article-image-how-recover-active-directory-failure
Packt
22 Oct 2009
19 min read
Save for later

How to Recover from an Active Directory Failure

Packt
22 Oct 2009
19 min read
Active Directory (AD) failure, which includes corruption, is something that is dreaded by any administrator. Simply put, it means that the directory service can no longer read the Active Directory database that it has locally. This will prevent logon and authentication as well as any directory-dependent services. Essentially, it renders the domain controller (DC) useless. What's worse, though, is that several times, the replication keeps going so that the corrupted database spreads out to the other DCs. In this article by Florian Rommel, we will look at the different options and approaches available to recover a DC that has a database corruption. In addition, this article outlines, symptoms, causes, and solutions for this scenario. Problems and Symptoms Let's take a look at the symptoms and then the causes. Symptoms The database gets corrupted and the DC is no longer able to process and authenticate or perform directory lookups. This becomes apparent when replication fails for some reason, or a large amount of specific event log errors appear. Another symptom could be that the AD services don't start on the DC. Causes This scenario can be caused by:      A software glitch, which can due to upgrading the schema, but having customized schema entries      An unclean AD write      A replication that has been interrupted      An accidental or malicious change in the AD schema with low level tools such as ADSIedit or something similar. Solution Process The recovery process is as follows:      You will have to verify that it is, in fact, a failure or corruption within the AD database, and not a network-related problem or other problem.      You will then have to perform a specific directory restore mode recovery, where you have to decide between an authoritative and a non-authoritative mode.      Once the recovery is complete, you will need to verify whether the DC is replicating and functioning properly. Solution Details What follows is a complete outline of what to do during each part of the solution process. Verification of Corruption If the AD on a domain controller becomes corrupted, or stops replicating to other DC's, or both, find out the root cause for this. A good starting point is to check again that the DNS is in order, and revert any manual changes that may have been made recently. Also, ensure that it is not a network-related problem—this means that no routers and routes have been changed, or firewalls re-configured, and the connection is not down. These are more often than not the main causes, and not an actual corruption. If you can safely rule out those causes, you can use utilities such as ReplMon and DCDiag, which are included in the Windows 2003 support tools, available free from Microsoft's website or on your install CD. Although ReplMon is a graphical utility, it is pretty small, and one of the best tool for checking whether or not there are replication errors within an entire domain. It shows which DCs are not replicating and why. The other utility, DCDiag, scans every DC, and determines if and why they have replication and other errors. When you have checked that all other DCs replicate just fine, you should check the Event log for specific event IDs (467 and 1018), that only occur when you have a real database corruption, and the AD jet database, which AD uses, is unreadable. Tools for Verification The Windows 2003 support tools (found on your installation CD under the SUPPORT folder) and the Windows 2003 resource kit tools (found at http://www.microsoft.com/downloads/details.aspx?FamilyID=9d467a69-57ff-4ae7-96ee-b18c4790cffd&displaylang=en ) provide a variety of tools to verify if the DC is still operational, if there is actually a problem, and where the problem lies. Although the usage of these programs can also be described as part of an AD health check, in this case, we will focus on a single DC. The output from some of these tools is fairly long, and for brevity's sake, we will focus on only the relevant parts. ReplMon ReplMon, short for Replication Monitor, is essential in your arsenal of tools for detecting replication errors within a domain. It can also provide you with a good view of replication partners for each DC, and allows you to run a check against an entire domain for replication errors. The following screenshot shows the default ReplMon window, while the next screenshot shows the right-click action menu for a monitored DC. To run ReplMon,  simply type replmon from the command line. An example of where ReplMon becomes very useful is to detect errors while trying to replicate to other DCs. There are different errors for different scenarios. but, for example, a server that cannot be reached, or is offline, would show the following error in the domain-wide search for errors: DCDiag DCDiag is a command line utility that performs a full check of the DC as regards AD. These tests include forest DNS tests, to check that the DNS is okay at the forest level, domain DNS tests, to do the same on a domain level, configuration test, schema test, and the FSMO test to check that all FSMO servers are available. Running it is as simple as typing dcdiag at the command line, and watching the output. You might want to add an output file such as dcdiag > c:dcdiag_output.txt at the end, to save everything to a text file that you can read easily. A successful DCDiag test would have the following results: NetDiag and DNSDiag Both of these utilities check the network connectivity of the DC on which they are installed. DNSDiag is more geared towards Exchange, and checks that all of the essential DNS records are valid and are working, by pretending to follow the MX records. It also gives us a lot of output with regard to the domain DNS structure, and identifies if there are any problems. NetDiag checks the local networking stack. It tests all the installed protocols and services, including WINS, NetBT, and TCP/IP. Sonar The Sonar.exe utility provides a GUI, which shows exactly where a replication failed, and what the state of the File Replication Service (FRS) is on each of the DCs. This is particularly useful for large environments. It also has different views that allow you to troubleshoot your FRS on a DC. Options to Recover and Stop the Spread of Corruption If, in effect, you have a corrupted or failed AD database, and it hasn't spread yet, meaning it is only on one DC, you should remove that DC from the replication chain as quickly as possible. A good way is, of course, to disconnect the network connection, though this will have an impact if you are working remotely on the machine. The other option is to isolate the DC with firewall rules. This is also the safest way that still gives you access to the machine remotely. If neither option is available to you, you can use the Repadmin utility to stop outbound and inbound replication. The two options are repadmin /options DCNAME +DISABLE_INBOUND_REPL and repadmin /options DCNAME +DISABLE_OUTBOUND_REPL, where DCNAME is the name of the DC that should be disabled. If you want to enable the replication, simply retype the command with a - instead of a + so as repadmin /options DCNAME -DISABLE_INBOUND_REPL. The following screenshot shows the command line for disabling outbound replication with Repadmin. Please note that when you disable outbound replication, errors with Event ID 1115 will appear in the event log, just as errors with event ID 1113 will show when inbound replication is disabled. When either one is re-enabled, informative Event IDs 1116 (outbound) and 1114 (inbound) will appear in the event log. The fastest way to recover from a corrupted AD database is to forcefully demote the DC to become a member server, and to promote it again to replicate off of another DC, if there is one in the same network. You should take this step only if you are sure, and have verified that the AD is actually working on the other DC. You can use DCDiag to easily verify whether everything is correctly order. You should not replicate from another DC within the same site if you cannot verify whether the DC is actually operational. Make sure that the event log does not contain any Jet database, or FRS errors such as 1173, which would indicate an internal error in the Jet Database. You will have to perform a metadata cleanup if you force-demote the DC because a clean demotion is not an option. If you use Dcpromo normally, it will replicate its "change" out to another DC. In other words, you are spreading the corrupted or non-functioning database this way. You can easily force-demote it by running  dcpromo /forceremoval, and then do a metadata cleanup. Non-Authoritative and Authoritative Restore To speed up the data replication of the AD, for example, for sites that have a slow or saturated network connection, or to make the whole restore process much quicker, you should perform a non-authoritative restore of the AD database. Non-authoritative restore means that you are restoring a database, but that database will not assert authority in the AD. This means that it will take all changes that the replication partners send it, where an authoritative restore is set to be the master replication in your domain. It will restore itself, and give itself such high update sequence numbers to objects that every other DC in the domain replicates from it because it is assumed it has the newest copy. Effectively, you are restoring a backup of the AD database to its original location, overwriting the current database. This can only be done when you are in Directory Restore mode. After the database is restored, reboot your DC normally. If you have changed the boot order, reverse the order again. Please make sure your backups are run regularly, whether it is on tape or on disk, according to your organization's backup policy. After the restore is complete, and you reboot the DC, the next replication will be much faster because only changed objects will be replicated to the server. This is because it has most of the AD database already, depending on how much the AD has changed, and how recent the backup was. In a non-authoritative restore, data that is restored includes AD objects with original update sequence numbers. This requires a lot of caution, as any data that is restored with a non-authoritative restore will appear to be outdated to the AD replication system. Hence, the data will not get replicated to other domain controllers. You therefore run the risk of having the data overwritten from DCs that have not yet been restored to, and contain somewhat newer data, including other corruptions. Since this article is meant for a corruption or AD failure on a single DC only, with a replication partner relatively close by, we will skip the authoritative restore. The following figure illustrates how authoritative and non-authoritative restores work. Option One: Restoring AD from a Backup In order to restore an AD database on a Domain Controller, you have to go into the "Directory Services Restore" mode. To do this, reboot the DC, and at the boot prompt, which is where the boot process waits for a second before the splash screen of Windows 2003 comes up, press the F8 key, after which you will be presented with a menu similar to the following: Windows 2003 Advanced Options MenuPlease select an option: Safe ModeSafe Mode with NetworkingSafe Mode with Command Prompt Enable Boot LoggingEnable VGA ModeLast Known Good ConfigurationDirectory Services Restore Mode (Windows NT domain controllers only)Debugging Mode Use | and | to move the highlight to your choice.Press Enter to choose. At the menu, select the Directory Services Restore Mode, by moving up and down with the arrow keys on your keyboard, and then press ENTER. You will then be in the same menu again. Press ENTER again, after which, your display will show Directory Restore Mode at the bottom of the screen. Your DC will now boot, but no AD-related services will be started. Once booted, restore the AD/system state from a trusted and recent backup, and reboot the machine. After the machine is rebooted, you should have working AD services with a slightly outdated AD database. Wait for the replication to take effect, and your AD will be updated. No Physical Access to the Machine If you do not have physical access to the machine in question, you can achieve the same effect by editing the boot.ini file, which is located in c:. This file, bydefault, is hidden and may be write-protected. To see the hidden files and system files, you will need to change your Windows Explorer settings, as shown in the following screenshot. When you can see the file, check its properties, that is, whether it is read-only or not, by right-clicking on it and selecting Properties. However, instead of changing file permissions of system files, you can edit this quite easily with the Graphical User Interface. Some people are not comfortable editing protected files, and then you can also run into problems if you forget to un-mark or re-mark the read-only flag. To get to the GUI editor, right-click on My Computer in your Start menu, or click the icon with the server name in the right hand upper-right, and then select Properties. Once you have done this, the following screen will appear: On the properties window, go to the Advanced tab, as shown in the following screenshot: And then click on Settings in the Startup and Recovery section of the  Advanced tab, as shown in the screenshot below: In the resulting window, which contains quite a few options and sections, you can click on the  Edit button and the boot.ini file will be opened in Notepad. Once editing is completed, and the file has been saved and closed, all permissions will be reset to their original settings. The editing of this file is pretty straightforward, yet not easily understandable for someone who has never done this. In the following screenshot, you can see the actual file displayed: To edit the file in order to reboot in Directory Restore Mode, it is recommended to just copy and paste the last line again, and perform two changes:      Change the display name by adding something like Recovery.      Add the /safeboot:dsrepair at the end of the line, as shown in the following screenshot: When editing is done, save and close the file as you would with any other text file, and click  OK in the Startup and Recovery window so that it closes. To select the option on which you want to boot, click SETTINGS in the  Startup and Recovery section again. And As you can see in the following screenshot, you can now select the boot default from the drop-down menu in the opening window: Select the Recovery line and click OK. Then click OK again to reboot the server. It will now boot automatically into the Directory Restore Mode. Restoring from a Backup Once you are in Directory Restore Mode (DRM), you can use your company's backup software to recover the AD database. If you use Windows backup, you can safely backup and restore the system state of the server, as you can see in the following screenshot. This will allow you to fully revert to a completely working system. However, if you do not need to, or do not want to, recover the whole system state, you can easily choose to restore only AD from the list, as shown in the following screenshot. We assume that we have a complete backup of the server, and that we need to restore the AD database. You can restore the AD or system state to a different location, or overwrite the original files, depending on whether you want to perform an authoritative or a non-authoritative restore, or even an install from media. Option Two: Replication In case you have no valid recent backup media from which to restore the AD, or you need to act very fast, you can restore the AD by first force-demoting the DC in question with dcpromo /forceremoval. This way, demoting will prevent replication to any replication partners, but will successfully demote the DC from the domain. If you have any important Flexible Server Master Operations (FSMO) roles running on the DC that you are removing by force, you will be presented with the following warning: Make sure that you seize the FSMO roles after you have demoted the DC, or after you disconnect its network cable. Do not forget this though, as the FSMO roles are quite important. As you can see in the following screenshot, the Active Directory wizard will now proceed to remove AD without updating the forest. This means that it will not replicate out its own data or changes. Once demoted, reboot the server, and remove its leftover information from AD as a DC. Then, verify whether the network connection is fully functional, and promote the DC again with the same name. You should encounter no problems in completing the promotion. The AD will replicate to it during the next scheduled replication. If your AD is very large and contains many records or files, such as pictures in the user information, the replication to a "blank" DC will take a long time and stress your network quite heavily. The replication option in a larger environment with a large dataset can take along time if the other DC is either busy, or the replication link is not a very high-speed connection. The size of the AD databases can start from 1 GB and may run up to several gigabytes. If your DC has to replicate from a DC over a leased line, or an average consumer Internet class connection (2 to 8mbit), you may have to consider the other option of  restoring from a backup, or performing a non-authoritative restore. Option Three: Rebuild DC with Install from Media Starting from Windows 2000 SP3, there is a dcpromo option available called "IFM" (install from media). This option adds a step to the DC promotion that will pre-populate the AD database on the promoted DC from a recent system state backup of a working DC, which is restored to a disk, CD or DVD. IFM is the fastest option if you have to install, recover, or re-install a DC that is connected by only a slow link to its closest replication partner. As you are pre-populating the local AD, the replication changes it will get from its partner are much smaller, and therefore will replicate much faster. To use IFM, restore the backup to the server BEFORE you make it a DC, and before you run dcpromo. A good way to do this, if you want to, or need to, restore several DCs with IFM, is to restore it to a network drive.. To prepare a backup and restore for IFM, please see Microsoft's Knowledge base article http://support.microsoft.com/kb/311078 IFM can only be used for additional DCs within a domain, not for the first DC. You can also only pre-populate, or restore a DC with a backup of the domain that you are building in. A good option for this is to force-demote the DC, reboot it, and start it as a normal member server. You then need to clean up all of the records from the AD regarding the DC, and then perform an install from Media. Restore the most recent backup to a separate directory, for example, c:restore. This restore should be a restore of the System State of a DC, and the directory will contain sub-directories such as AD, Boot Files, Registry and so on. Once the restore has succeeded, click on Start, Run, and enter dcpromo /adv as shown: The /adv flag in the Dcpromo command gives you advanced options within Dcpromo, such as the install from media options. Follow the dialog as you would normally do for a dcpromo. However, when you see the screen regarding Copying Domain Information, select the option:  From these restored backup files, and navigate to and select the directory to which you restored the system state. As you can see in this screenshot, you also have the option to pre-populate immediately from an existing DC. This is, of course, an option, but if your link is slow, or if your replication partner is busy serving other partners, it will be much slower than from media. Not just for disaster recovery! If you have to install several DCs for any reason, the /adv switch can save you the time spent waiting for the first replication, if you have a fast link, or a backup, that you can use to pre-populate. When following the wizard, you will be asked if you want to make this DC a Global Catalog, and then you may be required to use an account with Administrator privileges in order to proceed with the dcpromo. Once the Dcpromo wizard completes its task, and the AD records are copied from a previous backup, at the next replication only the changes since the last backup will be replicated. This saves a lot of bandwidth and time. This is especially useful for sites with a slow or saturated network connection, where replication would take far too long, so if you need to recover or install fast, this is probably your best bet. Summary In this article, we looked at the recovery procedure in the event of a corruption or failure of the AD database on a single domain controller. This is a scenario that happens more frequently than one might expect. So, instead of "just re-installing", these options and procedures will help you get a healthy DC back in no time while limiting the amount of errors in the event log. We also looked at some tools that will help you not only diagnose AD problems, but also allow you to perform, when used regularly, AD health tests to make sure your AD is always in perfect working order. This way, you might prevent failures because you might find the symptoms earlier.  
Read more
  • 0
  • 0
  • 17690

article-image-google-earth-google-maps-and-your-photos-tutorial
Packt
22 Oct 2009
38 min read
Save for later

Google Earth, Google Maps and Your Photos: a Tutorial

Packt
22 Oct 2009
38 min read
Introduction A scholar who never travels but stays at home is not worthy to be accounted a scholar. From my youth on I had the ambition to travel, but could not afford to wander over the three hundred counties of Korea, much less the whole world. So, carrying out an ancient practise, I drew a geographical atlas. And while gazing at it for long stretches at a time I feel as though I was carrying out my ambition . . . Morning and evening while bending over my small study table, I meditate on it and play with it and there in one vast panorama are the districts, the prefectures and the four seas, and endless stretches of thousands of miles. WON HAK-SAENG Korean Student Preface to his untitled manuscript atlas of China during the Ming Dynasty, dated 1721. To say that maps are important is an understatement almost as big as the world itself. Maps quite literally connect us to the world in which we all live, and by extension, they link us to one another. The oldest preserved maps date back nearly 4500 years. In addition to connecting us to our past, they chart much of human progress and expose the relationships among people through time. Unfortunately, as a work of humankind, maps share many of the same shortcomings of all human endeavors. They are to some degree inaccurate and they reflect the bias of the map maker. Advancements in technology help to address the former issue, and offer us the opportunity to resist the latter. To the extent that it's possible for all of us to participate in the map making, the bias of a select few becomes less meaningful. Google Earth  and Google Maps  are two applications that allow each of us to assert our own place in the world and contribute our own unique perspective. I can think of no better way to accomplish this than by combining maps and photography. Photos reveal much about who we are, the places we have been, the people with whom we have shared those experiences, and the significant events in our lives. Pinning our photos to a map allows us to see them in their proper geographic context, a valuable way to explore and share them with friends and family. Photos can reveal the true character of a place, and afford others the opportunity to experience these destinations, perhaps faraway and unreachable for some of us, from the perspective of someone who has actually been there. In this tutorial I'll show you how to 'pin' your photos using Google Earth and Google Maps. Both applications are free, and available for Windows, Mac OS X, and Linux. In the case of Google Earth there are requirements associated with installing and running what is a local application. Google Maps has its own requirements: primary among them a compatible web browser (the highly regarded FireFox is recommended). In Google Earth, your photos show up on the map within the application complete with titles, descriptions, and other relevant information. You can choose to share your photos with everyone, only people you know, or even reserve them strictly for yourself. Google Maps offers the flexibility to present maps outside of a traditional application. For example you can embed a map on a webpage pinpointing the location of one particular photo, or mapping a collection of photos to present along with a photo gallery, or even collecting all of your digital photos together on one dynamic map. Over the course of a short couple of articles we'll cover everything you need to take advantage of both applications. I've put together two scripts to help us accomplish that goal. The first is a Perl script that works through your photos and generates a file in the proper format with all of the data necessary to include those photos in Google Earth. The second is a short bit of Javascript that works with the first file and builds a dynamic Google Map of those same photos. Both scripts are available for you to download, after which you are free to use them as is, or modify them to suit your own projects. I've purposefully kept the code as simple as possible to make them accessible to the widest audience, even those of you reading this who may be new to programming or unfamiliar with Perl, Javascript or both. I've taken care to comment the code generously so that everything is plainly obvious. I'm hopeful that you will be surprised at just how simple it is. There a couple of preliminary topics to examine briefly before we go any further. In the preceding paragraph I mentioned that the result of the first of our two scripts is a 'file in the proper format...'. This file, or more to the point the file format, is a very important part of the project. KML  (Keyhole Markup Language) is a fairly simple XML-based format that can be considered the native "language" of Google Earth. That description begs the question, 'What is XML?'. To oversimplify, because even a cursory discussion of XML is outside of the scope of this article, XML  (Extensible Markup Language) is an open data format (in contrast to proprietary formats) which allows us to present information in such way that we communicate not only the data itself, but also descriptive information about the data and the relationships among elements of the data. One of the technical terms that applies is 'metalanguage', which approximately translates to mean a language that makes it possible to describe other languages. If you're unfamiliar with the concept, it may be difficult to grasp at first or it may not seem impressive. However, metalanguages, and specifically XML, are an important innovation (I don't mean to suggest that it's a new concept. In fact XML has roots that are quite old, relative to the brief history of computing). These metalanguages make it possible for us to imbue data with meaning such that software can make use of that data. Let's look at an example taken from the Wikipedia entry for KML. <?xml version="1.0" encoding="UTF-8"?> <kml > <Placemark>  <description>New York City</description>  <name>New York City</name>  <Point>  <coordinates>-74.006393,40.714172,0</coordinates>  </Point> </Placemark> </kml>   Ignore all of the pro forma stuff before and after the <Placemark> tags and you might be able to guess what this bit of data represents. More importantly, a computer can be made to understand what it represents in some sense. Without the tags and structure, "New York City" is just a string, i.e. a sequence of characters. Considering the tags we can see that we're dealing with a place, (a Placemark), and that "New York City" is the name of this place (and in this example also its description). With all of this formal structure, programs can be made to roughly understand the concept of a Placemark, which is a useful thing for a mapping application. Let's think about this for a moment. There are a very large number of recognizable places on the planet, and a provably infinite number of strings. Given a block of text, how could a computer be made to identify a place from, for example the scientific name of a particular flower, or a person's name? It would be extremely difficult. We could try to create a database of every recognizable place and have the program check the database every time it encounters a string. That assumes it's possible to agree on a 'complete list of every place', which is almost certainly untrue. Keep in mind that we could be talking about informal places that are significant only to a small number of people or even a single person, e.g. 'the park where, as a child, I first learned to fly a kite'. It would be a very long list if we were going to include these sorts of places, and incredibly time-consuming to search. Relying on the structure of the fragment above, we can easily write a program that can identify 'New York City' as the name of a place, or for that matter 'The park where I first learned to fly a kite'. In fact, I could write a program that pulls all of the place names from a file like this, along with a description, and a pair of coordinate points for each, and includes them on a map. That's exactly what we're going to do. KML makes it possible. If I haven't made it clear, the structural bits of the file must be standardized. KML supports a limited set of elements (e.g. 'Placemark' is a supported element, as are 'Point' and 'coordinates'), and all of the elements used in a file must adhere to the standard for it to be considered valid. The second point we need to address before we begin is, appropriately enough... where to begin? Lewis Carroll famously tells us to "Begin at the beginning and go on till you come to the end: then stop."  Of course, Mr. Carroll  was writing a book at the time. If "Alice's Adventures in Wonderland" were an article, he might have had different advice. From the beginning to the end there is a lot of ground to cover. We're going to start somewhere further along, and make up the difference with the following assumptions. For the purposes of this discussion, I am going to assume that you have: Perl Access to Phil Harvey's excellent ExifTool , a Perl library and command-line application for reading, writing, and editing metadata in images (among other file types). We will be using this library in our first script A publicly accessible web server. Google requires the use of an API key  by which it can monitor the use of its map services. Google must be able to validate your key, and so your site must be publicly available. Note that this a requirement for Google Maps only Photos, preferably in a photo management application. Essentially, all you need is an app capable of generating both thumbnails and reduced size copies of your original photos . An app that can export a nice gallery for use on the web is even better Coordinate data as part of the EXIF metadata  embedded in your photos. If that sounds unfamiliar to you, then most likely you will have to take some additional steps before you can make use of this tutorial. I'm not aware of any digital cameras that automatically include this information at the time the photo is created. There are devices that can be used in combination with digital cameras, and there are a number of ways that you can 'tag' your photos with geographic data much the same way you would add keywords and other annotations. Let's begin!   Part 1: Google Earth Section 1: Introduction to Part 1 Time to begin the project in earnest. As I've already mentioned we'll spend the first half of this tutorial looking at Google Earth and putting together a Perl script whichthat, given a collection of geotagged photos, will build a set of KML files so that we can browse our photos in Google Earth. These same files will serve as the data source for our Google Maps application later on. Section 2: Some Advice Before We Begin Take your time to make sure you understand each topic before you move on to the next. Think of this as the first step in debugging your completed code. If you go slowly enough that you are to be able to identify aspects of the project that you don't quite understand, then you'll have some idea where to start looking for problems should things not go as expected. Furthermore, going slowly will give you the opportunity to identify those parts that you may want to modify to better fit the script to your own needs. If this is new to you, follow along as faithfully as possible with what I do here the first time through. Feel free to make notes for yourself as you go, but making changes on the first pass may make it difficult for you to catch back on to the narrative and piece together a functional script. After you have a working solution, it will be a simple matter to implement changes one at a time until you have something that works for you. Following this approach it will be easy to identify the silly mistakes that tend to creep in once you start making changes. There is also the issue of trust. This is probably the first time we're meeting each other, in which case you should have some doubt that my code works properly to begin with. If you minimize the number of changes you make, you can confirm that this works for you before blaming yourself or your changes for my mistakes. I will tell you up front that I'm building this project myself as we go. You can be certain at least that it functions as described for me as of the date attached to the article. I realize that this is quite different from being certain that the project will work for you, but at least it's something. The entirety of my project is available for you to download. You are free to use all of it for any legal purpose whatsoever,  including my photos in addition to all of the code, icons, etc. This is so you have some samples to use before you involve your own content. I don't promise that they are the most beautiful images you have ever seen, but they are all decent photos and properly annotated with the necessary metadata, including geographic tags. Section 3: Photos, Metadata and ExifTool To begin, we must have a clear understanding of what the Perl script will require from us. Essentially, we need to provide it with a selection of annotated image files, and information about how to reference those files. A simple folder of files is sufficient, and will be convenient for us, both as the programmer and end user. The script will be capable of negotiating nested folders, and if a directory contains both images and other file types, non-image types will be ignored. Typically, after a day of taking photos I'll have 100 to 200 that I want to keep. I delete the rest immediately after offloading them from the camera. For the files that are left, I preserve the original grouping, keeping all of the files together in a single folder. I place this folder of images in an appropriate location according to a scheme that serves to keep my complete collection of photos neatly organized. These are my digital 'negatives'. I handle all subsequent organization, editing, enhancements, and annotations within my photo management application. I use Apple Inc.'s Aperture; but there are many others that do the job equally well. Annotating your photos is well worth the investment of time and effort, but it's important that you have some strategy in mind so that you don't create meaningless tags that are difficult to use to your advantage. For the purposes of this project the tags we'll need are quite limited, which means that going forward we will be able to continue adding photos to our maps with a reasonable amount of work. The annotations we need are: Caption Latitude Longitude Image Date * Location/Sub-Location City Province/State Country Name Event People ImageWidth * ImageHeight * * Values for these Exif tags are generated by your camera. Note that these are labels used in Aperture, and are not necessarily consistent from one application to the next. Some of them are more likely than others to be used reliably. 'City' for example should be dependable, while the labels 'People', 'Events', and 'Location', among others, are more likely to differ. One explanation for these variations is that the meaning of these fields are more open to interpretation. Location, for example, is likely to be used to narrow down the area where the photo was taken within a particular city, but it is left to the person who is annotating the photo to decide that the field should name a specific street address, an informal place (e.g. 'home' or 'school'), or a larger area, for example a district or neighborhood. Fortunately things aren't so arbitrary as they seem. Each of these fields corresponds to a specific tag name that adheres to one of the common metadata formats (Exif1, IPTC2, XMP3, and there are others). These tag names are consistent as required by the standards. The trick is in determining the labels used in your application that correspond to the well-defined tag names. Our script relies on these metadata tags, so it is important that you know which fields to use in your application. This gives us an excuse to get acquainted with ExifTool4. From the project's website, we have this description of the application: ExifTool is a platform-independent Perl library plus a command-line application for reading, writing, and editing meta information in image, audio, and video files... ExifTool can seem a little intimidating at first. Just keep in mind that we will need to understand just a small part of it for this project, and then be happy that such a useful and flexible tool is freely available for you to use. The brief description above states in part that ExifTool is a Perl library and command line application that we can use to extract metadata from image files. With a single short command, we can have the app print all of the metadata contained in one of our image files. First, make sure ExifTool is installed. You can test for this by typing the name of the application at the command line. $ exiftool If it is installed, then running it with no options should prompt the tool to print its documentation. If this works, there will be more than one screen of text. You can page through it by pressing the spacebar. Press the 'q' key at any time to stop. If the tool is not installed, you will need to add it before continuing. See the appendix at the end of this tutorial for more information. Having confirmed that ExifTool is installed, typing the following command will result in a listing of the metadata for the named image: $ exiftool -f -s -g2 /path/image.jpg Where 'path' is an absolute path to image.jpg or a relative path from the current directory, and 'image.jpg' is the name of one of your tagged image files. We'll have more to say about ExifTool later, but because I believe that no tutorial should ask the reader to blindly enter commands as if they were magic incantations, I'll briefly describe each of the options used in the command above: -f, forces printing of tags even if their values are not found. This gives us a better idea about all of the available tag names, whether or not there are currently defined values for those tags. -s, prints tag names instead of descriptions. This is important for our purposes. We need to know the tag names so that we can request them in our Perl code. Descriptions, which are expanded, more human-readable forms of the same names obscure details we need. For example, compare the tag name 'GPSLatitude' to the description 'GPS Latitude'. We can use the tag name, but not the description to extract the latitude value from our files. -g2, organizes output by category. All location specific information is grouped together, as is all information related to the camera, date and time tags, etc.  You may feel, as I do, that this grouping makes it easier to examine the output. Also, this organization is more likely to reflect the grouping of field names used by your photo management application. If you prefer to save the output to a file, you can add ExifTool's -w option with a file extension. $ exiftool -f -s -g2 -w txt path/image.jpg This command will produce the same result but write the output to the file 'image.txt' in the current directory; again, where 'image.jpg' is the name of the image file. The -w option appends the named extension to the image file's basename, creates a new file with that name, and sets the new file as the destination for output. The tag names that correspond to the list of Aperture fields presented above are: metadata tag name Aperture field label Caption-Abstract Caption GPSLatitude Latitude GPSLongitude Longitude DateTimeOriginal Image Date Sub-location Location/Sub-Location City City Province-State Province/State Country-PrimaryLocationName Country Name FixtureIdentifier Event Contact People ImageWidth Pixel Width ImageHeight Pixel Height     Section 4: Making Photos Available to Google Earth We will use some of the metadata tags from our image files to locate our photos on the map (e.g. GPSLatitude, GPSLongitude), and others to describe the photos. For example, we will include the value of the People tag in the information window that accompanies each marker to identify friends and family who appear in the associated photo. Because we want to display and link to photos on the map, not just indicate their position, we need to include references to the location of our image files on a publicly accessible web server. You have some choice about how to do this, but for the implementation described here we will (1) Display a thumbnail in the info window of each map marker and (2) include a link to the details page for the image in a gallery created in our photo management app. When a visitor clicks on a map marker they will see a thumbnail photo along with other brief descriptive information. Clicking a link included as part of the description will open the viewer's web browser to a page displaying a large size image and additional details. Furthermore, because the page is part of a gallery, viewers can jump to an index page and step forward and back through the complete collection. This is a complementary way to browse our photos. Taking this one step further, we could add a comments section to the gallery pages or replace the gallery altogether, instead choosing to display each photo as a weblog post for example. The structure of the gallery created from my photo app is as follows... / (the root of the gallery directory) index.html large-1.html large-2.html large-3.html ... large-n.html assets/ css/ img/ catalog/ pictures/ picture-1.jpg picture-2.jpg picture-3.jpg ... picture-n.jpg thumbnails/ thumb-1.jpg thumb-2.jpg thumb-3.jpg ... thumb-n.jpg The application creates a root directory containing the complete gallery. Assuming we do not want to make any manual changes to the finished site, publishing is as easy as copying the entire directory to a location within the web server's document root. assets/: Is a subfolder containing files related to the theme itself. We don't need to concern ourselves with this sub-directory. catalog/: Contains a single catalog.plist file which is specific to Aperture and not relevant to this discussion. pictures/: Contains the large size images included on the detail gallery pages. thumbnails/: This subfolder contains the thumbnail images corresponding to the large size images in pictures/. Finally, there are a number of files at the root of the gallery. These include index pages and files named 'large-n.html', where n is a number starting at 1 and increasing sequentially e.g. large-1.html, large-2.html, large-3.html, ... The index files are the index pages of our gallery. The number of index pages generated will be dictated by the number of image files in the gallery, as well as the gallery's layout and design. index.html is always the first gallery page. The large-n.html files are the details pages of our gallery. Each page features an individual photo with links to the previous and next photos in sequence and a link back to the index. You can see the gallery I have created for this tutorial here: http://robreed.net/photos/tutorial_gallery/ If you take the time to look through the gallery, maybe you can appreciate the value of viewing these photos on a map. Web-based photo galleries like this one are nice enough, but the photos are more interesting when viewed in some meaningful context. There are a couple of things to notice about this gallery. Firstly, picture-1.jpg, thumb-1.jpg, and large-1.html all refer to the same image. So if we pick one of the three files we can easily predict the names of the other two. This relationship will be useful when it comes to writing our script. There is another important issue I need to call to your attention because it will not be apparent from looking only at the gallery structure. Aperture has renamed all of my photos in the process of exporting them. The name of the original file from which picture-1.jpg was generated (as well as large-1.html and thumb-1.jpg) is 'IMG_0277.JPG', which is the filename produced by my camera. Because I want to link to these gallery files, not my original photos which will stay safely tucked away on a local drive, I must run the script against the photos in the gallery. I cannot run it against the original image files because the filenames referenced in the output are unrelated to the files in the gallery. If my photo management app provided me the option of preserving the original filenames for the corresponding photos in the gallery, then I could run the script against the original image files or the gallery photos because all of the filenames would be consistent, but this is not the case. I don't have a problem as long as I run the script on the exported photos. However, if I'm running the script against the photos in the web gallery, either the pictures or thumbnail images must contain the same metadata as the original image files. Aperture preserves the metadata in both. Your application may not. A simple, dependable way to confirm that the metadata is present in the gallery files is to run ExifTool first against the original file and then the same photo in the pictures/ and thumbnails/ directories in the gallery. If ExifTool reports identical metadata, then you will have no trouble using one of pictures/ or thumbnails/ as your source directory. If the metadata is not present or not complete in the gallery files, you may need to use the script on your original image files. As has already been explained, this isn't a problem unless the gallery produces filenames that are inconsistent with the original filenames, as Aperture does. In this case you have a problem. You won't be able to run the script on the original image files because of the naming issue or on the gallery photos because they don't contain metadata. Make sure that you understand this point. If you find yourself in this situation, then your best bet is to generate files to use with your maps from your original photos in some other way, bypassing your photo management app's web gallery features altogether in favor of a solution that preserves the filenames, the metadata, or both. There is another option which involves setting up a relationship between the names of the original files and the gallery filenames. This tutorial does not include details about how to set up this association. Finally, keep in mind that though we've looked at the structure of a gallery generated by Aperture, virtually all photo management apps produce galleries with a similar structure. Regardless of the application used, you should find: A group of html files including index and details pages A folder of large size image files A folder of thumbnails Once you have identified the structure used by your application, as we have done here, it will be a simple task to translate these instructions. Section 5: Referencing Files Over the Internet Now we can talk about how to reference these files and gallery pages so that we can create a script to generate a KML file that includes these references. When we identify a file over the internet, it is not enough to use the filename, e.g. 'thumb-1.jpg', or even the absolute and relative path to the file on the local computer. In fact these paths are most likely not valid as far as your web server is concerned. Instead we need to know how to reference our files such that they can be accessed over the global internet, and the web more specifically. In other words, we need to be able to generate a URL (Uniform Resource Locator) which unambiguously describes the location of our files. The formal details of exactly what comprises a URL5; are more complicated than may be obvious at first, but most of us are familiar with the typical URL, like this one: http://www.ietf.org/rfc/rfc1630.txt which describes the location of a document titled "Universal Resource Identifiers in WWW" that just so happens to define the formal details of what comprises a URL. http://www.ietf.org This portion of the address is enough to describe the location of a particular web server over the public internet. In fact it does a little more than just specify the location of a machine. The http:// portion is called the scheme and it identifies a particular protocol (i.e. a set of rules governing communication) and a related application, namely the web. What I just said isn't quite correct; at one time, HTTP was used exclusively by the web, but that's no longer true. Many internet-based applications use the protocol because the popularity of the web ensures that data sent via HTTP isn't blocked or otherwise disrupted. You may not be accustomed to thinking of it as such, but the web itself is a highly-distributed, network-based application. /rfc/ This portion of the address specifies a directory on the server. It is equivalent to an absolute path on your local computer. The leading forward slash is the root of the web server's public document directory. Assuming no trickiness on the part of the server, all content lives under the document root. This tells us that rfc/ is a sub-directory contained within the document root. Though this directory happens to be located immediately under the root, this certainly need not be the case. In fact these paths can get quite long. We have now discussed all of the URL except for: rfc1630.txt which is the name of a specific file. The filename is no different than the filenames on your local computer. Let's manually construct a path to one of the large-n.html pages of the gallery we have created. The address of my server is robreed.net, so I know that the beginning of my URL will be: http://robreed.net I keep all of my galleries together within a photos/ directory, which is contained in the document root. http://robreed.net/photos/ Within photos/, each gallery is given its own folder. The name of the folder I have created for this tutorial is 'tutorial_gallery'. Putting this all together, the following URL brings me to the root of my photo gallery: http://robreed.net/photos/tutorial_gallery/ We've already gone over the directory structure of the gallery, so it should make sense you to that when referring to the 'large-1.html' detail page, the complete URL will be: http://robreed.net/photos/tutorial_gallery/large-1.html the URL of the image that corresponds to that detail page is: http://robreed.net/photos/tutorial_gallery/pictures/picture-1.jpg and the thumbnail can be found at: http://robreed.net/photos/tutorial_gallery/thumbnails/thumb-1.jpg Notice that the address of the gallery is shared among all of these resources. Also, notice that resources of each type (e.g. the large images, thumbnails, and html pages) share a more specific address with files of that same type. If we use the term 'base address' to refer to the shared portions of these URLs, then we can talk about several significant base addresses: The gallery base address: http://robreed.net/photos/tutorial_gallery/ The html pages base address: http://robreed.net/photos/tutorial_gallery/ The images base address: http://robreed.net/photos/tutorial_gallery/pictures/ The thumbnails base address: http://robreed.net/photos/tutorial_gallery/thumbnails/ Note that given the structure of this particular gallery, the html pages base address and the gallery base address are identical. This need not be the case, and may not be for the gallery produced by your application. We can hard-code the base addresses into our script. For each photo, we need only append the associated filename to construct valid URLs to any of these resources. As the script runs, it will have access to the name of the file that it is currently evaluating, and so it will be a simple matter to generate the references we need as we go. At this point we have discussed almost everything we need to put together our script. We have: Created a gallery at our server, which includes our photos with metadata in tow Identified all of the metadata tags we need to extract from our photos with the script and the corresponding field names in our photo management application Determined all of the base addresses we need to generate references to our image files Section 6: KML The last thing we need to understand is the format of the KML files we want to produce. We've already looked at a fragment of KML. The full details can be found on Google's KML documentation pages , which include samples, tutorials and a complete reference for the format. A quick look at the reference is enough to see that the language includes many elements and attributes, the majority of which we will not be including in our files. That statement correctly implies it is not necessary for every KML file to include all elements and attributes. The converse however is not true, which is to say that every element and attribute contained in any KML file must be a part of the standard. A small subset of KML will get us most, if not all, of what you will typically see in Google Earth from other applications. Many of the features we will not be using deal with aspects of the language that are either: Not relevant to this project, e.g. ground overlays (GroundOverlay) which "draw an image overlay draped onto the terrain" Minute details for which the default values are sensible There is no need to feel shortchanged because we are covering only a subset of the language. With the basic structure in place and a solid understanding of how to script the generation of KML, you will be able to extend the project to include any of the other components of the language as you see fit. The structure of our KML file is as follows:  1.    <?xml version="1.0" encoding="UTF-8"?> 2.    <kml > 3.        <Document> 4.            <Folder> 5.                <name>$folder_name</name> 6.                <description>$folder_description</description> 7.                <Placemark> 8.                    <name>$placemark_name</name> 9.                    <Snippet maxLines="1"> 10.                        $placemark_snippet 11.                    </Snippet> 12.                    <description><![CDATA[ 13.                        $placemark_description 14.                    ]]></description> 15.                    <Point> 16.                        <coordinates>$longitude,$latitude </coordinates> 17.                    </Point> 18.                </Placemark> 19.            </Folder> 20.        </Document> 21.    </kml> Line 1: XML header Every valid KML file must start with this line and nothing else is allowed to appear before it. As I've already mentioned, KML is an XML-based language and XML requires this header. Line 2: Namespace declaration More specifically this is the KML namespace declaration, and it is another formality. The value of the >Line 3: <Document> is a container element representing the KML file itself. If we do not explicitly name the document by including a name element then Google Earth will use the name of the KML file as the Document element <name>. The Document container will appear on the Google Earth 'Sidebar' within the 'Places' panel. Optionally we can control whether the container is closed or open by default. (This setting can be toggled in Google Earth using a typical disclosure triangle.) There are many other elements and attributes that can be applied to the Document element. Refer to the KML Reference for the full details. Line 4: <Folder> is another container element. The files we produce will include a single <Folder> containing all of our Placemarks, where each Placemark represents a single image. We could create multiple Folder elements to group our Placemarks according to some significant criteria. Think of the Folder element as being similar to your operating system's concept of a folder. At this point, note the structure of the fragment. The majority of it is contained within the Folder element. Folder, in turn, is an element of Document which is itself within the <kml> container. It should make sense that everything in the file that is considered part of the language must be contained within the kml element. From the KML reference: A Folder is used to arrange other Features hierarchically (Folders, Placemarks, NetworkLinks, or Overlays). A Feature is visible only if it and all its ancestors are visible. Line 5: The name element identifies an object, in this case the Folder object. The text that appears between the name tags can be any plain text that will serve as an appropriate label.   Line 6: <description> is any text that seems to adequately describe the object. The description element supports both plain text and a subset of HTML. We'll consider issues related to using HTML in <description> at the discussion of Placemark, lines 12 - 14. Lines 7 - 17 define a <Placemark> element. Note that Placemark contains a number of elements that also appear in Folder, including <name> (line 8), and <description> (lines 12 - 14). These elements serve the same purpose for Placemark as they do for the Folder element, but of course they refer to a different object. I've said that <description> can include a subset of HTML in addition to plain text. Under XML, some characters have special meaning. You may need to use these characters as part of the HTML included in your descriptions. Angle brackets (<, >) for example surround tag names in HTML, but serve a similar purpose in XML. When they are used strictly as part of the content, we want the XML parser to ignore these characters. We can accomplish this a few different ways: We can use entity references, either numeric character references or character entity references, to indicate that the symbol appears as part of the data and should not be treated as part of the syntax of the language. The character '<', which is required to include an image as part of the description (something we will be doing), (e.g. <img src=... />) can safely be included as the character entity reference '&lt;' or the numeric character reference '&#60'. The character entity references may be easier to remember and recognize on sight but are limited to the small subset of characters for which they have been defined. The numeric references on the other hand can specify any ASCII character. Of the two types, numeric character references should be preferred. There are also Unicode entity references which can specify any character at all. In the simple case of embedding short bits of HTML in KML descriptions, we can avoid the complication of these references altogether by enclosing the entire description in a CDATA6 element, which instructs the XML parser to ignore any special characters that appear until the end of the block set off by the CDATA tags. Notice the string '<![CDATA[' immediately after the opening <description> tag within <Placemark>, and the string ]]> immediately before the closing </description> tag. If we simply place all of our HTML and plain text between those two strings, it will all be ignored by the parser and we are not required to deal with special characters individually. This is how we'll handle the issue. Lines 9 - 11: <Snippet> The KML reference does a good job of clearly describing this element. From the KML reference: In Google Earth, this description is displayed in the Places panel under the name of the feature. If a Snippet is not supplied, the first two lines of the <description> are used. In Google Earth, if a Placemark contains both a description and a Snippet, the <Snippet> appears beneath the Placemark in the Places panel, and the <description> appears in the Placemark's description balloon. This tag does not support HTML markup. <Snippet> has a maxLines attribute, an integer that specifies the maximum number of lines to display. Default for maxLines is 2. Notice that in the block above at line 9, I have included a 'maxLines' attribute with a value of 1. Of course, you are free to substitute your own value for maxLines, or you can omit the attribute entirely to use the default value. The only element we have yet to review is <Point>, and again we need only look to the official reference for a nice description. From the KML reference: A geographic location defined by longitude, latitude, and (optional) altitude. When a Point is contained by a Placemark, the point itself determines the position of the Placemark's name and icon. <Point> in turn contains the element <coordinates> which is required. From the KML reference: A single tuple consisting of floating point values for longitude, latitude, and altitude (in that order). The reference also informs us that altitude is optional, and in fact we will not be generating altitude values. Finally the reference warns: Do not include spaces between the three values that describe a coordinate. This seems like an easy mistake to make. We'll need to be careful to avoid it. There will be a number of <Placemark> elements, one for each of our images. The question is how to handle these elements. The answer is that we'll treat KML as a 'fill in the blanks' style template. All of the structural and syntactic bits will be hard-coded, e.g. the XML header, namespace declaration, all of the element and attribute labels, and even the whitespace, which is not strictly required but will make it much easier for us to inspect the resulting files in a text editor. These components will form our template. The blanks are all of the text and html values of the various elements and attributes. We will use variables as place-holders everywhere we need dynamic data, i.e. values that change from one file to the next or one execution of the script to the next. Take a look at the strings I've used in the block above. $folder_name, $folder_description, $placemark_name, etc. For those of you unfamiliar with Perl, these are all valid variable names chosen to indicate where the variables slot into the structure. These are the same names used in the source file distributed with the tutorial. Section 7: Introduction to the Code At this point, having discussed every aspect of the project, we can succinctly describe how to write the code. We'll do this in 3 stages of increasing granularity. Firstly, we'll finish this tutorial with a natural language walk-through of the execution of the script. Secondly, if you look at the source file included with the project, you will notice immediately that comments dominate the code. Because instruction is as important an objective as the actual operation of the script, I use comments in the source to provide a running narration. For those of you who find this superabundant level of commenting a distraction, I'm distributing a second copy of the source file with much of the comments removed. Finally, there is the code itself. After all, source code is nothing more than a rigorously formal set of instructions that describe how to complete a task. Most programs, including this one, are a matter of transforming input in one form to output in another. In this very ge
Read more
  • 0
  • 0
  • 8682

article-image-visual-mysql-database-design-mysql-workbench
Packt
21 Oct 2009
3 min read
Save for later

Visual MySQL Database Design in MySQL Workbench

Packt
21 Oct 2009
3 min read
MySQL Workbench is a visual database design tool recently released by MySQL AB. The tool is specifically for designing MySQL database. What you build in MySQL Workbench is called physical data model. A physical data model is a data model for a specific RDBMS product; the model in this article will have some MySQL unique specifications. We can generate (forward-engineer) the database objects from its physical model, which in addition to tables and their columns, can also include other objects such as view. MySQL Workbench has many functions and features; this article by Djoni Darmawikarta shows some of them by way of an example. We’ll build a physical data model for an order system where an order can be a sale order or a purchase order; and then, forward-engineer our model into an MySQL database. The physical model of our example in EER diagram will look like in the following MySQL Workbench screenshot. Creating ORDER Schema Let’s first create a schema where we want to store our order physical model. Click the + button (circled in red). Change the new schema’s default name to ORDER. Notice that when you’re typing in the schema name, its tab name on the Physical Schemata also changes accordingly—a nice feature. The order schema is added to the Catalog (I circled the order schema and its objects in red). Close the schema window. Confirm to rename the schema when prompted. Creating Order Tables We’ll now create three tables that model the order: ORDER table and its two subtype tables: SALES_ORDER and PURCHASE_ORDER, in the ORDER schema. First of all, make sure you select the ORDER schema tab, so that the tables we’ll create will be in this schema. We’ll create our tables as EER diagram (EER = Enhanced Entity Relationship). So, double-click the Add Diagram button. Select (click) the Table icon, and then move your mouse onto the EER Diagram canvas and click on the location you want to place the first table. Repeat for the other two tables. You can move around the tables by dragging and dropping. Next, we’ll work on table1, which we’ll do so using the Workbench’s table editor. We start the table editor by right-clicking the table1 and selecting Edit Table. Next, we’ll work on table1, which we’ll do so using the Workbench’s table editor. We start the table editor by right-clicking the table1 and selecting Edit Table. Rename the table by typing in ORDER over table1. We’ll next add its columns, so select the Columns tab. Replace idORDER column name with ORDER_NO. Select INT as the data type from the drop-down list. We’d like this ORDER_NO column to be valued incrementally by MySQL database, so we specify it as AI column (Auto Increment). AI is a specific feature of MySQL database. You can also specify other physical attributes of the table, such as its Collation; as well as other advanced options, such as its trigger and partioning (the Trigger and Partioning tabs). Notice that on the diagram our table1 has changed to ORDER, and it has its first column, ORDER_NO. In the Catalog you can also see the three tables. The black dots on the right of the tables indicate that they’ve been included in an diagram.  
Read more
  • 0
  • 0
  • 13050

article-image-customizing-plugins-joomla-15x-part-1
Packt
21 Oct 2009
9 min read
Save for later

Customizing Plugins in Joomla! 1.5x (Part 1)

Packt
21 Oct 2009
9 min read
The code used in this article can be downloaded from here. Plugin composition and operation     Like a module, in its simplest form, a plugin can consist of only two files, a PHP file with the actual code for the plugin, and an XML manifest that tells Joomla! what to do with the plugin. Despite this apparent simplicity, plugins are very powerful, and more difficult to understand than modules and components. Plugins are designed to run at certain times during the execution of our site, and they perform actions that can only be done at these times. For example, in our sample site we want to hide some of our content from guests, and only show it to paid subscribers. This action can only be performed when we are actually preparing the content to be displayed, because we need to wait until we identify if the viewer is a guest or subscriber, and then make the changes to the content dynamically. In a different example, checking if a subscriber's subscription is valid is something that only needs to be done when they try to login, and not on every page load. Plugin types Plugins are divided into eight types, as follows: Authentication Content Editors Editors-XTD Search System User XML-RPC Authentication Authentication plugins are designed to allow the site to check the user's authentication against a variety of sources. The default is to check the user's authentication against a username and password stored in the Joomla! database, which, as of Joomla! 1.5, will be the username and password fields in the #__user table (#__ is the table prefix we chose when setting up Joomla!). However, any source with a public API can be used to verify someone's authentication details. Common uses are LDAP, OpenID, a Google account, a subscription, community component, and more. On our site, for example, we are already using an authentication plugin to verify the subscriptions of users when they attempt to login. Content Possibly the most commonly used of all plugins, content plugins allow content items to be modified or have additional features added to them. We could, for example, use content plugins to cloak email addresses, embed audio or video into our pages, or do text replacements. We can even embed components and modules into our pages via plugins. We will later look at a content plugin that we will use to hide and show content depending on a user's subscription. Editors Editors plugins add WYSIWYG editors that we can use when editing our content. We installed JCE on our site earlier, which is the most popular Joomla! editor plugin as of this publication according to Joomla.org. Editors-XTD Editors-XTD (extended) plugins allow us to add additional buttons to the editors. The Image, Read more, and Pagebreak buttons on the default Joomla! WYSIWYG editor, for example, are actually plugins. Search Search plugins allow us to search through the data from different components. By default, Joomla! comes with plugins that search through content articles and the Contacts and Weblinks components. These can be expanded upon by creating or installing search plugins for other extensions. System System plugins are arguably the most powerful and most flexible types of plugins for Joomla!, as they can execute at several different pre-defined points during the execution of a Joomla! page plugin. They can be used to perform a vast array of functions, such as loading extra scripts or CSS into the header of a web page, redirecting people away from pages, logging statistics, and more. User User plugins allow us to perform actions at different times with respect to users. Such times include logging in and out, and also when saving changes to a user's profile. User plugins are often used to create a "bridge" between Joomla! and other web applications (such as the phpBB forum or the osCommerce e-commerce platform.). XML-RPC XML-RPC plugins are for communicating between our Joomla! site and other external applications, such as a desktop application or a different web site. Plugin events As a Joomla! site loads a page, it steps through a series of events as part of that process. The events it steps through are determined by the type of page it is loading. Plugins are always tied to one or more of these events, and are executed during those events as required. When loading a page of content, for example, we would step through a mix of the system and some of the content events. When loading the same page for editing, we will step through the system events, different content events, and also possibly editor events. The events triggered in Joomla! are: Authentication onAuthenticate Content onPrepareContent onAfterDisplayTitle onBeforeDisplayContent onBeforeContentSave onAfterContentSave Editors onInit onGetContent onSetContent onSave onDisplay onGetInsertMethod Editors XTD (Extended) onDisplay Search onSearch onSearchAreas System onAfterInitialise onAfterRoute onAfterDispatch onAfterRender User onLoginUsexr onLoginFailure onLogoutUser onLogoutFailure onBeforeStoreUser onAfterStoreUser onBeforeDeleteUser onAfterDeleteUser XML-RPC onGetWebServices Most of these events are easy to understand from their name, but just in case, more information can be found on the Joomla! documentation wiki at http://docs.joomla.org/CategoryPlugins. Some events are only activated at specific times, such as onAuthenticate, which is only activated when someone logs into their account. Others are activated on every page load. Content events are activated on all content pages and only on content pages, not on pages with components other than com_content. Content plugins are also only executed on the main body content itself and don't have access to the template or other module data. So a text replacement content plugin, for example, wouldn't change any text in modules or the template, only in the main content itself. It is actually possible for modules and components to manually activate plugin events with clever programming, but this is not the default Joomla! behavior. It is usually done when a developer wants to apply content plugin restrictions/changes to a module. Plugin order Aside from the events and types, there is a third important factor to consider when setting up our plugins. That is the order in which the plugins of a particular type are executed. This order is best observed on the Plugin Manager screen that can be found under the Extensions menu. The order in which the plugins execute is something not many people think about, but is really quite powerful and useful. This is because the plugins which execute later, can then use the output or effects of the earlier executing plugins as input. For example, imagine we have a plugin that displays different text for different user types, and we have another plugin that reads certain text values and replaces them with embedded video or audio. If we wanted to be able to show different videos to different groups, then we could use the first plugin to generate the different command strings for the second plugin, and have it generate them based on the user type. The second plugin, our media embedding plugin, doesn't even know that the first plugin exists. All it knows is which videos it needs to display based on what is in the content item. If the media plugin executes first, then it will generate both videos regardless of the user type. As another example, imagine we have some sort of external application and we log users into it after they authenticate via an authentication plugin. We need to make sure that this plugin is executed after all of our other authentication plugins that may check a user's credentials or account status. Otherwise, someone may get logged into our external application even though they were prevented from login into our Joomla! site. So a hacker, for example, could get access to our external application without needing to even successfully get into our main site. This was all because we had the order of our plugins wrong. So when we install and activate plugins, it is well worth taking the time to double check that everything happens in the order it is meant to be in. If one of our plugins is not behaving how it should, it might be worth checking the order to see if another plugin is conflicting with it. Customizing a Plugin Now that we have a better understanding of how our plugins work and fit together, we are going to try our hand at customizing one for our site. This will hopefully give us the understanding and confidence to make any other customizations we need in the future. As with modules, it is often easier to find a plugin that does most of what we want it to do and then make changes to it so that it meets our needs more completely. Looking back over our goals, one that requires a plugin is that we want to limit access to certain parts of our content to only our paying subscribers. This effect is going to be best achieved via content plugin, so we chose the Ninja Access plugin to fill this need. To use Ninja Access we first need to mark the content we want to restrict with special tags and indicate the user groups we want to see what is contained within the tags. When the page is produced, the plugin reads the visitor's user group and then compares it to the ones in the list provided by the tag. If the user groups match, then the content is displayed, if not, then it is hidden. For example: {njaccess 0}shows only to guest users{/njaccess}{njaccess 18,19,20,21,23,24,25}shows to all users who are not a guest{/njaccess} The numbers in the examples above indicate the default Joomla! user group ids. The most important ones are: 0 = Guests 18 = Registered 24 = Administrators 25 = Super Administrators We could use this as it is, but as we don't have a component installed to create new access groups, it won't be very flexible. We could get into trouble in the future if we decide to let people register without getting a subscription, or create a free subscription. In this instance, we will have paying and free subscribers all in the number 18 user group. Also, as we are always going to be restricting the same groups, we don't really need to type the parameters in every single time. Making our plugins always restrict the same groups automatically will save us some time and reduce mistakes. Lastly, do we really need to type njaccess every time? Let's shorten it to something like soc—subscriber only content. For our first dilemma, a better idea than groups might be to associate the access to certain AEC subscriptions that are currently active. That way if people's subscriptions expire, or they get a free account, the content is still properly controlled regardless of their user groups.
Read more
  • 0
  • 0
  • 2042

article-image-setting-plone-development-environment
Packt
21 Oct 2009
13 min read
Save for later

Setting up the Plone Development Environment

Packt
21 Oct 2009
13 min read
(For more resources on Plone, see here.) Prerequisites Pre-built packages for Zope and Plone are available for many operating systems. These can be tempting, but as developers it is normally better to configure the environment ourselves, in order to fully understand and control it. During development, we need write access to the Python code and configuration files. We may also need to run different versions of Zope and Plone in parallel. We will assume that you have at least the following as part of your regular development environment:   Python 2.4. We will assume that Python 2.4 gets invoked when you run python on the command line. Unfortunately, Zope, at version 2.10, does not yet support Python 2.5. Note that many Linux distributions have two packages for Python—one containing the binaries, and one containing header files (typically called python-dev or something similar). You will need both in order to compile Zope. PIL, the Python Imaging Library (http://www.pythonware.com/products/pil), should be installed for this Python interpreter. elementtree, an XML processing library, is required for Plone to start up. Most operating systems have packages for this. It can also be downloaded from http://effbot.org/zone/element-index.htm. A programmer's text editor. Preferably one with Python, XML/HTML and CSS syntax highlighting. You should set up your editor so that a tab/indent is output as four spaces. This makes Python development a lot more predictable. A shell. Most examples in this book will show a Bash interpreter shell, though we will cover Windows syntax when it differs significantly. Bear in mind that path separators on Windows are backslashes (), while other environments use forward slashes (/). Also, environment variables on Windows are referred to as %NAME%, while in most Unix shells, including Bash, variables are dereferenced with $NAME. A Subversion client. We will show the command line client syntax, but you can use a graphical client if you are more comfortable with that. Subversion can be obtained from http://subversion.tigris.org. A C compiler. You will need this to compile Zope. The venerable gcc is fine on UNIX-like systems. On Windows you probably want mingw32 (http://www.mingw.org). Alternatively, you can use a Zope Windows installer to get a binary Zope distribution.   Quick Start Understanding your development environment is an important step in becoming a productive developer. If you need to get up and running quickly, however, and you have the prerequisites outlined above in order, here are the key steps. We will assume you have Python 2.4 as your main Python interpreter. First download easy_install if you do not have it already, and use it to install ZopeSkel:   $ wget http://peak.telecommunity.com/dist/ez_setup.py   $ python ez_setup.py   $ easy_install ZopeSkel Note: If you do not have wget (e.g. because you are using Windows), you can just as easily download the ez_setup.py script using a web browser. When you run this script, it will install the easy_install binary to a directory that may not already be in your system PATH. Watch the ez_setup.py output to identify this directory. If it is not in your PATH, you should add it, allowing you to run easy_install as shown. Then, use paster, which was installed as a dependency of ZopeSkel, to create a new buildout. This folder holds our source code and dependencies, including the Zope application server:   $ paster create -t plone3_buildout myproject You can accept the defaults for all the questions, except for the password, which you must enter. Then, build the environment like so:   $ cd myproject   $ python bootstrap.py   $ ./bin/buildout Note: If you are using Windows, be sure to read the README.txt file that is generated in the myproject directory before running the buildout command. This last step may take some time, and you will need a live Internet connection.When it is complete, you can start Zope with:   $ ./bin/instance fg Go to http://localhost:8080/manage and you should see the Zope Management Interface. Use the drop-down box to add a Plone Site. If you call this mysite, it will beaccessible from http://localhost:8080/mysite. Glossary Let us now take a step back and consider our development environment in more detail. The table below summarizes the various terms and technologies that you will encounter in this chapter. It pays to be familiar with these names, because you will find them again not only throughout this book, but also in other Plone documentation. Term Definition Zope installation Zope consists of Python code, C extensions, configuration files, documentation, scripts, and utilities. Collectively, these are known as the Zope installation. Software home The part of the Zope installation that contains the main Zope runtime. This is found in the lib/python directory of the Zope installation. The full path is assigned to the $SOFTWARE_HOME environment variable when Zope is run. Zope instance The same Zope installation can be used to power multiple Zope servers, possibly running concurrently on different ports. Each instance has a directory containing a configuration file, instance-specific software components (e.g. an installation of Plone), and the local Zope database storage. Instance home When a Zope instance is running, the $INSTANCE_HOME environment variable refers to the directory where the instance is set up. Package A generic term for a distributable bundle of Python modules and supporting files. Product The traditional way to redistribute software for Zope 2 is in a "Product", which we will sometimes refer to as an "old-style Product". Products are placed in a special directory (Products/) and automatically discovered by Zope. The term "product" is also used more generally to refer to add-on components that extend Plone, even if they are not actually packaged as old-style Products. Egg A more generic and flexible alternative to products. Eggs are not specifi c to Zope, and Zope has only recently been made egg-aware. In addition to code, eggs contain metadata such as version, dependencies, and license information. Egg management tools can use this information to manage concurrent versions or automatically fetch dependencies, for example. $PYTHONPATH The $PYTHONPATH environment variable lists the directories containing Python packages that should be available at run time. It can also reference specific eggs. You should not have to set this manually. setuptools A Python library, which extends Python's built-in distutils package to support extended egg metadata and offers enhanced functionality when using software packaged as eggs. The Cheese Shop Also known as PyPI (Python Package Index). An online repository of Python eggs. Anyone can upload a package here. Egg-aware tools can use the Cheese Shop to locate dependencies and download them automatically when installing other eggs. easy_install A command-line tool, which searches the Cheese Shop for a given package, and downloads and installs it. Note that, by default, easy_install puts packages in the global site-packages folder for the Python interpreter that was used to install easy_install itself. Normally, we want our Plone packages to be local to a specific Zope instance, necessitating different tools. paster (Paste Script) paster, part of the Python Paste package, is a command runner. The paster create command invokes Paste Script templates, which are used to create skeleton packages based on command-line options and questions. ZopeSkel A collection of Paste Script templates for Zope and Plone development. We will use this to create new egg-ready packages, as well as the buildout that manages our development environment. Buildout (zc.buildout) A "buildout", using the zc.buildout toolset, is a self-contained environment that is controlled through a single configuration file (buildout.cfg). We will use it to download, install, and configure Zope, Plone, and other dependencies. Buildouts are "repeatable", meaning that they can be used to replicate a particular setup across multiple servers or developers' machines.     Understanding Eggs and Setuptools Python eggs are not specific to Zope or Plone. However, since Zope has only recently become egg-aware, they are new to many developers. Traditionally, almost all Zope add-on products, including Plone, have been distributed as Zope products. These are fairly easy to manage—you typically copy or symlink them into $INSTANCE_HOME/Products. Zope will scan this directory on startup, taking care of any product initialization and registration. However, code inside products is nearly impossible to re-use outside Zope because Zope does magic things with the Products.* namespace. Further, the namespace quickly becomes crowded, which deters developers from breaking up functionality into smaller, more re-usable and better-isolated packages. The Zope 3 philosophy is to be as close to "plain Python" as possible, and that means distributing code as such small packages. So long as its dependencies are in order, any package should be able to run in any environment. For example, the zope.interface package is used by the Twisted project, which is not otherwise dependent on Zope. This design goal has made it much easier to adopt Zope 3 packages in Zope2 and Plone. Starting with Plone 3, the Plone community has also embraced "plain Python" packages and uses them wherever possible. A number of packages, such as plone.memoize and plone.portlets are generic enough to work without any dependencies on the rest of Plone. Others are more specific to Plone and live in the plone.app namespace, such as plone.app.layout and plone.app.portlets, thelatter containing Plone-centric extensions to the generic plone.portlets package. All that is needed to use these packages is a sensible $PYTHONPATH. Thus, we can copy or link packages into $INSTANCE_HOME/lib/python/plone/memoize, lib/python/plone/portlets, lib/python/plone/app/portlets, and so forth for Zope to find them. This works, but it is pretty tedious when there are many packages, and it can become outright confusing when there are nested namespaces being used bymultiple packages. Luckily, other Python programmers have solved these problems, first creating distutils, then its successor setuptools and with setuptools, Python eggs. Note: For the setuptools documentation, see http://peak.telecommunity.com/DevCenter/setuptools. Installing Eggs When using setuptools, each project or package lives in a directory that has a toplevel setup.py file. This contains metadata about the package itself, and declares its current version as well as any dependencies. Dependencies can be specifieddown to particular versions (e.g. ">=0.2,<1.0" means "later than version 0.2 but before version 1.0"). When a package is installed, setuptools will attempt to fulfilldependencies by downloading and installing them if necessary. If you have a setuptools-enabled package, you can use setup.py to install it globally, by running:   $ python setup.py install This will copy the source code to the system-wide Python site-packages directory. Having to re-run this command each time you make a change can make development a little awkward, so while you are working on a particular package,you can install a development egg. This is essentially a link to the package's source code that ensures it is added to the $PYTHONPATH. To install a development egg, run:   $ python setup.py develop New packages can be released as binary eggs for distribution, which are just ZIP files of the package with some additional metadata. You can build an egg from within apackage by running:   $ python setup.py bdist_egg The new egg will be placed in the dist sub-directory, which will be created if necessary. Eggs can be uploaded to the Cheese Shop, also known as PyPI (the PythonPackage Index). This central repository makes it easy to fi nd packages. You canbrowse packages at http://cheeseshop.python.org/pypi. New packages can beuploaded via this website, or directly from the command line:   $ python setup.py egg_info -RDb "" sdist bdist_egg register upload You will be asked to specify or create a Cheese Shop account if this is the first time you run this command. A script called easy_install lets you search the Cheese Shop (or a similar index, if you specify a URL) for packages that it can download and install into the global Python environment. Dependencies will be included automatically. This is great for simple libraries and end-user applications, but less great when you are working on multiple Zope projects that may have different version requirements. This is why we tend to manage our eggs inside $INSTANCE_HOME or, as you will see in the next section, as part of a controlled buildout. Note: A tool called workingenv.py can create a mini-environment where global commands are restricted to a particular directory. We will not cover workingenv in this book, but you can download and read more about it at http://cheeseshop.python.org/pypi/workingenv.py. When eggs are activated (either explicitly, or implicitly by being unambiguouslyfound in the $PYTHONPATH), they can be discovered by other packages listening for plug-ins, using a mechanism called entry points (see http://peak.telecommunity.com/DevCenter/setuptools#dynamic-discovery-of-services-and-plugins). Zope does not yet directly use entry points, so we will not be covering them inany detail here. However, entry points are a very powerful system, and there areproposals to let Zope's discovery of packages use entry points instead of scanningmagic directories. With eggs, we therefore have the tools to manage multiple packages, from differentdevelopers and repositories, possibly across multiple versions. By using the packagemanagement tools that the rest of the Python community employs, we also make iteasier to re-use other libraries and share our own code with outside developers. Automating the Build Process with zc.buildout Creating a Zope instance and copying or linking packages into $INSTANCE_HOME/lib/python as is done in the manual process is not too difficult, but this approach has a few limitations. The process is manual and cumbersome to repeat across multiple environments. Multiple developers working on the same project may share the code in eggs and products by using a version control system such as Subversion. However, each developer would be responsible for setting up their development environment, and subtle differences may cause problems that are difficult to debug. Packages are installed manually, and so cannot benefit from setuptools' ability to manage dependencies and updates. Complex deployments that include other libraries, non-python code, or specific configurations will also need to be taken care of manually. Luckily, there are tools to make deployment easier. zc.buildout is one such tool, written largely by Zope founder Jim Fulton at Zope Corporation. It makes heavyuse of eggs and setuptools and is very flexible in supporting a wide range of deployment scenarios. Central to a buildout (i.e. what zc.buildout is managing for us) is a file called buildout.cfg. This specifies various options, including a list of parts, which will be executed when the buildout is run. Each part is associated with a recipe—a named egg, which will be called upon to parse the options provided, and perform a particular task, such as building a Zope instance or downloading Plone. A project-specific buildout directory can be checked into a version control system and shared among developers. It can also be used to replicate a particular environment across different servers with a high degree of predictability. By writing custom recipes, you can make zc.buildout do almost anything. Writing a recipe is not particularly hard, and there are plenty of examples and generic solutions available. However, we will not cover creating new recipes in this book, because all the recipes we need already exist.
Read more
  • 0
  • 0
  • 2318

article-image-installing-and-using-openfire
Packt
21 Oct 2009
6 min read
Save for later

Installing and Using Openfire

Packt
21 Oct 2009
6 min read
The Openfire instant messaging server is very easy to install. In fact, it's totally newbie-proof. So much so, that unlike other complex server software, even if you've never setup up Openfire before, you'll be able to get it up and running on your first try. If you're sceptical, by the time we are done with this short article, we'll have ourselves a fully-functional Openfire server that will register users and connect with clients. Preparing Your System Openfire is a cross-platform server and can be installed under Linux, Solaris, Mac, or Windows operating system environments. Openfire reserves its enormity for its users. When it comes to system requirements, Openfire is very suave and a perfect gentleman who has very moderate demands. You don't need to spend much time preparing your system for installing Openfire. Just pick out the environment you're comfortable with—Windows or one of the popular Linux distributions such as Fedora, Debian, or Ubuntu, and you're good to go. You don't have to run around getting obscure libraries or worry about mismatched versions. But like any hard-working gentleman, Openfire has a thing for caffeine, so make sure you have Java on your system. No need to run to the kitchen—this isn't the Java in the cupboard. Openfire is written in the Java programming language, so it'll need a Java Runtime Environment (JRE) installed on your system. A JRE creates a simple (breathable, so to say) environment for Java applications to live and function in. It's available as a free download and is very easy to install. If you're installing under Windows, just skip to the "Installing Under Windows" section later in the article. Linux Users Get Your Cuppa! Sun's Java Runtime Environment is available as a free download from Sun's website (http://www.java.com/en/download/linux_manual.jsp) or it can also be installed from your distribution's software management repositories. Users of RPM-based systems can safely skip this section because the Openfire installer for their distribution already includes a JRE. On the other hand, users of Debian-based systems such as Ubuntu will have to install the JRE before installing Openfire. Thanks to the popular apt-get package management system, there isn't much to installing the JRE. Because Sun's JRE isn't free and is also not an open source software, most Linux distributions make the JRE package available in their non-free tree. If the following command doesn't work, check out the detailed installation instructions for your specific distribution, at  https://jdk-distros.dev.java.net. Open a console and issue the following command: $ sudo apt-get install sun-java6-jre Now the apt-get system will automatically fetch, install, and activate the JRE for you! Meet The Protagonists This article is about making sure that you have no trouble installing one file. This one file is the Openfire installer and it is available in multiple flavors. The four flavors we're concerned with aren't as exotic as Baskin Robbins' 31 flavors but that doesn't make the decision any easier. The Openfire project releases several installers. The four flavors we're concerned with are: Openfire-3.5.2-1.i386.rpm: RPM package for Fedora Linux and other RPM-based variants Openfire_3.5.2_all.deb: DEB package for Debian, Ubuntu Linux and their derivates Openfire_3_5_2.tar.gz: Compressed "tarball" archive that'll work on any Linux distribution Openfire_3_5_2.exe: Openfire installer for Windows We'll cover installing Openfire from all of these files, so that you may use Openfire from your favorite Linux distribution or from within Windows. Just to reiterate here, the Windows installer and the RPM Linux installer both bundle the JRE, while the other other versions do not. The Actual Install-Bit Alright, so you have the Java JRE setup and you've downloaded the Openfire installer. In this section, we'll install Openfire server from the various versions we discussed in the last section. Let's first install from the source tarball. The first step when dealing with .tar.gz source archive is to extract the files. Let's extract ours under /tmp and then move the extracted directory under /opt. # tar zxvf openfire_3_5_2.tar.gz# mv openfire /opt Now we'll create a non-priviledged user and group for running Openfire. # groupadd openfire# useradd -d /opt/openfire -g openfire openfire Next, we'll change ownership of the openfire/directory to the newly-created user and group. # chown -R openfire:openfire /opt/openfire Believe it or not, that's it! You've just installed Openfire server. Surprised? Get ready for more. It gets even simpler if you install using the precompiled RPM or DEB binaries. In the case of RPM, Openfire is installed under /opt/openfire and in case of the DEB file, Openfire resides under /etc/openfire. On RPM-based systems such as Fedora and its derivates (as root), use: # rpm -ivh openfire-3.5.2-1.i386.rpm On DEB-based systems such as Debian, Ubuntu, and so on, use: $ sudo dpkg -i openfire_3.5.2_all.deb Voila! You're done. Now, who thought my "installing Openfire is totally newbie-proof" comment was an exaggeration? Running Openfire on Linux/Unix So, we now have Openfire on our favourite Linux distribution, whichever distribution this may be. Now it's time to fire it up and get going. Depending on how you installed Openfire, the procedure to start it varies a little. If you've installed Openfire from the RPM or DEB, you'll be pleased to know that the Openfire developers have already done most of the hard work for you. These binaries contain some custom handling for the RedHat/Debian-like environments. You can start and stop Openfire just like any other service on your system: # /etc/init.d/openfire startStarting Openfire: You can also view the other options available: # /etc/init.d/openfireUsage /etc/init.d/Openfire {start|stop|restart|status|condrestart|reload} On the other hand, if you've installed Openfire using the .tar.gz archive, you can start and stop Openfire using the bin/openfire script in your Openfire installation directory. First, change to the user that owns the /opt/openfire directory: # su - openfire# cd /opt/openfire/bin/# ./openfire startStarting Openfire And now you have Openfire up and running! If you are using a firewall, which you most probably are, make sure to forward traffic on ports 5222 and 5223 (for SSL) which clients use for connecting with the Openfire server. Also forward traffic on port 7777 for file transfer. Linux users can skip the next section on installing Openfire under Windows and move directly to the section that discusses the preliminary Openfire setup.
Read more
  • 0
  • 0
  • 5933
Unlock access to the largest independent learning library in Tech for FREE!
Get unlimited access to 7500+ expert-authored eBooks and video courses covering every tech area you can think of.
Renews at €18.99/month. Cancel anytime
article-image-access-control-php5-cms-part-2
Packt
21 Oct 2009
17 min read
Save for later

Access Control in PHP5 CMS - Part 2

Packt
21 Oct 2009
17 min read
Framework Solution The implementation of access control falls into three classes. One is the class that is asked questions about who can do what. Closely associated with this is another class that caches general information applicable to all users. It is made a separate class to aid implementation of the split of cache between general and user specific. The third class handles administration operations. Before looking at the classes, though, let's figure out the database design. Database for RBAC All that is required to implement basic RBAC is two tables. A third table is required to extend to a hierarchical model. An optional extra table can be implemented to hold role descriptions. Thinking back to the design considerations, the first need is for a way to record the operations that can be done on the subjects, that is the permissions. They are the targets for our access control system. You'll recall that a permission consists of an action and a subject, where a subject is defined by a type, and an identifier. For ease of handling, a simple auto-increment ID number is added. But we also need a couple of other things. To make our RBAC system general, it is important to be able to control not only the actual permissions, but also who can grant those permissions, and whether they can grant that right to others. So an extra control field is added with one bit for each of those three possibilities. It therefore becomes possible to grant the right to access something with or without the ability to pass on that right. The other extra data item that is useful is a "system" flag. It is used to make some permissions incapable of deletion. Although not being a logical requirement, this is certainly a practical requirement. We want to give administrators a lot of power over the configuration of access rights, but at the same time, we want to avoid any catastrophes. The sort of thing that would be highly undesirable would be for the top level administrator to remove all of their own rights to the system. In practice, most systems will have a critical central structure of rights, which should not be altered even by the highest administrator. So now the permissions table can be seen to be as shown in the following screenshot: Note that the character strings for role, action, and subject_type are given generous lengths of 60, which should be more than adequate. The subject ID will often be quite short, but to avoid constraining generality, it is made a text field, so that the RBAC system can still handle very complex identifiers, if required. Of course, there will be some performance penalties if this field is very long, but it is better to have a design trade-off than a limitation. If we restricted the subject ID to being a number, then more complex identifiers would be a special case. This would destroy the generality of our scheme, and might ultimately reduce overall efficiency. In addition to the auto-increment primary key ID, two indices are created, as shown in the following screenshot. They involve overhead during update operations but are likely to speed access operations. Since far more accesses will typically be made than updates, this makes sense. If for some reason an index does not give a benefit, it is always possible to drop it. Note that the index on the subject ID has to be constrained in length to avoid breaking limits on key size. The value chosen is a compromise between efficiency through short keys, and efficiency through the use of fine grained keys. In a heavily used system, it would be worth reviewing the chosen figure carefully, and perhaps modifying it in the light of studies into actual data. The other main database table is even simpler, and holds information about assignment of accessors to roles. Again, an auto-increment ID is added for convenience. Apart from the ID, the only fields required are the role, the accessor type, and the accessor ID. This time a single index, additional to the primary key, is sufficient. The assignment table is shown in the following screenshot, and its index is shown in the screenshot after that: Adding hierarchy to RBAC requires only a very simple table, where each row contains two fields: a role, and an implied role. Both fields constitute the primary key, neither field on its own being necessarily unique. An index is not required for efficiency, since the volume of hierarchy information is assumed to be small, and whenever it is needed, the whole table is read. But it is still a good principle to have a primary key, and it also guarantees that there will not be redundant entries. For the example given earlier, a typical entry might have consultant as the role, and doctor as the implied role. At present, Aliro implements hierarchy only for backwards compatibility, but it is a relatively easy development to make hierarchical relationships generally available. Optionally, an extra table can be used to hold a description of the roles in use. This has no functional purpose, and is simply an option to aid administrators of the system. The table should have the role as its primary key. As it does not affect the functionality of the RBAC at all, no further detail is given here. With the database design settled, let's look at the classes. The simplest is the administration class, so we'll start there. Administering RBAC The administration of the system could be done by writing directly to the database, since that is what most of the operations involve. There are strong reasons not to do so. Although the operations are simple, it is vital that they be handled correctly. It is generally a poor principle to allow access to the mechanisms of a system rather than providing an interface through class methods. The latter approach ideally allows the creation of a robust interface that changes relatively infrequently, while details of implementation can be modified without affecting the rest of the system. The administration class is kept separate from the classes handling questions about access because for most CMS requests, administration will not be needed, and the administration class will not load at all. As a central service, the class is implemented as a standard singleton, but it is not cached because information generally needs to be written immediately to the database. In fact, the administration class frequently requests the authorization cache class to clear its cache so that the changes in the database can be effective immediately. The class starts off: class aliroAuthorisationAdmin { private static $instance = __CLASS__; private $handler = null; private $authoriser = null; private $database = null; private function __construct() { $this->handler =& aliroAuthoriserCache::getInstance(); $this->authoriser =& aliroAuthoriser::getInstance(); $this->database = aliroCoreDatabase::getInstance(); } private function __clone() { // Enforce singleton } public static function getInstance() { return is_object(self::$instance) ? self::$instance : (self::$instance = new self::$instance()); } private function doSQL($sql, $clear=false) { $this->database->doSQL($sql); if ($clear) $this->clearCache(); } private function clearCache() { $this->handler->clearCache(); } Apart from the instance property that is used to implement the singleton pattern, the other private properties are related objects that are acquired in the constructor to help other methods. Getting an instance operates in the usual fashion for a singleton, with the private constructor, and clone methods enforcing access solely via getInstance. The doSQL method also simplifies other methods by combining a call to the database with an optional clearing of cache through the class's clearCache method. Clearly the latter is simple enough that it could be eliminated. But it is better to have the method in place so that if changes were made to the implementation such that different actions were needed when any relevant cache is to be cleared, the changes would be isolated to the clearCache method. Next we have a couple of useful methods that simply refer to one of the other RBAC classes: public function getAllRoles($addSpecial=false) { return $this->authoriser->getAllRoles($addSpecial); }public function getTranslatedRole($role) { return $this->authoriser->getTranslatedRole($role); } Again, these are provided so as to simplify the future evolution of the code so that implementation details are concentrated in easily identified locations. The general idea of getAllRoles is obvious from the name, and the parameter determines whether the special roles such as visitor, registered, and nobody will be included. Since those roles are built into the system in English, it would be useful to be able to get local translations for them. So the method getTranslatedRole will return a translation for any of the special roles; for other roles it will return the parameter unchanged, since roles are created dynamically as text strings, and will therefore normally be in a local language from the outset. Now we are ready to look at the first meaty method: public function permittedRoles ($action, $subject_type, $subject_id) { $nonspecific = true; foreach ($this->permissionHolders ($subject_type, $subject_id) as $possible) { if ('*' == $possible->action OR $action == $possible->action) { $result[$possible->role] = $this->getTranslatedRole ($possible->role); if ('*' != $possible->subject_type AND '*' != $possible_subject_id) $nonspecific = false; } } if (!isset($result)) { if ($nonspecific) $result = array('Visitor' => $this->getTranslatedRole('Visitor')); else return array(); } return $result; }private function &permissionHolders ($subject_type, $subject_id) { $sql = "SELECT DISTINCT role, action, control, subject_type, subject_id FROM #__permissions"; if ($subject_type != '*') $where[] = "(subject_type='$subject_type' OR subject_type='*')"; if ($subject_id != '*') $where[] = "(subject_id='$subject_id' OR subject_id='*')"; if (isset($where)) $sql .= " WHERE ".implode(' AND ', $where); return $this->database->doSQLget($sql); } Any code that is providing an RBAC administration function for some part of the CMS is likely to want to know what roles already have a particular permission so as to show this to the administrator in preparation for any changes. The private method permissionHolders uses the parameters to create a SQL statement that will obtain the minimum relevant permission entries. This is complicated by the fact that in most contexts, asterisk can be used as a wild card. The public method permittedRoles uses the private method to obtain relevant database rows from the permissions table. These are checked against the action parameter to see which of them are relevant. If there are no results, or if none of the results refer specifically to the subject, without the use of wild cards, then it is assumed that all visitors can access the subject, so the special role of visitor is added to the results. When actual permission is to be granted we need the following methods: public function permit ($role, $control, $action, $subject_type, $subject_id) { $sql = $this->permitSQL($role, $control, $action, $subject_type, $subject_id); $this->doSQL($sql, true); }private function permitSQL ($role, $control, $action, $subject_type, $subject_id) { $this->database->setQuery("SELECT id FROM #__permissions WHERE role='$role' AND action='$action' AND subject_type='$subject_type' AND subject_id='$subject_id'"); $id = $this->database->loadResult(); if ($id) return "UPDATE #__permissions SET control=$control WHERE id=$id"; else return "INSERT INTO #__permissions (role, control, action, subject_type, subject_id) VALUES ('$role', '$control', '$action', '$subject_type', '$subject_id')"; } The public method permit grants permission to a role. The control bits are set in the parameter $control. The action is part of permission, and the subject of the action is identified by the subject type and identity parameters. Most of the work is done by the private method that generates the SQL; it is kept separate so that it can be used by other methods. Once the SQL is obtained, it can be passed to the database, and since it will normally result in changes, the option to clear the cache is set.   The SQL generated depends on whether there is already a permission with the same parameters, in which case only the control bits are updated. Otherwise an insertion occurs. The reason for having to do a SELECT first, and then decide on INSERT or UPDATE is that the index on the relevant fields is not guaranteed to be unique, and also because the subject ID is allowed to be much longer than can be included within an index. It is therefore not possible to use ON DUPLICATE KEY UPDATE. Wherever possible, it aids efficiency to use the MySQL option for ON DUPLICATE KEY UPDATE. This is added to the end of an INSERT statement, and if the INSERT fails by virtue of the key already existing in the table, then the alternative actions that follow ON DUPLICATE KEY UPDATE are carried out. They consist of one or more assignments, separated by commas, just as in an UPDATE statement. No WHERE is permitted since the condition for the assignments is already determined by the duplicate key situation. A simple method allows deletion of all permissions for a particular action and subject: public function dropPermissions ($action, $subject_type, $subject_id) { $sql = "DELETE FROM #__permissions WHERE action='$action' AND subject_type='$subject_type'AND subject_id='$subject_id' AND system=0"; $this->doSQL($sql, true); } The final set of methods relates to assigning accessors to roles. Two of them reflect the obvious need to be able to remove all roles from an accessor (possibly preparatory to assigning new roles) and the granting of a role to an accessor. Where the need is to assign a whole set of roles, it is better to have a method especially for the purpose. Partly this is convenient, but it also provides an extra operation, minimization of the set of roles. The method is: public function assign ($role, $access_type, $access_id, $clear=true) { if ($this->handler->barredRole($role)) return false; $this->database->setQuery("SELECT id FROM #__assignments WHERE role='$role' AND access_type='$access_type' AND access_id='$access_id'"); if ($this->database->loadResult()) return true; $sql = "INSERT INTO #__assignments (role, access_type, access_id) VALUES ('$role', '$access_type', '$access_id')"; $this->doSQL($sql, $clear); return true; }public function assignRoleSet ($roleset, $access_type, $access_id) { $this->dropAccess ($access_type, $access_id); $roleset = $this->authoriser->minimizeRoleSet($roleset); foreach ($roleset as $role) $this->assign ($role, $access_type, $access_id, false); $this->clearCache(); }public function dropAccess ($access_type, $access_id) { $sql = "DELETE FROM #__assignments WHERE access_type='$access_type' AND access_id='$access_id'"; $this->doSQL($sql, true); } The method assign links a role to an accessor. It checks for barred roles first, these are simply the special roles discussed earlier, which cannot be allocated to any accessor. As with the permitSQL method, it is not possible to use ON DUPLICATE KEY UPDATE because the full length of the accessor ID is not part of an index, so again the existence of an assignment is checked first. If the role assignment is already in the database, there is nothing to do. Otherwise a row is inserted, and the cache is cleared. Getting rid of all role assignments for an accessor is a simple database deletion, and is implemented in the dropAccess method. The higher level method assignRoleSet uses dropAccess to clear out any existing assignments. The call to the authorizer object to minimize the role set reflects the implementation of a hierarchical model. Once there is a hierarchy, it is possible for one role to imply another as consultant implied doctor in our earlier example. This means that a role set may contain redundancy. For example, someone who has been allocated the role of consultant does not need to be allocated the role of doctor. The minimizeRoleSet method weeds out any roles that are superfluous. Once that has been done, each role is dealt with using the assign method, with the clearing of the cache saved until the very end. The General RBAC Cache As outlined earlier, the information needed to deal with RBAC questions is cached in two ways. The file system cache is handled by the aliroAuthoriserCache singleton class, which inherits from the cachedSingleton class. This means that the data of the singleton object will be automatically stored in the file system whenever possible, with the usual provisions for timing out an old cache, or clearing the cache when an update has occurred. It is highly desirable to cache the data both to avoid database operations and to avoid repeating the processing needed in the constructor. So the intention is that the constructor method will run only infrequently. It contains this code: protected function __construct() { // Making private enforces singleton $database = aliroCoreDatabase::getInstance(); $database->setQuery("SELECT role, implied FROM #__role_link UNION SELECT DISTINCT role, role AS implied FROM #__assignments UNION SELECT DISTINCT role,role AS implied FROM #__permissions"); $links = $database->loadObjectList(); if ($links) foreach ($links as $link) { $this->all_roles[$link->role] = $link->role; $this->linked_roles[$link->role][$link->implied] = 1; foreach ($this->linked_roles as $role=>$impliedarray) { foreach ($impliedarray as $implied=>$marker) { if ($implied == $link->role OR $implied == $link->implied) { $this->linked_roles[$role][$link->implied] = 1; if (isset($this->linked_roles[$link->implied])) foreach ($this->linked_roles[$link->implied] as $more=>$marker) { $this->linked_roles[$role][$more] = 1; } } } } } $database->setQuery("SELECT role, access_id FROM #__assignments WHERE access_type = 'aUser' AND (access_id = '*' OR access_id = '0')"); $user_roles = $database->loadObjectList(); if ($user_roles) foreach ($user_roles as $role) $this- >user_roles[$role->access_id][$role->role] = 1; if (!isset($this->user_roles['0'])) $this->user_roles['0'] = array(); if (isset($this->user_roles['*'])) $this->user_roles['0'] = array_merge($this->user_roles['0'], $this->user_roles['*']); } All possible roles are derived by a UNION of selections from the permissions, assignments, and linked roles database tables. The union operation has overheads, so that alone is one reason for favoring the use of a cache. The processing of linked roles is also complex, and therefore worth running as infrequently as possible. Rather than working through the code in detail, it is more useful to describe what it is doing. The concept is much simpler than the detail! If we take an example from the backwards compatibility features of Aliro, there is a role hierarchy that includes the role Publisher, which implies membership of the role Editor. The role Editor also implies membership of the role Author. In the general case, it is unreasonable to expect the administrator to figure out the implied relationships. In this case, it is clear that the role Publisher must also imply membership of the role Editor. But these linked relationships can plainly become quite complex. The code in the constructor therefore assumes that only the least number of connections have been entered into the database, and it figures out all the implications. The other operation where the code is less than transparent is the setting of the user_roles property. The Aliro RBAC system permits the use of wild cards for specification of identities within accessor, or subject types. An asterisk indicates any identity. For accessors whose accessor type is user, another wild card available is zero. This means any user who is logged in, and is not an unregistered visitor. Given the relatively small number of role assignments of this kind, it saves a good deal of processing if all of them are cached. Hence the user_roles processing is done in the constructor. Other methods in the cache class are simple enough to be mentioned rather than given in detail. They include the actual implementation of the getTranslatedRole method, which provides local translations for the special roles. Other actual implementations are getAllRoles with the option to include the special roles, getTranslatedRole, which translates a role if it turns out to be one of the special ones and barredRole, which in turn, tests to see if the passed role is in the special group. It may therefore not be assigned to an accessor.
Read more
  • 0
  • 0
  • 2174

article-image-supporting-editorial-team-drupal-6
Packt
21 Oct 2009
16 min read
Save for later

Supporting an Editorial Team in Drupal 6

Packt
21 Oct 2009
16 min read
What you will do In this article, you will: Create a team Add Roles to support the team Define new Node Content types Configure permissions to support the Roles Handle a former (and disgruntled) team member The Creative team Let's take a quick look at Drupal's jargon regarding teams. Users—the logins of the individuals that make up a team Roles—the different 'job descriptions' based on a person's responsibilities Permissions—the granting of authorization to perform a Drupal function As the system administrator, you are authorized to perform any action within the Drupal environment, but you would not want every member of a team to have this absolute capability, or else you would soon have chaos. Let's first create a team. Then, we will look at assimilating that team into the Drupal environment. Our Creative team will be made up of individuals, each having one or more of the responsibilities mentioned below (Note: the titles are not Drupal terms): Copy Writers—are the writers of short articles Feature Writers—are the writers of long pieces, in which style matters a much as content Ad Writers—are the writers of internal and external advertising that will appear in blocks Proofreaders—are the reviewers who check pieces for spelling, grammar and usage errors Associate Editors—are the reviewers that are concerned with style, readability, and continuity Style Editors—are responsible for the formatting of content Graphic Artists—are the creators of the illustrations and images that are used as copy Senior Editor—is responsible for the quality of all of the above Moderator—manages postings by site visitors, such as comments and blog posts Blogger—creates blog entries Administrator—addresses the aspects of the site unrelated to content With our team assembled, let's move on to creating the roles in our site. Roles Drupal comes with three roles installed: creator (also known as userID1), authenticated user and anonymous user. Only the latter two are listed when assigning permissions, because the creator role can do everything, including things that you might not want the administrator to be able to do. It's best not to use the creator's login as the administrator login. A separate administrator role should be created and granted the appropriate permissions. So, looking at the list above, we will need to create roles for all of our team members. Creating roles in Drupal is a quick and easy process. Let's create them. Activity 1: Creating Roles The Name of the role is assigned as per the responsibilities of the team member. Login as the administrator. Select the User management option. Select the Roles option. Enter the name of the role in the text box, as shown in the following screenshot, and then click on the Add role button. We'll add the rest of the roles in the same way. After a couple of minutes, we have the entire team added, as seen in following screenshot. The edit role links are locked for anonymous user and authenticated user, because those roles should remain constant and never be edited or deleted. Node Content types The default installation of Drupal contains two Node Content types namely: Page and Story. Some modules, when activated, create additional Node Content types. One such example is the Blog entry, and another is an Event, which is used when using an event calendar. We're using the term Node Content to differentiate content nodes in Drupal, such as Pages and Stories, from other non-node types of content, such as Blocks, which is the generic term for anything on the page. What is the purpose of having different Node Content types? If we want a feature  writer to be able to create Features, then how do we accomplish that? Currently, we have Stories and Pages as our Node Content types. So, if we give the Feature writer the ability to create a Page, then what differentiates that Page from any other Page on our site? If we consider a Page as a Feature, then anyone who can create a Page has created a Feature, but that's not right, because not every Page is a Feature. Activity 2: Node Content for our Roles Because we have role types that we want to limit to working with their respective Node Content types, we will need to create those Node Content types. We will assign a Node Content type of Feature for Feature Writers, Ads for Ad Writers, and so on. Let's create them. From the admin menu, we'll select Content management. On the Content management page, we'll choose Content types. The Node Content types are listed, and from the top of the page we'll select Add content type. We're going to start with the Feature writer, so in the Name field we'll enter Feature. The next field, Type, determines the term that will be used to construct the default URL for this Node Content type. We'll enter feature as the text value for this field. In the Description field, we'll enter a short description, which will appear next to the Node Content type's link on the admin page, as follows: Next, we'll click on the Workflow settings link to display the additional workflow fields. When our Feature Writer completes a piece, it will not be published immediately. It will have to be proofread and undergo an editorial review. So, we'll deselect the Published and Promote to front page boxes. At this point we've configured the new Node Content type as per our needs, so we'll click on the Save button, and then we can see it listed, as shown in the screenshot below. We already have a Node Content type of Blog entry, which was created by the Blog module. The only other Role that requires its own Node Content type is the Ad Writer. This is because the other Roles defined will only edit existing Node Content,  as opposed to creating it. It is here that we run into trouble. The pieces that are 'grabbed' by Drupal to appear (usually) at the center of the screen, which we have been referring to as Node Content, are nodes, whether a Page, a Story, or now a Feature. The small blocks that appear around the sides, or on top, or at the bottom, are Blocks. Because they are placed in those positions, and are not available for selection as Node Content, they are not nodes. The Benefit of BlocksWhen looking at a typical web page of a CMS site, you will see a main body area with Node Content, such as articles, and also small blocks of information elsewhere on the page, such as in the left and right margins, or along the top or bottom. The main content, nodes, are limited, as to where they appear. However, each of the blocks can be configured to appear on any or every page of the site. That is why ads are best created as blocks, so that they can be placed where they will be the most effective. Nodes are created via the Create content function, and that function is available from the front page to anyone who is granted the permission. Using the admin menu is not necessary. On the other hand, blocks are created and edited from the Block page, which is an admin function. Although we can grant that capability to a user without granting any other admin capabilities, it would be much better if we could have an Ad Writer create ads in the same way that they create other Node Content. The reason for this is that with nodes, separate permission can be given to create a node and to administer a node. With  blocks, there is only one permission. You can create, edit, delete, and rearrange all of the blocks, or none. This opens the door to an accidental disaster. We don't want the Ad Writer doing anything but creating ad copy. So, in order to address this concern, we've added a module to our site: Node blocks. This module allows us designate a Node Content type (other than Page and Story) to be used as a Block. With that in mind, let's create our final Node Content type. Where can you find this module? This module, as well as other modules, can be found at http://drupal.org/project/modules. Activity 3—creating a Block Node Content type We'll start by repeating Steps 1 to 3 from the previous activity. In the Title field, we'll type in Ad. In the Type field, we'll type in ad. For the description, we'll enter Advertisement copy that will be used as blocks. We'll click on Workflow settings and deselect Published and Promoted to front page, as we did with the Feature. There is a new heading in this dialog, Available as Block, as seen in the following screenshot. This comes from the module that we've added. We'll select Enabled, which will make any piece created with this Node Content type available as a Block. That's all we need to do, so now we'll save our new Node Content type   Permissions The way that we enable one user to do something that the other cannot is by creating different user types (which we have done), different Node Content types—where necessary—(which again has been done), and then assign permissions to the user types (which we'll do now). The administrator will not be listed as a user type under Permissions, because if permissions were accidentally removed from the administrator, there might be no other user type that has the permissions to restore them. Activity 4: Granting Permissions Let's now assign to the members of the Creative team the Permissions that suit them best. From the admin menu we'll select User management. On the User management page we'll choose Permissions. The screenshot below shows us the upper portion of the screen. There are numerous permissions, and we now have numerous User types, so the resulting grid is very large. Rather than step-by-step illustrations, I'll simply list each Role and the Permissions that should be enabled in the form of Heading→Permission. Ad Writer node module→access content node module→create ad content node module→delete any ad content node module→delete own ad content node module→edit any ad content node module→edit own ad content node module→view revisions fckeditor module→access fckeditor Because of the number of Node Content types, each having several permissions as seen above, combined with the permissions being alphabetical by verb within the heading, instead of Content type, the necessary permissions are somewhat distant from each other and require scrolling to find them all. Feature Writer node module→access content node module→create feature content node module→delete any feature content node module→delete own feature content node module→edit any feature content node module→edit own feature content node module→view revisions fckeditor module→access fckeditor Blogger blog module→create blog entries blog module→delete own blog entries blog module→edit own blog entries node module→access content node module→view revisions fckeditor module→access fckeditor Associate Editor—The Associate Editor is concerned with content, which means editing it. The ability to create or delete content, to affect where the content appears, and so on, is not required for this Role. fckeditor module→access fckeditor node module→access content node module→edit any ad content node module→edit any feature content node module→edit any page content node module→edit any story content node module→revert revisions node module→view revisions path module→create URL aliases Copy Writer fckeditor module→access fckeditor node module→access content node module→create page content node module→create story content node module→delete own page content node module→delete own story content node module→edit own page content node module→edit own story content node module→view revisions Graphic Artist blog module→edit any blog entry fckeditor module→access fckeditor fckeditor module→allow fckeditor fle uploads node module→access content node module→edit any ad content node module→edit any feature content node module→edit any page content node module→edit any story content Moderator blog module→edit any blog entry comment module→access comments comment module→administer comments fckeditor module→access fckeditor node module→access content node module→edit any ad content node module→edit any feature content node module→edit any page content node module→edit any story content Proofreader blog module→edit any blog entry fckeditor module→access fckeditor node module→access content node module→edit any ad content node module→edit any feature content node module→edit any page content node module→edit any story content Style Editor block module→administer blocks fckeditor module→access fckeditor fckeditor module→allow fckeditor fle uploads node module→access content node module→edit any ad content node module→edit any feature content node module→edit any page content node module→edit any story content Senior Editor block module→administer blocks blog module→delete any blog entry blog module→edit any blog entry comment module→access comments comment module→administer comments fckeditor module→access fckeditor fckeditor module→allow fckeditor fle upload node module→access content node module→delete any ad content node module→delete any feature content node module→delete any page content node module→delete any story content node module→delete revisions node module→edit any ad content node module→edit any feature content node module→edit any page content node module→edit any story content node module→revert revisions node module→view revisions path module→create URL aliases view module→access all views view module→administer views With that, we have assigned the required permissions to all of our team members, which will allow them to do their jobs, but keep them out of trouble! However, what do you do when someone intentionally gets into trouble? The disgruntled team member So, we've been marching along as one big happy team, and then it happens. Someone gets let go, and that someone isn't happy about it, to say the least. Of course, we'll remove that person's login, but there is public access to our site as well, in the form of comments. Is there a way for us to stop this person from looking for ways to annoy us, or worse? Yes! Activity 5: Blocking Let's now perform the tasks necessary to keep disgruntled employees (and trouble-makers) at bay. From the admin menu, select User management. On the User management page, we'll select the Access rules option. We'll choose the Add rule option on the Access rules page. On the Add rule page, we have the option to deny access to a user, email address, or host. The username and email address options will block someone from registering, but will not affect someone already registered. The host name will stop anyone with that host name from accessing the system at all. Wild cards can be used: % will match any number of characters, and _ will match one character. Allow rules can be used to give access to someone who would otherwise be blocked by a host or wild card rule. In our case, let's say that the disgruntled former team member is spamming our comments from a host called spamalot.com, and is doing it from many emails. The first thing we want to do is create a 'deny' rule that will deny access to anyone from that host, as shown in the following figure, and then click on the Add rule button. We're also going to create an email deny rule for %@spamalot.com. We shouldn't have to (as we've already denied the host, which in turn would include all of the emails from that host), but we need to, because the rules testing logic ignores that hierarchy at this time. Let's also say that we've received an email from someone whose email address is its_not_me@spamalot.com, who would like to be a member of our site, and we verify that this person is not our former team member. In such a scenario, we will need to create an Allow rule, as shown in the following screenshot, so that this person can get past our previous Deny rule. Our rules now appear, as shown below, when we click on the List button, which is at the top of the page. It's always good to check and make certain that we've created the rule(s) correctly. If we don't do this, then we might inadvertently block the wrong users. Let's click on the Check rules tab at the top of the Access rules page. In the email box, we'll first try disgruntled@spamalot.com. Next, we'll try its_not_me@spamalot.com. In this last activity we have created some access rules. Drupal uses these access rules to determine who can and cannot access the site. In some cases, you may be having difficulty with a particular user adding comments to your site. Of course, if you set comments to require moderation, then the questionable ones won't appear, but it can still be a pain having to review a steady stream of them. In that case, you can block a specific user. You might be having difficulty with comments from more than one user at a given email domain. You can, if you like, block everyone from that location. On the other hand, your site might be meant for users of a particular domain, perhaps a university. In that case, you can allow users from that domain and only them. Summary In this article we learned about: Roles—defining types of users Permissions—defining capabilities for each role Node Content types—as they apply to Roles Access Rules—for those pesky, misbehaving users These features have been explained and learned with the help of activities where we have: Created a team Added Roles to enable the team Defined new Node Content types to suit the requirements of some team members Configured permissions to support the Roles and Node Content types Handled a former (and disgruntled) team member
Read more
  • 0
  • 0
  • 2529

article-image-sakai-web-services-connecting-enterprise-part-1
Packt
21 Oct 2009
17 min read
Save for later

Sakai Web Services: Connecting to the Enterprise (Part 1)

Packt
21 Oct 2009
17 min read
Connecting to Sakai is straightforward, and simple tasks, such as automatic course creation, take only a few tens of lines of programming effort. There are significant advantages to having web services in the enterprise. If a developer writes an application that calls a number of web services, then the application does not need to know the hidden details behind the services. It just needs to agree on what data to send. This loosely couples the application to the services. Later, you can replace one web service with another. Programmers do not need to change the code on the application side. SOAP works well with most organizations' firewalls (http://en.wikipedia.org/wiki/Firewall), as SOAP uses the same protocol as web browsers. System administrators have a tendency to protect an organization's network by closing unused ports to the outside world. This means that most of the time there is no extra network configuration effort required to enable web services. Another simplifying factor is that a programmer does not need to know the details of SOAP or REST, as there are libraries and frameworks that hide the underlying magic. For the Sakai implementation of SOAP, to add a new service is as simple as writing a small amount of Java code within a text file, which then is automatically compiled and run the first time the service is called. This is great for rapid application development and deployment, as the system administrator does not need to restart Sakai for each change. Just as importantly, the Sakai services use the well-known libraries from the Apache Axis project (http://ws.apache.org/axis/). SOAP is an XML message passing protocol that, in the case of Sakai sites, sits on top of the Hyper Text Transfer Protocol (HTTP). HTTP is the protocol used by web browsers to obtain web pages from a server. The client sends messages in XML format to a service, including the information that the service needs, and then the service returns a message with the results or an error message. A readable reference to this interchange is the book Pro Apache XML by Poornachandra Sarang, PhD (http://www.freesoftwaremagazine.com/articles/book_review_pro_apache_xml). The full definition of HTTP is given at http://www.w3.org/TR/soap12-part1. The architects introduced SOAP-based web services first to Sakai and later RESTful services. Unlike SOAP, instead of sending XML via HTTP posts to one URL that points to a service, REST sends to a URL that includes information about the entity, such as a user, with which the client wishes to interact. For example, a REST URL for viewing an address book item could look similar to http://host/direct/addressbook_item/15. Applying URLs in this way makes understandable address spaces that are easier for a human to read. This more intuitive approach simplifies coding. Further, SOAP XML passing requires that the client and server parse the XML and at times, the parsing effort is expensive in CPU cycles and response times. The Entity Broker is an internal service that makes life easier for programmers and helps them manipulate entities. Entities in Sakai are managed pieces of data such as representations of courses, users, grade books, and so on. In the newer versions of Sakai, the Entity Broker has the power to expose entities as RESTful services. In contrast, for SOAP services, if you wanted a new service, you would need to write it yourself. Over time, the Entity Broker exposes more and more entities RESTfully, delivering more hooks free to integrate with other enterprise systems. Both SOAP and REST services sit on top of the HTTP protocol, which is explained in the next section of this article. Protocols This section explains how web browsers talk to servers in order to gather web pages. It explains how to use the telnet command and a visual tool called TCPMON (http://ws.apache.org/commons/tcpmon/tcpmontutorial.html) to gain insight into how web services and Web 2.0 technologies work. Playing with Telnet It turns out that message passing occurs via text commands between the browser and the server. Web browsers use HTTP (http://www.w3.org/Protocols/rfc2616/rfc2616.html) to get web pages and the embedded content from the server and to send form information to the server. HTTP talks between the client and server via text (7 bit ASCII) commands. When humans talk with each other, they have a wide vocabulary. However, HTTP uses fewer than twenty words. You can experiment directly with HTTP using a Telnet client to send your commands to a web server. For example, if your demonstration Sakai instance is running on port 8080, the following command will get you the login page: telnet localhost 8080GET /portal/login The GET command does what it sounds like and gets a web page. Forms can use the GET verb to send data at the end of the URL. For example, GET /portal/login?name=alan&age=15 is sending the variables name=alan and age=15 to the server. Installing TCPMON You can use the TCPMON tool to view requests and responses from a web browser such as Firefox. One of TCPMON's abilities is that it can act as an invisible man in the middle, recording the messages between the web browser and the server. Once set up, the requests sent from the browser go to TCPMON and TCPMON passes the request on to the server. The server passes back a response and then TCPMON, a transparent proxy (http://en.wikipedia.org/wiki/Proxy_server), returns the response to the web browser. This allows us to look at all requests and responses graphically. First, you can set TCPMON up to listen on a given port number—by convention, normally, port 8888—and then you can configure your web browser to send its requests through the proxy. Then, you can type the address of a given page into the web browser, but instead of going directly to the relevant server, the browser sends the request to the proxy, which then passes it on and passes the response back. TCPMON displays both the request and responses in a window. You can download TCPMON from http://ws.apache.org/commons/tcpmon/download.cgi. After downloading and unpacking, you can, from within the build directory, run either tcpmon.bat for the Windows environment or tcpmon.sh for Unix/Linux environments. To configure a proxy, you can click the Admin tab and then set the Listen Port to 8888 and select the Proxy radio button. After that, clicking Add will create a new tab, where the requests and responses will later be displayed. Your favorite web browser now has to recognize the newly set up proxy. For Firefox 3, you can do this by selecting the menu option Edit/Preferences and then choosing the advanced tab and the network tab, as shown next. You will need to set the proxy options HTTP proxy to 127.0.0.1 and the port number to 8888. If you do this, you will need to ensure that the No proxies text input is blank. Clicking the OK button enables the new settings. To use the Proxy from within Internet Explorer 7 for a Local Area Network (LAN), you can edit the dialog box found under Tools | Internet Options | Connections | LAN settings. Once the proxy is working, typing http://localhost:8080/portal/login in the address bar will seamlessly return the login page of your local Sakai instance. Otherwise, you will see an error message similar to Proxy Server Refused Connection for Firefox or Internet Explorer cannot display the webpage. To turn the proxy settings off, simply select the No Proxies radio box and click OK for Firefox 3, or unselect the Use the proxy server for the LAN tick box in Internet Explorer 7 and click OK. Requests and returned status codes When TCPMON is running a proxy on port 8888, it allows you to view the requests from the browser and the response in an extra tab, as shown in the following screen grab. Notice the extra information that the browser sends as part of the request. HTTP/1.1 defines the protocol and version level and the lines below the GET are header variables. The User-Agent defines which client sent the request. The Accept headers tell the server what the capabilities of the browser are, and the Cookie header defines the value stored in a cookie. HTTP is stateless, that is, in principle; each response is based only on the current request. However, to get around this, persistent information can be stored in cookies. Web browsers normally store their representation of a cookie as a little text file or in a small database on the end users' computers. Sakai uses the supporting features of a servlet container, such as Tomcat, to maintain state in cookies. A cookie stores a session ID, and when the server sees the session ID, it can look up the request's server-side state. Server-side state contains information such as whether the user is logged in or what he or she has ordered. The web browser deletes the local representation of the cookie each time the browser closes. A cookie that is deleted when a web browser closes is known as a session cookie. The server response starts with the protocol followed by a status number. HTTP/1.1 200 OK tells the web browser that the server is using HTTP version 1.1 and it was able to return the requested web page successfully. 2xx status codes imply success. 3xx status codes imply some form of redirection and tell the web browser where to try to pick up the requested resource. 4xx status codes are for client errors, such as malformed requests or lack of permission to obtain the resource. 4xx states are fertile grounds for security managers to look in log files for attempted hacking. 5xx status codes mostly have to do with a failure of the server itself and are mostly of interest to system administrators and programmers during the debugging cycle. In most cases, 5xx status numbers are about either high server load or a broken piece of code. Sakai is changing rapidly and even with the most vigorous testing, there are bound to be the occasional hiccups. You will find accurate details of the full range of status codes at: http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html. Another important part of the response is the Content-Type, which tells the web browser which type of material the response is returning so the browser knows how to handle it. For example, the web browser may want to run a plug-in for video types and display text natively. The Content-Length in characters is normally also given. After the header information is finished, there is a newline followed by the content. Web browsers interpret any redirects that are returned by sending extra requests. Web browsers also interpret any HTML pages and make multiple requests for resources such as JavaScript files and images. Modern browsers do not wait until the server returns all the requests, but render the HTML page live as the server returns the parts. The GET verb is not very efficient for posting a large amount of data, as the URL has a length limit of around 2000 characters. Further, the end user can see the form data, and the browser may encode entities such as spaces to make the URL unreadable. There is also a security aspect: if you are typing in passwords in forms using GET, others may see your password or other details. This is not a good idea, especially at Internet Cafés where the next user who logs on can see the password in the browsing history. The POST verb is a better choice. Let us take as an example the Sakai demonstration login page http://localhost:8080/portal/login. The login page itself contains a form tag that points with the POST method to the relogin page. <form method="post" action="http://localhost:8080/portal/relogin" enctype="application/x-www-form-urlencoded"> Notice the HTML tag also defines the content type. Key features of the Post request compared to the GET are: the form values are stored as content after the header values, there is a newline between the end of the header and the data, and the request mentions data and the amount of data by the use of the Content-Length header value. The essential POST values for a login form with user admin (eid=admin) and password admin (pw=admin) will look like: POST http://localhost:8080/portal/relogin HTTP/1.1Content-Type: application/x-www-form-urlencodedContent-Length: 31eid=admin&pw=admin&submit=Login POSTs can contain much more information than GETs, and the request hides the values from the Address bar of the web browser. This is not secure. The header is just as visible as the URL, so POST values are also neither hidden nor secure. The only viable solution is for your web browser to encrypt your transactions using SSL/TLS (http://www.ietf.org/rfc/rfc2246.txt) for security, and this occurs every time you connect to a server using an HTTPS URL. SOAP Sakai uses the Apache Axis framework, which the developers have configured to accept SOAP calls via POST. SOAP sends messages in a specific XML format with the Content-Type, otherwise known as MIME type, application/soap+xml. A programmer does not need to know much more than that, as client libraries take care of the majority of the excruciating low-level details. An example SOAP message generated by the Perl module SOAP::Lite (http://www.soaplite.com/) for creating a login session in Sakai will look like the following Post data: <?xml version="1.0" encoding="UTF-8"?><soap:Envelope soap:encodingStyle="http://schemas.xmlsoap.org/soap/encoding/" ><soap:Body><login ><c-gensym3 xsi_type="xsd:string">admin</c-gensym3><c-gensym5 xsi_type="xsd:string">admin</c-gensym5></login></soap:Body></soap:Envelope> There is an envelope with a body containing data for the service to consume. The important point to remember is that both the client and the server have to be able to parse the specific XML schema. SOAP messages can include extra security features, but Sakai does not require these. The architects expect organizations to encrypt web services using SSL/TSL. The last extra SOAP-related complexity is the Web Service Description Language (http://www.w3.org/TR/wsdl). Web services may change location or exist in multiple locations for redundancy. The service writer can define the location of the services and the data types involved with those services in another file, in XML format. JSON Also worth mentioning is JavaScript Object Notation (JSON) (http://tools.ietf.org/html/rfc4627), which is another popular format passed using HTTP. A significant improvement in the quality of the end user experience during web browsing occurred when web developers realized that they could force browsers to load parts of a web page in at a time. This asynchronous loading enables all kinds of whiz-bang features, such as when you type in a search term and can choose from a set of search term completions before pressing submit. Asynchronous loading delivers more responsive and richer web pages that feel more like traditional applications than a plain old web page. JSON is one of the formats of choice for passing asynchronous requests and responses. The asynchronous communication normally occurs through HTTP GET or POST, but with a specific content structure that is designed to be human readable and script language parser-friendly. JSON calls have the file extension .json as part of the URL. As mentioned in RFC 4627, an example image object communicated in JSON looks like: { "Image": { "Width": 800, "Height": 600, "Title": "View from 15th Floor", "Thumbnail": { "Url": "http://www.example.com/image/481989943", "Height": 125, "Width": "100" }, "IDs": [116, 943, 234, 38793] }} To confuse the boundaries between client and server, a lot of the presentation and business logic is locked on the client side in scripting languages such as JavaScript. The scripting language orchestrates the loading of parts of pages and the generation of widget sets. Frameworks such as jQuery (http://jquery.com/) and MyFaces (http://myfaces.apache.org/) significantly ease the client-side programming burden. REST To understand REST, you need to understand the other verbs in HTTP (http://www.w3.org/Protocols/rfc2616/rfc2616-sec9.html). The full HTTP set is OPTIONS, GET, HEAD, POST, PUT, DELETE, and TRACE. The HEAD verb returns from the server only the headers of the response without the content, and is useful for clients that want to see if the content has changed since the last request. PUT requests that the content in the request be stored at the particular location mentioned in the request. DELETE is for deleting the entity. REST uses the URL of the request to route to the resource, and the HTTP verb GET is used to get a resource, PUT to update, DELETE to delete, and POST to add a new resource. In general, POST=create an item, PUT=update an item, DELETE=delete an item, and GET=return information on the item. In SOAP, you are pointing directly towards the service the client calls or indirectly via the web service description. However, in REST, part of the URL describes the resource or resources you wish to work with. For example, a hypothetical address book application that lists all email addresses in HTML format would look similar to the following: GET /email To list the addresses in XML format or JSON format: GET /email.xmlGET /email.json To get the first email address in the list: GET /email/1 To create a new email address, of course remembering to add the rest of email details to the end of the GET: POST /email And to delete address 5 in the list: DELETE /email/5 To obtain address 5 in other formats such as JSON or XML, then use file extensions at the end of the URL, for example: GET /email/5.jsonGET /email/5.xml RESTful services are more intuitively descriptive than SOAP services and they enable easy switching of the format from HTML to JSON to fuel dynamic, asynchronously-loaded web sites. Due to the direct use of HTTP verbs by REST, this methodology also fits well with the most common application type: CRUD (Create, Read, Update, Delete) applications, such as the site or user tools within Sakai. Now that we have discussed the theory, in the next section, we shall discuss which Sakai-related SOAP services already exist. Existing web services Sakai has built in, by default, the most community-requested web services, and there are also a few more services in the contributed section of the source code repository. This section describes the currently available services and the next section explains an example use, creating a new user. Recapping terminology In general, developers write web services for other developer's code to connect to (consume). Therefore, terminology can be confusing. In Sakai, a realm is a set of roles and their associated permissions. When you create a site, a copy is made from a specific realm template for that particular site type. The permissions can then be modified for the roles in the site, and members added to the site with one or other of the specific roles. Internally, Sakai uses AuthzGroups to keep track of groups of users. An AuthzGroup is an authorization group (a group of users, each with a role and a set of permissions of functions assigned to each role). A site contains pages; when you click on the tool menu for a given tool, normally, you will see one tool displayed in a page. However, for the home page tool, you will see more tools contained within a page.
Read more
  • 0
  • 0
  • 2015

article-image-selinux-highly-secured-web-hosting-python-based-web-applications
Packt
21 Oct 2009
10 min read
Save for later

SELinux - Highly Secured Web Hosting for Python-based Web Applications

Packt
21 Oct 2009
10 min read
When contemplating the security of a web application, there are several attack vectors that you must consider. An outsider may attack the operating system by planting a remote exploit, exercising insecure operating system settings, or brandishing some other method of privilege escalation. Or, the outsider may attack other sites contained in the same server without escalating privileges. (Note that this particular discussion does not touch upon the conditions under which an attack steals data from a single site. Instead, I'm focusing on the ability to attack different applications on the same server.) With hosts providing space for large numbers of PHP-based sites, security can be difficult as the httpd daemon traditionally runs under the same Unix user for all sites. In order to prevent these kinds of attacks from occurring, you need to concentrate on two areas: Preventing the site from reading or modifying the data of another site, and Preventing the site from escalating privileges to tamper with the operating system and bypass user-based restrictions. There are two toolboxes you use to accomplish this. In the first case, you need to find a way to run all of your sites under different Linux users. This allows the traditional Linux filesystem security model to provide protection against a hacked site attacking other sites on the same server. In the second case, you need to find a way to prevent a privilege escalation to begin with and barring that, prevent damage to the operating system should an escalation occur. Let's first take a look at a method to run different sites under different users. The Python web framework provides several versatile methods by which applications can run. There are three common methods: first, using Python's built-in http server; second, running the script as a CGI application; and third, using mod_python under Apache (similar to what mod_perl and mod_php do). These methods have various disadvantages: respectively, a lack of scalability, performance issues due to CGI application loading, and the aforementioned “all sites under one user” problem. To provide a scalable, secure, high-performance framework, you can turn to a relatively new delivery method: mod_wsgi. This Apache module, created by Graham Dumpleton, provides several methods by which you can run Python applications. In this case, we'll be focusing on the “daemon” mode of mod_wsgi. Much like mod_python, the daemon mode of mod_wsgi embeds a Python interpreter (and the requisite script) into a httpd instance. Much like with mod_python, you can configure sites based on mod_wsgi to appear at various locations in the virtual directory tree and under different virtual servers. You can also configure the number and behavior of child daemons on a per-site basis. However, there is one important difference: with mod_wsgi, you can configure each httpd instance to run as a different Linux user. During operation, the main httpd instance dispatches requests to the already-running mod_wsgi children, producing performance results that rival mod_python. But most importantly, since each httpd instance is running under a different Linux user, you can apply Linux security mechanisms to different sites running on one server. Once you have your sites running on a per-user basis, you should next turn your attention to preventing privilege escalation and protecting the operating system. By default, the Targeted mode of SELinux provided by RedHat Enterprise Linux 5 (and its free cousins such as CentOS) provides strong protection against intrusions from httpd-based applications. Because of this, you will need to configure SELinux to allow access to resources such as databases and files that reside outside of the normal httpd directories. To illustrate these concepts, I'll guide you as you install a Trac instance under mod_wsgi. The platform is CentOS 5. As a side note, it's highly recommended that you perform the installation and SELinux debugging in a XEN instance so that your environment only contains the software that is needed. The sidebar explains how to easily install the environment that was originally used to perform this exercise, and I will assume that is your primary environment. There are a few steps that require the use of a C compiler – namely, the installation of Trac – and I'll guide you through migrating these packages to your XEN-based test environment. Installing Trac In this example, you'll use a standard installation of Trac. Following the instructions provided in the URL in the Resource section, begin by installing Trac 0.10.4 with ClearSilver 0.10.5 and SilverCity 0.9.7. (Note that with many Python web applications such as Trac and Django, “installing” the application means that you're actually installing the libraries necessary for Python to run the application. You'll need to run a script to create the actual site.) Next, create a PostgreSQL user and database on a different machine. If you are using XEN for your development machine, you can use a PostgreSQL database running in your main DOM0 instance; all we are concerned with is that the PostgreSQL instance is accessed on a different machine over the network. (Note that MySQL will also work in this example, but SQLite will not. In this case, we need a database engine that is accessed over the network, not as a disk file.) After that's done, you'll need to create an actual Trac site. Create a directory under /opt, such as /opt/trac. Next, run the trac_admin command and enter the information prompted. trac-admin /opt/trac initenv Installing mod_wsgi You can find mod_wsgi at the source listed in the Resources. After you make sure the httpd_devel package is installed, installing mod_wsgi is as simple as extracting the tarball and issuing the normal ./configure and 'make install' commands. Running Trac under mod_wsgi If you look under /opt/trac, you'll notice two directories: one labeled apache, and one with the label of the project that you assigned when you installed this instance of Trac. You'll start by creating an application script in the apache directory. The application script is listed in Listing 1. Listing 1: /opt/trac/apache/trac.wsgi #!/usr/bin/python import sys sys.stdout = sys.stderr import os os.environ['TRAC_ENV'] = '/opt/trac/test_proj' import trac.web.main application = trac.web.main.dispatch_request (Note the 'sys.stdout = sys.stderr' line. This is necessary due to the way WSGI handles communications between the Python script and the httpd instance. If there is any code in the script that prints to STDOUT (such as debug messages), then the httpd instance can crash.) After creating the application script, you'll modify httpd.conf to load the wsgi module and set up the Trac application. After the LoadModule lines, insert a line for mod_wsgi: LoadModule wsgi_module modules/mod_wsgi.so Next, go to the bottom of httpd.conf and insert the text in Listing 2. This text configures the wsgi module for one particular site; it can be used under the default httpd configuration as well as under VirtualHost directives. Listing 2: Excerpt from httpd.conf: WSGIDaemonProcess trac user=trac_user group=trac_user threads=25 WSGIScriptAlias /trac /opt/trac/apache/trac.wsgi WSGIProcessGroup trac WSGISocketPrefix run/wsgi <Directory /opt/trac/apache> WSGIApplicationGroup %{GLOBAL} Order deny,allow Allow from all </Directory> Note the WSGIScriptAlias identifier. The /trac keyword (first parameter) specifies where in the directory tree the application will exist. With this configuration, If you go to your server's root address, you'll see the default CenOS splash page. If you add /trac after the address, you'll hit your Trac instance. Save the httpd.conf file. Finally, add a Linux user called trac_user. It is important that this user should not have login privileges. When the root httpd instance runs and encounters the WSGIDaemonProcess directive noted above, it will fork itself as the user specified in the directive; the fork will then load Python and the indicated script.     Securing Your Site In this section, I'll focus on the two areas noted in the introduction: User based security and SELinux. I will touch briefly on the theory of SELinux and explain the nuts and bolts of this particular implementation in more depth. I highly recommend that you read the RedHat Enterprise Linux Deployment Guide for the particulars about how RedHat implements SELinux. As with all activities involving some risk, if you plan to implement these methods, you should retain the services of a qualified security consultant to advise you about your particular situation. Setting up the user-based security is not difficult. Because the HTTPD instance containing Python and the Trac instance will run under the Trac user, you can safely set everything under /opt/trac/test_project for read and execute (for directories) for user and none for group/all. By doing this, you will isolate this site from other sites and users on the system. Now, let's configure SELinux. First, you should verify that your system is running the proper Policy and Mode. On your development system, you'll be using the Targeted policy in its Permissive mode. If you choose to move your Python applications to a production machine, you would run under the Targeted policy, in the Enforcing mode. The Targeted policy is limited to protecting the most popular network services without making the system so complex as to prevent user-level work from being done. It is the only mode that ships with RedHat 5, and by extension, CentOS 5. In Permissive mode, SELinux policy violations are trapped and sent to the audit log, but the behavior is allowed. In enforcing mode, the violation is trapped and the behavior is not allowed. To verify the Mode, run the Security Level Configuration tool from the Administration menu. The SELinux tab, shown in Figure 1, allows you to adjust the Mode. After you have verified that SELinux is running in Permissive mode, you need to do two things. First, you need to change the Type of the files under /opt/trac. Second, you need to allow Trac to connect to the Postgres database that you configured when you installed Trac. First, you need to tweak the SELinux file types attached to the files in your Trac instance. These file types dictate what processes are allowed to access them. For example, /etc/shadow has a very restrictive 'shadow' type that only allows a few applications to read and write it. By default, SELinux expects web-based applications – indeed, anything using Apache – to reside under /var/www. Files created under this directory have the SELinux Type httpd_sys_content_t. When you created the Trac instance under /opt/trac, the files were created as type usr_t. Figure 2 shows the difference between these labels To properly label the files under /opt, issue the following commands as root: cd /optchcon -R -t httpd_user_content_t trac/ After the file types are configured, there is one final step to do: allow Trac to connect to PostgreSQL. In its default state, SELinux disallows outbound network connections for the httpd type. To allow database connections, issue the following command: setsebool -P httpd_can_network_connect_db=1 In this case, we are using the -P option to make this setting persistent. If you omit this option, then the setting will be reset to its default state upon the next reboot. After the setsebool command has been run, start HTTPD by issuing the following command: /sbin/service httpd start If you visit the url http://127.0.0.1/trac, you should see the Trac screen such as that in Figure 3.    
Read more
  • 0
  • 0
  • 8965
article-image-gateways-sipxecs-40-part-1
Packt
21 Oct 2009
5 min read
Save for later

Gateways in sipXecs 4.0: Part 1

Packt
21 Oct 2009
5 min read
Gateways provide the connectivity required to reach other systems. These systems can be other sipXecs PBX's, traditional phone lines, or Internet Telephony Service Providers (ITSPs).Connecting the IP phone system to the outside world is one of the most difficult tasks in making the phone system work. If the network infrastructure is configured properly for quality of service, the connection to the outside world can most likely be the source of any call quality problems. Traditional analog Plain Old Telephone Service (POTS) lines are the largest source of frustration. If you can avoid them by utilizing a digital type of service or an ITSP, by all means take that avenue. For those not so lucky, you'll learn more about them then you ever thought you needed to. Typically, volume levels, line disconnect, and echo are the most common problems. Most gateways will have some advanced settings for dealing with these issues but they are different for every manufacturer. Adding gateways There are three types of gateways that can be configured to work in sipXecs; managed, unmanaged, and SIP Trunks. A managed gateway is a hardware device that connects to a traditional phone line. sipXecs knows how to generate configuration files (plug and play) for it. An unmanaged gateway is either a hardware device for which sipXecs doesn't know how to generate configuration files, or it may be another SIP PBX. A SIP Trunk is a connection to an ITSP. Managed gateways At present, there are eight gateways for which sipXecs generates configuration information (ACME 1000 and AudioCodes Models MP114, MP118, Mediant 1000/2000/3000/BRI, and TP260). This is just a small cross section of gateways available in the market. If your gateway is not in this list, see the following Unmanaged gateways subsection. The following detailed information about managed gateways may prove to be useful in setting up an unmanaged gateway. For the following example screens, we'll utilize an AudioCodes MP114 FXO (Foreign Exchange Office) gateway. This particular gateway has four analog ports for connecting to POTS lines. Information on the gateway is available at http://www.audiocodes.com/products/mediapack-1xx. To  add the gateway, click on the Gateways menu item in the  Devices menu. As shown in the following screenshot, there are no gateways configured by default. To add the gateway, click on the Gateways menu item in the Devices menu. As shown in the following screenshot, there are no gateways configured by default. To add a managed gateway, click on the Add new gateway drop-down box and select the appropriate gateway. The gateway configuration page will be displayed as follows: The following configuration information can be configured on this page (click on the Show Advanced Settings hyperlink to display all configuration items): Name: A name given to the gateway (no spaces). Address: The IP address of the gateway or the fully qualified hostname of the gateway (see manufacturer's documentation for information on configuring IP address and other basic settings). Port: An optional setting for UDP or TCP port if a non-standard port is used. Set to 0 to ignore this field. Transport protocol : This can be manually configured to UDP or TCP to force the SIP transport protocol. If it is set to Auto, the transport is determined through a DNS query. Serial Number : This is the Ethernet MAC address of the gateway. Firmware Version : Certain gateways may have different configuration file information or formats depending on the version of firmware in the device. Select the version of firmware that is loaded in the gateway (see manufacturer's documentation). Location: It is possible to restrict the gateway by selecting a specific location for which it can be used. A location is represented by a group of users. A user group must be created for every location that needs to be distinguished (remember that users can be in more than one group). This setting allows routing of calls based on the location or the user from which the call originates (source routing). This is useful if users located in a branch office would like to have a gateway preference so that calls are routed through their local gateway, for example, to preserve WAN bandwidth or to use caller ID offered by an analog gateway based on the PSTN number assigned to it. Only if that gateway is not available, will call routing fall back to other gateways specified for the corresponding dialing rule. Shared: If this setting is checked, this gateway can be used by any user in any location, even if a specific location is selected. This setting is checked by default so that users in an identified location still use their preferred gateway, but the gateway can also be used by other users in other locations. Description : This is for documenting the system configuration. Information about the lines connected to the gateway is very useful here. With all of the configuration information entered, click on the OK button and the Gateway page will be displayed as follows with the new gateway on it. Click on the gateway name to reveal more configuration options, as shown in the following screenshot: With all of the configuration information entered, click on the OK button and the Gateway page will be displayed as follows with the new gateway on it. Click on the gateway name to reveal more configuration options, as shown in the following screenshot: In the following subsections, we'll explore the managed gateway settings available.
Read more
  • 0
  • 0
  • 4509

article-image-installing-drupal-themes
Packt
21 Oct 2009
5 min read
Save for later

Installing Drupal Themes

Packt
21 Oct 2009
5 min read
The large and active community of developers that has formed around Drupal guarantees a steady flow of themes for this popular CMS. The diversity of that community also assures that there will be a wide variety of themes produced. Add into the equation the existence of a growing number of commercial and open source web designs and you can be certain that somewhere out there is a design that is close to what you want. The issue becomes identifying the sources of themes and designs, and determining how much work you want to do yourself. You can find both design ideas and complete themes on the Web. You need to decide whether you want to work with an existing theme, or convert a design into a theme, or whether you want to start from scratch, unburdened by any preliminary constraints or alien code. For purposes of this article, we will be dealing with finding, installing, and then uninstalling an existing and current Drupal theme. This article assumes you have a working Drupal installation, and that you have access to the files on your server. Finding Additional Themes There are several factors to consider when determining the suitability of an existing theme. The first issue is compatibility. Due to changes made to Drupal in the 5.x series, older themes will not work properly with Drupal 5.x. Accordingly, your first step is to determine which version of Drupal you are running. To find the version information for your installation, go to Administer | Logs | Status Report. The first line of the Status Report tabular data will show your version number. If you do not see the Status Report option, then you are probably using a Drupal version earlier than 5.x. We suggest you upgrade as this book is for Drupal 5.x. If you know your Drupal version, you can confirm whether the theme you are considering is usable on your system. If the theme you are looking at doesn't provide versioning information, assume the worst and make sure you back up your site before you install the questionable theme. Once you're past the compatibility hurdle, your next concern is system requirements; does the theme require any additional extensions to work properly? Some themes are ready to run with no additional extensions required. Many themes require that your Drupal installation include a particular templating engine. The most commonly required templating engine is PHPTemplate. If you are running a recent instance of Drupal, you will find that the PHPTemplate engine is installed by default. You can also download a variety of other popular templating engines, including Smarty and PHPTal from http://drupal.org/project/Theme+engines.Check carefully whether the theme you've chosen requires you to download and install other extensions. If so, track down the additional extensions and install them first, before you install your theme. A good place to start looking for a complete Drupal theme is, perhaps not surprisingly, the official Drupal site. At Drupal.org, you can find a variety of downloads, including both themes and template engines. Go to http://drupal.org/project/Themes to find a listing of the current collection of themes. All the themes state very clearly the version compatibility and whether there are any prerequisites to run the theme. In addition to the resources on the official Drupal site, there is an assortment of fan sites providing themes. Some sites are open source, others commercial, and a fair number are running unusual licenses (most frequently asking that footers be left intact with links back to their sites). Some of the themes available are great; most are average. If your firm is brand sensitive, or your design idiosyncratic, you will probably find yourself working from scratch. Regardless of your particular needs, the theme repositories are a good place to start gathering ideas. Even if you cannot find exactly what you need, you sometimes find something with which you can work. An existing set of properly formed theme files can jump start your efforts and save you a ton of time. If you wish to use an existing theme, pay attention to the terms of usage. You can save yourself (or your clients) major headaches by catching any unusual licensing provisions early in the process. There's nothing worse than spending hours on a theme only to discover its use is somehow restricted. One source for designs with livable usage policies is the Open Source Web Design site, http://www.oswd.org, which includes a repository of designs, all governed by open source licensing terms. The down side of this resource is that all you get is the design—not the code, not a ready-made theme. You will need to convert the design into a usable theme. For this article, let's search out a completed theme and for the sake of simplicity, let's take one from the official Drupal site. I am going to download the Gagarin theme from Drupal.org. I'll refer to this theme as a working example of some ofthe steps below. You can either grab a copy of the same theme or you can use another—the principles are the same regardless. Gagarin is an elegant little theme from Garamond of the Russian Drupal community. Gagarin is set up for a two-column site (though it can be run in three columns) and works particularly well for a blog site.
Read more
  • 0
  • 0
  • 2836

article-image-social-bookmarking-mediawiki
Packt
21 Oct 2009
4 min read
Save for later

Social Bookmarking - MediaWiki

Packt
21 Oct 2009
4 min read
Traditionally, bookmarking was done through Internet browsing software, such as Internet Explorer, Safari, Firefox, or Opera. With social bookmarking, your bookmarks are not confined to one browser, but are stored online. The following two options are available for enabling social bookmarking on your website: Link to each individual social bookmarking service that you wish to use Use a social bookmarking aggregator such as Socializer or AddThis Even if you do not have links to allow visitors to bookmark your website, many services allow their users to install toolbars in their browser, which allows your website to be added anyway. Adding these links will help to spread your wiki and its new skin very fast. Individual Social Bookmarking Services There are huge numbers of social bookmarking services on the Internet, and quite a number of these have become reasonably popular. We will look at some of the more popular bookmarking services such as: Mister Wong Furl Facebook Your Wiki's Audience One thing to consider before adding social bookmarking service links to your wiki is your audience. For instance, if your wiki is technology-related, you may find it better to use Digg than Facebook, as Digg is more popular than Facebook for your wiki's intended audience. If your wiki's visitors are primarily from Germany, you may find Mister Wong more useful than Furl, because Mister Wong is more popular with the German users. There are many other social bookmarking services available, including Del.icio.us (http://del.icio.us) and StumbleUpon (http://stumbleupon.com), which you can use after asking your wiki's visitors by means of a poll, or following simple experimentation. Example of Audience durhamStudent (http://durhamStudent.co.uk) is a niche website aimed at students of Durham University in the UK: The durhamStudent website uses both the methods of social bookmarking discussed earlier, providing links for eKstreme's Socializer and a link to an individual bookmarking service, Facebook: Facebook was linked individually here because it is incredibly popular among the students at Durham University (indeed, the Durham network is only open to those with a university email address). Although Facebook's bookmarking service is accessible through Socializer, it is also linked separately as the website's target audience is more likely to use that service than any other. Mister Wong Mister Wong, http://www.mister-wong.com, is popular with German and other European users (though not so much in Great Britain), and allows the users to store their bookmarks while maintaining their privacy, with the ability to set bookmarks as "public" or "private". Generally, the social bookmarking services ask for two pieces of information when creating a link from your website to them: the URL (address) of the page or website you want to add, and the title of that page or website, as you wish it to be posted. Linking to Mister Wong To link to Mister Wong, create a link where you want your social bookmark to be shown in your MediaWiki template in the following format: http://www.mister-wong.com/index.php?action=addurl&bm_url=www.example. com &bm_description=Your+Website Spaces are automatically escaped with a "+" sign by the majority of social bookmarking services. You don't have to worry about properly escaping spaces in the title of your link when linking to bookmarking services. To use these services with MediaWiki, we will be required to use some PHP. In particular, we will need the following: The page's title, which we can get with <?php $this->text('pagetitle') ?>. The website's address, retrievable with <?php $this->urlencode('serverurl') ?>. For simplicity, we will assume that our visitors will always want to bookmark JazzMeet's homepage, http://www.jazzmeet.com. Thus, the code in our MediaWiki template would appear as follows: <a href= "http://www.mister-wong.com/index.php?action=addurl&bm_url=www.jazzmeet.com &bm_description=JazzMeet" title="Bookmark JazzMeet with Mister Wong">Bookmark JazzMeet with Mister Wong</a> What Mister Wong Users See If a visitor to your wiki decides to bookmark your wiki with Mister Wong, they will be greeted with a screen similar to the following, with fields for the address of the website (URL), title, related keywords ("tags"), and a comment about the website: They are also given the option to bookmark as either "public", allowing other Mister Wong users to see it, or "private", which restricts the bookmarked website to their account only.
Read more
  • 0
  • 0
  • 2078
article-image-theming-modules-drupal-6
Packt
21 Oct 2009
5 min read
Save for later

Theming Modules in Drupal 6

Packt
21 Oct 2009
5 min read
Our Target Module: What We Want Before we begin developing a module, here's a brief overview of what we want to accomplish. The module we will write in this article is the Philosophy Quotes module (philquotes will be our machine-readable name). The goal of this module will be to create a block that displays pithy philosophical quotes. We will implement the following features: Quotes should be stored along with other basic content, making it possible to add, modify, and delete this content in exactly the same way that we create other articles. Since our existing themes aren't aware of this quotes module, it must provide some default styling. We will progress through the creation of this module by first generating a new "quote" content type, and then building a theme-aware module. Creating a Custom Content Type As Drupal evolved, it incorporated an increasingly sophisticated method for defining content. Central to this system is the idea of the content type. A content type is a definition, stored in Drupal's database, of how a particular class of content should be displayed and what functionality it ought to support. Out of the box, Drupal has two defined content types: Page and Story. Pages are intended to contain content that is static, like an "About Us" or "Contact Us" page. Stories, on the other hand, are intended to contain more transient content—news items, blog postings, and so on. Creating new pages or stories is as simple as clicking on the Create Content link in the default menu. Obviously, not all content will be classified as either a page or a story, and many sites will need specialized content types to adequately represent a specific class of content. Descriptions of events, products, component descriptions, and so on might all be better accomplished with specialized content types. Our module is going to display brief quotes. These quotes shouldn't be treated like either articles or pages. For example, we wouldn't want a new quote to be displayed along with site news in the center column of our front page. Thus, our quotes module needs a custom content type. This content type will be very simple. It will have two parts: the text of the quote and the origin of the quote. For example, here's a famous quote: The life of man [is] solitary, poor, nasty, brutish, and short.—Thomas Hobbes. The text of this quote is "The life of man [is] solitary, poor, nasty, brutish, and short", and the origin in this example is Thomas Hobbes. We could have been more specific and included the title of the work (Leviathan) or even the exact page reference, edition, and so on. But all this information, in our simple example, would be treated as the quote's origin. Given the simplicity of our content type, we can simply use the built-in Drupal content type tool to create the new type. To generate even more sophisticated content types, we could install the CCK (Content Creation Kit) module, and perhaps some of the CCK extension modules. CCK provides a robust set of tools for defining custom fields, data types, and features. But here our needs are simple, so we won't need any additional modules or even any custom code to create this new content type. Using the Administration Interface to Create a Content Type The process of creating our custom content type is as simple as logging into Drupal and filling out a form. The content type tool is in Administer | Content management | Content types. There are a couple of tabs at the top of the page: Clicking the Add content type tab will load the form used to create our new content type. On this form, we need to complete the Name and Type fields—the first with a human-friendly name, and the second with a computer-readable name. Description is often helpful. In addition to these fields, there are a few other form fields under the Submission form settings and Workflow settings that we need to change. In the Submission form settings section, we will change the labels to match the terminology we have been using. Instead of Title and Body, our sections will be Origin and Text. Changing labels is a superficial change. While it changes the text that is displayed to users, the underlying data model will still refer to these fields as title and body. We will see this later in the article. In the Workflow settings section, we need to make sure that only Published is checked. By default, Promoted to front page is selected. That should be disabled unless you want new quotes to show up as content in the main section of the front page. Once the form is complete, pressing the Save content type button will create the new content type. That's all there is to it. The Create content menu should now have the option of creating a new quote: As we continue, we will create a module that displays content of type quote in a block. Before moving on, we want a few pieces of content. Otherwise, our module would have no data to display. Here's the list of quotes (as displayed on Administer | Content management | Content) that will constitute our pool of quotations for our module.  
Read more
  • 0
  • 0
  • 1855

article-image-handler-and-phase-apache-axis2
Packt
21 Oct 2009
5 min read
Save for later

Handler and Phase in Apache Axis2

Packt
21 Oct 2009
5 min read
(For more resources on Axis2, see here.) Handler In any messaging system, the interceptor has its factual meaning in the context of messaging, where it intercepts the flow of messaging and does whatever task it is assigned to do. In fact, an interceptor is the smallest execution unit in a messaging system, and an Axis2 handler is also an interceptor. Handlers in Axis are stateless, that is, they do not keep their pass execution states in the memory. A handler can be considered as a logic invoker with the input for the logic evaluation taken from the MessageContext. A Handler has both read and write access permissions to MessageContext (MC) or to an incoming SOAP message. We can consider MessageContext as a property bag that keeps incoming or outgoing messages (maybe both) and other required parameters. It may also include properties to carry the message through the execution chain. On the other hand, we can access the whole system including the system runtime, global parameters, and property service operations via the MC. In most cases, a handler only touches the header block part of the SOAP message, which will either read a header (or headers), add a header(s), or remove a header(s). (This does not mean that the handler cannot touch the SOAP body, nor does it mean that it is not going to touch the SOAP body.) During reading, if a header is targeted to a handler and is not executing properly (the message might be faulty), then it should throw an exception, and the next driver in the chain (in Axis2, it is the Axis engine) would take the necessary action. A typical SOAP message with few headers is shown in the figure given below: Any handler in Axis2 has the capability to pause the message execution, which means that the handler can terminate the message flow if it cannot continue. Reliable messaging (RM) is a good example or use case for that scenario, when it needs to pause the flow depending on some of the preconditions and the postconditions as well and it works on a message sequence. If a service invocation consists of more than one message, and if the second message comes before the first one, then the RM handler will stop (or rather pause) the execution of the message invocation corresponding to the second message until it gets the first one. And when it gets, the first message is invoked, and thereafter it invokes or resumes the second message. Writing a Simple Handler Just learning the concepts will not help us in remembering what we have discussed. For that, we need to write a handler and see how it works. Writing a handler in Axis2 is very simple. If you want to write a handler, you either have to extend the AbstractHandler class or implement the Handler interface. A simple handler that extends the AbstractHandler class will appear as follows: public class SimpleHandler extends AbstractHandler{ public SimpleHandler() { }public InvocationResponse invoke(MessageContext msgContext) throws AxisFault { //Write the processing logic here // DO something return InvocationResponse.CONTINUE; }} Note the return value of the invoke method. We can have the following three values as the return value of the invoke method: Continue: The handler thinks that the message is ready to go forward. Suspend: The handler thinks that the message cannot be sent forward since some conditions are not satisfied; so the execution is suspended. Abort: The handler thinks that there is something wrong with the message, and cannot therefore allow the message to go forward. In most cases, handlers will return InvocationResponse.CONTINUE as the return value. When a message is received by the Axis engine, it calls the invoke method of each of the handlers by passing the argument to the corresponding MessageContext. As a result of this, we can implement all the processing logic inside that method. A handler author has full access to the SOAP message, and also has the required properties to process the message via the MessageContext. In addition, if the handler is not satisfied with the invocation of some precondition, the invocation can be paused as we have discussed earlier (Suspend). If some handler suspends the execution, then it is its responsibility to store the message context, and to forward the message when the conditions are satisfied. For example, the RM handler performs in a similar manner. Phase The concept of phase is introduced by Axis2, mainly to support the dynamic ordering of handlers. A phase can be defined in a number of ways: It can be considered a logical collection of handlers. It can be considered a specific time interval in the message execution. It can be considered a bucket into which to put a handler. One can consider a phase as a handler too. A flow or an execution chain can be considered as a collection of phases. Even though it was mentioned earlier that an Axis engine calls the invoke method of a handler, that is not totally correct. In fact, what the engine really does is call the invoke method of each phase in a given flow, and then the phase will sequentially invoke all the handlers in it (refer to the following figure). As we know, we can extend AbstractHandler and create a new handler; in the same way one can extend the Phase class and then create a new phase. But remember that we need not always extend the Phase class to create a new phase. We can do it by just adding an entry into axis2.xml (All the configuration that requires starting axis2 is obtained from axis2.xml). A phase has two important methods—precondition checking and postcondition checking. Therefore, if we are writing a custom phase, we need to consider the methods that have been mentioned. However, writing a phase is not a common case; you need to know how to write a handler.
Read more
  • 0
  • 0
  • 5045
Modal Close icon
Modal Close icon