Search icon CANCEL
Subscription
0
Cart icon
Your Cart (0 item)
Close icon
You have no products in your basket yet
Arrow left icon
Explore Products
Best Sellers
New Releases
Books
Events
Videos
Audiobooks
Packt Hub
Free Learning
Arrow right icon
timer SALE ENDS IN
0 Days
:
00 Hours
:
00 Minutes
:
00 Seconds

How-To Tutorials

7019 Articles
Packt
24 Oct 2013
13 min read
Save for later

Using Media Files – playing audio files

Packt
24 Oct 2013
13 min read
(For more resources related to this topic, see here.) Playing audio files JUCE provides a sophisticated set of classes for dealing with audio. This includes: sound file reading and writing utilities, interfacing with the native audio hardware, audio data conversion functions, and a cross-platform framework for creating audio plugins for a range of well-known host applications. Covering all of these aspects is beyond the scope of this article, but the examples in this section will outline the principles of playing sound files and communicating with the audio hardware. In addition to showing the audio features of JUCE, in this section we will also create the GUI and autogenerate some other aspects of the code using the Introjucer application. Creating a GUI to control audio file playback Create a new GUI application Introjucer project of your choice, selecting the option to create a basic window. In the Introjucer application, select the Config panel, and select Modules in the hierarchy. For this project we need the juce_audio_utils module (which contains a special Component class for configuring the audio device hardware); therefore, turn ON this module. Even though we created a basic window and a basic component, we are going to create the GUI using the Introjucer application. Navigate to the Files panel and right-click (on the Mac, press control and click) on the Source folder in the hierarchy, and select Add New GUI Component… from the contextual menu. When asked, name the header MediaPlayer.h and click on Save. In the Files hierarchy, select the MediaPlayer.cpp file. First select the Class panel and change the Class name from NewComponent to MediaPlayer. We will need four buttons for this basic project: a button to open an audio file, a Play button, a Stop button, and an audio device settings button. Select the Subcomponents panel, and add four TextButton components to the editor by right-clicking to access the contextual menu. Space the buttons equally near the top of the editor, and configure each button as outlined in the table as follows: Purpose member name name text background (normal) Open file openButton open Open... Default Play/pause file playButton play Play Green Stop playback stopButton stop Stop Red Configure audio settingsButton settings Audio Settings... Default Arrange the buttons as shown in the following screenshot: For each button, access the mode pop-up menu for the width setting, and choose Subtracted from width of parent. This will keep the right-hand side of the buttons the same distance from the right-hand side of the window if the window is resized. There are more customizations to be done in the Introjucer project, but for now, make sure that you have saved the MediaPlayer.h file, the MediaPlayer.cpp file, and the Introjucer project before you open your native IDE project. Make sure that you have saved all of these files in the Introjucer application; otherwise the files may not get correctly updated in the file system when the project is opened in the IDE. In the IDE we need to replace the MainContentComponent class code to place a MediaPlayer object within it. Change the MainComponent.h file as follows: #ifndef __MAINCOMPONENT_H__#define __MAINCOMPONENT_H__#include "../JuceLibraryCode/JuceHeader.h"#include "MediaPlayer.h"class MainContentComponent : public Component{public:MainContentComponent();void resized();private:MediaPlayer player;};#endif Then, change the MainComponent.cpp file to: #include "MainComponent.h"MainContentComponent::MainContentComponent(){addAndMakeVisible (&player);setSize (player.getWidth(),player.getHeight());}void MainContentComponent::resized(){player.setBounds (0, 0, getWidth(), getHeight());} Finally, make the window resizable in the Main.cpp file, and build and run the project to check that the window appears as expected. Adding audio file playback support Quit the application and return to the Introjucer project. Select the MediaPlayer.cpp file in the Files panel hierarchy and select its Class panel. The Parent classes setting already contains public Component. We are going to be listening for state changes from two of our member objects that are ChangeBroadcaster objects. To do this, we need our MediaPlayer class to inherit from the ChangeListener class. Change the Parent classes setting such that it reads: public Component, public ChangeListener Save the MediaPlayer.h file, the MediaPlayer.cpp file, and the Introjucer project again, and open it into your IDE. Notice in the MediaPlayer.h file that the parent classes have been updated to reflect this change. For convenience, we are going to add some enumerated constants to reflect the current playback state of our MediaPlayer object, and a function to centralize the change of this state (which will, in turn, update the state of various objects, such as the text displayed on the buttons). The ChangeListener class also has one pure virtual function, which we need to add. Add the following code to the [UserMethods] section of MediaPlayer.h: //[UserMethods]-- You can add your own custom methods...enum TransportState {Stopped,Starting,Playing,Pausing,Paused,Stopping};void changeState (TransportState newState);void changeListenerCallback (ChangeBroadcaster* source);//[/UserMethods] We also need some additional member variables to support our audio playback. Add these to the [UserVariables] section: //[UserVariables] -- You can add your own custom variables...AudioDeviceManager deviceManager;AudioFormatManager formatManager;ScopedPointer<AudioFormatReaderSource> readerSource;AudioTransportSource transportSource;AudioSourcePlayer sourcePlayer;TransportState state;//[/UserVariables] The AudioDeviceManager object will manage our interface between the application and the audio hardware. The AudioFormatManager object will assist in creating an object that will read and decode the audio data from an audio file. This object will be stored in the ScopedPointer<AudioFormatReaderSource> object. The AudioTransportSource object will control the playback of the audio file and perform any sampling rate conversion that may be required (if the sampling rate of the audio file differs from the audio hardware sampling rate). The AudioSourcePlayer object will stream audio from the AudioTransportSource object to the AudioDeviceManager object. The state variable will store one of our enumerated constants to reflect the current playback state of our MediaPlayer object. Now add some code to the MediaPlayer.cpp file. In the [Constructor] section of the constructor, add following two lines: playButton->setEnabled (false);stopButton->setEnabled (false); This sets the Play and Stop buttons to be disabled (and grayed out) initially. Later, we enable the Play button once a valid file is loaded, and change the state of each button and the text displayed on the buttons, depending on whether the file is currently playing or not. In this [Constructor] section you should also initialize the AudioFormatManager as follows: formatManager.registerBasicFormats(); This allows the AudioFormatManager object to detect different audio file formats and create appropriate file reader objects. We also need to connect the AudioSourcePlayer, AudioTransportSource and AudioDeviceManager objects together, and initialize the AudioDeviceManager object. To do this, add the following lines to the [Constructor] section: sourcePlayer.setSource (&transportSource);deviceManager.addAudioCallback (&sourcePlayer);deviceManager.initialise (0, 2, nullptr, true); The first line connects the AudioTransportSource object to the AudioSourcePlayer object. The second line connects the AudioSourcePlayer object to the AudioDeviceManager object. The final line initializes the AudioDeviceManager object with: The number of required audio input channels (0 in this case). The number of required audio output channels (2 in this case, for stereo output). An optional "saved state" for the AudioDeviceManager object (nullptr initializes from scratch). Whether to open the default device if the saved state fails to open. As we are not using a saved state, this argument is irrelevant, but it is useful to set this to true in any case. The final three lines to add to the [Constructor] section to configure our MediaPlayer object as a listener to the AudioDeviceManager and AudioTransportSource objects, and sets the current state to Stopped: deviceManager.addChangeListener (this);transportSource.addChangeListener (this);state = Stopped; In the buttonClicked() function we need to add some code to the various sections. In the [UserButtonCode_openButton] section, add: //[UserButtonCode_openButton] -- add your button handler...FileChooser chooser ("Select a Wave file to play...",File::nonexistent,"*.wav");if (chooser.browseForFileToOpen()) {File file (chooser.getResult());readerSource = new AudioFormatReaderSource(formatManager.createReaderFor (file), true);transportSource.setSource (readerSource);playButton->setEnabled (true);}//[/UserButtonCode_openButton] When the openButton button is clicked, this will create a FileChooser object that allows the user to select a file using the native interface for the platform. The types of files that are allowed to be selected are limited using the wildcard *.wav to allow only files with the .wav file extension to be selected. If the user actually selects a file (rather than cancels the operation), the code can call the FileChooser::getResult() function to retrieve a reference to the file that was selected. This file is then passed to the AudioFormatManager object to create a file reader object, which in turn is passed to create an AudioFormatReaderSource object that will manage and own this file reader object. Finally, the AudioFormatReaderSource object is connected to the AudioTransportSource object and the Play button is enabled. The handlers for the playButton and stopButton objects will make a call to our changeState() function depending on the current transport state. We will define the changeState() function in a moment where its purpose should become clear. In the [UserButtonCode_playButton] section, add the following code: //[UserButtonCode_playButton] -- add your button handler...if ((Stopped == state) || (Paused == state))changeState (Starting);else if (Playing == state)changeState (Pausing);//[/UserButtonCode_playButton] This changes the state to Starting if the current state is either Stopped or Paused, and changes the state to Pausing if the current state is Playing. This is in order to have a button with combined play and pause functionality. In the [UserButtonCode_stopButton] section, add the following code: //[UserButtonCode_stopButton] -- add your button handler...if (Paused == state)changeState (Stopped);elsechangeState (Stopping);//[/UserButtonCode_stopButton] This sets the state to Stopped if the current state is Paused, and sets it to Stopping in other cases. Again, we will add the changeState() function in a moment, where these state changes update various objects. In the [UserButtonCode_settingsButton] section add the following code: //[UserButtonCode_settingsButton] -- add your button handler...bool showMidiInputOptions = false;bool showMidiOutputSelector = false;bool showChannelsAsStereoPairs = true;bool hideAdvancedOptions = false;AudioDeviceSelectorComponent settings (deviceManager,0, 0, 1, 2,showMidiInputOptions,showMidiOutputSelector,showChannelsAsStereoPairs,hideAdvancedOptions);settings.setSize (500, 400);DialogWindow::showModalDialog(String ("Audio Settings"),&settings,TopLevelWindow::getTopLevelWindow (0),Colours::white,true); //[/UserButtonCode_settingsButton] This presents a useful interface to configure the audio device settings. We need to add the changeListenerCallback() function to respond to changes in the AudioDeviceManager and AudioTransportSource objects. Add the following to the [MiscUserCode] section of the MediaPlayer.cpp file: //[MiscUserCode] You can add your own definitions...void MediaPlayer::changeListenerCallback (ChangeBroadcaster* src){if (&deviceManager == src) {AudioDeviceManager::AudioDeviceSetup setup;deviceManager.getAudioDeviceSetup (setup);if (setup.outputChannels.isZero())sourcePlayer.setSource (nullptr);elsesourcePlayer.setSource (&transportSource);} else if (&transportSource == src) {if (transportSource.isPlaying()) {changeState (Playing);} else {if ((Stopping == state) || (Playing == state))changeState (Stopped);else if (Pausing == state)changeState (Paused);}}}//[/MiscUserCode] If our MediaPlayer object receives a message that the AudioDeviceManager object changed in some way, we need to check that this change wasn't to disable all of the audio output channels, by obtaining the setup information from the device manager. If the number of output channels is zero, we disconnect our AudioSourcePlayer object from the AudioTransportSource object (otherwise our application may crash) by setting the source to nullptr. If the number of output channels becomes nonzero again, we reconnect these objects. If our AudioTransportSource object has changed, this is likely to be a change in its playback state. It is important to note the difference between requesting the transport to start or stop, and this change actually taking place. This is why we created the enumerated constants for all the other states (including transitional states). Again we issue calls to the changeState() function depending on the current value of our state variable and the state of the AudioTransportSource object. Finally, add the important changeState() function to the [MiscUserCode] section of the MediaPlayer.cpp file that handles all of these state changes: void MediaPlayer::changeState (TransportState newState){if (state != newState) {state = newState;switch (state) {case Stopped:playButton->setButtonText ("Play");stopButton->setButtonText ("Stop");stopButton->setEnabled (false);transportSource.setPosition (0.0);break;case Starting:transportSource.start();break;case Playing:playButton->setButtonText ("Pause");stopButton->setButtonText ("Stop");stopButton->setEnabled (true);break;case Pausing:transportSource.stop();break;case Paused:playButton->setButtonText ("Resume");stopButton->setButtonText ("Return to Zero");break;case Stopping:transportSource.stop();break;}}} After checking that the newState value is different from the current value of the state variable, we update the state variable with the new value. Then, we perform the appropriate actions for this particular point in the cycle of state changes. These are summarized as follows: In the Stopped state, the buttons are configured with the Play and Stop labels, the Stop button is disabled, and the transport is positioned to the start of the audio file. In the Starting state, the AudioTransportSource object is told to start. Once the AudioTransportSource object has actually started playing, the system will be in the Playing state. Here we update the playButton button to display the text Pause, ensure the stopButton button displays the text Stop, and we enable the Stop button. If the Pause button is clicked, the state becomes Pausing, and the transport is told to stop. Once the transport has actually stopped, the state changes to Paused, the playButton button is updated to display the text Resume and the stopButton button is updated to display Return to Zero. If the Stop button is clicked, the state is changed to Stopping, and the transport is told to stop. Once the transport has actually stopped, the state changes to Stopped (as described in the first point). If the Return to Zero button is clicked, the state is changed directly to Stopped (again, as previously described). When the audio file reaches the end of the file, the state is also changed to Stopped. Build and run the application. You should be able to select a .wav audio file after clicking the Open... button, play, pause, resume, and stop the audio file using the respective buttons, and configure the audio device using the Audio Settings… button. The audio settings window allows you to select the input and output device, the sample rate, and the hardware buffer size. It also provides a Test button that plays a tone through the selected output device. Summary This article has covered a few of the techniques for dealing with audio files in JUCE. The article has given only an introduction to get you started; there are many other options and alternative approaches, which may suit different circumstances. The JUCE documentation will take you through each of these and point you to related classes and functions. Resources for Article: Further resources on this subject: Quick start – media files and XBMC [Article] Audio Playback [Article] Automating the Audio Parameters – How it Works [Article]
Read more
  • 0
  • 0
  • 2441

article-image-planning-characters-look
Packt
23 Dec 2013
10 min read
Save for later

Planning a character's look

Packt
23 Dec 2013
10 min read
(For more resources related to this topic, see here.) Creating a character is not an easy job, it can be done by creative insight or careful calculation. The main hero is the soul of the game and whose avatar is used by a player to explore the game world. Players identify themselves with that picture on the screen, empathizing with it, enjoying, grumbling, and taking all the situations to heart. Therefore, the objective is to generate an emotional connection between the character and the player. It is not necessary to provoke only positive feelings; ironically, sometimes the player can hate his avatar. A graphic look of the character can induce a wide spectrum of emotions, and these emotions can be calculated in advance. This is based on the fact that humans usually have nearly the same reactions; in this case, the rules of their behavior are pretty conspicuous. To illustrate the principles, I created the Scale of attractiveness, made of four main entries. At the very left, there is the Cute mark, in the middle is Brutal, followed by Human likeness, and to the right we see Scary. So, the chart begins with something very adorable and sweet but end with a scary character. Each position has its own collection of qualities. Making characters cute Cuteness is one of the most popular and demanded features that game designers want for their creations. Protagonists of majority casual games are cute to some degree; they look and act as sweet and comely creatures. It is simply impossible to ignore them and not fall in love with them. Recall the famous image of Nintendo's Mario, he is 100 percent cute. What is the secret of such popularity? First, the cuteness is not about beauty (it is hard to call an alligator's baby truly attractive, but it definitely is cute), which depends on personal taste and preferences, but about some basic patterns and proportions. Here is a citation from Natalie Angie's article The Cute Factor published in The New York Times: "Scientists who study the evolution of visual signaling have identified a wide and still expanding assortment of features and behaviors that make something look cute: bright forward-facing eyes set low on a big round face, a pair of big round ears, floppy limbs and a side-to-side, teeter-totter gait, among many others." All listed features are general descriptions of one class of creatures on Earth: little babies and cubs. They are small, their heads are noticeably bigger than their body, their limbs are short, eyes are large, and so on. When we see something like that, a special system inside us tends to react commonly. It says that probably in front of us is a defenseless young creature that needs protection, care, and tenderness; a list of positive senses is switched on. Figuratively, we are filled with light. By introducing a cute character in a game or other media, the authors simply exploits one of the natural human reactions. This is possible because it is pretty unconditional, the brain only needs some basic patterns, and the factual meaning of an object is totally irrelevant in this case. Thus, we consider something as cute despite the fact it is not a baby at all. Kittens are super cute, but adult cats can be cute too because they are small, have round and smooth bodies, and big eyes. Another popular example is owls, they have big round heads and large expressive eyes, making them one of the cutest birds on the planet. Moreover, some mechanical objects are cute as well: majority European compact cars from the 1950s are adorable, remember the BMW Isetta, Fiat 500, original Mini, and VW Beetle? All of them look so nice and sweet, that you want to hug them, cover them with a plaid, and give some milk in a plate, as though they are small mechanical babies of bigger adult cars. The industrial design in that period was inclined toward cuteness (may be it correlated with the baby boom). Even utility vehicle such as buses and trucks were cute, in addition to household devices such as radio sets and refrigerators. But the most amazing thing was cute weapons. Of course, I'm not talking about the real ones, but imaginary ray guns that appeared in sci-fi art and in the form of toys were definitely adorable. It is clear why illustrators prefer to use objects from previous decades in their pictures, the final illustrations look warmer. Therefore, to create a cute character, you need to follow some evident visual rules: Short body Rounded angles Curved contours and chubby figures Smooth surfaces without folds and wrinkles Big head with large forehead and small low jaw Small mouth and teeth Large eyes (or they stand wide) with big pupils Wide-open eyes with eyebrows lifted up Animals with big nose Short arms, legs, and fingers without visual joints Making characters scary Generally, cuteness is necessary for a protagonist to have a corner in the player's heart. An antagonist must give birth to opposite feelings: loath and fear. Good enemies are creepy characters. To choose their visual appearance, let's again exploit some ancient mechanics from the human brain. There are a lot of alarm systems that alert us when something looks or behaves suspiciously. Deep-seated fears of various forms are inside most people. A game, of course, should not provoke a real panic action, but can tickle some sensitive zones, playing with associations. Creepiness is the complete opposite of cuteness, it gives not a feeling of warmth, but that of cold. To find an effective scary factor, it is good to look at common fears. Traditionally, many people try to avoid insects or even have phobias. Attributes of such creatures are interpreted as unpleasant or frightening. The only exception is ladybugs (they are round with white dots on their head and large eyes) and butterflies, which are primarily associated with petals of bright flowers. It is important to note that in most cases, insects are not aggressive and harmful, but they remind us of creatures from our worst nightmares, giving us the creeps (few examples are mole crickets and earwigs, eek!) Besides them, there are other types of arthropods with high potential of creepiness (and some of them are really dangerous!): spiders, scorpions, and centipedes. They have adverse visual features such as jointed legs, spikes, exoskeletons, tails, multiple eyes, mandibles, and pincers. The key visual element is a gad, associated with cuts, injuries, and so on, that contradicts a cute creature's properties, which has no acute angles. Moreover, fishes, reptiles, birds, and mammals look dangerous when they show their teeth, canines, tusks, horns, clutches, or sharp beaks. A predator is frightening because it shows it threatens with its weaponry: the potential danger is pretty obvious and their current intention is questionable. Now, the eyes comes into play. If they are fixed at you and are not blinking, it is most likely that the predator is paying attention to you and that is super scary. Furthermore, the eyes can be red because the reflection of light creates such an image. Dangerous creatures are fast so they can move and attack quickly; this means their limbs are pretty long, but bodies are narrow and streamlined. The following figure shows a scary creature: Besides aggressive elements, other unpleasant properties can be used to increase the emotional impact; for example, the character can be additionally disgusting if it is covered with strange skin and even mucus. That turns on the dread of biological substances and the fear of germs and parasites. Squeamishness is one of the protection systems of a human, and sometimes it is very unconditional. Such an approach was used in Ridley Scott's science fiction classics Alien: apparently the xenomorph was inspired by various creepy creatures, including arthropods and reptiles. In addition, it had a very disgusting feature: toxic saliva was always dripping from his mouth, causing the viewers to feel terror and revulsion, a doubled negative emotion. While creating a scary character, remember some basic features it needs to express through its design: Long and skinny body Nonhumanoid structure Many angles Many legs or at least noticeable joints Acute elements, such as spikes, clutches, and horns Small head Naked eyes that stay very close and can be red Weaponry openly displayed (biological tools such as pincers, real guns, and grenades) Unpleasant skin with verruca, folds, wrinkles, and some mucus Warning color that can mean that the character is venomous Making characters brutal Brutality, at the middle of the scale of attractiveness, describes the properties of a character that should exercise some heroic duties, being a soldier or a mercenary. It is obvious that such a person cannot have cute characteristics, otherwise it would look comical. Adorable creatures are associated with something very young, but the heroic character should be an adult. He must demonstrate strength and confidence with a little aggression. So, his look should be a little scary, but only a little, as far as he is not a creepy creature from the end of the scale. Since the brutal hero performs various acts of bravery, he must be fast and agile. So, his anatomical proportions should be close to hyperbolic athletic ones like heavy action heroes from the 1980s, featuring a well-developed muscular system and military toys. The following figure shows such a heavy action hero: The apparent illustration of a brutal character is Duke Nukem, a protagonist of the game series of the same name originally developed by Apogee Software. He is brutal and fearless, and definitely not cute. A bunch of good examples is included in the game Gears of War from Epic Games. The members of Delta Squad are canonically brutal and tough guys. Such type of characters generally are used in action games such as shooters. Figuratively speaking, they are mix made of a human and an armed vehicle, since they hold heavy weapons and armor. The following are the basic visual properties of such a hero (it is important to note that the brutality is gender independent, and although many such characters are men, nobody has forbidden you from creating a strong woman protagonist): Figures with no acute angles or rounded corners, but ones that are roughed down Strong legs Heavy feet to lean on the ground reliably Big hands with tenacious fingers to hold weapons and other objects Wide chest with ram-like powerful shoulders that are bigger than the legs Normal head with mid-sized forehead and a big low jaw Naked big eyes It is pretty apparent that the extreme position on the scale is too categorical a benchmark to follow, whereas in most cases, nobody needs super cute or extremely scary characters. Something more universal is a mix of different properties, which is far more expressive. It is interesting that, as a rule, designs of a protagonist are situated between cuteness and brutality, so the character is anatomically adapted to make complex actions: move fast, climb ladders, fight, among others, and at the same time being pretty attractive. Summary The game lets you create a universe and settles it with characters, describing the rules of their behavior. This is a real gift for your imagination! And it is not about realism. The game universe can be based on your own principles and artistic taste. It can be cartoon like or gloomy and dark. It all depends on the mood of the story you are going to tell the players. Now, you know how to create a cute character and why you should be careful with real anatomical proportions. If the animation does not look scary, you can easily animate the protagonist and other characters. Resources for Article: Further resources on this subject: Development of iPhone Applications [Article] iPhone JavaScript: Installing Frameworks [Article] iPhone User Interface: Starting, Stopping, and Multitasking Applications [Article]
Read more
  • 0
  • 0
  • 2440

article-image-amazon-simpledb-versus-rdbms
Packt
08 Jun 2010
7 min read
Save for later

Amazon SimpleDB versus RDBMS

Packt
08 Jun 2010
7 min read
(For more resources on SimpleDB, see here.) We have all used a Relational Database Management System (RDBMS) at some point in our careers. These relational databases are ubiquitous and are available from a wide range of companies such as Oracle, Microsoft, IBM, and so on. These databases have served us well for our application needs. However, there is a new breed of applications coming to the forefront in the current Internet-driven and socially networked economy. The new applications require large scaling to meet demand peaks that can quickly reach massive levels. This is a scenario that is hard to satisfy using a traditional relational database, as it is impossible to requisition and provision the hardware and software resources that will be needed to service the demand peaks. It is also non-trivial and difficult to scale a normal RDBMS to hundreds or thousands of nodes. The overwhelming complexity of doing this makes the RDBMS not viable for these kinds of applications. SimpleDB provides a great alternative to an RDBMS and can provide a solution to all these problems. However, in order to provide this solution, SimpleDB makes some choices and design decisions that you need to understand in order to make an informed choice about the data storage for your application domain. No normalization Normalization is a process of organizing data efficiently in a relational database by eliminating redundant data, while at the same time ensuring that the data dependencies make sense. SimpleDB data models do not conform to any of the normalization forms, and tend to be completely de-normalized. The lack of need for normalization in SimpleDB allows you a great deal of flexibility with your model, and enables you to use the power of multi-valued attributes in your data. Let's look at a simple example of a database starting with a basic spreadsheet structure and then design it for an RDBMS and a SimpleDB. In this example, we will create a simple contact database, with contact information as raw data. ID First_Name Last_Name Phone_Num 101 John Smith 555-845-7854 101 John Smith 555-854-9885 101 John Smith 555-695-7485 102 Bill Jones 555-748-7854 102 Bill Jones 555-874-8654 The obvious issue is the repetition of the name data. The table is inefficient and would require care to update to keep the name data in sync. To find a person by his or her phone number is easy. SELECT * FROM Contact_Info WHERE Phone_Num = '555-854-9885' So let's analyze the strengths and weaknesses of this database design. SCORE-Raw data Strength Weakness Efficient storage   No Efficient search by phone number Yes   Efficient search by name   No Easy-to-add another phone number Yes   The design is simple, but as the name data is repeated, it would require care to keep the data in sync. Searching for phone numbers by name would be ugly if the names got out of sync. To improve the design, we can rationalize the data. One approach would be to create multiple phone number fields such as the following. While this is a simple solution, it does limit the phone numbers to three. Add e-mail and Twitter, and the table becomes wider and wider. ID First_Name Last_Name Phone_Num_1 Phone_Num_2 Phone_Num_3 101 John Smith 555-845-7854 555-854-9885 555-695-7485 102 Bill Jones 555-748-7854 555-874-8654   Finding a person by a phone number is ugly. SELECT * FROM Contact_Info WHERE Phone_Num_1 = '555-854-9885'OR Phone_Num_2 = '555-854-9885'OR Phone_Num_3 = '555-854-9885' Now let's analyze the strengths and weaknesses of this database design. SCORE-Rationalize data Strength Weakness Efficient storage Yes   Efficient search by phone number   No Efficient search by name Yes   Easy to add another phone number   No The design is simple, but the phone numbers are limited to three, and searching by phone number involves three index searches. Another approach would be to use a delimited list for the phone number as follows: ID First_Name Last_Name Phone_Nums 101 John Smith 555-845-7854;555-854-9885;555-695-7485 102 Bill Jones 555-748-7854;555-874-8654 This approach has the advantage of no data repetition and is easy to maintain, compact, and extendable, but the only way to find a record by the phone number is with a substring search. SELECT * FROM Contact_Info WHERE Phone_Nums LIKE %555-854-9885% This type of SQL forces a complete table scan. Do this with a small table and no one will notice, but try this on a large database with millions of records, and the performance of the database will suffer. SCORE-Delimited Data Strength Weakness Efficient storage Yes   Efficient search by phone number   No Efficient search by name Yes   Easy to add another phone number Yes   A delimited field is good for data that is of one type and will only be retrieved. The normalization for relational databases results in splitting up your data into separate tables that are related to one another by keys. A join is an operation that allows you to retrieve the data back easily across the multiple tables. Let's first normalize this data. This is the Person_Info table: ID First_Name Last_Name 101 John Smith 102 Bill Jones And this is the Phone_Info table: ID Phone_Num 101 555-845-7854 101 555-854-9885 101 555-695-7485 102 555-748-7854 102 555-874-8654 Now a join of the Person_Info table with the Phone_Info can retrieve the list of phone numbers as well as the e-mail addresses. The table structure is clean and other than the ID primary key, no data is duplicated. Provided Phone_Num is indexed, retrieving a contact by the phone number is efficient. SELECT First_Name, Last_Name, Phone_num, Person_Info.IDFROM Person_Info JOIN Phone_InfoON Person_Info.ID = Phone_Info.IDWHERE Phone_Num = '555-854-9885' So if we analyze the strengths and weaknesses of this database design, we get: SCORE-Relational Data Strength Weakness Efficient storage Yes   Efficient search by phone number Yes   Efficient search by name Yes   Easy to add another phone number Yes   While this is an efficient relational model, there is no join command in SimpleDB. Using two tables would force two selects to retrieve the complete contact information. Let's look at how this would be done using the SimpleDB principles. No joins SimpleDB does not support the concept of joins. Instead, SimpleDB provides you with the ability to store multiple values for an attribute, thus avoiding the necessity to perform a join to retrieve all the values. ID       101 First_Name=John Last_Name=Smith Phone_Num =555-845-7854Phone_Num =555-854-9885Phone_Num =555-695-7485 102 First_Name=Bill Last_Name=Jones Phone_Num =555-748-7854Phone_Num =555-874-8654 In the SimpleDB table, each record is stored as an item with attribute/value pairs. The difference here is that the Phone_Num field has multiple values. Unlike a delimited list field, SimpleDB indexes all values enabling an efficient search each value. SELECT * FROM Contact_Info WHERE Phone_Num = '555-854-9885' This SELECT is very quick and efficient. It is even possible to use Phone_Num multiple times such as follows: SELECT * FROM Contact_Info WHERE Phone_Num = '555-854-9885'OR Phone_Num = '555-748-7854' Let's analyze the strengths and weaknesses of this approach: SCORE-SimpleDB Data Strength Weakness Efficient storage Yes   Efficient search by phone number Yes   Efficient search by name Yes   Easy to add another phone number Yes  
Read more
  • 0
  • 0
  • 2437

article-image-slider-dynamic-applications-using-scriptaculous-part-1
Packt
08 Oct 2009
5 min read
Save for later

Slider for Dynamic Applications using script.aculo.us (part 1)

Packt
08 Oct 2009
5 min read
Before we start exploring the slider, let me try to give you a complete picture of its functionality with a simple example. Google Finance uses a horizontal slider, showing the price at a given day, month, and year. Although this particular module is built in Flash, we can build a similar module using the script.aculo.us slider too. To understand the concept and how it works, look at the following screenshot: Now that we have a clear understanding of what the slider is and how it appears in UI, let's get started! First steps with slider As just explained, a slider can handle a single value or a set of values. It's important to understand at this point of time that unlike other features of script.aculo.us, a slider is used in very niche applications for a specific functionality. The slider is not just mere functionality, but is the behavior of the users and the application. A typical constructor syntax definition for the slider is shown as follows: new Control.Slider(handle, track [ , options ] ); Track mostly represents the <div> element. Handle represents the element inside the track and, as usual, a large number of options for us to fully customize our slider. For now, we will focus on understanding the concepts and fundamentals of the slider. We will surely have fun playing with code in our Code usage for the slider section. Parameters for the slider definition In this section we will look at the parameters required to define the slider constructor: track in a slider represents a range handle in a slider represents the sliding along the track, that is, within a particular range and holding the current value options in a slider are provided to fully customize our slider's look and feel as well as functionality It's time to put the theory into action. We need the appropriate markup for working with the slider. We have <div> for the track and one <div> for each handle. The resulting code should look like the snippet shown as follows: <div id="track"><div id="handle1"></div></div> It is possible to have multiple handles inside a single track. The following code snippet is a simple example: <div id="track"><div id="handle1"></div><div id="handle2"></div></div> Options with the slider Like all the wonderful features of script.aculo.us, the slider too comes with a large number of options that allow us to create multiple behaviours for the slider. They are: Axis: This defines the orientation of the slider. The direction of movement could be horizontal or vertical. By default it is horizontal. Increment: This defines the relation between value and pixels. Maximum: This is the maximum value set for the slider to move to. While using a vertical slider from top-to-bottom, the bottom most value will be the maximum. And for a horizontal slider from left-to-right, the right most value will be the maximum value. Minimum: This is the minimum value set for the slider to move to. While using a vertical slider from top-to-bottom, the top most value will be the minimum. And for a horizontal slider from left-to-right, the left most value will be the minimum value approach for horizontal slider. Range: This is the fixed bandwidth allowed for the values. Define the minimum and maximum values. Values: Instead of a range, pass a set of values as an array. SliderValue: This sets the initial value of the slider. If not set, will take the extreme value of the slide as the default value. Disabled: As the name suggests, this disables the slider functionality. Some of the functions offered by the slider are: setValue:This will set the value of the slider directly and move it to the value position. setDisabled: This defines that the slider is disabled at runtime. setEnabled: This can enable the slider at runtime. Some of the callbacks supported by the slider are: onSlide: This is initiated on every slide movement. The called function would get the current slider value as parameter onChange: Whenever the value of the slider is changed, the called function is invoked. The value can change due to the slider movement or by passing the setValue function. Types of slider script.aculo.us provides us the flexibility and comfort of two different orientations for the slider: Vertical slider Horizontal slider Vertical slider When the axis orientation of a slider is defined as vertical, the slider becomes and acts as a vertical slider. Horizontal slider When the axis orientation of a slider is defined as horizontal, the slider becomes and acts as a horizontal slider. So let's get our hands dirty with code and start defining the constructors for horizontal and vertical slider with options. Trust me this will be fun.
Read more
  • 0
  • 0
  • 2437

article-image-building-content-management-system
Packt
25 Sep 2014
25 min read
Save for later

Building a Content Management System

Packt
25 Sep 2014
25 min read
In this article by Charles R. Portwood II, the author of Yii Project Blueprints, we will look at how to create a feature-complete content management system and blogging platform. (For more resources related to this topic, see here.) Describing the project Our CMS can be broken down into several different components: Users who will be responsible for viewing and managing the content Content to be managed Categories for our content to be placed into Metadata to help us further define our content and users Search engine optimizations Users The first component of our application is the users who will perform all the tasks in our application. For this application, we're going to largely reuse the user database and authentication system. In this article, we'll enhance this functionality by allowing social authentication. Our CMS will allow users to register new accounts from the data provided by Twitter; after they have registered, the CMS will allow them to sign-in to our application by signing in to Twitter. To enable us to know if a user is a socially authenticated user, we have to make several changes to both our database and our authentication scheme. First, we're going to need a way to indicate whether a user is a socially authenticated user. Rather than hardcoding a isAuthenticatedViaTwitter column in our database, we'll create a new database table called user_metadata, which will be a simple table that contains the user's ID, a unique key, and a value. This will allow us to store additional information about our users without having to explicitly change our user's database table every time we want to make a change: ID INTEGER PRIMARY KEYuser_id INTEGERkey STRINGvalue STRINGcreated INTEGERupdated INTEGER We'll also need to modify our UserIdentity class to allow socially authenticated users to sign in. To do this, we'll be expanding upon this class to create a RemoteUserIdentity class that will work off the OAuth codes that Twitter (or any other third-party source that works with HybridAuth) provide to us rather than authenticating against a username and password. Content At the core of our CMS is our content that we'll manage. For this project, we'll manage simple blog posts that can have additional metadata associated with them. Each post will have a title, a body, an author, a category, a unique URI or slug, and an indication whether it has been published or not. Our database structure for this table will look as follows: ID INTEGER PRIMARY KEYtitle STRINGbody TEXTpublished INTEGERauthor_id INTEGERcategory_id INTEGERslug STRINGcreated INTEGERupdated INTEGER Each post will also have one or many metadata columns that will further describe the posts we'll be creating. We can use this table (we’ll call it content_metadata) to have our system store information about each post automatically for us, or add information to our posts ourselves, thereby eliminating the need to constantly migrate our database every time we want to add a new attribute to our content: ID INTEGER PRIMARY KEYcontent_id INTEGERkey STRINGvalue STRINGcreated INTEGERupdated INTEGER Categories Each post will be associated with a category in our system. These categories will help us further refine our posts. As with our content, each category will have its own slug. Before either a post or a category is saved, we'll need to verify that the slug is not already in use. Our table structure will look as follows: ID INTEGER PRIMARY KEYname STRINGdescription TEXTslug STRINGcreated INTEGERupdated INTEGER Search engine optimizations The last core component of our application is optimization for search engines so that our content can be indexed quickly. SEO is important because it increases our discoverability and availability both on search engines and on other marketing materials. In our application, there are a couple of things we'll perform to improve our SEO: The first SEO enhancement we'll add is a sitemap.xml file, which we can submit to popular search engines to index. Rather than crawl our content, search engines can very quickly index our sitemap.xml file, which means that our content will show up in search engines faster. The second enhancement we'll be adding is the slugs that we discussed earlier. Slugs allow us to indicate what a particular post is about directly from a URL. So rather than have a URL that looks like http://chapter6.example.com/content/post/id/5, we can have URL's that look like: http://chapter6.example.com/my-awesome-article. These types of URLs allow search engines and our users to know what our content is about without even looking at the content itself, such as when a user is browsing through their bookmarks or browsing a search engine. Initializing the project To provide us with a common starting ground, a skeleton project has been included with the project resources for this article. Included with this skeleton project are the necessary migrations, data files, controllers, and views to get us started with developing. Also included in this skeleton project are the user authentication classes. Copy this skeleton project to your web server, configure it so that it responds to chapter6.example.com as outlined at the beginning of the article, and then perform the following steps to make sure everything is set up: Adjust the permissions on the assets and protected/runtime folders so that they are writable by your web server. In this article, we'll once again use the latest version of MySQL (at the time of writing MySQL 5.6). Make sure that your MySQL server is set up and running on your server. Then, create a username, password, and database for our project to use, and update your protected/config/main.php file accordingly. For simplicity, you can use ch6_cms for each value. Install our Composer dependencies: Composer install Run the migrate command and install our mock data: php protected/yiic.php migrate up --interactive=0psql ch6_cms -f protected/data/postgres.sql Finally, add your SendGrid credentials to your protected/config/params.php file: 'username' => '<username>','password' => '<password>','from' => 'noreply@ch6.home.erianna.net') If everything is loaded correctly, you should see a 404 page similar to the following: Exploring the skeleton project There are actually a lot of different things going on in the background to make this work even if this is just a 404 error. Before we start doing any development, let's take a look at a few of the classes that have been provided in our skeleton project in the protected/components folder. Extending models from a common class The first class that has been provided to us is an ActiveRecord extension called CMSActiveRecord that all of our models will stem from. This class allows us to reduce the amount of code that we have to write in each class. For now, we'll simply add CTimestampBehavior and the afterFind() method to store the old attributes for the time the need arises to compare the changed attributes with the new attributes: class CMSActiveRecordCMSActiveRecord extends CActiveRecord{public $_oldAttributes = array();public function behaviors(){return array('CTimestampBehavior' => array('class' => 'zii.behaviors.CTimestampBehavior','createAttribute' => 'created','updateAttribute' => 'updated','setUpdateOnCreate' => true));}public function afterFind(){if ($this !== NULL)$this->_oldAttributes = $this->attributes;return parent::afterFind();}} Creating a custom validator for slugs Since both Content and Category classes have slugs, we'll need to add a custom validator to each class that will enable us to ensure that the slug is not already in use by either a post or a category. To do this, we have another class called CMSSlugActiveRecord that extends CMSActiveRecord with a validateSlug() method that we'll implement as follows: class CMSSLugActiveRecord extends CMSActiveRecord{public function validateSlug($attributes, $params){// Fetch any records that have that slug$content = Content::model()->findByAttributes(array('slug' =>$this->slug));$category = Category::model()->findByAttributes(array('slug' =>$this->slug));$class = strtolower(get_class($this));if ($content == NULL && $category == NULL)return true;else if (($content == NULL && $category != NULL) || ($content !=NULL && $category == NULL)){$this->addError('slug', 'That slug is already in use');return false;}else{if ($this->id == $$class->id)return true;}$this->addError('slug', 'That slug is already in use');return false;}} This implementation simply checks the database for any item that has that slug. If nothing is found, or if the current item is the item that is being modified, then the validator will return true. Otherwise, it will add an error to the slug attribute and return false. Both our Content model and Category model will extend from this class. View management with themes One of the largest challenges of working with larger applications is changing their appearance without locking functionality into our views. One way to further separate our business logic from our presentation logic is to use themes. Using themes in Yii, we can dynamically change the presentation layer of our application simply utilizing the Yii::app()->setTheme('themename) method. Once this method is called, Yii will look for view files in themes/themename/views rather than protected/views. Throughout the rest of the article, we'll be adding views to a custom theme called main, which is located in the themes folder. To set this theme globally, we'll be creating a custom class called CMSController, which all of our controllers will extend from. For now, our theme name will be hardcoded within our application. This value could easily be retrieved from a database though, allowing us to dynamically change themes from a cached or database value rather than changing it in our controller. Have a look at the following lines of code: class CMSController extends CController{public function beforeAction($action){Yii::app()->setTheme('main');return parent::beforeAction($action);}} Truly dynamic routing In our previous applications, we had long, boring URL's that had lots of IDs and parameters in them. These URLs provided a terrible user experience and prevented search engines and users from knowing what the content was about at a glance, which in turn would hurt our SEO rankings on many search engines. To get around this, we're going to heavily modify our UrlManager class to allow truly dynamic routing, which means that, every time we create or update a post or a category, our URL rules will be updated. Telling Yii to use our custom UrlManager Before we can start working on our controllers, we need to create a custom UrlManager to handle routing of our content so that we can access our content by its slug. The steps are as follows: The first change we need to make to allow for this routing is to update the components section of our protected/config/main.php file. This will tell Yii what class to use for the UrlManager component: 'urlManager' => array('class' => 'application.components.CMSUrlManager','urlFormat' => 'path','showScriptName' => false) Next, within our protected/components folder, we need to create CMSUrlManager.php: class CMSUrlManager extends CUrlManager {} CUrlManager works by populating a rules array. When Yii is bootstrapped, it will trigger the processRules() method to determine which route should be executed. We can overload this method to inject our own rules, which will ensure that the action that we want to be executed is executed. To get started, let's first define a set of default routes that we want loaded. The routes defined in the following code snippet will allow for pagination on our search and home page, enable a static path for our sitemap.xml file, and provide a route for HybridAuth to use for social authentication: public $defaultRules = array('/sitemap.xml' => '/content/sitemap','/search/<page:d+>' => '/content/search','/search' => '/content/search','/blog/<page:d+>' => '/content/index','/blog' => '/content/index','/' => '/content/index','/hybrid/<provider:w+>' => '/hybrid/index',); Then, we'll implement our processRules() method: protected function processRules() {} CUrlManager already has a public property that we can interface to modify the rules, so we'll inject our own rules into this. The rules property is the same property that can be accessed from within our config file. Since processRules() gets called on every page load, we'll also utilize caching so that our rules don't have to be generated every time. We'll start by trying to load any of our pregenerated rules from our cache, depending upon whether we are in debug mode or not: $this->rules = !YII_DEBUG ? Yii::app()->cache->get('Routes') : array(); If the rules we get back are already set up, we'll simple return them; otherwise, we'll generate the rules, put them into our cache, and then append our basic URL rules: if ($this->rules == false || empty($this->rules)) { $this->rules = array(); $this->rules = $this->generateClientRules(); $this->rules = CMap::mergearray($this->addRssRules(), $this- >rules); Yii::app()->cache->set('Routes', $this->rules); } $this->rules['<controller:w+>/<action:w+>/<id:w+>'] = '/'; $this->rules['<controller:w+>/<action:w+>'] = '/'; return parent::processRules(); For abstraction purposes, within our processRules() method, we've utilized two methods we'll need to create: generateClientRules, which will generate the rules for content and categories, and addRSSRules, which will generate the RSS routes for each category. The first method, generateClientRules(), simply loads our default rules that we defined earlier with the rules generated from our content and categories, which are populated by the generateRules() method: private function generateClientRules() { $rules = CMap::mergeArray($this->defaultRules, $this->rules); return CMap::mergeArray($this->generateRules(), $rules); } private function generateRules() { return CMap::mergeArray($this->generateContentRules(), $this- >generateCategoryRules()); } The generateRules() method, that we just defined, actually calls the methods that build our routes. Each route is a key-value pair that will take the following form: array( '<slug>' => '<controller>/<action>/id/<id>' ) Content rules will consist of an entry that is published. Have a look at the following code: private function generateContentRules(){$rules = array();$criteria = new CDbCriteria;$criteria->addCondition('published = 1');$content = Content::model()->findAll($criteria);foreach ($content as $el){if ($el->slug == NULL)continue;$pageRule = $el->slug.'/<page:d+>';$rule = $el->slug;if ($el->slug == '/')$pageRule = $rule = '';$pageRule = $el->slug . '/<page:d+>';$rule = $el->slug;$rules[$pageRule] = "content/view/id/{$el->id}";$rules[$rule] = "content/view/id/{$el->id}";}return $rules;} Our category rules will consist of all categories in our database. Have a look at the following code: private function generateCategoryRules() { $rules = array(); $categories = Category::model()->findAll(); foreach ($categories as $el) { if ($el->slug == NULL) continue; $pageRule = $el->slug.'/<page:d+>'; $rule = $el->slug; if ($el->slug == '/') $pageRule = $rule = ''; $pageRule = $el->slug . '/<page:d+>'; $rule = $el->slug; $rules[$pageRule] = "category/index/id/{$el->id}"; $rules[$rule] = "category/index/id/{$el->id}"; } return $rules; } Finally, we'll add our RSS rules that will allow RSS readers to read all content for the entire site or for a particular category, as follows: private function addRSSRules() { $categories = Category::model()->findAll(); foreach ($categories as $category) $routes[$category->slug.'.rss'] = "category/rss/id/ {$category->id}"; $routes['blog.rss'] = '/category/rss'; return $routes; } Displaying and managing content Now that Yii knows how to route our content, we can begin work on displaying and managing it. Begin by creating a new controller called ContentController in protected/controllers that extends CMSController. Have a look at the following line of code: class ContentController extends CMSController {} To start with, we'll define our accessRules() method and the default layout that we're going to use. Here's how: public $layout = 'default';public function filters(){return array('accessControl',);}public function accessRules(){return array(array('allow','actions' => array('index', 'view', 'search'),'users' => array('*')),array('allow','actions' => array('admin', 'save', 'delete'),'users'=>array('@'),'expression' => 'Yii::app()->user->role==2'),array('deny', // deny all users'users'=>array('*'),),);} Rendering the sitemap The first method we'll be implementing is our sitemap action. In our ContentController, create a new method called actionSitemap(): public function actionSitemap() {} The steps to be performed are as follows: Since sitemaps come in XML formatting, we'll start by disabling WebLogRoute defined in our protected/config/main.php file. This will ensure that our XML validates when search engines attempt to index it: Yii::app()->log->routes[0]->enabled = false; We'll then send the appropriate XML headers, disable the rendering of the layout, and flush any content that may have been queued to be sent to the browser: ob_end_clean();header('Content-type: text/xml; charset=utf-8');$this->layout = false; Then, we'll load all the published entries and categories and send them to our sitemap view: $content = Content::model()->findAllByAttributes(array('published'=> 1));$categories = Category::model()->findAll();$this->renderPartial('sitemap', array('content' => $content,'categories' => $categories,'url' => 'http://'.Yii::app()->request->serverName .Yii::app()->baseUrl)) Finally, we have two options to render this view. We can either make it a part of our theme in themes/main/views/content/sitemap.php, or we can place it in protected/views/content/sitemap.php. Since a sitemap's structure is unlikely to change, let's put it in the protected/views folder: <?php echo '<?xml version="1.0" encoding="UTF-8"?>'; ?><urlset ><?php foreach ($content as $v): ?><url><loc><?php echo $url .'/'. htmlspecialchars(str_replace('/', '', $v['slug']), ENT_QUOTES, "utf-8"); ?></loc><lastmod><?php echo date('c',strtotime($v['updated']));?></lastmod><changefreq>weekly</changefreq><priority>1</priority></url><?php endforeach; ?><?php foreach ($categories as $v): ?><url><loc><?php echo $url .'/'. htmlspecialchars(str_replace('/', '', $v['slug']), ENT_QUOTES, "utf-8"); ?></loc><lastmod><?php echo date('c',strtotime($v['updated']));?></lastmod><changefreq>weekly</changefreq><priority>0.7</priority></url><?php endforeach; ?></urlset> You can now load http://chapter6.example.com/sitemap.xml in your browser to see the sitemap. Before you make your site live, be sure to submit this file to search engines for them to index. Displaying a list view of content Next, we'll implement the actions necessary to display all of our content and a particular post. We'll start by providing a paginated view of our posts. Since CListView and the Content model's search() method already provide this functionality, we can utilize those classes to generate and display this data: To begin with, open protected/models/Content.php and modify the return value of the search() method as follows. This will ensure that Yii's pagination uses the correct variable in our CListView, and tells Yii how many results to load per page. return new CActiveDataProvider($this, array('criteria' =>$criteria,'pagination' => array('pageSize' => 5,'pageVar' =>'page'))); Next, implement the actionIndex() method with the $page parameter. We've already told our UrlManager how to handle this, which means that we'll get pretty URI's for pagination (for example, /blog, /blog/2, /blog/3, and so on): public function actionIndex($page=1){// Model Search without $_GET params$model = new Content('search');$model->unsetAttributes();$model->published = 1;$this->render('//content/all', array('dataprovider' => $model->search()));} Then we'll create a view in themes/main/views/content/all.php, that will display the data within our dataProvider: <?php $this->widget('zii.widgets.CListView', array('dataProvider'=>$dataprovider,'itemView'=>'//content/list','summaryText' => '','pager' => array('htmlOptions' => array('class' => 'pager'),'header' => '','firstPageCssClass'=>'hide','lastPageCssClass'=>'hide','maxButtonCount' => 0))); Finally, copy themes/main/views/content/all.php from the project resources folder so that our views can render. Since our database has already been populated with some sample data, you can start playing around with the results right away, as shown in the following screenshot: Displaying content by ID Since our routing rules are already set up, displaying our content is extremely simple. All that we have to do is search for a published model with the ID passed to the view action and render it: public function actionView($id=NULL){// Retrieve the data$content = Content::model()->findByPk($id);// beforeViewAction should catch thisif ($content == NULL || !$content->published)throw new CHttpException(404, 'The article you specified doesnot exist.');$this->render('view', array('id' => $id,'post' => $content));} After copying themes/main/views/content/view.php from the project resources folder into your project, you'll be able to click into a particular post from the home page. In its actions present form, this action has introduced an interesting side effect that could negatively impact our SEO rankings on search engines—the same entry can now be accessed from two URI's. For example, http://chapter6.example.com/content/view/id/1 and http://chapter6.example.com/quis-condimentum-tortor now bring up the same post. Fortunately, correcting this bug is fairly easy. Since the goal of our slugs is to provide more descriptive URI's, we'll simply block access to the view if a user tries to access it from the non-slugged URI. We'll do this by creating a new method called beforeViewAction() that takes the entry ID as a parameter and gets called right after the actionView() method is called. This private method will simply check the URI from CHttpRequest to determine how actionView was accessed and return a 404 if it's not through our beautiful slugs: private function beforeViewAction($id=NULL){// If we do not have an ID, consider it to be null, and throw a 404errorif ($id == NULL)throw new CHttpException(404,'The specified post cannot befound.');// Retrieve the HTTP Request$r = new CHttpRequest();// Retrieve what the actual URI$requestUri = str_replace($r->baseUrl, '', $r->requestUri);// Retrieve the route$route = '/' . $this->getRoute() . '/' . $id;$requestUri = preg_replace('/?(.*)/','',$requestUri);// If the route and the uri are the same, then a direct accessattempt was made, and we need to block access to the controllerif ($requestUri == $route)throw new CHttpException(404, 'The requested post cannot befound.');return str_replace($r->baseUrl, '', $r->requestUri);} Then right after our actionView starts, we can simultaneously set the correct return URL and block access to the content if it wasn't accessed through the slug as follows: Yii::app()->user->setReturnUrl($this->beforeViewAction($id)); Adding comments to our CMS with Disqus Presently, our content is only informative in nature—we have no way for our users to communicate with us what they thought about our entry. To encourage engagement, we can add a commenting system to our CMS to further engage with our readers. Rather than writing our own commenting system, we can leverage comment through Disqus, a free, third-party commenting system. Even through Disqus, comments are implemented in JavaScript and we can create a custom widget wrapper for it to display comments on our site. The steps are as follows: To begin with, log in to the Disqus account you created at the beginning of this article as outlined in the prerequisites section. Then, navigate to http://disqus.com/admin/create/ and fill out the form fields as prompted and as shown in the following screenshot: Then, add a disqus section to your protected/config/params.php file with your site shortname: 'disqus' => array('shortname' => 'ch6disqusexample',) Next, create a new widget in protected/components called DisqusWidget.php. This widget will be loaded within our view and will be populated by our Content model: class DisqusWidget extends CWidget {} Begin by specifying the public properties that our view will be able to inject into as follows: public $shortname = NULL; public $identifier = NULL; public $url = NULL; public $title = NULL; Then, overload the init() method to load the Disqus JavaScript callback and to populate the JavaScript variables with those populated to the widget as follows:public function init() public function init(){parent::init();if ($this->shortname == NULL)throw new CHttpException(500, 'Disqus shortname isrequired');echo "<div id='disqus_thread'></div>";Yii::app()->clientScript->registerScript('disqus', "var disqus_shortname = '{$this->shortname}';var disqus_identifier = '{$this->identifier}';var disqus_url = '{$this->url}';var disqus_title = '{$this->title}';/* * * DON'T EDIT BELOW THIS LINE * * */(function() {var dsq = document.createElement('script'); dsq.type ='text/javascript'; dsq.async = true;dsq.src = '//' + disqus_shortname + '.disqus.com/embed.js';(document.getElementsByTagName('head')[0] || document.getElementsByTagName('body')[0]).appendChild(dsq);})();");} Finally, within our themes/main/views/content/view.php file, load the widget as follows: <?php $this->widget('DisqusWidget', array('shortname' => Yii::app()->params['includes']['disqus']['shortname'],'url' => $this->createAbsoluteUrl('/'.$post->slug),'title' => $post->title,'identifier' => $post->id)); ?> Now, when you load any given post, Disqus comments will also be loaded with that post. Go ahead and give it a try! Searching for content Next, we'll implement a search method so that our users can search for posts. To do this, we'll implement an instance of CActiveDataProvider and pass that data to our themes/main/views/content/all.php view to be rendered and paginated: public function actionSearch(){$param = Yii::app()->request->getParam('q');$criteria = new CDbCriteria;$criteria->addSearchCondition('title',$param,'OR');$criteria->addSearchCondition('body',$param,'OR');$dataprovider = new CActiveDataProvider('Content', array('criteria'=>$criteria,'pagination' => array('pageSize' => 5,'pageVar'=>'page')));$this->render('//content/all', array('dataprovider' => $dataprovider));} Since our view file already exists, we can now search for content in our CMS. Managing content Next, we'll implement a basic set of management tools that will allow us to create, update, and delete entries: We'll start by defining our loadModel() method and the actionDelete() method: private function loadModel($id=NULL){if ($id == NULL)throw new CHttpException(404, 'No category with that IDexists');$model = Content::model()->findByPk($id);if ($model == NULL)throw new CHttpException(404, 'No category with that IDexists');return $model;}public function actionDelete($id){$this->loadModel($id)->delete();$this->redirect($this->createUrl('content/admin'));} Next, we can implement our admin view, which will allow us to view all the content in our system and to create new entries. Be sure to copy the themes/main/views/content/admin.php file from the project resources folder into your project before using this view: public function actionAdmin(){$model = new Content('search');$model->unsetAttributes();if (isset($_GET['Content']))$model->attributes = $_GET;$this->render('admin', array('model' => $model));} Finally, we'll implement a save view to create and update entries. Saving content will simply pass it through our content model's validation rules. The only override we'll be adding is ensuring that the author is assigned to the user editing the entry. Before using this view, be sure to copy the themes/main/views/content/save.php file from the project resources folder into your project: public function actionSave($id=NULL){if ($id == NULL)$model = new Content;else$model = $this->loadModel($id);if (isset($_POST['Content'])){$model->attributes = $_POST['Content'];$model->author_id = Yii::app()->user->id;if ($model->save()){Yii::app()->user->setFlash('info', 'The articles wassaved');$this->redirect($this->createUrl('content/admin'));}}$this->render('save', array('model' => $model));} At this point, you can now log in to the system using the credentials provided in the following table and start managing entries: Username Password user1@example.com test user2@example.com test Summary In this article, we dug deeper into Yii framework by manipulating our CUrlManager class to generate completely dynamic and clean URIs. We also covered the use of Yii's built-in theming to dynamically change the frontend appearance of our site by simply changing a configuration value. Resources for Article: Further resources on this subject: Creating an Extension in Yii 2 [Article] Yii 1.1: Using Zii Components [Article] Agile with Yii 1.1 and PHP5: The TrackStar Application [Article]
Read more
  • 0
  • 0
  • 2435

article-image-introduction-spring-web-application-no-time
Packt
10 Sep 2015
8 min read
Save for later

Introduction to Spring Web Application in No Time

Packt
10 Sep 2015
8 min read
 Many official Spring tutorials have both a Gradle build and a Maven build, so you will find examples easily if you decide to stick with Maven. Spring 4 is fully compatible with Java 8, so it would be a shame not to take advantage of lambdas to simplify our code base. In this article by Geoffroy Warin, author of the book Mastering Spring MVC 4, we will see some Git commands. It's a good idea to keep track of your progress and commit when you are in a stable state. (For more resources related to this topic, see here.) Getting started with Spring Tool Suite One of the best ways to get started with Spring and discover the numerous tutorials and starter projects that the Spring community offers is to download Spring Tool Suite (STS). STS is a custom version of eclipse designed to work with various Spring projects, as well as Groovy and Gradle. Even if, like me, you have another IDE that you would rather work with, we recommend that you give STS a shot because it gives you the opportunity to explore Spring's vast ecosystem in a matter of minutes with the "Getting Started" projects. So, let's visit https://Spring.io/tools/sts/all and download the latest release of STS. Before we generate our first Spring Boot project we will need to install the Gradle support for STS. You can find a Manage IDE Extensions button on the dashboard. You will then need to download the Gradle Support software in the Language and framework tooling section. Its recommend installing the Groovy Eclipse plugin along with the Groovy 2.4 compiler, as shown in the following screenshot. These will be needed later in this article when we set up acceptance tests with geb: We now have two main options to get started. The first option is to navigate to File | New | Spring Starter Project, as shown in the following screenshot. This will give you the same options as http://start.Spring.io, embedded in your IDE: The second way is to navigate to File | New | Import Getting Started Content. This will give you access to all the tutorials available on Spring.io. You will have the choice of working with either Gradle or Maven, as shown in the following screenshot: You can also check out the starter code to follow along with the tutorial, or get the complete code directly. There is a lot of very interesting content available in the Getting Started Content. It will demonstrate the integration of Spring with various technologies that you might be interested in. For the moment, we will generate a web project as shown in the preceding image. It will be a Gradle application, producing a JAR file and using Java 8. Here is the configuration we want to use: Property Value Name masterSpringMvc Type Gradle project Packaging Jar Java version 1.8 Language Java Group masterSpringMvc Artifact masterSpringMvc Version 0.0.1-SNAPSHOT Description Be creative! Package masterSpringMvc On the second screen you will be asked for the Spring Boot version you want to use and the the dependencies that should be added to the project. At the time of writing this, the latest version of Spring boot was 1.2.5. Ensure that you always check out the latest release. The latest snapshot version of Spring boot will also be available by the time you read this. If Spring boot 1.3 isn't released by then, you can probably give it a shot. One of its big features is the awesome devs tools. Refer to https://spring.io/blog/2015/06/17/devtools-in-spring-boot-1-3 for more details. At the bottom the configuration window you will see a number of checkboxes representing the various boot starter libraries. These are dependencies that can be appended to your build file. They provide autoconfigurations for various Spring projects. We are only interested in Spring MVC for the moment, so we will check only the Web checkbox. A JAR for a web application? Some of you might find it odd to package your web application as a JAR file. While it is still possible to use WAR files for packaging, it is not always the recommended practice. By default, Spring boot will create a fat JAR, which will include all the application's dependencies and provide a convenient way to start a web server using Java -jar. Our application will be packaged as a JAR file. If you want to create a war file, refer to http://spring.io/guides/gs/convert-jar-to-war/. Have you clicked on Finish yet? If you have, you should get the following project structure: We can see our main class MasterSpringMvcApplication and its test suite MasterSpringMvcApplicationTests. There are also two empty folders, static and templates, where we will put our static web assets (images, styles, and so on) and obviously our templates (jsp, freemarker, Thymeleaf). The last file is an empty application.properties file, which is the default Spring boot configuration file. It's a very handy file and we'll see how Spring boot uses it throughout this article. The last is build.gradle file, the build file that we will detail in a moment. If you feel ready to go, run the main method of the application. This will launch a web server for us. To do this, go to the main method of the application and navigate to Run as | Spring Application in the toolbar either by right-clicking on the class or clicking on the green play button in the toolbar. Doing so and navigating to http://localhost:8080 will produce an error. Don't worry, and read on. Now we will show you how to generate the same project without STS, and we will come back to all these files. Getting started with IntelliJ IntelliJ IDEA is a very popular tool among Java developers. For the past few years I've been very pleased to pay Jetbrains a yearly fee for this awesome editor. IntelliJ also has a way of creating Spring boot projects very quickly. Go to the new project menu and select the Spring Initializr project type: This will give us exactly the same options as STS. You will need to import the Gradle project into IntelliJ. we recommend generating the Gradle wrapper first (refer to the following Gradle build section). If needed, you can reimport the project by opening its build.gradle file again. Getting started with start.Spring.io Go to http://start.Spring.io to get started with start.Spring.io. The system behind this remarkable Bootstrap-like website should be familiar to you! You will see the following screenshot when you go to the previously mentioned link: Indeed, the same options available with STS can be found here. Clicking on Generate Project will download a ZIP file containing our starter project. Getting started with the command line For those of you who are addicted to the console, it is possible to curl http://start.Spring.io. Doing so will display instructions on how to structure your curl request. For instance, to generate the same project as earlier, you can issue the following command: $ curl http://start.Spring.io/starter.tgz -d name=masterSpringMvc -d dependencies=web -d language=java -d JavaVersion=1.8 -d type=gradle-project -d packageName=masterSpringMvc -d packaging=jar -d baseDir=app | tar -xzvf - % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 1255 100 1119 100 136 1014 123 0:00:01 0:00:01 --:--:-- 1015 x app/ x app/src/ x app/src/main/ x app/src/main/Java/ x app/src/main/Java/com/ x app/src/main/Java/com/geowarin/ x app/src/main/resources/ x app/src/main/resources/static/ x app/src/main/resources/templates/ x app/src/test/ x app/src/test/Java/ x app/src/test/Java/com/ x app/src/test/Java/com/geowarin/ x app/build.Gradle x app/src/main/Java/com/geowarin/AppApplication.Java x app/src/main/resources/application.properties x app/src/test/Java/com/geowarin/AppApplicationTests.Java And viola! You are now ready to get started with Spring without leaving the console, a dream come true. You might consider creating an alias with the previous command, it will help you prototype the Spring application very quickly. Summary In this article, we leveraged Spring Boot's autoconfiguration capabilities to build an application with zero boilerplate or configuration files. We configured Spring Boot tool suite, IntelliJ,and start.spring.io and how to configure it! Resources for Article: Further resources on this subject: Welcome to the Spring Framework[article] Mailing with Spring Mail[article] Creating a Spring Application [article]
Read more
  • 0
  • 0
  • 2433
Unlock access to the largest independent learning library in Tech for FREE!
Get unlimited access to 7500+ expert-authored eBooks and video courses covering every tech area you can think of.
Renews at $19.99/month. Cancel anytime
article-image-oracle-using-metadata-service-share-xml-artifacts
Packt
23 Jan 2013
11 min read
Save for later

Oracle: Using the Metadata Service to Share XML Artifacts

Packt
23 Jan 2013
11 min read
(For more resources related to this topic, see here.) The WSDL of a web service is made up of the following XML artifacts: WSDL Definition: It defines the various operations that constitute a service, their input and output parameters, and the protocols (bindings) they support. XML Schema Definition (XSD): It is either embedded within the WSDL definition or referenced as a standalone component; this defines the XML elements and types that constitute the input and output parameters. To better facilitate the exchange of data between services, as well as achieve better interoperability and re-usability, it is good practice to de?ne a common set of XML Schemas, often referred to as the canonical data model, which can be referenced by multiple services (or WSDL De?nitions). This means, we will need to share the same XML schema across multiple composites. While typically a service (or WSDL) will only be implemented by a single composite, it will often be invoked by multiple composites; so the corresponding WSDL will be shared across multiple composites. Within JDeveloper, the default behavior, when referencing a predefined schema or WSDL, is for it to add a copy of the file to our SOA project. However, if we have several composites, each referencing their own local copy of the same WSDL or XML schema, then every time that we need to change either the schema or WSDL, we will be required to update every copy. This can be a time-consuming and error-prone approach; a better approach is to have a single copy of each WSDL and schema that is referenced by all composites. The SOA infrastructure incorporates a Metadata Service (MDS), which allows us to create a library of XML artifacts that we can share across SOA composites. MDS supports two types of repositories: File-based repository: This is quicker and easier to set up, and so is typically used as the design-time MDS by JDeveloper. Database repository: It is installed as part of the SOA infrastructure. This is used at runtime by the SOA infrastructure. As you move projects from one environment to another (for example, from test to production), you must typically modify several environment-specific values embedded within your composites, such as the location of a schema or the endpoint of a referenced web service. By placing all this information within the XML artifacts deployed to MDS, you can make your composites completely agnostic of the environment they are to be deployed to. The other advantage of placing all your referenced artifacts in MDS is that it removes any direct dependencies between composites, which means that they can be deployed and started in any order (once you have deployed the artifacts to MDS). In addition, an SOA composite leverages many other XML artifacts, such as fault policies, XSLT Transformations, EDLs for event EDN event definitions, and Schematrons for validation, each of which may need to be shared across multiple composites. These can also be shared between composites by placing them in MDS. Defining a project structure Before placing all our XML artifacts into MDS, we need to define a standard file structure for our XML library. This allows us to ensure that if any XML artifact within our XML library needs to reference another XML artifact (for example a WSDL importing a schema), it can do so via a relative reference; in other words, the XML artifact doesn't include any reference to MDS and is portable. This has a number of benefits, including: OSB compatibility; the same schemas and WSDLs can be deployed to the Oracle Service Bus without modification Third-party tool compatibility; often we will use a variety of tools that have no knowledge of MDS to create/edit XML schemas, WSDLs, and so on (for example XML Spy, Oxygen) In this article, we will assume that we have defined the following directory structure under our <src> directory. Under the xmllib folder, we have defined multiple <solution> directories, where a solution (or project) is made up of one or more related composite applications. This allows each solution to maintain its XML artifacts independently. However, it is also likely that there will be a number of XML artifacts that need to be shared between different solutions (for example, the canonical data model for the organization), which in this example would go under <core>. Where we have XML artifacts shared between multiple solutions, appropriate governance is required to manage the changes to these artifacts. For the purpose of this article, the directory structure is over simpli?ed. In reality, a more comprehensive structure should be de?ned as part of the naming and deployment standards for your SOA Reference Architecture. The other consideration here is versioning; over time it is likely that multiple versions of the same schema, WSDL and so on, will require to be deployed side by side. To support this, we typically recommend appending the version number to the filename. We would also recommend that you place this under some form of version control, as it makes it far simpler to ensure that everyone is using an up-to-date version of the XML library. For the purpose of this article, we will assume that you are using Subversion. Creating a file-based MDS repository for JDeveloper Before we can reference this with JDeveloper, we need to define a connection to the file-based MDS. Getting ready By default, a file-based repository is installed with JDeveloper and sits under the directory structure: <JDeveloper Home>/jdeveloper/integration/seed This already contains the subdirectory soa, which is reserved for, and contains, artifacts used by the SOA infrastructure For artifacts that we wish to share across our applications in JDeveloper, we should create the subdirectory apps (under the seed directory); this is critical, as when we deploy the artifacts to the SOA infrastructure, they will be placed in the apps namespace We need to ensure that the content of the apps directory always contains the latest version of our XML library; as these are stored under Subversion, we simply need to check out the right portion of the Subversion project structure. How to do it... First, we need to create and populate our file-based repository. Navigate to the seed directory, and right-click and select SVN Checkout..., this will launch the Subversion Checkout window. For URL of repository, ensure that you specify the path to the apps subdirectory. For Checkout directory, specify the full pathname of the seed directory and append /apps at the end. Leave the other default values, as shown in the following screenshot, and then click on OK: Subversion will check out a working copy of the apps subfolder within Subversion into the seed directory. Before we can reference our XML library with JDeveloper, we need to define a connection to the file-based MDS. Within JDeveloper, from the File menu select New to launch the Gallery, and under Categories select General | Connections | SOA-MDS Connection from the Items list. This will launch the MDS Connection Wizard. Enter File Based MDS for Connection Name and select a Connection Type of File Based MDS. We then need to specify the MDS root folder on our local filesystem; this will be the directory that contains the apps directory, namely: <JDeveloper Home>jdeveloperintegrationseed Click on Test Connection; the Status box should be updated to Success!. Click on OK. This will create a file-based MDS connection in JDeveloper. Browse the File Based MDS connection in JDeveloper. Within JDeveloper, open the Resource Palette and expand SOA-MDS. This should contain the File Based MDS connection that we just created. Expand all the nodes down to the xsd directory, as shown in the following screenshot: If you double-click on one of the schema files, it will open in JDeveloper (in read-only mode). There's more... Once the apps directory has been checked out, it will contain a snapshot of the MDS artifacts at the point in time that you created the checkpoint. Over time, the artifacts in MDS will be modified or new ones will be created. It is important that you ensure that your local version of MDS is updated with the current version. To do this, navigate to the seed directory, right-click on apps, and select SVN Update. Creating Mediator using a WSDL in MDS In this recipe, we will show how we can create Mediator using an interface definition from a WSDL held in MDS. This approach enables us to separate the implementation of a service (a composite) from the definition of its contract (WSDL). Getting ready Make sure you have created a file-based MDS repository for JDeveloper, as described in the first recipe. Create an SOA application with a project containing an empty composite. How to do it... Drag Mediator from SOA Component Palette onto your composite. This will launch the Create Mediator wizard; specify an appropriate name (EmployeeOnBoarding in the following example), and for the Template select Interface Definition from WSDL Click on the Find Existing WSDLs icon (circled in the previous screenshot); this will launch the SOA Resource Browser. Select Resource Palette from the drop-down list (circled in the following screenshot). Select the WSDL that you wish to import and click on OK. This will return you to the Create Mediator wizard window; ensure that the Port Type is populated and click on OK. This will create Mediator based on the specified WSDL within our composite. How it works... When we import the WSDL in this fashion, JDeveloper doesn't actually make a copy of the schema; rather within the componentType file, it sets the wsdlLocation attribute to reference the location of the WSDL in MDS (as highlighted in the following screenshot). For WSDLs in MDS, the wsdlLocation attribute uses the following format: oramds:/apps/<wsdl name> Where oramds indicates that it is located in MDS, apps indicates that it is in the application namespace and <wsdl name> is the full pathname of the WSDL in MDS. The wsdlLocation doesn't specify the physical location of the WSDL; rather it is relative to MDS, which is specific to the environment in which the composite is deployed. This means that when the composite is open in JDeveloper, it will reference the WSDL in the file-based MDS, and when deployed to the SOA infrastructure, it will reference the WSDL deployed to the MDS database repository, which is installed as part of the SOA infrastructure. There's more... This method can be used equally well to create a BPEL process based on the WSDL from within the Create BPEL Process wizard; for Template select Base on a WSDL and follow the same steps. This approach works well with Contract First Design as it enables the contract for a composite to be designed first, and when ready for implementation, be checked into Subversion. The SOA developer can then perform a Subversion update on their file-based MDS repository, and then use the WSDL to implement the composite Creating Mediator that subscribes to EDL in MDS In this recipe, we will show how we can create Mediator that subscribes to an EDN event whose EDL is defined in MDS. This approach enables us to separate the definition of an event from the implementation of a composite that either subscribes to, or publishes, the event. Getting ready Make sure you have created a file-based MDS repository for JDeveloper, as described in the initial recipe. Create an SOA application with a project containing an empty composite. How to do it... Drag Mediator from SOA Component Palette onto your composite. This will launch the Create Mediator wizard; specify an appropriate name for it (UserRegistration in the following example), and for the Template select Subscribe to Events. Click on the Subscribe to new event icon (circled in the previous screenshot); this will launch the Event Chooser window. Click on the Browse for Event Definition (edl) files icon (circled in the previous screenshot); this will launch SOA Resource Browser. Select Resource Palette from the drop-down list. Select the EDL that you wish to import and click on OK. This will return you to the Event Chooser window; ensure that the required event is selected and click on OK. This will return you to the Create Mediator window; ensure that the required event is configured as needed, and click on OK. This will create an event subscription based on the EDL specified within our composite. How it works... When we reference an EDL in MDS, JDeveloper doesn't actually make a copy of the EDL; rather within the composite.xml file, it creates an import statement to reference the location of the EDL in MDS. There's more... This approach can be used equally well to subscribe to an event within a BPEL process or publish an event using either Mediator or BPEL.
Read more
  • 0
  • 0
  • 2433

article-image-photo-manipulation-gimp-26
Packt
21 Mar 2011
5 min read
Save for later

Photo Manipulation with GIMP 2.6

Packt
21 Mar 2011
5 min read
Introduction When working with photos we need to be sure that colors always look the way we want. Monitors, LCDs, and laptop screens, all render colors differently, and they don't look the same. There's no way you can control how people see their own screen, but you can calibrate your own using some kind of color profiling. The problem extends to different devices like scanners, printers, and cellphones, among others, because each device has its own color characteristics. Also, each device can handle different ranges of color. To avoid these problems, color profiles were implemented. It's a table that matches a specific color of a device to a device-independent color. Color profiles are created by each manufacturer, but the ICC (International Color Consortium) created its own standard. Calibration, image modes, and colour profiles This is one of the first things to do before working with photos. The Color Management menu (Edit | Preferences | Color Management) enables you to load any preset for your monitor. The presets can be created using any hardware calibration and measurement tool. For Linux, you can use LPROF and ARGYLL. For more information on color management in Linux, check the Linux color management page on Wikipedia. Apart from the color profile in use, you should set up your monitor and video card to render colors as close as possible to the standard. For Linux you can use XCALIB or DISPWIN. You can also simulate how an image will look when printed by selecting Print Simulation from the Mode of Operation in the Color Management menu. Scaling an image Image scaling is one of the easiest tasks, and also one of the most useful. In just a few seconds you can reduce an image's size, making e-mail transfers quicker. How to do it... To scale an image follow these steps: Open a file, go to Image | Scale Image. Change the size to whatever you want: As you change one value, you can see the other changing too. That's to keep the original file aspect ratio. If you click on any of the chain icons you can change any value independently. How it works... Image scaling works by changing the number of pixels of an image. The way pixels are created/deleted can be changed by choosing a different interpolation method in the Quality section of the Scale Image window. The best (but slowest) method is the Cubic interpolation, which calculates the color of each pixel by estimating an average of the eight closest ones. On the opposite corner, if you choose None as the interpolation method, each pixel gets its own color from its closest neighbor. This is the fastest method, but it also generates unwanted aliasing that can give you headaches if you are working with transparencies and highcontrasts. The resolution fields are useful when printing the image, they say how many pixels are per inch. If your image is too big and the resolution too low, the printed image can look pixelated. You can change the print size of an image without changing its resolution or size by going to Image | Print Size. Cropping an image There's another quick way to rescale your image and select just a section of it. Use the Crop Tool. How to do it... The following steps will show you how to crop an existing image: Open an image, and select the Crop Tool: Quickly click and drag around the area you want. Then, just press Return (Enter in Windows), and anything around the selection area will be cropped. Flipping or rotating an image Sometimes you scan and old photo or take a picture with the camera tilted. Following is how to quickly rotate or flip them. How to do it... To rotate or flip an image easily, carry out the following steps: Open an image, for example the following: Select the Flip Tool from the Toolbox: Use the left-mouse button along with the Ctrl key to change between horizontal and vertical flipping. Now, to rotate the image, select the Rotate Tool from the Toolbox: Click and drag your image to rotate it. Use the Ctrl key to constrain rotation to 15 degrees steps. Press the Return key when you've finished: How it works... Flip and rotate are basic tools. You can use the mouse to perform the operations after selecting any of them, but using their respective tool options gives you a lot more control. In both cases, you can control what to rotate (a layer, a selection, or a path). Also, in the Rotate tool, you can change the middle point around which the image rotates. This allows you to rotate the image with more freedom. To change it, select the Rotate Tool, and click and drag the small circle that appears in the center of the image: You can also set its coordinates from the Rotate window: You can also flip and rotate an image by using the menu items in Image | transform.
Read more
  • 0
  • 0
  • 2432

article-image-getting-started-drupal-6-panels
Packt
20 Aug 2010
7 min read
Save for later

Getting Started with Drupal 6 Panels

Packt
20 Aug 2010
7 min read
(For more resources on Drupal, see here.) Introduction Drupal Panels are distinct pieces of rectangular content that create a custom layout of the page—where different Panels are more visible and presentable as a structured web page. Panels is a freely-distributed, open source module developed for Drupal 6. With Panels, you can display various content in a customizable grid layout on one page. Each page created by Panels can include a unique structure and content. Using the drag-and-drop user interface, you select a design for the layout and position various kinds of content (or add custom content) within that layout. Panels integrates with other Drupal modules like Views and CCK. Permissions, deciding which users can view which elements, are also integrated into Panels. You can even override system pages such as the display of keywords (taxonomy) and individual content pages (nodes). In the next section, we will see what the Panels can actually do, as defined on drupal.org: http://drupal.org/project/panels. Basically, Panels will help you to arrange a large content on a single page. While Panels can be used to arrange a lot of content on a single page, it is equally useful for small amounts of related content and/or teasers. Panels support styles, which control how individual content's panes, regions within a Panel, and the entire Panels will be rendered. While Panels ship with few styles, styles can be provided as plugins by modules, as well as by themes: The User Interface is nice for visually designing a layout, but a real HTML guru doesn't want the somewhat weighty HTML that this will create. Modules and themes can provide custom layouts that can fit a designer's exacting specifications, but still allow the site builders to place content wherever they like. Panels include a pluggable caching mechanism: a single cache type is included, the 'simple' cache, which is time-based. Since most sites have very specific caching needs based upon the content and traffic patterns, this system was designed to let sites that need to devise their own triggers for cache clearing and implement plugins that will work with Panels. A cache mechanism can be defined for each pane or region with the Panel page. Simple caching is a time-based cache. This is a hard limit, and once cached, it will remain that way until the time limit expires. If "arguments" are selected, this content will be cached per individual argument to the entire display; if "contexts" are selected, this content will be cached per unique context in the pane or display; if "neither", there will be only one cache for this pane. Panels can also be cached as a whole, meaning the entire output of the Panels can be cached, or individual content panes that are heavy, like large views, can be cached. Panels can be integrated with the Drupal module Organic Groups through the #og_Panels module to allow individual groups to have their own customized layouts. Panels integrates Views to allow administrators to add any view as content. We will discuss Module Integration in the coming recipes. Shown in the previous screenshot is one of the example sites that use Panels 3 for their home page (http://concernfast.org). The home page is built using a custom Panels 3 layout with a couple of dedicated Content types that are used to build nodes to drop into the various Panels areas. The case study can be found at: http://drupal.org/node/629860. Panels arrange your site content into an easy navigational pattern, which can be clearly seen in the following screenshot. There are several terms often used within Panels that administrators should become familiar with as we will be using the same throughout the recipes. The common terms in Panels are: Panels page: The page that will display your Panels. This could be the front page of a site, a news page, and so on. These pages are given a path just like any other node. Panels: A container for content. A Panel can have several pieces of content within it, and can be styled. Pane: A unit of content in a Panel. This can be a node, view, arbitrary HTML code, and so on. Panes can be shifted up and down within a Panel and moved from one Panel to another. Layout: Provides a pre-defined collection of Panels that you can select from. A layout might have two columns, a header, footer, or three columns in the middle, or even seven Panels stacked like bricks. Setting up Ctools and Panels We will now set up Ctools, which is required for Panels. "Chaos tools" is a centralized library, which is used by the most powerful modules of Drupal Panels and views. Most functions in Panels are inherited from the chaos library. Getting ready Download the Panels modules for the Drupal website: http://drupal.org/project/Panels You would need Ctools as a dependency module, which can be downloaded from: http://drupal.org/project/ctools How to do it... Upload both the files, Ctools and Panels, into /sites/all/modules. It is always a best practice to keep the installed modules separate from the "core" (the files that install with Drupal) into the /sites/all/modules folder. This makes it easy to upgrade the modules at a later stage when your site becomes complex and has too many modules. Go to the modules page in admin (Admin| Site Building | Modules) and enable Ctools, then enable Panels. Go to permissions (Admin | User Management | Permissions) and give site builders permission to use Panels. Enable the Page manager module in the Chaos tools suite. This module enables the page manager for Panels. To integrate views with Panels, enable the Views content panes module too. We will discuss more about views later on. Enable Panels and set the permissions. You will need to enable Panel nodes, the Panel module, and Mini panels too (as shown in the following screenshot) as we will use the same in our advanced recipes. Go to administer by module in the Site building | Modules. Here, you find the Panels User Interface. There is more Chaos tools suite includes the following tools that form the base of the Panels module. You do not need to go into the details of it to use Panels but it is good to know what it includes. This is the powerhouse that makes Panels the most efficient tool to design complex layouts: Plugins—tools to make it easy for modules to let other modules implement plugins from .inc files. Exportables—tools to make it easier for modules to have objects that live in database or live in code, such as 'default views'. AJAX responder—tools to make it easier for the server to handle AJAX requests and tell the client what to do with them. Form tools—tools to make it easier for forms to deal with AJAX. Object caching—tool to make it easier to edit an object across multiple page requests and cache the editing work. Contexts—the notion of wrapping objects in a unified wrapper and providing an API to create and accept these contexts as input. Modal dialog—tool to make it simple to put a form in a modal dialog. Dependent—a simple form widget to make form items appear and disappear based upon the selections in another item. Content—pluggable Content types used as panes in Panels and other modules like Dashboard. Form wizard—an API to make multi-step forms much easier. CSS tools—tools to cache and sanitize CSS easily to make user input CSS safe. How it works... Now, we have our Panels UI ready to generate layouts. We will discuss each of them in the following recipes. The Panels dashboard will help you to generate the layouts for Drupal with ease.
Read more
  • 0
  • 0
  • 2432

article-image-building-facebook-clone-using-ruby
Packt
25 Aug 2010
8 min read
Save for later

Building the Facebook Clone using Ruby

Packt
25 Aug 2010
8 min read
(For more resources on Ruby, see here.) This is the largest clone and has many components. Some of the less interesting parts of the code are not listed or described here. To get access to the full source code please go to http://github.com/sausheong/saushengine. Configuring the clone We use a few external APIs in Colony so we need to configure our access to these APIs. In a Colony all these API keys and settings are stored in a Ruby file called config.rb as below. S3_CONFIG = {}S3_CONFIG['AWS_ACCESS_KEY'] = '<AWS ACCESS KEY>'S3_CONFIG['AWS_SECRET_KEY'] = '<AWS SECRET KEY>'RPX_API_KEY = '<RPX API KEY>' Modeling the data You will find a large number of classes and relationships in this article. The following diagram shows how the clone is modeled: User The first class we look at is the User class. There are more relationships with other classes and the relationship with other users follows that of a friends model rather than a followers model. class User include DataMapper::Resource property :id, Serial property :email, String, :length => 255 property :nickname, String, :length => 255 property :formatted_name, String, :length => 255 property :sex, String, :length => 6 property :relationship_status, String property :provider, String, :length => 255 property :identifier, String, :length => 255 property :photo_url, String, :length => 255 property :location, String, :length => 255 property :description, String, :length => 255 property :interests, Text property :education, Text has n, :relationships has n, :followers, :through => :relationships, :class_name => 'User', :child_key => [:user_id] has n, :follows, :through => :relationships, :class_name => 'User', :remote_name => :user, :child_key => [:follower_id] has n, :statuses belongs_to :wall has n, :groups, :through => Resource has n, :sent_messages, :class_name => 'Message', :child_key => [:user_id] has n, :received_messages, :class_name => 'Message', :child_key => [:recipient_id] has n, :confirms has n, :confirmed_events, :through => :confirms, :class_name => 'Event', :child_key => [:user_id], :date.gte => Date.today has n, :pendings has n, :pending_events, :through => :pendings, :class_name => 'Event', :child_key => [:user_id], :date.gte => Date.today has n, :requests has n, :albums has n, :photos, :through => :albums has n, :comments has n, :activities has n, :pages validates_is_unique :nickname, :message => "Someone else has taken up this nickname, try something else!" after :create, :create_s3_bucket after :create, :create_wall def add_friend(user) Relationship.create(:user => user, :follower => self) end def friends (followers + follows).uniq end def self.find(identifier) u = first(:identifier => identifier) u = new(:identifier => identifier) if u.nil? return u end def feed feed = [] + activities friends.each do |friend| feed += friend.activities end return feed.sort {|x,y| y.created_at <=> x.created_at} end def possessive_pronoun sex.downcase == 'male' ? 'his' : 'her' end def pronoun sex.downcase == 'male' ? 'he' : 'she' end def create_s3_bucket S3.create_bucket("fc.#{id}") end def create_wall self.wall = Wall.create self.save end def all_events confirmed_events + pending_events end def friend_events events = [] friends.each do |friend| events += friend.confirmed_events end return events.sort {|x,y| y.time <=> x.time} end def friend_groups groups = [] friends.each do |friend| groups += friend.groups end groups - self.groups endend As mentioned in the design section above, the data used in Colony is user-centric. All data in Colony eventually links up to a user. A user has following relationships with other models: A user has none, one, or more status updates A user is associated with a wall A user belongs to none, one, or more groups A user has none, one, or more sent and received messages A user has none, one, or more confirmed and pending attendances at events A user has none, one, or more user invitations A user has none, one, or more albums and in each album there are none, one, or more photos A user makes none, one, or more comments A user has none, one, or more pages A user has none, one, or more activities Finally of course, a user has one or more friends Once a user is created, there are two actions we need to take. Firstly, we need to create an Amazon S3 bucket for this user, to store his photos. after :create, :create_s3_bucketdef create_s3_bucket S3.create_bucket("fc.#{id}")end We also need to create a wall for the user where he or his friends can post to. after :create, :create_walldef create_wall self.wall = Wall.create self.saveend Adding a friend means creating a relationship between the user and the friend. def add_friend(user) Relationship.create(:user => user, :follower => self) end Colony treats the following relationship as a friends relationship. The question here is who will initiate the request to join? This is why when we ask the User object to give us its friends, it will add both followers and follows together and return a unique array representing all the user's friends. def friends (followers + follows).uniqend In the Relationship class, each time a new relationship is created, an Activity object is also created to indicate that both users are now friends. class Relationship include DataMapper::Resource property :user_id, Integer, :key => true property :follower_id, Integer, :key => true belongs_to :user, :child_key => [:user_id] belongs_to :follower, :class_name => 'User', :child_key => [:follower_id] after :save, :add_activity def add_activity Activity.create(:user => user, :activity_type => 'relationship', :text => "<a href='/user/#{user.nickname}'>#{user.formatted_name}</a> and <a href='/user/#{follower.nickname}'>#{follower.formatted_name}</a> are now friends.") end end Finally we get the user's news feed by taking the user's activities and going through each of the user's friends, their activities as well. def feed feed = [] + activities friends.each do |friend| feed += friend.activities end return feed.sort {|x,y| y.created_at <=> x.created_at}end Request We use a simple mechanism for users to invite other users to be their friends. The mechanism goes like this: Alice identifies another Bob whom she wants to befriend and sends him an invitation This creates a Request class which is then attached to Bob When Bob approves the request to be a friend, Alice is added as a friend (which is essentially making Alice follow Bob, since the definition of a friend in Colony is either a follower or follows another user) class Request include DataMapper::Resource property :id, Serial property :text, Text property :created_at, DateTime belongs_to :from, :class_name => User, :child_key => [:from_id] belongs_to :user def approve self.user.add_friend(self.from) endend Message Messages in Colony are private messages that are sent between users of Colony. As a result, messages sent or received are not tracked as activities in the user's activity feed. class Message include DataMapper::Resource property :id, Serial property :subject, String property :text, Text property :created_at, DateTime property :read, Boolean, :default => false property :thread, Integer belongs_to :sender, :class_name => 'User', :child_key => [:user_id] belongs_to :recipient, :class_name => 'User', :child_key => [:recipient_id] end A message must have a sender and a recipient, both of which are users. has n, :sent_messages, :class_name => 'Message', :child_key => [:user_id]has n, :received_messages, :class_name => 'Message', :child_key => [:recipient_id] The read property tells us if the message has been read by the recipient, while the thread property tells us how to group messages together for display. Album An activity is logged, each time an album is created. class Album include DataMapper::Resource property :id, Serial property :name, String, :length => 255 property :description, Text property :created_at, DateTime belongs_to :user has n, :photos belongs_to :cover_photo, :class_name => 'Photo', :child_key => [:cover_photo_id] after :save, :add_activity def add_activity Activity.create(:user => user, :activity_type => 'album', :text => "<a href='/user/#{user.nickname}'>#{user.formatted_name}</a> created a new album <a href='/album/#{self.id}'>#{self.name}</a>") end end
Read more
  • 0
  • 0
  • 2431
article-image-2d-graphics
Packt
04 Mar 2013
8 min read
Save for later

2D Graphics

Packt
04 Mar 2013
8 min read
(For more resources related to this topic, see here.) Adding content Create a new project and call it Chapter2Demo. XNA Game Studio created a class called Game1. Rename it to MainGame so it has a proper name. When we take a look at our solution, we can see two projects. A game project called Chapter2Demo that contains all our code, and a content project called Chapter2DemoContent. This content project will hold all our assets, and compile them to an intermediate file format (xnb). This is often done in game development to make sure our games start faster. The resulting files are uncompressed, and thus larger, but can be read directly into memory without extra processing. Note that we can have more than one content project in a solution. We might add one per platform, but this is beyond the scope of this article. Navigate to the content project using Windows Explorer, and place our textures in there. The start files can be downloaded from the previously mentioned link. Then add the files to the content project by right-clicking on it in the Solution Explorer and choosing the Add | Existing Item.... Make sure to place the assets in a folder called Game2D. When we click on the hero texture in the content project, we can see several properties. First of all, our texture has a name, Hero. We can use that name to load our texture in code. Note that this has no extension, because the files will be compiled to an intermediate format anyway. We can also specify a Content Importer and Content Processor. Our .png file gets recognized as texture so XNA Game studio automatically selects the Texture importer and processor for us. An importer will convert our assets into the "Content Document Object Model", a format that can be read by the processor. The processor will compile the asset into a managed code object, which can then be serialized into the intermediate .xnb file. That file will then be loaded at runtime. Drawing sprites Everything is set up for us to begin. Let's start drawing some images. We'll draw a background, an enemy, and our hero. Adding fields At the top of our MainGame, we need to add a field for each of our objects.The type used here is Texture2D. Texture2D _background, _enemy, _hero; Loading textures In the LoadContent method, we need to load our textures using the content manager. // TODO: use this.Content to load your game content here _background = Content.Load<Texture2D>("Game2D/Background"); _enemy = Content.Load<Texture2D>("Game2D/Enemy"); _hero = Content.Load<Texture2D>("Game2D/Hero"); The content manager has a generic method called Load. Generic meaning we can specify a type, in this case Texture2D. It has one argument, being the asset name. Note that you do not specify an extension, the asset name corresponds with the folder structure and then the name of the asset that you specified in the properties. This is because the content is compiled to .xnb format by our content project anyway, so the files we load with the content manager all have the same extension. Also note that we do not specify the root directory of our content, because we've set it in the game's constructor. Drawing textures Before we start drawing textures, we need to make sure our game runs in full screen. This is because the emulator has a bug and our sprites wouldn't show up correctly. You can enable full screen by adding the following code to the constructor: graphics.IsFullScreen = true; Now we can go to the Draw method. Rendering textures is always done in a specific way: First we call the SpriteBatch.Begin() method. This will make sure all the correct states necessary for drawing 2D images are set properly. Next we draw all our sprites using the Draw method of the sprite batch. This method has several overloads. The first is the texture to draw. The second an object of type Vector2D that will store the position of the object. And the last argument is a color that will tint your texture. Specify Color.White if you don't want to tint your texture. Finally we call the SpriteBatch.End() method. This will sort all sprites we've rendered (according the the specified sort mode) and actually draw them. If we apply the previous steps, they result in the following code: // TODO: Add your drawing code here spriteBatch.Begin(); spriteBatch.Draw(_background, new Vector2(0, 0), Color.White); spriteBatch.Draw(_enemy, new Vector2(10, 10), Color.White); spriteBatch.Draw(_hero, new Vector2(10, 348), Color.White); spriteBatch.End(); Run the game by pressing F5. The result is shown in the following screenshot: Refactoring our code In the previous code, we've drawn three textures from our game class. We hardcoded the positions, something we shouldn't do. None of the textures were moving but if we want to add movement now, our game class would get cluttered, especially if we have many sprites. Therefore we will refactor our code and introduce some classes. We will create two classes: a GameObject2D class that is the base class for all 2D objects, and a GameSprite class, that will represent a sprite. We will also create a RenderContext class. This class will hold our graphics device, sprite batch, and game time objects. We will use all these classes even more extensively when we begin building our own framework. Render context Create a class called RenderContext. To create a new class, do the following: Right-click on your solution. Click on Add | New Item. Select the Code template on the left. Select Class and name it RenderContext. Click on OK. This class will contain three properties: SpriteBatch, GraphicsDevice, and GameTime. We will use an instance of this class to pass to the Update and Draw methods of all our objects. That way they can access the necessary information. Make sure the class has public as access specifier. The class is very simple: public class RenderContext { public SpriteBatch SpriteBatch { get; set; } public GraphicsDevice GraphicsDevice { get; set; } public GameTime GameTime { get; set; } } When you build this class, it will not recognize the terms SpriteBatch, GraphicsDevice, and GameTime. This is because they are stored in certain namespaces and we haven't told the compiler to look for them. Luckily, XNA Game Studio can find them for us automatically. If you hover over SpriteBatch, an icon like the one in the following screenshot will appear on the left-hand side. Click on it and choose the using Microsoft.Xna.Framework.Graphics; option. This will fix the using statement for you. Do it each time such a problem arises. The base class The base class is called GameObject2D. The only thing it does is store the position, scale, and rotation of the object and a Boolean that determines if the object should be drawn. It also contains four methods: Initialize, LoadContent, Draw, and Update. These methods currently have an empty body, but objects that will inherit from this base class later on will add an implementation. We will also use this base class for our scene graph, so don't worry if it still looks a bit empty. Properties We need to create four automatic properties. The Position and the Scale parameters are of type Vector2. The rotation is a float and the property that determines if the object should be drawn is a bool. public Vector2 Position { get; set; } public Vector2 Scale { get; set; } public float Rotation { get; set; } public bool CanDraw { get; set; } Constructor In the constructor, we will set the Scale parameter to one (no scaling) and set the CanDraw parameter to true. public GameObject2D() { Scale = Vector2.One; CanDraw = true; } Methods This class has four methods. Initialize: We will create all our new objects in this method. LoadContent: This method will be used for loading our content. It has one argument, being the content manager. Update: This method shall be used for updating our positions and game logic. It also has one argument, the render context. Draw: We will use this method to draw our 2D objects. It has one argument, the render context. public virtual void Initialize() { } public virtual void LoadContent(ContentManager contentManager) { } public virtual void Update(RenderContext renderContext) { } public virtual void Draw(RenderContext renderContext) { } Summary In this Article we have got used to the 2D coordinate system. Resources for Article : Further resources on this subject: 3D Animation Techniques with XNA Game Studio 4.0 [Article] Advanced Lighting in 3D Graphics with XNA Game Studio 4.0 [Article] Environmental Effects in 3D Graphics with XNA Game Studio 4.0 [Article]
Read more
  • 0
  • 0
  • 2431

article-image-scraping-web-page
Packt
20 Jun 2017
11 min read
Save for later

Scraping a Web Page

Packt
20 Jun 2017
11 min read
In this article by Katharine Jarmul author of the book Python Web Scraping - Second Edition we can look at some example as suppose I have a shop selling shoes and want to keep track of my competitor's prices. I could go to my competitor's website each day and compare each shoe's price with my own, however this will take a lot of time and will not scale well if I sell thousands of shoes or need to check price changes frequently. Or maybe I just want to buy a shoe when it's on sale. I could come back and check the shoe website each day until I get lucky, but the shoe I want might not be on sale for months. These repetitive manual processes could instead be replaced with an automated solution using the web scraping techniques covered in this book. In an ideal world, web scraping wouldn't be necessary and each website would provide an API to share the data in a structured format. Indeed, some websites do provide APIs, but they typically restrict the data that is available and how frequently it can be accessed. Additionally, a website developer might change, remove or restrict the backend API. In short, we cannot rely on APIs to access the online data we may want and therefore, we need to learn about web scraping techniques. (For more resources related to this topic, see here.) Three approaches to scrape a web page Now that we understand the structure of this web page we will investigate three different approaches to scraping its data, first with regular expressions, then with the popular BeautifulSoup module, and finally with the powerful lxml module. Regular expressions If you are unfamiliar with regular expressions or need a reminder, there is a thorough overview available at (https://docs.python.org/3/howto/regex.html). Even if you use regular expressions (or regex) with another programming language, I recommend stepping through it for a refresher on regex with Python. To scrape the country area using regular expressions, we will first try matching the contents of the <td> element, as follows: >>> import re >>> from advanced_link_crawler import download >>> url = 'http://example.webscraping.com/view/UnitedKingdom-239' >>> html = download(url) >>> re.findall(r'(.*?)', html) ['<'img src="/places/static/images/flags/gb.png" />', '244,820 square kilometres', '62,348,447', 'GB', 'United Kingdom', 'London', 'EU', '.uk', 'GBP', 'Pound', '44', '@# #@@|@## #@@|@@# #@@|@@## #@@|@#@ #@@|@@#@ #@@|GIR0AA', '^(([A-Z]\d{2}[A-Z]{2})|([A-Z]\d{3}[A-Z]{2})|([A-Z]{2}\d{2} [A-Z]{2})|([A-Z]{2}\d{3}[A-Z]{2})|([A-Z]\d[A-Z]\d[A-Z]{2}) |([A-Z]{2}\d[A-Z]\d[A-Z]{2})|(GIR0AA))$', 'en-GB,cy-GB,gd', 'IE '] This result shows that thetag is used for multiple country attributes. If we simply wanted to scrape the country area, we can select the second matching element, as follows: >>> re.findall('(.*?)', html)[1]'244,820 square kilometres' This solution works but could easily fail if the web page is updated. Consider if this table is changed and the area is no longer in the second matching element. If we just need to scrape the data now, future changes can be ignored. However, if we want to rescrape this data at some point, we want our solution to be as robust against layout changes as possible. To make this regular expression more specific, we can include the parentelement, which has an ID, so it ought to be unique: >>> re.findall(' Area: (.*?) ', html) ['244,820 square kilometres'] This iteration is better; however, there are many other ways the web page could be updated in a way that still breaks the regular expression. For example, double quotation marks might be changed to single, extra spaces could be added between the tags, or the area_label could be changed. Here is an improved version to try and support these various possibilities: >>> re.findall('''.*?<tds*class=["']w2p_fw["']>(.*?) ''', html) ['244,820 square kilometres'] This regular expression is more future-proof but is difficult to construct, and quite unreadable. Also, there are still plenty of other minor layout changes that would break it, such as if a title attribute was added to the <td> tag or if the tr or td elements changed their CSS classes or IDs. From this example, it is clear that regular expressions provide a quick way to scrape data but are too brittle and easily break when a web page is updated. Fortunately, there are better data extraction solutions such as. Beautiful Soup Beautiful Soup is a popular library that parses a web page and provides a convenient interface to navigate content. If you do not already have this module, the latest version can be installed using this command: pip install beautifulsoup4 The first step with Beautiful Soup is to parse the downloaded HTML into a soup document. Many web pages do not contain perfectly valid HTML and Beautiful Soup needs to correct improper open and close tags. For example, consider this simple web page containing a list with missing attribute quotes and closing tags: <ul class=country> <li>Area <li>Population </ul> If the Population item is interpreted as a child of the Area item instead of the list, we could get unexpected results when scraping. Let us see how Beautiful Soup handles this: >>> from bs4 import BeautifulSoup >>> broken_html = '<ul class=country><li>Area<li>Population</ul>' >>> # parse the HTML >>> soup = BeautifulSoup(broken_html, 'html.parser') >>> fixed_html = soup.prettify() >>> print(fixed_html) <ul class="country"> <li> Area <li> Population </li> </li> </ul> We can see that using the default html.parser did not result in properly parsed HTML. We can see from the previous snippet that it has used nested li elements, which might make it difficult to navigate. Luckily there are more options for parsers. We can install LXML or we can also use html5lib. To install html5lib, simply use pip: pip install html5lib Now, we can repeat this code, changing only the parser like so: >>> soup = BeautifulSoup(broken_html, 'html5lib') >>> fixed_html = soup.prettify() >>> print(fixed_html) <html> <head> </head> <body> <ul class="country"> <li> Area </li> <li> Population </li> </ul> </body> </html>  Here, BeautifulSoup using html5lib was able to correctly interpret the missing attribute quotes and closing tags, as well as add the <html> and <body> tags to form a complete HTML document. You should see similar results if you used lxml. Now, we can navigate to the elements we want using the find() and find_all() methods: >>> ul = soup.find('ul', attrs={'class':'country'}) >>> ul.find('li') # returns just the first match <li>Area</li> >>> ul.find_all('li') # returns all matches [<li>Area</li>, <li>Population</li>] For a full list of available methods and parameters, the official documentation is available at http://www.crummy.com/software/BeautifulSoup/bs4/doc/. Now, using these techniques, here is a full example to extract the country area from our example website: >>> from bs4 import BeautifulSoup >>> url = 'http://example.webscraping.com/places/view/United-Kingdom-239' >>> html = download(url) >>> soup = BeautifulSoup(html) >>> # locate the area row >>> tr = soup.find(attrs={'id':'places_area__row'}) >>> td = tr.find(attrs={'class':'w2p_fw'}) # locate the data element >>> area = td.text # extract the text from the data element >>> print(area) 244,820 square kilometres This code is more verbose than regular expressions but easier to construct and understand. Also, we no longer need to worry about problems in minor layout changes, such as extra whitespace or tag attributes. We also know if the page contains broken HTML that BeautifulSoup can help clean the page and allow us to extract data from very broken website code. Lxml Lxml is a Python library built on top of the libxml2 XML parsing library written in C, which helps make it faster than Beautiful Soup but also harder to install on some computers, specifically Windows. The latest installation instructions are available at http://lxml.de/installation.html. If you run into difficulties installing the library on your own, you can also use Anaconda to do so:  https://anaconda.org/anaconda/lxml. If you are unfamiliar with Anaconda, it is a package and environment manager primarily focused on open data science packages built by the folks at Continuum Analytics. You can download and install Anaconda by following their setup instructions here: https://www.continuum.io/downloads. Note that using the Anaconda quick install will set your PYTHON_PATH to the Conda installation of Python. As with Beautiful Soup, the first step when using lxml is parsing the potentially invalid HTML into a consistent format. Here is an example of parsing the same broken HTML: >>> from lxml.html import fromstring, tostring >>> broken_html = '<ul class=country><li>Area<li>Population</ul>' >>> tree = fromstring(broken_html) # parse the HTML >>> fixed_html = tostring(tree, pretty_print=True) >>> print(fixed_html) <ul class="country"> <li>Area</li> <li>Population</li> </ul> As with BeautifulSoup, lxml was able to correctly parse the missing attribute quotes and closing tags, although it did not add the <html> and <body> tags. These are not requirements for standard XML and so are unnecessary for lxml to insert. After parsing the input, lxml has a number of different options to select elements, such as XPath selectors and a find() method similar to Beautiful Soup. Instead, we will use CSS selectors here, because they are more compact and can be reused later when parsing dynamic content. Some readers will already be familiar with them from their experience with jQuery selectors or use in front-end web application development. We will compare performance of these selectors with XPath. To use CSS selectors, you might need to install the cssselect library like so: pip install cssselect Now we can use the lxml CSS selectors to extract the area data from the example page: >>> tree = fromstring(html) >>> td = tree.cssselect('tr#places_area__row > td.w2p_fw')[0] >>> area = td.text_content() >>> print(area) 244,820 square kilometres By using the cssselect method on our tree, we can utilize CSS syntax to select a table row element with the places_area__row ID, and then the child table data tag with the w2p_fw class. Since cssselect returns a list, we then index the first result and call the text_content method, which will iterate over all child elements and return concatenated text of each element. In this case, we only have one element, but this functionality is useful to know for more complex extraction examples. Summary We have walked through a variety of ways to scrape data from a web page. Regular expressions can be useful for a one-off scrape or to avoid the overhead of parsing the entire web page, and BeautifulSoup provides a high-level interface while avoiding any difficult dependencies. However, in general, lxml will be the best choice because of its speed and extensive functionality, so we will use it in future examples. Resources for Article: Further resources on this subject: Web scraping with Python (Part 2) [article] Scraping the Web with Python - Quick Start [article] Scraping the Data [article]
Read more
  • 0
  • 0
  • 2431

article-image-administrating-mysql-server
Packt
09 Feb 2012
10 min read
Save for later

Administrating the MySQL Server

Packt
09 Feb 2012
10 min read
(For more resources on MySQL, see here.) Managing users and their privileges The Privileges sub-page (visible only if we are logged in as a privileged user) contains dialogs to manage MySQL user accounts. It also contains dialogs to manage privileges on global, database, and table levels. This sub-page is hierarchical. When editing a user's privileges, we can see the global privileges as well as the database-specific privileges. Then, when viewing database-specific privileges for a user, we can view and edit this user's privileges for any table within this database. The user overview The first page displayed when we enter the Privileges sub-page is called User overview. This shows all user accounts and a summary of their global privileges, as shown in the following screenshot: (Move the mouse over the image to enlarge.) From this page, we can: Edit a user's privileges, via the Edit Privileges link for this user Export a user's privileges definition, via the Export link for this user Use the checkboxes to remove users, via the Remove selected users dialog Access the page where the Add a new User dialog is available The displayed users' list has columns with the following characteristics: Column Characteristic User The user account we are defining. Host The machine name or IP address, from which this user account will be connecting to the MySQL server. A % value here indicates all hosts. Password Contains Yes if a password is defined and No if it isn't. The password itself cannot be seen from phpMyAdmin's interface or by directly looking at the mysql.user table, as it is encrypted with a one-way hashing algorithm. Global privileges A list of the user's global privileges. Grant Contains Yes if the user can grant his/her privileges to others. Action Contains a link to edit this user's privileges or export them. Exporting privileges This feature can be useful when we need to create a user with the same password and privileges on another MySQL server. Clicking on Export for user marc produces the following panel: Then it's only a matter of selecting these GRANT statements and pasting them in the SQL box of another phpMyAdmin window, where we have logged in on another MySQL server. Privileges reload At the bottom of User overview page, this message is displayed: Note: phpMyAdmin gets the users' privileges directly from MySQL's privilege tables. The content of these tables may differ from the privileges the server uses, if they have been changed manually. In this case, you should reload the privileges before you continue. Here, the text reload the privileges is clickable. The effective privileges (the ones against which the server bases its access decisions) are the privileges that are located in the server's memory. Privilege modifications that are made from the User overview page are made both in memory and on disk in the mysql database. Modifications made directly to the mysql database do not have immediate effect. The reload the privileges operation reads the privileges from the database and makes them effective in memory. Adding a user The Add a new User link opens a dialog for user account creation. First, we see the panel where we will describe the account itself, as shown in the following screenshot: The second part of the Add a new User dialog is where we will specify the user's global privileges, which apply to the server as a whole (see the Assigning global privileges section of this article), as shown in the following screenshot: Entering the username The User name menu offers two choices. We can choose Use text field: and enter a username in the box, or we can choose Any user to create an anonymous user (the blank user). More details about the anonymous user are available at http://dev.mysql.com/doc/refman/5.5/en/connection-access.html. Let us choose Use text field: and enter bill. Assigning a host value By default, this menu is set to Any host, with % as the host value. The Local choice means localhost. The Use host table choice (which creates a blank value in the host field) means to look in the mysql.host table for database-specific privileges. Choosing Use text field: allows us to enter the exact host value we want. Let us choose Local. Setting passwords Even though it's possible to create a user without a password (by selecting the No password option), it's best to have a password. We have to enter it twice (as we cannot see what is entered) to confirm the intended password. A secure password should have more than eight characters, and should contain a mixture of uppercase and lowercase characters, digits, and special characters. Therefore, it's recommended to have phpMyAdmin generate a password—this is possible in JavaScript-enabled browsers. In the Generate password dialog, clicking on Generate button enters a random password (in clear text) on the screen and fills the Password and Re-type input fields with the generated password. At this point, we should note the password so that we can pass it on to the user. Understanding rights for database creation A frequent convention is to assign a user the rights to a database having the same name as this user. To accomplish this, the Database for user section offers the Create database with same name and grant all privileges radio button. Selecting this checkbox automates the process by creating both the database (if it does not already exist) and assigning the corresponding rights. Please note that, with this method, each user would be limited to one database (user bill, database bill). Another possibility is to allow users to create databases that have the same prefix as their usernames. Therefore, the other choice Grant all privileges on wildcard name (username_%) performs this function by assigning a wildcard privilege. With this in place, user bill could create the databases bill_test, bill_2, bill_payroll, and so on; phpMyAdmin does not pre-create the databases in this case. Assigning global privileges Global privileges determine the user's access to all databases. Hence, these are sometimes known as superuser privileges. A normal user should not have any of these privileges unless there is a good reason for this. Moreover, should a user account that has global privileges become compromised, the damage could be far greater. If we are really creating a superuser, we will select every global privilege that he or she needs. These privileges are further divided into Data, Structure, and Administration groups. In our example, bill will not have any global privileges. Limiting the resources used We can limit the resources used by this user on this server (for example, the maximum queries per hour). Zero means no limit. We will not impose any resources limits on bill. The following screenshot shows the status of the screen just before hitting Create user to create this user's definition (with the remaining fields being set to default): Editing a user profile The page used to edit a user's profile appears whenever we click on Edit Privileges for a user in the User overview page. Let us try it for our newly created user bill. There are four sections on this page, each with its own Go button. Hence, each section is operated independently and has a distinct purpose. Editing global privileges The section for editing the user's privileges has the same look as the Add a new User dialog, and is used to view and to change global privileges. Assigning database-specific privileges In this section, we define the databases to which our user has access, and his or her exact privileges on these databases. As shown in the previous screenshot, we see None because we haven't defined any privileges yet. There are two ways of defining database privileges. First, we can choose one of the existing databases from the drop-down menu as shown in the following screenshot: This assigns privileges only for the chosen database. Secondly, we can also choose Use text field: and enter a database name. We could enter a non-existent database name, so that the user can create it later (provided we give him/her the CREATE privilege in the next panel). We can also use special characters, such as the underscore and the percent sign, for wildcards. For example, entering bill here would enable him to create a bill database, and entering bill% would enable him to create a database with any name that starts with bill. For our example, we will enter bill and click on Go. The next screen is used to set bill's privileges on the bill database, and create table-specific privileges. To learn more about the meaning of a specific privilege, we can hover the mouse over a privilege name (which is always in English), and an explanation about this privilege appears in the current language. We give SELECT, INSERT, UPDATE, DELETE, CREATE, ALTER, INDEX, and DROP privileges to bill on this database. We then click on Go. After the privileges have been assigned, the interface stays at the same place, so that we can refine these privileges further. We cannot assign table-specific privileges for the moment, as the database does not yet exist. To go back to the general privileges page of bill, click on the 'bill'@'localhost' title. This brings us back to the following, familiar page except for a change in one section: We see the existing privileges (we could click on Edit Privileges link to edit or on Revoke link to revoke them) on the bill database for user bill, and we can add privileges for bill on another database. We can also see that bill has no table-specific privilege on the bill database. Changing the password The Change password dialog is part of the Edit user page, and we can use it either to change bill's password or to remove it. Removing the password will enable bill to log in without a password. The dialog offers a choice of password hashing options, and it's recommended to keep the default of MySQL 4.1+ hashing. For more details about hashing, please visit http://dev.mysql.com/doc/refman/5.1/en/password-hashing.html. Changing login information or copying a user This dialog can be used to change the user's login information, or to copy his or her login information to a new user. For example, suppose that Bill calls and tells us that he prefers the login name billy instead of bill. We just have to add a y to the username, and then select delete the old one from the user tables radio button, as shown in the following screenshot: After clicking on Go, bill no longer exists in the mysql database. Also, all of his privileges, including the privileges on the bill database, will have been transferred to the new user—billy. However, the user definition of bill will still exist in memory, and hence it's still effective. If we had chosen the delete the old one from the user tables and reload the privileges afterwards option instead, the user definition of bill would immediately have ceased to be valid. Alternatively, we could have created another user based on bill, by making use of the keep the old one choice. We can transfer the password to the new user by choosing Do not change the password option, or change it by entering a new password twice. The revoke all active privileges… option immediately terminates the effective current privileges for this user, even if he or she is currently logged in. Removing a user Removing a user is done from the User overview section of the Privileges page. We select the user to be removed. Then (in Remove selected users) we can select the Drop the databases that have the same names as the users option to remove any databases that are named after the users we are deleting. A click on Go effectively removes the selected users.
Read more
  • 0
  • 0
  • 2430
article-image-introduction-online-analytical-processing-oracle-essbase
Packt
07 Oct 2009
6 min read
Save for later

Introduction to Online Analytical Processing in Oracle Essbase

Packt
07 Oct 2009
6 min read
Introduction to OLAP OLAP is the common term for Online Analytical Processing and is generally known to be a multidimensional, client-server computing environment. The differences between OLAP analytical solutions and traditional data analysis applications containing backend relational databases are stark. The most obvious being an OLAP analytical application's ability to provide speedy analysis of broad slices of data. Programs which are complex and expensive to write would be required to perform even a fraction of the functionality provided by a simple Oracle Essbase OLAP application. Another notable difference is OLAP's ability to drill-down to the lowest level of granularity with ease. You will even hear phrases like, slice and dice, and multidimensionality, which means having the ability to view the data from virtually any perspective. Finally, the ability to calculate large amounts of data on the fly gives users a superior advantage over traditional applications with relational databases when it comes to "what if" and "cause-and-effect" data analysis and reporting. Oracle Essbase is widely known as a financial analytical tool. We want to change the mindset just a bit, right here and now. Oracle Essbase absolutely is a superior financial OLAP tool, but it is an equally superior OLAP tool for just about any type of data analysis Determine data storage options Get ready to toss out everything you've ever learned about storing data in a typical relational database with tables, rows, and columns. Keeping the above example of the Essbase outline in mind, we will now begin covering how the data is stored in Essbase and the various options available to you (the Essbase programmer). An Essbase cube usually stores less physical data than a typical relational database must store to deliver the same results to the user. Usually, the greatest saving is in the expense of data retrieval times. The results returned from a typical Essbase database require less processing overhead than the similar results being delivered as the result of queries performed against relational database tables. Essbase stores data in what is commonly referred to as a multidimensional array. Inside the multidimensional array are the data cells. It is these data cells where the data is actually stored. The smallest vehicle Essbase uses to store data is a cell. A data cell however, cannot stand alone. The smallest usable vehicle to store data, contained in an Essbase database, is the data block (see the following figure). These data blocks are the building blocks of the Essbase cube: A simplified explanation is that the data blocks are made up of data cells. The number of data cells are, for the most part, in direct relation to the number of dimensions in the Essbase outline (the data attributes explained previously), and the number of possible data combinations or intersections that can be created. In a traditional relational database, one new element of data may require an entire new row of data in one to manytables. Looking at the previous image, you can see that if you need to add stock information on a vehicle, you will need to insert a new row in the Stock table of your relational database. In Essbase, that same new piece of data is plugged into the waiting data cell that was created in the data block, when the database outline was structured or restructured. You can add a new dimension to the database outline or add new members to an existing dimension at any time. By adding dimensions to the database outline you are actually increasing the size of the data block. When a data block is created by Essbase, it contains cells for all of the various dimensions whether you have the data at that point or not. In our example, the data block created by the database would already contain a cell for stock, even if you did not yet have a value to store there. When you have a value for stock, it just gets plugged into its data cell and the size of the database is unaffected. When you add or remove information from the outline and save the outline, Essbase will automatically restructure the database and modify the data blocks (add/remove data cells) to incorporate the new outline information as necessary. In Oracle Essbase there are two distinct storage options that can be used when creating a database. These storage options are known as the Block Storage Option (BSO) and the Aggregate Storage Option (ASO). For most transactional Essbase applications, the more suitable of the two options is the BSO. For our example in this article, we will create an application/database using the BSO. It should be mentioned that the size of the data blocks can have a dramatic effect on the performance of the system. It is always best to try to avoid extremely large and complex database outlines. As we explained previously, the data blocks are structured roughly in relation to the possible combinations of data based on the number of members in the database outline. More members = larger data blocks. Less members = smaller data blocks. Oracle Essbase offers an extremely valuable option to help keep block sizes to a minimum in order to help keep your database running at peak performance. The dynamically calculated database member! The dynamically calculated member is a measure typically derived from other data elements in the database. It is not physically stored in the database. Instead, it is only created (calculated) at the time you ask for it. There are three great benefits for building your database with dynamically calculated members: There is a huge potential to create many new measures without adding new sources of data or writing expensive programs to derive the values. While the dynamically calculated member occupies a place in the database outline; it does not affect the block size in the database, therefore, it does not affect performance. The resultant measure is always accurate to the other measures in the database and will always tally (the derived number will always equal the result of the stored component numbers). There is never a question of "where did this number come from?"
Read more
  • 0
  • 0
  • 2429

article-image-ai-in-the-real-world-real-estate
Julian Melanson
20 Jul 2023
4 min read
Save for later

AI in the Real World: Real Estate

Julian Melanson
20 Jul 2023
4 min read
The fast-paced development of Artificial Intelligence has already started reshaping various sectors, with the real estate industry standing out as a prominent beneficiary. Of particular interest is the potential AI presents in streamlining property valuation, a critical process that underlies a myriad of real estate activities, including setting sale prices, making investment decisions, and optimizing home insurance premiums. While the conventional means of property valuation have their merits, they are far from perfect. This article delves into the potential of AI, specifically OpenAI's ChatGPT, in transforming property valuation in the real estate sector, discussing the challenges inherent to traditional approaches and exploring the benefits offered by this AI-driven approach.The Current State of Property ValuationProperty valuation is a meticulous process that draws on a variety of data sources, both public and private. Depending on the valuation's purpose, the time and effort committed to research can differ significantly. For instance, real estate brokers might base their Broker Price Opinions on a limited set of comparable properties, while appraisers might undertake a thorough firsthand inspection to understand a property's condition, quality, and value comprehensively.Despite the evolution of valuation methodologies over the years, traditional approaches still grapple with certain obstacles. One of the primary issues is data inconsistency, mainly arising from the dispersed and scattered nature of relevant property data across various sources. While attempts have been made to centralize information on property features, ownership changes, and other key insights, consistency in data remains elusive. The result is disparities in the Automated Valuation Models (AVMs) currently used, which can lead to divergent valuations for the same property.Moreover, human bias forms a significant challenge in property appraisals. It's often difficult to find identical properties for comparison, leading to inevitable subjectivity in adjustments made to reconcile price differences. Studies show that appraised values fall below the agreed purchase price in just 10% of cases, suggesting a propensity towards price confirmation bias, a situation that calls for greater objectivity in home appraisals.Integrating AI into Property Valuation: The Role of ChatGPTIn response to these challenges, the integration of AI into the property valuation process presents a promising solution. The application of AI, especially advanced language models like ChatGPT, can offer consistent examinations of a property’s condition and quality, mitigating issues associated with data inconsistencies and human bias.ChatGPT, a generative pre-trained transformer, has been designed to understand and generate human-like text based on given input. In the context of real estate, it offers tremendous potential in data analysis and, consequently, in generating accurate property valuations. Traditionally, property valuations have been conducted by human appraisers who assess a property’s worth based on a range of factors such as location, size, and condition. However, this approach can be time-consuming, costly, and susceptible to human error.By incorporating ChatGPT into the valuation process, real estate professionals can input relevant data into the AI model, which can then analyze the data and supply a detailed valuation report. The implications of this are transformative for the industry: it offers considerable time savings, reduces the potential for errors, and enhances the transparency of the valuation process.A Practical Application of ChatGPT in Property ValuationHere’s a very simple prompt that illuminates how ChatGPT can be a great guide in the property valuation process:    The evolution of AI has unlocked numerous opportunities for innovation and efficiency across a variety of sectors, with the real estate industry being no exception. Particularly, the advent of AI models like ChatGPT has opened new avenues for enhancing the accuracy and efficiency of property valuations. By surmounting the obstacles inherent to traditional valuation methodologies, such as data inconsistencies and human bias, AI offers a more streamlined, transparent, and precise approach to property valuation. Author BioJulian Melanson is one of the founders of Leap Year Learning. Leap Year Learning is a cutting-edge online school that specializes in teaching creative disciplines and integrating AI tools. We believe that creativity and AI are the keys to a successful future and our courses help equip students with the skills they need to succeed in a continuously evolving world. Our seasoned instructors bring real-world experience to the virtual classroom and our interactive lessons help students reinforce their learning with hands-on activities.No matter your background, from beginners to experts, hobbyists to professionals, Leap Year Learning is here to bring in the future of creativity, productivity, and learning!
Read more
  • 0
  • 0
  • 2429
Modal Close icon
Modal Close icon