How-To Tutorials

If you were following along in Part 1, you will have seen how we used Keras to create our model for tackling The German Traffic Sign Recognition Benchmark(GTSRB). Now in Part 2 you will see how we achieve performance close to human-level performance. You will also see how to improve the accuracy of the model using augmentation of the training data. Training Now, our model is ready to train. During the training, our model will iterate over batches of the training set, each of size batch_size. For each batch, gradients will be computed and updates will be made to the weights of the network automatically. One iteration over all of the training set is referred to as an epoch. Training is usually run until the loss converges to a constant. We will add a couple of features to our training: Learning rate scheduler: Decaying learning rate over the epochs usually helps the model learn better. Model checkpoint: We will save the model with best validation accuracy. This is useful because our network might start overfitting after a certain number of epochs, but we want the best model. These are not necessary but they improve the model accuracy. These features are implemented via the callback feature of Keras. callback are a set of functions that will applied at given stages of training procedure like end of an epoch of training. Keras provides inbuilt functions for both learning rate scheduling and model checkpointing. fromkeras.callbacks import LearningRateScheduler, ModelCheckpoint deflr_schedule(epoch): returnlr*(0.1**int(epoch/10)) batch_size = 32 nb_epoch = 30 model.fit(X, Y, batch_size=batch_size, nb_epoch=nb_epoch, validation_split=0.2, callbacks=[LearningRateScheduler(lr_schedule), ModelCheckpoint('model.h5',save_best_only=True)] ) You'll see that model starts training and logs the losses and accuracies: Train on 31367 samples, validate on 7842 samples Epoch 1/30 31367/31367 [==============================] - 30s - loss: 1.1502 - acc: 0.6723 - val_loss: 0.1262 - val_acc: 0.9616 Epoch 2/30 31367/31367 [==============================] - 32s - loss: 0.2143 - acc: 0.9359 - val_loss: 0.0653 - val_acc: 0.9809 Epoch 3/30 31367/31367 [==============================] - 31s - loss: 0.1342 - acc: 0.9604 - val_loss: 0.0590 - val_acc: 0.9825 ... Now this might take a bit of time, especially if you are running on a CPU. If you have anNvidiaGPU, you should install cuda. It speeds up the training dramatically. For example, on my Macbook air, it takes 10 minutes per epoch while on a machine with Nvidia Titan X GPU, it takes 30 seconds. Even modest GPUs offer impressive speedup because of the inherent parallelizability of the neural networks. This makes GPUs necessary for deep learning if anything big has to be done. Grab a coffee while you wait for training to complete ;). Congratulations! You have just trained your first deep learning model. Evaluation Let's quickly load test data and evaluate our model on it: import pandas as pd test = pd.read_csv('GT-final_test.csv',sep=';') # Load test dataset X_test = [] y_test = [] i = 0 forfile_name, class_id in zip(list(test['Filename']), list(test['ClassId'])): img_path = os.path.join('GTSRB/Final_Test/Images/',file_name) X_test.append(preprocess_img(io.imread(img_path))) y_test.append(class_id) X_test = np.array(X_test) y_test = np.array(y_test) # predict and evaluate y_pred = model.predict_classes(X_test) acc = np.sum(y_pred==y_test)/np.size(y_pred) print("Test accuracy = {}".format(acc)) Which outputs on my system (Results may change a bit because the weights of the neural network are randomly initialized): 12630/12630 [==============================] - 2s Test accuracy = 0.9792557403008709 97.92%! That's great! It's not far from average human performance (98.84%)[1]. A lot of things can be done to squeeze out extra performance from the neural net. I'll implement one such improvement in the next section. Data Augmentation You might think 40000 images is a lot of images. Think about it again. Our model has 1358155 parameters (try model.count_params() or model.summary()). That's 4X the number of training images. If we can generate new images for training from the existing images, that will be a great way to increase the size of the dataset. This can be done by slightly: Translating theimage Rotating theimage Shearing the image Zooming in/out of the image Rather than generating and saving such images to hard disk, we will generate them on the fly during training. This can be done directly using built-in functionality of Keras. fromkeras.preprocessing.image import ImageDataGenerator fromkeras.preprocessing.image import ImageDataGenerator fromsklearn.cross_validation import train_test_split X_train, X_val, Y_train, Y_val = train_test_split(X, Y, test_size=0.2, random_state=42) datagen = ImageDataGenerator(featurewise_center=False, featurewise_std_normalization=False, width_shift_range=0.1, height_shift_range=0.1, zoom_range=0.2, shear_range=0.1, rotation_range=10.,) datagen.fit(X_train) # Reinitialize model and compile model = cnn_model() model.compile(loss='categorical_crossentropy', optimizer=sgd, metrics=['accuracy']) # Train again nb_epoch = 30 model.fit_generator(datagen.flow(X_train, Y_train, batch_size=batch_size), samples_per_epoch=X_train.shape[0], nb_epoch=nb_epoch, validation_data=(X_val, Y_val), callbacks=[LearningRateScheduler(lr_schedule), ModelCheckpoint('model.h5',save_best_only=True)] ) With this model, I get 98.29% accuracy on the test set. Frankly, I haven't done much parameter tuning. I'll make a small list of things which can be tried to improve the model: Try different network architectures. Try deeper and shallower networks Try adding BatchNormalization layers to the network Experiment with different weight initializations Try different learning rates and schedules Make an ensemble of models Try normalization of input images More aggressive data augmentation This is but a model for beginners. For state-of-the-art solutions of the problem, you can have a look at this, where the authors achieve 99.61% accuracy with a specialized layer called Spatial Transformer layer. Conclusion In this two-part post, you have learned how to use convolutional networks to solve a computer vision problem. We used the Keras deep learning framework to implement CNNs in Python. We have achieved performance close to human-level performance. We also have seen a way to improve the accuracy of the model using augmentation of the training data. References: Stallkamp, Johannes, et al. "Man vs. computer: Benchmarking machine learning algorithms for traffic sign recognition." Neural networks 32 (2012): 323-332. About the author Sasank Chilamkurthy works at Qure.ai. His work involves deep learning on medical images obtained from radiology and pathology. He completed his UG in Mumbai at the Indian Institute of Technology, Bombay. He can be found on Github at here.

(For more resources related to this topic, see here.) Easy To run a Yii version 1.x-powered web application, all you need are the core framework files and a web server supporting PHP 5.1.0 or higher. To develop with Yii, you only need to know PHP and object-oriented programming. You are not required to learn any new configuration or templating language. Building a Yii application mainly involves writing and maintaining your own custom PHP classes, some of which will extend from the core, Yii framework component classes. Yii incorporates many of the great ideas and work from other well-known web programming frameworks and applications. So if you are coming to Yii from using other web development frameworks, it is likely that you will find it familiar and easy to navigate. Yii also embraces a convention over configuration philosophy, which contributes to its ease of use. This means that Yii has sensible defaults for almost all the aspects that are used for configuring your application. Following the prescribed conventions, you can write less code and spend less time developing your application. However, Yii does not force your hand. It allows you to customize all of its defaults and makes it easy to override all of these conventions. Efficient Yii is a high-performance, component-based framework that can be used for developing web applications on any scale. It encourages maximum code reuse in web programming and can significantly accelerate the development process. As mentioned previously, if you stick with Yii's built-in conventions, you can get your application up and running with little or no manual configuration. Yii is also designed to help you with DRY development. DRY stands for Don't Repeat Yourself , a key concept of agile application development. All Yii applications are built using the Model-View-Controller (MVC) architecture. Yiienforces this development pattern by providing a place to keep each piece of your MVC code. This minimizes duplication and helps promote code reuse and ease of maintainability. The less code you need to write, the less time it takes to get your application to market. The easier it is to maintain your application, the longer it will stay on the market. Of course, the framework is not just efficient to use, it is remarkably fast and performance optimized. Yii has been developed with performance optimization in mind from the very beginning, and the result is one of the most efficient PHP frameworks around. So any additional overhead that Yii adds to applications written on top of it is extremely negligible. Extensible Yii has been carefully designed to allow nearly every piece of its code to be extended and customized to meet any project requirement. In fact, it is difficult not to take advantage of Yii's ease of extensibility, since a primary activity when developing a Yii application is extending the core framework classes. And if you want to turn your extended code into useful tools for other developers, Yii provides easy-to-follow steps and guidelines to help you create such third-party extensions. This allows you to contribute to Yii's ever-growing list of features and actively participate in extending Yii itself. Remarkably, this ease-of-use, superior performance, and depth of extensibility does not come at the cost of sacrificing its features. Yii is packed with features to help you meet those high demands placed on today's web applications. AJAX-enabled widgets, RESTful and SOAP Web services integration, enforcement of an MVC architecture, DAO and relational ActiveRecord database layer, sophisticated caching, hierarchical role-based access control, theming, internationalization (I18N), and localization (L10N) are just the tip of the Yii iceberg. As of version 1.1, the core framework is now packaged with an official extension library called Zii. These extensions are developed and maintained by the core framework team members, and continue to extend Yii's core feature set. And with a deep community of users who are also contributing by writing Yiiextensions, the overall feature set available to a Yii-powered application is growing daily. A list of available, user-contributed extensions on the Yii framework website can be found at http://www.yiiframework.com/extensions. There is also an unofficial extension repository of great extensions that can be found at http://yiiext.github.com/, which really demonstrates the strength of the community and the extensibility of this framework. MVC architecture As mentioned earlier, Yii is an MVC framework and provides an explicit directory structure for each piece of model, view, and controller code. Before we get started with building our first Yii application, we need to define a few key terms and look at how Yii implements and enforces this MVC architecture. Model Typically in an MVC architecture, the model is responsible for maintaining the state, and should encapsulate the business rules that apply to the data that defines this state. A model in Yii is any instance of the framework class CModel or its child class. A model class is typically comprised of data attributes that can have separate labels (something user friendly for the purpose of display), and can be validated against a set of rules defined in the model. The data that makes up the attributes in the model class could come from a row of a database table or from the fields in a user input form. Yii implements two kinds of models, namely the form model (a CFormModel class) and active record (a CActiveRecord class). They both extend from the same base class CModel. The class CFormModel represents a data model that collects HTML form inputs. It encapsulates all the logic for form field validation, and any other business logic that may need to be applied to the form field data. It can then store this data in memory or, with the help of an active record model, store data in a database. Active Record (AR) is a design pattern used to abstract database access in an objectoriented fashion. Each AR object in Yii is an instance of CActiveRecord or its child class, which wraps a single row in a database table or view, that encapsulates all the logic and details around database access, and houses much of the business logic that is required to be applied to that data. The data field values for each column in the table row are represented as properties of the active record object. View Typically the view is responsible for rendering the user interface, often based on the data in the model. A view in Yii is a PHP script that contains user interface-related elements, often built using HTML, but can also contain PHP statements. Usually, any PHP statements within the view are very simple, conditional or looping statements, or refer to other Yii UI-related elements such as HTML helper class methods or prebuilt widgets. More sophisticated logic should be separated from the view and placed appropriately in either the model, if dealing directly with the data, or the controller, for more general business logic. Controller The controller is our main director of a routed request, and is responsible for taking user input, interacting with the model, and instructing the view to update and display appropriately. A controller in Yii is an instance of CController or a child class thereof. When a controller runs, it performs the requested action, which then interacts with the necessary models, and renders an appropriate view. An action, in its simplest form, is a controller class method whose name starts with the word action

(For more resources on WordPress, see here.) Blog design principles Blogs tend to have a fairly simple, minimalist layout and design. This has always been one of their key characteristics. Blogs are all about frequently updated content, so the main purpose of their design is to present that content as efficiently and conveniently as possible. The vast majority of blogs present their most recent content on the page that visitors arrive at; hence, the front page contains the latest posts. There's no home page with a verbose welcome message and a long navigation menu to click through to the important stuff. The visitor gets straight into the meat of the blog. By default, this is the structure that WordPress provides. It is possible to set a static page as your blog's front page, but, in the vast majority of cases, I wouldn't recommend it. So when considering the architecture of a blog, unlike other types of website, we don't have to worry too much about a complex navigation structure. There is a convention that we can follow. Yes, we may want to add some extra static pages, but probably only a few of these. What we are most concerned with in blog design is not a complicated navigation structure and how all the pages link together, but how the actual blog page should look and feel. This can be broken down into four key components, which we will examine, one by one: Layout Color Typography Usability and accessibility Layout Good design is all about making something easy for people to use. Designers achieve this by employing standards and conventions. For example, cars have a standard design: four wheels, a chassis, a steering wheel, gas pedal, brake, gear shift, and so on. Car designers have stuck to this convention for many years. First, because it works well and second, because it enables us to drive any car we choose. When you sit down in any standard road car, you know how it works. You turn the key in the ignition, select a gear, hit the gas, and off you go. It's certainly not beyond the ken of car designers to come up with new ways for getting a car in motion (a joystick maybe, or a hand-operated brake) but this would make it more difficult for people to drive. Cars work reasonably safely and efficiently because we are all familiar with these conventions. The layout of blog pages also tends to follow certain conventions. As with cars, this helps people to use blogs efficiently. They know how they work, because they're familiar with the conventions. Most blogs have a header and a footer with the main content arranged into columns. This columnar layout works very well for the type of chronological content presented in blogs. Because of these conventions, the decisions about our blog layout are fairly simple. It's basically a case of deciding where we want to place all our page elements and content within this standard columnar layout. The set of page elements we have to choose from is also based on fairly well entrenched blogging conventions. The list of things we may want on our page includes: Header Posts Comments Static content (for example, the About page) Links to static pages (simple navigation) RSS feeds Search Categories Archives Blogroll Widgets and plugins Footer If we look at this list in more detail, we can see that these page elements can be grouped in a sensible way. For example: Group 1 Header Links to static pages Group 2 Posts Comments Static content Group 3 RSS Feeds Search Categories Group 4 Archives Blogroll Widgets and plugins Group 5 Footer This isn't the only possible grouping scheme we might come up with. For example, we may place the items in Groups 3 and 4 into a single larger group, or we may have widgets and plugins in a group on their own. From this grouping, we can see that the type of items in Group 2 are likely to be the main content on our page, with Groups 3 and 4 being placed in sidebars . Sidebars are areas on the page where we place ancillary content. Having considered the elements we want on the page and how they might be grouped, we can think about possible layouts. Within the conventional columnar structure of blogs there are quite a few possible layout variations. We'll look at four of the most common. The first is a three-column layout. Here, we have two sidebars, one on either side of the main content. Using this type of layout, we would probably place the items in Groups 3 and 4 in the sidebars and Group 2 in the main content area. A variation on the three-column layout is to have the two sidebars next to each other on one side of the page (usually the right), as shown in the following diagram. This is a popular layout for blogs, not just for aesthetics, but because the search engine spiders encounter the post column first as that content is at the top of the template. Using two sidebars is useful if you anticipate having a lot of ancillary content on your blog. The list of page elements given earlier is really the bare minimum you would want on your page. However, if you decide to use lots of widgets or have a long blogroll, it's a good idea to spread them across two sidebars. This means that more of your content can be placed above the fold. The concept of above the fold in web design applies to content in a web page which is visible without having to scroll down, that is, the stuff in the top part of the page. It's a good idea to place the most important content above the fold so that readers can see it immediately. This is particularly true if you plan to monetize your blog by displaying adverts. Adverts that appear above the fold get the most exposure, and therefore, generate the most revenue Another popular layout amongst bloggers has just two columns. In this layout, we would place the items in Groups 3 and 4 together in the one sidebar. It doesn't really matter which side of the page the sidebar is placed, but it seems more common to have it on the right. Studies have shown that a web user's eyes are most often focused on the top-left region of a web page, when they first open any page. So it makes sense to place your main content there, with your sidebar on the right. Also, remember that the search engine spiders will find the leftmost content first. You want them to find your main content quickly, which is a good reason for placing your sidebar on the right, out of their way. An important benefit of a two-column layout is that it allows more room for your main content area. This may be important, if you intend to use a lot of video or images within your blog posts. The extra room allows you to display this visual content bigger. Many blogs place some of their ancillary content just above the footer, below the main content. This also has the advantage of leaving more space for the main content, as with the two-column layout. The following diagram shows this type of layout. Here, the content just above the footer isn't strictly speaking a sidebar, but I've labeled it this way because it's the terminology most often applied to this type of ancillary content. Wireframing The layout diagrams we've just seen are referred to as wireframes by web designers. They give a simple overview of where the elements of a page should be placed. It would be a good idea for you to create your own wireframe for your blog design. This can be done using most graphic software packages or something like Microsoft Visio , or a simple pen and paper does the job equally well! Color This is the next design principle we need to consider. It may be that you already have a corporate color scheme based on your company logo, stationery, or existing website. In this case, you'll probably want to continue that theme through your blog design. Even if you already have your corporate color scheme, this section may still be useful in case you decide to change your blog colors in the future. The subject of color in design is a large one. Design students spend a great deal of time learning about the psychology and science of colors and techniques for achieving the best color schemes. Obviously, we don't have enough space to go into that kind of detail, but I will try to give you a few pointers. The first thing to think about is the psychology of color, in particular, color associations. This is the idea that different colors evoke different feelings and emotions in the eye of the beholder. To a certain extent this can be rather subjective and it can also depend on cultural influences, but there are some generalities that can be applied. For example, red is often perceived as being exciting, passionate, or dramatic. Yellow is an active and highly visible color, which is why it is used in warning signs. It is also associated with energy and happiness. Blue is sometimes thought of as being cold. It can also be a calming color and may sometimes be seen as corporate or conservative. White, for many people, gives the idea of cleanliness, purity, and perfection. Black can be seen as strong, elegant, and chic. Obviously, these color associations can vary from person to person, so designers don't rely on them solely in their color decisions, but they are certainly worth bearing in mind. There are more practical considerations regarding color that are probably more relevant than color psychology. For example, we all know that some color combinations don't work well together. There is a great deal of design theory aimed at devising good color combinations, but unless you're a professional designer, it's not really worth going into. Probably the best method for working out good color combinations is trial and error. If you're trying to figure out a background color and text color for your blog, simply test a few out. You could use a graphics package such as Photoshop or Microsoft Paint , or one of the many online color tools such as, http://colorschemedesigner.com/ or Adobe's Kuler at http://kuler.adobe.com. When choosing background and text colors you need to think about contrast. For example, yellow text on a white background can be very difficult to read. Some people also find light text on a black background a strain on their eyes. It's also important not to use too many colors in your design. Try to limit your palette to a maximum of three or four. Sometimes you may only need two colors to make an attractive design. One method for devising color combinations is to look for examples all around you, particularly in nature. Maybe look at a photograph of a landscape and pick out color combinations you like. Also consider the work of professional designers. Think about websites and blogs you like, and examine the color schemes they have used. You will also find good examples in offline design—pick up a book and see how colors have been used in the cover design. If you would like to base your blog's color scheme on your company logo, you could use lighter and darker versions of one color from the logo. Use the most vivid color in the logo for emphasis or headings. Web color theory At this point, it's worth looking at the technical theory behind colors on the Web. Web browsers use the Hexadecimal RGB color system to render colors in web pages. This is because computer monitors use an RGB color model, which means every pixel is colored using a combination of red, green, and blue light (hence RGB). There are 256 different levels of red light, 256 different levels of green light, and 256 different levels of blue light. These can be combined to create 16,277,216 different colors, which are all available for your web browser. The hexadecimal system gives us a way of counting from 0 to 255 using numbers and letters, which covers all 256 levels of RGB light. In the hexadecimal scale, 0 is 00 and 255 is FF. A six-character hexadecimal color code specifies the levels of red, green and blue, which form a particular color. For example, the color white combines red, green, and blue at their highest possible levels, that is 255. Remember that in hexadecimal 255 is FF, so the color code for white is FFFFFF (Red: FF, Green: FF, and Blue: FF). The color code for black is 000000 as the levels of red, green, and blue are set to their lowest, or 00 (in hexadecimal). The code for red is FF0000, blue is 0000FF, and yellow is FFFF00, and so on. We can use six-character Hexadecimal RGB codes to define all of the 16,277,216 web colors. So how do we know the hexadecimal code for a particular color? Well, there are many tools available that define the Hexadecimal RGB codes for the colors you choose. Some are standalone applications for PC or Mac, and others are online. Take a look at https://www.webpagefx.com/web-design/color-picker/ or do a quick search in Google on color picker . For more information on web colors , read the article at http://en.wikipedia.org/wiki/Web_colors. Typography Another important consideration for your blog design is the fonts you use. Your choice of font will have a big impact on the readability of your blog. It's important to bear in mind that although there are literally thousands of fonts to choose from, only a small number of them are practical for web design. This is because a web browser can only display fonts that are already installed on the user's computer. If you choose an obscure font for your blog, the chances are that most users won't have it installed on their computer. If this is the case the web browser will automatically select a substitute font. This may be smaller or far less readable than the font you had in mind. It's always safest to stick to the fonts that are commonly used in web design, which are known as web safe fonts. These include the following: Arial Verdana Times New Roman Georgia There are two types of font, serif and sans-serif. Serif fonts have little flourishes at the end of the strokes whereas sans-serif fonts don't have this extra decoration. Arial and Verdana are sans-serif fonts, whereas Times New Roman and Georgia are serif fonts. As you'll see later in the article, when we look at CSS, fonts are usually specified in groups or families. They are listed in the order of the designer's preference. For example, a designer may specify font-family:"Georgia, Times New Roman, serif". This means when the browser renders the page it will first look for the Georgia font; if it isn't installed, it will look for the Times New Roman font and if that isn't installed, it will look for the computer's default serif font. This method gives the designer more control over the font choices the browser will make. The size of your font is also an important factor. Generally speaking, the bigger it is, the easier it is to read. Computer displays are getting bigger and bigger but the default screen resolutions are tending to get smaller. In other words, the individual pixels on users' screens are getting smaller. This is a good reason for web designers to choose larger font sizes. This trend can be seen on many Web 2.0 sites, which tend to use large and clear fonts as part of their design, for example http://www.37signals.com. But be careful not to go too big as this can make your design look messy and a little childish. Remember that you're not limited to using just one font in your blog design. For example, you may decide to use a different font for your headings. This can be an effective design feature but don't go crazy by using too many fonts, as this will make your design look messy. Probably two, or at most three, fonts on a page are plenty. Font replacement Font replacement refers to a relatively new group of technologies that are pushing the envelope of web typography. In theory, they allow designers to use any font in their web page designs. In practice, things are a little more complicated. Issues around browser compatibility and font licensing make font replacement technologies a bit of a minefield for anyone who is new to web design. It's true that, thanks to font replacement technologies, professional designers are no longer constrained by the notion of web safe fonts. But, if you are a web design novice, I recommend you stick to web safe fonts until your skills improve and you are ready to learn a whole new technology. A full discussion on font replacement is way beyond the scope of this article; I mention it only to give you a better overview of the current state of web typography. But if you are interested in knowing more, three popular font replacement technologies are Cufón (http://cufon.shoqolate.com), Font Squirrel (http://www.fontsquirrel.com), and Google Fonts API (http://code.google.com/apis/webfonts/). There is also something known as @font-face, which is part of CSS3, the latest specification of CSS. Again, it offers the tantalizing possibility of giving designers free rein in their choice of fonts. Sadly, @font-face is also hindered by browser compatibility and font licensing issues. The Font Squirrel technology, mentioned previously, resolves these issues to a certain extent, so this is something to be aware of as your web design skills develop. But for the time being, I recommend you concentrate on the basics of web typography and don't worry about @font-face until you feel ready. Usability and accessibility This is another very important area to consider when designing your blog. Many people, who live with various disabilities, use a range of 'assistive' software to access the Web. For example, people with visual impairments use screen readers, which translate the text in a web browser into audible speech. There are also people who are unable to use a mouse, and instead rely on their keyboard to access web pages. It's the responsibility of web designers to ensure that their websites are accessible for these different methods of browsing. There's no sense in alienating this group of web surfers just because your blog is inaccessible to them. There are also many other circumstances when your blog might be accessed by means other than a standard web browser, for example, via mobile phones, PDAs, or tablets. Again, a good designer will ensure that these modes of browsing are catered for. The web design industry has been well aware of these accessibility issues for many years and has come up with guidelines and technologies to help conscientious designers build websites that are standards compliant . These web standards help ensure best practice and maximize accessibility and usability. Luckily, WordPress takes care of a lot of the accessibility issues simply by the way it's been developed and built. The code behind WordPress is valid XHTML and CSS, which means that it complies with web standards and is easily accessible. It's important, then, that you don't break the system by allowing bad practice to creep in. Some of the things to bear in mind relate to a couple of design principles we've already discussed, for example, using a color scheme and font size that makes your text easy to read. Other issues include keeping the number of navigation links on your page to a minimum—a whole load of useless links can be annoying for people who have to tab through them to get to your main content. You should also ensure that any third-party plugins you install are standards-compliant and don't throw up any accessibility problems. The same is true if you decide to use a ready-made theme for your blog design. Just make sure it's accessible and satisfies web standards. For more background reading on web standards, you could take a look at http://www.alistapart.com or the World Wide Web Consortium (W3C) website at http://www.w3.org. Implementing your blog design We've now considered the principles involved in designing our blog. The next thing to decide is how we actually carry out the design work. There are three main options available, each of which involves varying degrees of work. However, they all require knowledge of the design principles we just covered. The first approach is probably the easiest; it simply involves finding a readymade theme and installing it in WordPress. By working through the earlier design principles, you should have an idea of what you want your blog to look like and then you simply need to find a theme that matches your vision as closely as possible. A good place to start looking is the official WordPress Free Themes Directory at http://wordpress.org/extend/themes/. You'll also find many more theme resources by running a search through Google. There are hundreds of very attractive WordPress themes available for free and many others which you can pay for. However, if you adopt this approach to your blog design, you won't have a unique or original blog. The chances are the theme you choose will also be seen on many other blogs. At the other end of the scale, in terms of workload, is designing your own theme from scratch. This is a fairly complicated and technical process, and is well beyond the scope of this article. In fact, it's a subject that requires its own book. If you intend to build your own theme, I recommend WordPress 2.8 Theme Design by Tessa Blakeley Silver ISBN 978-1-849510-08-0 published by Packt Publishing. The third approach is to modify a readymade theme. You could do this with any theme you choose, even the default Twenty Ten theme that ships with WordPress. However, if you edit a fully developed theme, you spend a lot of time unpicking someone else's design work and you may still be left with elements that are not completely your own. A better method is to start with a theme framework, which has been specially designed to be a blank canvas for your own design. Over the last few years many WordPress theme frameworks have appeared, some free, some paid-for. Two of the most popular paid-for theme frameworks are Thesis (http://diythemes.com/) and Genesis (http://www.studiopress.com/themes/genesis), while popular free frameworks include Hybrid (http://themehybrid.com/), Carrington (http://carringtontheme.com/), and Thematic (see below).

Selenium is a portable software testing framework for web applications. It is open-source software, released under the Apache 2.0 license, and can be downloaded and used without charge. (For more resources related to this topic, see here.) Selenium WebDriver is the successor to Selenium RC. Selenium WebDriver accepts commands and sends them to a browser. This is implemented through a browser-specific browser driver, which sends commands to a browser, and retrieves results. Selenium WebDriver is a set of open source tools and libraries to automate browsers. It has gained a wider acceptance and has become a tool of choice for automated testing on web applications. Selenium WebDriver is now part of the W3C standard. The beauty of Selenium WebDriver is that the user can write automated tests in any language, thanks to its platform agnostic approach. It provides a number of client libraries in Java, C#, Python, Ruby, JavaScript, and more to write the tests. Over the years Selenium has become a very powerful testing platform and many organizations are adopting Selenium over the other commercial tools. In the book Learning Selenium WebDriver with Python by Unmesh Gundecha, you will learn the following topics: Creating Selenium WebDriver tests using Python the unittest module Using Selenium WebDriver for cross-browser testing Building reliable and robust tests using implicit and explicit waits Setting up and using Selenium Grid for a distributed run Testing web applications on mobile platforms such as iOS and Android using Appium Using various methods provided by Selenium WebDriver to locate the web elements and interact with them Capturing screenshot and video of the test execution This book is a practical guide on automated web testing with Selenium testing tools using Python and is written for users with previous Python experience, although any previous knowledge of Selenium WebDriver is not needed. The author has provided you with step-by-step tutorials including practical examples that will help you build automated tests for testing your web applications using the Selenium WebDriver Python client library. This book is an interactive guide on automated web testing with Selenium WebDriver using Python. With the help of this book you can use Selenium for automated testing in real world, explore the Selenium WebDriver API for easy implementation of small to complex operations on browsers and web applications, and its easy and practical examples will help you get started with Selenium WebDriver. Summary The main aim of this book is to cover the fundamentals related to Python Selenium testing. You will learn how the Selenium WebDriver Python API can be integrated with CI and Build tools to allow tests to be run while building applications. This book will guide you through using the Selenium WebDriver Python client library as well as other tools from the Selenium project. Towards the end of this book, you'll get to grips with Selenium Grid, which is used for running tests in parallel using nodes for cross-browser testing. It will also give you a basic overview of the concepts, while helping you improve your practical testing skills with Python and Selenium. Resources for Article: Further resources on this subject: BackTrack 4: Security with Penetration Testing Methodology [article] Improving Plone 3 Product Performance [article] Selenium Testing Tools [article]

Time for action - files and folders for a new theme We could start off from scratch, but let's not make it more complicated than necessary: Simply copy the BlackCandy theme to a new folder in the same directory. We'll just call it bar as the page should be for a bar business. Now reload the CMS backend and you should be able to switch to the new bar theme. As we won't use any of the available images we can delete all image files, but leave the folder intact—we'll add our own pictures later on. Basic layout Next, we'll create our page's general layout. Time to put our template skills into action! Here's the basic layout we want to create: So far it's a pretty basic page. We'll take a good look at the templates and basic structure but won't spend much time on the HTML and CSS specifics. Do, however pay attention on how to set up the different CSS files to make the most out of them. We won't list every line of code involved. If you'd prefer, you can simply copy the missing parts from the code provided here (Ch:2) File themes/bar/templates/Page.ss This page is pretty empty. We'll later add an intro page with a different structure, so the code these templates will share is rather limited. Time for action - the base page Add the following code to the file themes/bar/templates/Page.ss: <!doctype html> <html lang="$ContentLocale"> <head> <meta charset="utf-8"/> <% base_tag %> <title> <% if MetaTitle %> $MetaTitle <% else %> $Title <% end_if %> </title> $MetaTags(false) <link rel="shortcut icon" href="favicon.ico"/> <!--[if lt IE 9]> <script src="http://html5shim.googlecode.com/svn/trunk/html5.js"> </script> <![endif]--> </head> <body> $Layout <noscript> <br/>&nbsp;<br/>&nbsp;<br/>&nbsp;<br/>&nbsp;<br/>&nbsp;<br/>&nbsp; <div><p> <b>Please activate JavaScript.</b><br/> Otherwise you won't be able to use all available functions properly... </p></div> </noscript> </body> </html> What just happened? The highlighted parts are SilverStripe specific: First, we set the language of our content. Depending on what you've selected during the installation process this can vary. By default, it will be: <html lang="en-US"> Setting the page's base; this can be useful when referencing images and external files. Depending on how you set up your system this will vary. Setting the title just like in the BlackCandy theme. Next, adding meta tags without the title element. These are only set if you actually enter them in the CMS on the Metadata tab on the desired page's Content tab. Empty fields are simply left out in the output. Only the SilverStripe note is set in any case. Finally, including the page's specific layout. The JavaScript we include for Internet Explorer versions before 9 fixes their inability to correctly render HTML5 tags. To learn more about the specific code, head over to https://code.google.com/p/html5shim/. Why HTML5? The motivation behind it isn't to stay buzz-word compliant. While HTML5 is not a finished standard yet, it already adds new features and makes the development a little easier. Take a look at the doctype declaration for example: that's much shorter than XHTML. Now there's a real chance of actually remembering it and not needing to copy it every time you start from scratch. New features include some very handy tags, but we'll come to those in the next code sample. The output on the final HTML page for the second and third elements (we've already taken a look at the first) in the header looks like the following listing. Which part in the template is responsible for which line or lines? <title>home</title> <meta name="generator" content="SilverStripe - http://silverstripe.org" /> <meta http-equiv="Content-type" content="text/html; charset=utf-8" /> <meta name="keywords" content="some keyword" /> <meta name="description" content="the description of this page" /> Did you notice that there is no reference to a CSS file and the styling is still applied? Remember that CSS files are automagically included if you stick to the naming convention we've already discussed. File themes/bar/templates/Layout/Page.ss Now let's move beyond the basics and do something a bit more interesting. Time for action - the layout page Add the following code to the file themes/bar/templates/Layout/Page.ss: <div> <img src="$ThemeDir/images/background.jpg" alt="Background" id="background"/> </div> <section id="content" class="transparent rounded shadow"> <aside id="top"> <% include BasicInfo %> </aside> <% include Menu %> <section class="typography"> $Content $Form </section> </section> <% include Footer %> We rely on just three includes which are very easy to reuse on different pages. We then reference the page's content and possible forms (no comments will be required). The only template part we've not yet covered is the $ThemeDir. This is replaced by the path to our theme, in our case themes/bar/. So you don't need to worry about hard-coded paths when copying template files between different themes. You only need to take care of paths in the CSS files as they are not processed by the template engine. The includes: BasicInfo.ss, Menu.ss, and Footer.ss In the previous code segment, we've referenced three includes. Let's not forget to create them. Time for action - the includes The includes we're yet to explore are Menu.ss, BasicInfo.ss, and Footer.ss. The menu is very simple, we only include the first level, highlighting the currently active item, as we won't have more subpages: <nav id="menu"> <ul> <% control Menu(1) %> <li class="$Linkingmode"> <a href="$Link">$MenuTitle</a> </li> <% end_control %> </ul> </nav> BasicInfo.ss only contains the $ThemeDir placeholder besides good old HTML: <a href="home"> <img src="$ThemeDir/images/logo.png" alt="Logo" id="logo"/> </a> <ul id="details-first"> <li>Phone: <b>01 23456789</b></li> <li>Contact: <a href="contact">contact@bar.com</a></li> <li>Address: <a href="location">Bar Street 123</a></li> </ul> <div id="details-second"> <div class="left">Opening hours:</div> <div class="right"><p> <b>Mon - Thu 2pm to 2am</b><br/> <b>Fri - Sat 2pm to 4am</b> </p></div> </div> <a href="http://www.facebook.com/pages/"> <img src="$ThemeDir/images/facebook.png" alt="Facebook" id="facebook"/> </a> Be sure to replace bar.com with your own domain. However, using it here is quite fitting because we're making a page for a bar and http://www.bar.com is only a placeholder itself. It's derived from foobar or foo bar, a placeholder name often used in computer programming. But take a look at the site yourself for more information. Finally, the footer makes use of the $Now placeholder and the date formatter .Year to retrieve the current year: <footer> <span><a href="imprint">Imprint</a></span> <span>&copy; Bar $Now.Year</span> </footer> Have a go hero - create the layout and pages Now that we've gone over all of the content, it's time to build the page. Feel free to copy some code as we're not a typing class, but take careful note of every SilverStripe specific part, so you understand what you're doing. We use three includes, so create these files and remove the rest still available from BlackCandy as well as the unused templates/Layout/Page_results.ss. And don't forget to flush the cache by appending ?flush=all to the page's URL! When working with includes it will save you a lot of trouble. We've hardcoded some links to different pages in the template files. Make sure you add all necessary pages including the imprint, but don't add it to the menu.

OpenCart 1.4 Beginner's Guide Build and manage professional online shopping stores easily using OpenCart. Develop a professional, easy-to-use, attractive online store and shopping cart solution using OpenCart that meets today's modern e-commerce standards Easily integrate your online store with one of the more popular payment gateways like PayPal and shipping methods such as UPS and USPS Provide coupon codes, discounts, and wholesale options for your customers to increase demand on your online store With hands-on examples, step-by-step explanations, and illustrations Q: What are the system requirements for OpenCart? A: The following screenshot shows the minimum system requirements for OpenCart for installation and running without problems. You should contact your hosting provider if you are not sure whether these settings are set or not. Q: What are the methods to upload files to a web host? A: There are two common methods for uploading files to a web host: Using cPanel File Manager Utility Using an FTP Client Q: Can we run more than one store on a single OpenCart installation? A: Yes. We can run more than one store on a single OpenCart installation. Q: What are GeoZones? A: Geo Zones represent the groups of countries or smaller geo sections under these countries. A Geo Zone can include countries, states, cities, and regions depending on the type of country. OpenCart uses Geo Zones to identify shipping and tax rate price regulations for a customer's order. Here is an example: Q: What if we want to edit anything in Geo Zones? A: If we want to edit any Country and / or Zone definition in Geo Zones, we should visit System | Localisation | Zones menu in the administration pane. Q: What is SEO? A: SEO (Search Engine Optimization) is a group of processes which is applied for websites to increase their visibility in search engine results to get more qualified traffic. For an online store, it is very important to apply at least the basic SEO techniques. Q: Where can we find the new modules of OpenCart? A: www.OpenCart.com contributions and forum pages are the essential sources to find new modules and/or ask for new ones from developers. Q: What do you mean by Payment Gateway? A: A payment gateway is an online analogue of a physical credit card processing terminal that we can locate in retail shops. Its function is to process credit card information and return the results back to the store system. You can imagine the payment gateway as an element in the middle of an online store and credit card network. The software part of this service is included in OpenCart but we will have to use one of the payment gateway services. Q: What are the payment methods in OpenCart? A: The current OpenCart version supports many established payment systems, including PayPal services, Authorize.net, Moneybookers, 2Checkout, and so on, as well as basic payment options such as Cash on Delivery, Bank Transfer, Check/money order, etc. Q: In which currency is the total amount calculated? A: PayPal automatically localizes the total amount according to the PayPal owner's account currency. Q: Whats the difference between PayPal Website Payment Standard and PayPal Website Payment Pro? A: PayPal Website Payment Standard is the easiest method to implement accepting credit card payments on an online store. There are no monthly fees or setup costs charged by PayPal. PayPal Website Payment Pro is the paid PayPal solution for an online store as a payment gateway and merchant account. The biggest difference from PayPal Website Payment Standard is that customers do not leave the website for credit card processing. The credit card information is completely processed in the online store as it is the popular method of all established e-commerce websites. Q: Which of the two PayPal products is recommended? A: For a beginner OpenCart administrator who wants to use PayPal for the online store, it is recommended to get experience with the free Standard payment option and then upgrade to the Pro option.

In this article by Wojciech Kocjan and Piotr Beltowski, the authors of the book Learning Nagios - Third Edition, we will look at the brief introduction of Nagios, the IT monitoring and management software for problem solving. (For more resources related to this topic, see here.) Imagine you're an administrator of a large IT infrastructure. You have just started receiving e-mails that a web application has suddenly stopped working. When you try to access the same page, it just does not load. What are the possibilities? Is it the router? Maybe the firewall? Perhaps the machine hosting the page is down? The server process has crashed? Before you even start thinking rationally about what to do, your boss calls about the critical situation and demands explanations. In all this panic, you'll probably start plugging everything in and out of the network, rebooting the machine… and it still doesn't help. After hours of nervous digging into the issue, you've finally found the root cause: although the web server was working properly, it continuously timed out during communication with the database server. This is because the machine with the database did not get an IP address assigned. Your organization requires all IP addresses to be configured using the DHCP procotol and the local DHCP server ran out of memory and killed several processes, including the dhcpd process responsible for assigning IP addresses. Imagine how much time it would take to determine all this manually! To make things worse, the database server could be located in another branch of the company or in a different time zone, and it could be the middle of the night over there. But what if you had Nagios up and running across your entire company? You would just go to the web interface and see that there are no problems with the web server and the machine on which it is running. There would also be a list of issues—the machine serving IP addresses to the entire company does not do its job and the database is down. If the setup also monitored the DHCP server, you'd get a warning e-mail that little swap memory is available or too many processes are running. Maybe it would even have an event handler for such cases to just kill or restart non critical processes. Also, Nagios would try to restart the dhcpd process over the network in case it is down. In the worst case, Nagios would reduce hours of investigation to ten minutes. Ideally, you would just get an e-mail that there was such a problem and another e-mail that it's already fixed. You would just disable a few services and increase the swap size for the DHCP machine and solve the problem permanently. Hopefully, it would be solved fast enough so that nobody would notice that there was a problem in the first place! Understanding the basics of Nagios Nagios is an open source tool for system monitoring. It means that it watches servers or other devices on your network and makes sure that they are working properly. Nagios constantly checks if other machines are working properly. It also verifies that various services on those machines are working properly. In addition, Nagios can accept information from other processes or machines regarding their status; for example, a web server can send information to Nagios if it is overloaded. The main purpose of system monitoring is to detect if any system is not working properly as soon as possible and notify the appropriate staff, and if possible, try to resolve the error—such as by restarting system services if needed. System monitoring in Nagios is split into two categories of objects, hosts and services: Hosts represent a physical or virtual device on your network (servers, routers, workstations, printers, and so on) Services are particular functionalities, for example, a Secure Shell (SSH) server (sshd process on the machine) can be defined as a service to be monitored Each service is associated with a host on which it is running. In addition, machines can be grouped into host groups, as shown here: A major benefit of Nagios' performance checks is that it only uses four distinct states—Ok, Warning, Critical, and Unknown. Another advantage is that it is based on the framework of plugins, allowing you to develop your own plugin; this means if you want to check something that's not yet possible to do, you just need to write a simple piece of code and that's it! The approach to offer only three states (excluding Unknown as it informs about not being able to determine the current state) allows administrators to ignore monitoring values and just decide on what the warning/critical limits are. This is a proven concept, and is far more efficient than monitoring graphs and analyzing trends. This is similar to traffic lights, where green indicates Ok and means a service is working correctly, a Warning state is same as the yellow light, and a Critical state is same as the red traffic light. For example, system administrators tend to ignore things such as gradually declining storage space. People often simply ignore early warnings until a critical process runs out of disk space. Having a strict limit to watch is much better, because you always catch a problem regardless of whether it turns from warning to critical in fifteen minutes or in a week. This approach is exactly what Nagios does. Each check performed by Nagios is turned from numerical values (such as the amount of disk space or CPU usage) to one of the three possible states. Another benefit is a clear report stating that X services are up and running, Y are in the warning state, and Z are currently critical, which is much more readable than a matrix of values. It saves you the time of analyzing what's working and what's failing. It can also help prioritize what needs to be handled first and which problems can be handled later. Introducing plugins Nagios performs all of its checks using plugins. These are external components for which Nagios passes information on what should be checked and what the warning and critical limits are. Plugins are responsible for performing the checks and analyzing the results. The output from such a check is the status (Ok, Warning, Critical, or Unknown) and additional text describing information on the service in detail. This text is mainly intended for system administrators to be able to read the detailed status of a service. Nagios comes with a set of standard plugins that allow performance checks for almost all the services your company might use or offer. Moreover, if you need to perform a specific check (for example, connect to a web service and invoke methods), it is very easy to write your own plugins. And that's not all—they can be written in any language and it takes less than 15 minutes to write a complete check command! Benefits of monitoring There are many reasons for you to ensure that all your resources are working as expected. If you're still not convinced after reading the introduction to this article, here are a few important points why it is important to monitor your infrastructure. The main reason is quality improvement. If your IT staff can notice failures quicker by using a monitoring tool, they will also be able to respond to them much faster. Sometimes it takes hours or days to get the first report of a failure even if many users bump into errors. Nagios ensures that if something is not working, you'll know about it. In some cases, event handling can even be done so that Nagios can switch to the backup solution until the primary process is fixed. A typical case would be to start a dial-up connection and use it as a primary connection in cases when the company VPN is down. Another reason is, much better problem determination. Very often what the users report as a failure is far from the root cause of the problem, such as when an e-mail system is down due to the LDAP service not working correctly. If you define dependencies between hosts correctly, then Nagios will point out that the IMAP or POP3 e-mail server is assumed to be not working because the LDAP service that it depends upon has a problem. Nagios will start checking the e-mail server as soon as the problem with LDAP has been resolved. Nagios is also very flexible when it comes to notifying people of what isn't functioning correctly. In most cases, your company has a large IT team or multiple teams. Usually, you want some people to handle servers and others to handle network switches/routers/modems. There might also be a team responsible for network printers, or a division is made based on geographical location. You can instruct Nagios about who is responsible for particular machines or groups of machines, so that when something is wrong, the right people will get to know about it. You can also use Nagios' web interface to manage who is working on which issue. Monitoring resources is not only useful for identifying problems, but it also saves you from having them, as Nagios handles warnings and critical situations differently. This means that it's possible to be aware of situations that may become problems really soon. For example, if your disk storage on an e-mail server is running out, it's better to be aware of this situation before it becomes a critical issue. Monitoring can also be set up on multiple machines across various locations. These machines will then communicate all their results to a central Nagios server so that information on all hosts and services in your system can be accessed from a single machine. This gives you a more accurate picture of your IT infrastructure as well as allows testing more complex systems such as firewalls. For example, it is vital that a testing environment is accessible from a production environment, but not the other way around. It is also possible to set up a Nagios server outside the company's intranet (for example, over a dedicated DSL) to make sure that traffic from the Internet is properly blocked. It can be used to check if only certain services are available, for example, to verify that only SSH and Hypertext Transfer Protocol (HTTP) are accessible from external IP addresses, and that services such as databases are inaccessible to users. Main features Nagios' main strength is flexibility—it can be configured to monitor your IT infrastructure in the way you want it. It also has a mechanism to react automatically to problems and has a powerful notification system. All of this is based on a clear object definition system, which in turn is based on a few types of object, as follows: Commands: These are definitions of how Nagios should perform particular types of check. They are an abstraction layer on top of actual plugins that allow you to group similar types of operation. Time periods: Date and time spans during which an operation should or should not be performed. For example, Monday–Friday, 09:00 A.M. – 5:00 P.M. Hosts and host groups: These are devices along with the possibility of group hosts. A single host might be a member of more than one group. Services: Various functionalities or resources to monitor on a specific host. For example, CPU usage, storage space, or web server. Contacts and contact groups: People that should be notified with information on how and when they should be contacted; contacts can be grouped, and a single contact might be a member of more than one group. Notifications: These define who should be notified of what, for example, all errors related to the linux-servers host group should go to the linux-admins contact group during working hours and to the critsit-team (critical situations handling team) contact group outside of working hours. Notifications are not strictly an object, but a combination of all the preceding objects and are an essential part of Nagios. Escalations: Extension to notifications; escalations define these after an object is in the same state for a specific period of time, other people should get notified of certain events. For example, a critical server being down for more than four hours should alert IT management so that they track the issue. A beneficial feature of using Nagios is that it is a mature dependency system. For any administrator, it is obvious that if your router is down then all machines accessed via it will fail. Some systems don't take that into account, and in such cases, you get a list of several failing machines and services. Nagios allows you to define dependencies between hosts to reflect actual network topology. For example, if a router that connects you to the rest of your network is down, Nagios will not perform checks for the subsequent parts and machines that are dependent on the router. This is illustrated in the following image: You can also define that a particular service depends on another service, either on the same host or a different host. If one of the dependent services is down, a check for a service is not even performed. For example, in order for your company's intranet application to function properly, both the underlying web and database servers must be running properly. So, if a database service is not working properly, Nagios will not perform checks and/or not send notifications that your application is not working, because the root cause of the problem is that the database is not working properly. The database server might be on the same or on a different host. If the database is not working properly or the dependent machine is down or not accessible, all services dependent on the database service will not be checked as well. Nagios offers a consistent system of macro definitions. These are variables that can be put into all object definitions and depend on the context. They can be put inside commands, and depending on the host, service, and many other parameters, macro definitions are substituted accordingly. For example, a command definition might use an IP address of the host it is currently checking in all remote tests. It also makes it possible to put information such as the previous and current status of a service in a notification e-mail. Nagios also offers various extensions to macro definitions, which makes it an even more powerful mechanism. Additionally, there is a built-in mechanism for scheduling planned downtimes. This is mainly used when maintenance of the IT infrastructure is to be carried out, and servers and/or services they provide are out of order for a period of time. You can let Nagios know that such an event will occur, and it will not send notifications about problems with hosts and/or services that have a scheduled downtime. In such cases, dependencies are also taken into consideration—if a database has a scheduled downtime, notifications for the services that depend on it will not be sent out. Nagios can also notify people of planned downtimes automatically. This allows creating an integrated process of scheduling downtimes that will also handle informing users. Soft and hard states Nagios works by checking if a particular host or service is working correctly and storing its status. Because the status is only one of four possible values, it is crucial that it precisely reflects the actual situation. In order to avoid detecting random and temporary failures, Nagios uses soft and hard states to describe what the current status is for a host or service. Imagine that an administrator is restarting a web server, which in turn makes web pages inaccessible for around five seconds. Since such restarts are usually done at night to lower the number of users affected, this is an acceptable period of time. However, a problem might be that Nagios will try to connect to the server and notice that it is actually down. If it only relies on a single result, Nagios could trigger an alert that a web server is down. It would actually be up and running again in a few seconds, but Nagios would require another couple of minutes to find that out. To handle situations where a service is down for a very short time, or the test has temporarily failed, soft states were introduced. When a previous status of a check is unknown or is different from the previous one, Nagios will retest the host or service a couple of times to make sure the change is permanent. Nagios assumes that the new result is a soft state. After additional tests have verified that the new state is permanent, it is considered a hard one. Each host and service check defines the number of retries to perform before assuming a change to be permanent. This allows more flexibility over how many failures should be treated as an actual problem instead of a temporary one. Setting the number of checks to 1 will cause all changes to be treated as hard instantly. The following figure is an illustration of soft and hard state changes, assuming that number of checks to be performed is set to 3: This feature is very useful for checks that should skip short outages of a service or use a protocol that might fail in case of extensive traffic—such as ICMP or UDP. Monitoring devices over SNMP is also an example of a check that can fail in cases where a single check fails; nevertheless, the check will eventually succeed during the second or third check. Summary In this article, you learned the basics of Nagios as a tool for performing system monitoring. It can be used to ensure that services are working correctly, problems are detected earlier, and appropriate people are aware when something's wrong. You learned the basic types of objects in Nagios—commands, hosts, services, time periods, contacts, as well as object grouping. You also found out about notifications and escalations, which can be used to notify administrators about problems. The article also introduced the concept of dependencies that helps in understanding the root cause of complex problems. Resources for Article: Further resources on this subject: Passive Checks and NSCA (Nagios Service Check Acceptor)[article] Notifications and Events in Nagios 3.0-part1 [article] Notifications and Events in Nagios 3.0- part2 [article]

(For more resources related to this topic, see here.) Supervised learning for classification Like clustering, classification is also about categorizing data instances, but in this case, the categories are known and are termed as class labels. Thus, it aims at identifying the category that a new data point belongs to. It uses a dataset where the class labels are known to find the pattern. Classification is an instance of supervised learning where the learning algorithm takes a known set of input data and corresponding responses called class labels and builds a predictor model that generates reasonable predictions for the class labels in the unknown data. To illustrate, let's imagine that we have gene expression data from cancer patients as well as healthy patients. The gene expression pattern in these samples can define whether the patient has cancer or not. In this case, if we have a set of samples for which we know the type of tumor, the data can be used to learn a model that can identify the type of tumor. In simple terms, it is a predictive function used to determine the tumor type. Later, this model can be applied to predict the type of tumor in unknown cases. There are some do's and don'ts to keep in mind while learning a classifier. You need to make sure that you have enough data to learn the model. Learning with smaller datasets will not allow the model to learn the pattern in an unbiased manner and again, you will end up with an inaccurate classification. Furthermore, the preprocessing steps (such as normalization) for the training and test data should be the same. Another important thing that one should take care of is to keep the training and test data distinct. Learning on the entire data and then using a part of this data for testing will lead to a phenomenon called over fitting. It is always recommended that you take a look at it manually and understand the question that you need to answer via your classifier. There are several methods of classification. In this recipe, we will talk about some of these methods. We will discuss linear discriminant analysis (LDA), decision tree (DT), and support vector machine (SVM). Getting ready To perform the classification task, we need two preparations. First, a dataset with known class labels (training set), and second, the test data that the classifier has to be tested on (test set). Besides this, we will use some R packages, which will be discussed when required. As a dataset, we will use approximately 2300 gene from tumor cells. The data has ~83 data points with four different types of tumors. These will be used as our class labels. We will use 60 of the data points for the training and the remaining 23 for the test. To find out more about the dataset, refer to the Classification and diagnostic prediction of cancers using gene expression profiling and artificial neural networks article by Khan and others (http://research.nhgri.nih.gov/microarray/Supplement/). The set has been precompiled in a format that is readily usable in R and is available on the book's web page (code files) under the name cancer.rda. How to do it… To classify data points based on their features, perform the following steps: First, load the following MASS library as it has some of the classification functions: > library(MASS) Now, you need your data to learn and test the classifiers. Load the data from the code files available on the book's web page (cancer.rda) as follows: > load ("path/to/code/directory/cancer.rda") # located in the code file directory for the chapter, assign the path accordingly Randomly sample 60 data points for the training and the remaining 23 for the test set as follows—ensure that these two datasets do not overlap and are not biased towards any specific tumor type (random sampling): > train <- mldata[train_row,] # use sampled indexes to extract training data > test <- mldata[-train_row,] # test set is select by selecting all the other data points For the training data, retain the class labels, which are the tumor columns here, and remove this information from the test data. However, store this information for comparison purposes: > testClass <- test$tumor > test$tumor <- NULL Now, try the linear discriminate analysis classifier, as follows, to get the classifier model: > myLD <- lda(tumor ~ ., train) # might issue a warning Test this classifier to predict the labels on your test set, as follows: > testRes_lda <- predict(myLD, test) To check the number of correct and incorrect predictions, simply compare the predicted classes with the testClass object, which was created in step 4, as follows: > sum(testRes_lda$class == testClass) # correct prediction [1] 19 > sum(testRes_lda$class != testClass) # incorrect prediction [1] 4 Now, try another simple classifier called DT. For this, you need the rpart package: > library(rpart) Create the decision tree based on your training data, as follows: > myDT <- rpart(tumor~ ., data = train, control = rpart.control(minsplit = 10)) Plot your tree by typing the following commands, as shown in the next diagram: > plot(myDT) > text(myDT, use.n=T) The following screenshot shows the cut off for each feature (represented by the branches) to differentiate between the classes: The tree for DT-based learning Now, test the decision tree classifier on your test data using the following prediction function: > testRes_dt <- predict(myDT, newdata= test) Take a look at the species that each data instance is put in by the predicted classifier, as follows (1 if predicted in the class, else 0): > classes <- round(testRes_dt) > head(classes) BL EW NB RM 4 0 0 0 1 10 0 0 0 1 15 1 0 0 0 16 0 0 1 0 18 0 1 0 0 21 0 1 0 0 Finally, you'll work with SVMs. To be able to use them, you need another R package named e1071 as follows: > library(e1071) Create the svm classifier from the training data as follows: > mySVM <- svm(tumor ~ ., data = train) Then, use your classifier, the model (mySVM object) learned to predict for the test data. You will see the predicted labels for each instance as follows: > testRes_svm <- predict(mySVM, test) > testRes_svm How it works… We started our recipe by loading the input data on tumors. The supervised learning methods we saw in the recipe used two datasets: the training set and test set. The training set carries the class label information. The first part in most of the learning methods shown here, the training set is used to identify a pattern and model the pattern to find a distinction between the classes. This model is then applied on the test set that does not have the class label data to predict the class labels. To identify the training and test sets, we first randomly sample 60 indexes out of the entire data and use the remaining 23 for testing purposes. The supervised learning methods explained in this recipe follow a different principle. LDA attempts to model the difference between classes based on the linear combination of its features. This combination function forms the model based on the training set and is used to predict the classes in the test set. The LDA model trained on 60 samples is then used to predict for the remaining 23 cases. DT is, however, a different method. It forms regression trees that form a set of rules to distinguish one class from the other. The tree learned on a training set is applied to predict classes in test sets or other similar datasets. SVM is a relatively complex technique of classification. It aims to create a hyperplane(s) in the feature space, making the data points separable along these planes. This is done on a training set and is then used to assign classes to new data points. In general, LDA uses linear combination and SVM uses multiple dimensions as the hyperplane for data distinction. In this recipe, we used the svm functionality from the e1071 package, which has many other utilities for learning. We can compare the results obtained by the models we used in this recipe (they can be computed using the provided code on the book's web page). There's more... One of the most popular classifier tools in the machine learning community is WEKA. It is a Java-based tool and implements many libraries to perform classification tasks using DT, LDA, Random Forest, and so on. R supports an interface to the WEKA with a library named RWeka. It is available on the CRAN repository at http://cran.r-project.org/web/packages/RWeka/ . It uses RWekajars, a separate package, to use the Java libraries in it that implement different classifiers. See also The Elements of Statistical Learning book by Hastie, Tibshirani, and Friedman at http://statweb.stanford.edu/~tibs/ElemStatLearn/printings/ESLII_print10.pdf, which provides more information on LDA, DT, and SVM

In the first half of this 2-part article series, we looked at the service module so as to evaluate potential actions that are triggered, based on service-related information. We also introduced a concept which explained how to utilize the service module features to establish a guaranteed response time for customers. We also learnt about : Key terms - The common terminology related to service management were covered. Although nothing major , we went about learning what the terms entail with regards to the SAP system. Service module core functions – In this section, the available functions and features were put into perspective—what is available and how much we can expect from it.  You also learnt what service operations mean. Case study and your own project – The available features of the service module were implemented for the case study. By doing so, knowledge was provided to implement the service module in your own business. We also reviewed some guidelines which enabled you to translate the case study implementation into a set of activities for your own project. Let's begin with Service reports. Service reports The crucial element of each module is the information that can easily be extracted for reporting purposes. SAP provides a series of canned reports for the service module. The Service Calls report provides information about service call activities based on the selected criteria. You can filter this report by timeframe of service call creation and also by resolution time. Additional filter ranges are available for Customer Code, Handled By, Item, and also Queue ID. In addition, the report allows filtering by Problem Type, Priority, Call Type, Origin, Call Status, and Overdue Calls. You can see that there is a wide range of options to obtain information. It is important to note that reporting can utilize the information only if all of the data is properly collected using the SAP forms. In case no options are selected for filtering, the report defaults to select all of the available information. The following additional service reports are available and have almost identical filtering capabilities as the service calls report: Service Calls by Queue, Response Time by Assigned to, Average Closure Time. The Service Contracts report(seen below) helps you manage the status of all maintenance contracts. If you've ever had to manage maintenance contracts with customers, you will appreciate the ease of obtaining the information here. You can filter this report by Customer Code, Start Date Range, End Date Range and Termination Date Range. In addition, this report can be further filtered by Contract Type, Contract Status, and Service Type. The customer equipment card report allows information to be obtained about items sold to customers based on serial number tracking. Each serial number has its own contract with expiration which is usually based on the purchase date. This report is able to be filtered by customer and item code. In addition, a more global filter can be used such as item group. The Service Monitor report provides a more real-time view of the service pipeline. We already covered this report in the previous section. Finally, My Reports includes My Service Calls, My Open Service Calls, and My Overdue Service Calls. Those reports conveniently filter the information based on the current login name. Therefore, if you run the report, you will only see information that is relevant for you based on the login. Limitations I have already mentioned that you can look at SAP Business ONE as the operating system for your business. You can use industry add-ons to seamlessly transform the standard features into an industry solution that is specific to your requirements. Therefore, let's evaluate some add-ons I've worked with that are related to the service module—specifically, the Enprise Job Costing module and the solution from Navigator called ServiceONE. By looking at these add-ons, we can also learn the limitations of the standard service module. For example, since the Navigator promises to have all of the information available in one view, we realize that in SAP, we sometimes need to jump to different forms to get where we wanted to be initially. Let's further evaluate the features of these add-ons. The Enprise Job Costing solution introduces a web-based timesheet. This is an obvious feature that is not directly available in the SAP standard configuration. First, I will look at the Navigator solution and will then follow the Enprise offering. Often, there is more than one add-on providing industry-specific features. You then need to evaluate both solutions and decide which one best fi ts your requirements. Please note that I am presenting the add-on features to better define the limitations of the SAP Standard Business ONE features. Job Costing add-on by Enprise The Enprise Job Costing add-on is one of the first industry-specific solutions that gained widespread adoption as a standard for companies that required a detailed job costing solution. The advantage of this solution is that it is based on true expertise in the job costing area as it relates to the SAP service module. Let's look at a scenario that is very common for companies that work in the service industry and require what is known as job costing. However, I would first like to take the opportunity to explain job costing a bit. Job costing allows the profit and loss for specific services provided to be calculated. For example, if you have a company that sends out technicians to customer sites for performing equipment repairs, you need to make sure that the invoiced amount exceeds the cost you incur. The following workflow may be common in this environment: Serialized items are delivered to customers with each having a warranty contract that may or may not include services, parts, and replacements. Services may be performed on serialized items delivered by you or by another company. A service call may lead to a proposal (job) which will then be ordered. Technicians may use a timesheet to report the status and time. Timesheet entries must be possible via a mobile device or the Web. Furthermore, time entered needs to be approved before it is relevant for invoicing. A job may lead to subjobs that require unique management of related costs. As the number of jobs increase, you will require "work in progress" reporting. Estimating a job is crucial. Therefore, technicians need to be able to create estimates. It must then be possible to translate those estimates into orders and contracts. As services may require replacement parts, a feature is required that allows optimized picking of relevant items for a specific job. Complex jobs require milestone payments. This needs to be implemented in the contracts. The invoicing system needs to be integrated with the way services are completed. For example, milestone payments, fixed priced billing, and partial invoicing are common requirements in the service industry. The Enprise Job Costing add-on resides in its own menu item called Job Costing. As you can see below, the menu items are well defined and provide a quick overview of the available features. In addition, it is important to note that the features seamlessly appear within the SAP interface: By selecting the Job Entry form, the powerful features come to light. As you can see in the following screenshot, the form allows searching jobs based on Status,Type, Properties, Category, and Entered By. The resulting list is shown in the lower pane. We can use this interface to search for specific jobs, and then click on the Bulk Invoice button as highlighted in the screenshot. This automates the invoicing process based on a clear, uncluttered form. Please note that we do not need to jump between multiple forms. The Direct Time Entry form(seen below) is basically a timesheet. Therefore, technicians can use this to enter the time they spend on projects. Please note the buttons in the lower right that allow importing from the Web and also from an Excel clipboard. Enprise provides a web-based timesheet from which we can import data. However, it is important to note that we can also import from an Excel clipboard. This way, we can use the date that technicians entered in their laptops. The contract list provides a link where the Enprise-enhanced contract management surfaces: The contract management allows milestones for a contract. Each milestone to be established could lead to a milestone payment. In addition, we can directly jump to the related invoices by using the Show Invoices button: Enprise has adopted the concept of master data. For this purpose, the Job Master Data form was established. This is consistent with the SAP concept. Each job is defined and configured with specific parameters, which later drive the transactions that are based on this master data. For example, we can define the job parameters alongside a list of subjobs. In addition, documents can be attached as attachments: Advanced service functionality using ServiceONE by Navigator Navigator provides a wide range of valuable add-ons. Each add-on is valuable. However, the key advantage of Navigator is the comprehensive portfolio of add-ons the cover almost all aspects of SAP Business ONE. In particular, the fact that Navigator also provides a mobile solution, which connects handheld computers with SAP Business ONE, extends the reach of the available functionality beyond the boundaries of the SAP client interface. Therefore, a mobile field service does not need to use a web-based timesheet, but could directly interact using mobile devices. However, you may need to purchase another add-on to accomplish this.

In this article by Andrew Dennis, the author of Raspberry Pi Computer Architecture Essentials, we will discuss about Assembly language and the assembler. (For more resources related to this topic, see here.) Assembly language The Raspberry Pi comes equipped with an ARM v7 quad core processor. Each processor has its own set of specific machine code that it understands; this machine code is represented in the binary format. The machine code is different for each processor architecture, so the Raspberry Pi's ARM processor machine code will not work on an IBM or Intel CPU. Short of writing out 32-bit long binary machine code instructions, the lowest level of programming language we can find ourselves using is Assembler language, also known as Assembly language. The computer architecture's Assembly language is usually a one-to-one mapping between itself and the underlying machine code. This is achieved through using a mnemonic. A combination of these mnemonic codes will result in an operation, such as addition or subtraction. A program written in the Assembly language is compiled into machine code by the Assembler program. This program passes through the code one or more times and generates an object file as part of this process. The Assembler in some cases will also perform a variety of optimizations on the code in its subsequent passes. Following this a program called the Linker that generates an executable file you can run on your computer. Two important terms you will come across when writing Assembly language are opcode and operand. The opcode is an instruction (such as add) and the operand is data (such as an integer value). Each opcode and operand is created through the combination of sets of 8 bits (1 byte). In this article, we will write a simple program in Assembly language in order to understand the basics. The subject of the ARM v7 Assembly language is covered in more detail by the University of Michigan that hosts a useful guide to the ARM v7 architecture in PDF format at https://web.eecs.umich.edu/~prabal/teaching/eecs373-f10/readings/ARMv7-M_ARM.pdf. You may be interested in reviewing this as a supplement to the topics covered here. So, what do the mnemonic codes that make up Assembly language look like before being converted to machine code? Let's take a look at an example and see. Here, we demonstrate how we can take register 0 of the CPU and assign a number to it; in this case, 10. MOV R0, #10 The Assembly code MOV is short hand for assigning a value. The register is an example of the processor's internal memory storage location and, of course, 10 is an integer value. You can read more on CPU registers at Wikipedia: https://en.wikipedia.org/wiki/Processor_register As you explore the language further, you will become familiar with these types of commands, as they are the building blocks of your program. How about looking at another example? What do you think this does? ADD R0, R1, R2 This simple program introduces us to another mnemonic, ADD. Here, we are taking the values of registers 1 and 2, adding and assigning them to register 0. Running commands like this on the Raspberry Pi is very simple; we can add them to a file assemble and link them ourselves. We shall now explore a short Assembly language program that incorporates these two commands, MOV and ADD. Let's start by creating a new directory under the pi user: mkdir /home/pi/assem_programs This will be the place we store our Assembly code. Navigate into this directory, for example: cd /home/pi/assem_programs Next, we need to create a new file to place our code in. You can choose any text editor you are comfortable with in order to write the program. We have used Vim in the following example: vim first_assem_prog.s To this file, add the following block of code. Make sure that you include the spacing as demonstrated next:     .global main .func main main: MOV R0, #0 MOV R1, #10 MOV R2, #20 ADD R0, R1, R2 BX LR So, what does this program do? The first line in the program defines a directive called main. The prefix of .global tells the Assembler that the name is global and thus available to the C runtime. A directive is a code executed by the Assembler at assembly time, rather than the processor. We could have called this directive anything, but we have gone with main to keep it consistent with our C program. Assembler, unlike C, does not require the program entry point to be called main. As you will see, we will use the GCC compiler/linker to build an executable for our program, so the format we are writing the Assembly language in mimics that of a C program in some areas. This is why you will see references to the C runtime mentioned when discussing Assembly in this article. Following this, we then define that main is a function. Here, we can see another directive .func is used to specify this. So, now that we have main available, we can denote where this function starts, which in our case is the third line. Contained in the function are three lines of code for adding values to the registers. From our earlier examples, these should be familiar. What we have done is assigned the value 0 to R0, 10 to R1, 20 to R2 and then added the values together and stored the result (30) in R0. Finally, we call BX LR to return the value of register 0 back to the operating system. As you can see, this program is very simple, but demonstrates how to add numbers and store the results. Save the file and exit your text editor. You should now be back at the command line. This leads us to the next step of assembling and linking in order to generate a file we can run. Assembling and linking Now we have a program we need to test it. This is a two-step process that involves assembling the code and then linking it, which we touched upon at the start of this article. When you come to explore the C language next, you will see that linking is also a component there as well; in fact, we use the same tool for both C and Assembly—the GCC compiler. Briefly, these two steps to generate a runnable program that can be summed up as: Assembling is the process of generating the machine code object file from the Assembly mnemonics Linking is the process of creating an executable from one or more object files The first command we will run called as (the GNU assembler) will take the code we wrote previously and create an object file as its output. Run the following command from inside of the folder where you created your program: as –o first_assem_prog.o first_assem_prog.s If it assembled correctly, you should see no output. Following this, we need to run the linker, which is invoked with the gcc command. There is also another linker available called ld. However, since we are writing our Assembly in a C-like manner, we will use the gcc tool. You will also need to run this command in the same directory that you ran as in: gcc –o first_assem_prog first_assem_prog.o GCC stands for the GNU Compiler Collection If everything is successful, you shouldn't see any output. We now have an executable file we can run from the Linux command line. To do this, you can simply type: ./first_assem_prog You'll notice there is no output, however. So, how do we know whether the program executed correctly? We can use the Linux echo command, as follows: echo $? This displays the exit code of the previous process, which in our case is the result of program we just ran. You may remember that we wrote this value back using the BX LR code. As our program simply returned a value of 30 to register 0, this is the result we can see when using the echo command. You can try changing the values in your program and assembling and linking once more. The result you see when running echo should reflect your changes. Try changing the program to use R1 instead of R0 in the add function and see what happens. So, in a few easy steps, you have created an Assembly language program and learned how to assemble, link, and run it. Summary In this article, we explored the programming languages we will be using in this book. This included Assembler and C/C++. Resources for Article: Further resources on this subject: Raspberry Pi and 1-Wire [article] Raspberry Pi Gaming Operating Systems [article] Raspberry Pi LED Blueprints [article]

Within the past five years, there has been a major change in the type of content found on the World Wide Web. In just a few short years, content has evolved from being primarily text and images, into a multimedia experience! Drupal contributors have put much effort in making this integration with multimedia as easy as possible. However, one issue still remains: in order to present multimedia to your users, you cannot rely on Drupal alone. You must have another application layer to present that media. This is most typically a Flash application that allows the user to listen or watch that media from within their web browser. This article explores how to use Drupal to manage a list of audio nodes and also builds a Flash application to play that music. When it comes to multimedia, Flash is the portal of choice for playing audio on a web sites. Integrating audio in Drupal is surprisingly easy, thanks to the contribution of the Audio module. This module allows you to upload audio tracks to your Drupal website (typically in MP3 format), by creating an Audio node. It also comes with a very basic audio player that will play those audio tracks in the node that was created. To start, let's download and enable the Audio module along with the Token, Views, and getID3 modules, which are required for the Audio module. The modules that you will need to download and install are as follows: Audio—http://www.drupal.org/project/audio Views—http://www.drupal.org/project/views Token—http://www.drupal.org/project/token getID3—http://www.drupal.org/project/getid3 At the time of writing this article, the Audio module was still considered "unstable". Because of this, I would recommend downloading the development version until a stable release has been made. It is also recommended to use the development or "unstable" versions for testing purposes only. Once we have downloaded these modules and placed them in our site's modules folder, we can enable the Audio module by first navigating to the Administer | Modules section, and then enabling the checkboxes in the Audio group as follows: After you have enabled these modules, you will probably notice an error at the top of the Administrator section that says the following: This error is shown because we have not yet installed the necessary PHP library to extract the ID3 information from our audio files. The ID3 information is the track information that is embedded within each audio file, and can save us a lot of time from having to manually provide that information when attaching each audio file to our Audio nodes. So, our next step will be to install the getID3 library so that we can utilize this great feature. Installing the getID3 library The getID3 library is a very useful PHP library that will automatically extract audio information (called ID3) from any given audio track. We can install this useful utility by going to http://sourceforge.net/project/showfiles.php?group_id=55859, which is the getID3 library URL at SourceForge.net. Once we have done this, we should see the following: We can download this library by clicking on the Download link on the first row, which is the main release. This will then take us to a new page, where we can download the ZIP package for the latest release. We can download this package by clicking on the latest ZIP link, which at the time of writing this article was getid3-1.7.9.zip Once this package has finished downloading, we then need to make sure that we place the extracted library on the server where the getID3 module can use it. The default location for the getID3 module, for this library, is within our site's modules/getid3 directory. Within this directory, we will need to create another directory called getid3, and then place the getid3 directory from the downloaded package into this directory. To verify that we have installed the library correctly, we should have the getid3.php at the following location: Our next task is to remove the demos folder from within the getid3 library, so that we do not present any unnecessary security holes in our system. Once this library is in the correct spot, and the demos folder has been removed, we can refresh our Drupal Administrator section and see that the error has disappeared. If it hasn't, then verify that your getID3 library is in the correct location and try again. Now that we have the getID3 library installed, we are ready to set up the Audio content type. Setting up the Audio content type When we installed the Audio module, it automatically created an Audio content type that we can now use to add audio to our Drupal web site. But before we add any audio to our web site, let's take a few minutes to set up the Audio content type to the way we want it. We will do so by navigating to Administer | Content Types, and then clicking on the edit link, next to the Audio content type. Our goal here is to set up the Audio content type so that the default fields make sense to the Audio content type. Drupal adds the Body field to all new content types, which doesn't make much sense when creating an Audio content. We can easily change this by simply expanding the Submission form settings. We can then replace the Body label with Description, since it is easily understood when adding new Audio tracks to our system. We will save this content type by clicking on the Save content type button at the bottom of the page. Now, we are ready to start adding audio content to our Drupal web site. Creating an Audio node We will add audio content by going to Create Content, and then clicking on Audio, where we should then see the following on the page: You will probably notice that the Title of this form has already been filled out with some strange looking text (as shown in the previous screenshot). This text is a series of tags, which are used to represent track information that is extracted using the getID3 module that we installed earlier. Once this ID3 information is extracted, these tags will be replaced with the Title and Artist of that track, and then combined to form the title of this node. This will save a lot of time because we do not have to manually provide this information when submitting a new audio track to our site. We can now upload any audio track by clicking on the Browse button next to the Add a new audio file field. After it adds the file to the field, we can submit this audio track to Drupal by clicking on the Save button at the bottom of the page, which will then show you something like the following screenshot: After this node has been added, you will notice that there is a player already provided to play the audio track. Although this player is really cool, there are some key differences between the player provided by the Audio module and the player that we will create later in this article. How our player will be different (and better) The main difference between the player that is provided by the Audio module and the player that we are getting ready to build is how it determines which file to play. In the default player, it uses flash variables passed to the player to determine which file to play. This type of player-web site interaction places the burden on Drupal to provide the file that needs to be played. In a way, the default player is passive, where it does nothing unless someone tells it to do something. The player that we will be building is different because instead of Drupal telling our player what to play, we will take an active approach and query Drupal for the file we wish to play. This has several benefits, such as that the file path does not have to be exposed to the public in order for it to be played. So, let's create our custom player!

Kushal Sharma: What is the vision behind Miro? Nicholas Reville: There's an opportunity to build a new, open mass medium of online television. We're developing the Miro Internet TV platform so that watching Internet video channels will be as easy as watching TV and broadcasting a channel will be open to everyone. Unlike traditional TV, everyone will have a voice. KS: Does PCF finance the entire project or do you have any other contributors? NR: We have tons of help from volunteers – translating the software, coding, testing, and providing user support. We would not be able to do nearly enough without our community. KS: Are the developers full-time PCF employees or is it similar to other Open Source projects where people voluntarily contribute to the community in their spare-time? NR: We have 6 full-time developers and also volunteers. We're a hybrid model, like Mozilla. KS: Please highlight the most crucial features of Miro, and the idea behind having those features as part of this application. NR: The most crucial feature of Miro is the ability to download and play video RSS feeds. It's a truly open way to distribute video online. Using RSS means that creators can publish with any company they want and users can bring together video from multiple sources into one application. KS: How many languages is Miro translated into? NR: Miro is at least partially translated into more than 40 languages, but we always need helping revising and improving the translations. KS: How is Miro different from other players from the technological perspective? NR: Above all, Miro is open-source. That means that anyone can work on the code, improve, and customize it. Beyond this, Miro is unique in a number of ways. It runs on Mac, Windows, and Linux. It can play almost any video format. And it has more HD content available than any other Internet TV application. KS: Are the Torrent download capabilities as well developed as any other standalone Bit Torrent Client? Please tell us something more about it's download capabilities, the share ratio, configurable upload limits while downloading the torrents etc. compared to other software? NR: Our current version of Miro keeps the BitTorrent capabilities very simple and doesn't provide many options. We think that most users don't even notice when they are downloading a torrent feed, however, we want to expand our options and features for power users in future versions – expect much more bit torrent control in 3 months or so. KS: What type of support do you have for Miro? NR: Most of our support is provided user-to-user in the discussion forums – they are very useful, actually. In addition, because we are an open project, anyone can file a bug or even fix a bug. In the long-term this means we will have a more stable user experience than closed competitors. KS: With a host of TV channels and video content going online, viewers could really use a common solution for all their media needs rather than keeping tabs on multiple sources. How do you see Miro being instrumental in providing this? NR: We think that an open platform for video is the only way to create a unified experience for video. Miro is the only really open Internet video solution right now and we think we have a crucial role to play in unifying Internet TV in an open way. KS: Does Miro download Internet TV programs and store it on the hard disk or does it stream live media like any other online service? NR: For now, Miro downloads everything that's watched. This is useful in two key ways: first, you can watch HD video with no skipping or buffering delays. Second, you can move those files onto any external device, with no DRM or other restrictions. In the future we may add a streaming option for some special cases, or the ability to start watching while the download is in progress. KS: Subscription-free content is a great gift that viewers get with Internet TV, however, bandwidth issues could be a concern for some users. What minimum bandwidth requirements would you suggest for satisfactory Internet TV performance on Miro? Furthermore, does Miro have an alternate solution for users having low Internet bandwidth? NR: Miro actually works better than most Internet video solutions for low bandwidth users. On a dial-up connection, streaming video solutions are unusable. Miro will download videos in those circumstances – it may happen slowly, but once you have the video you can watch it full speed with no skipping. KS: I remember a statement on the PCF site saying “Miro is designed to eliminate gatekeepers”. Could you please elaborate on this? NR: Miro works with open standards and is designed to be decentralized, like the web. That means that users can use Miro to connect directly to any video publisher – they don't need our permission and the files don't travel through our servers. Companies like Joost design their software to be highly centralized so that they can control both users and creators. It's time to leave that model behind. KS: So far, what has the response been like? NR: Response has been great. Last month alone, we had more than 200,000 downloads and we've been growing with each release. I expect that when we release version 1.0 in November we'll see even faster growth. KS: What are your future plans for Miro in terms of releases, updates, functionality, etc.? NR: After version 1.0 is released, we'll be making major changes to Miro to improve performance and add an extension system that will give people new ways of customizing the software to fit their needs. KS: To me, Miro’s agenda is a lot more than simply creating a good player. You’re attempting to change the face of Internet Video and the way it’s being hosted right now. How would you describe the future of Miro and Internet Video to our readers? NR: We want Miro to push online video in an open direction. We're hoping to build the best video experience possible, something that can be a true substitute for traditional television. But that doesn't mean we want to control the future of online video – we want other people to build open video distribution platforms as well. Openness is vital to the future of our mass media. KS: What other projects is the PCF involved in? NR: Right now, PCF is exclusively focused on making video more open and Miro is at the center of that. KS: Thank you for your time Nicholas, and good luck developing Miro!

Spark is written in a language called Scala. It has interfaces to use from Java and Python and from the recent version 1.4.0; it also supports R. This is called SparkR, which we will describe in the next section. The four classes of libraries available in Spark are SQL and DataFrames, Spark Streaming, MLib (machine learning), and GraphX (graph algorithms). Currently, SparkR supports only SQL and DataFrames; others are definitely in the roadmap. Spark can be downloaded from the Apache project page at http://spark.apache.org/downloads.html. Starting from 1.4.0 version, SparkR is included in Spark and no separate download is required. (For more resources related to this topic, see here.) SparkR Similar to RHadoop, SparkR is an R package that allows R users to use Spark APIs through the RDD class. For example, using SparkR, users can run jobs on Spark from RStudio. SparkR can be evoked from RStudio. To enable this, include the following lines in your .Rprofile file that R uses at startup to initialize the environments: Sys.setenv(SPARK_HOME/.../spark-1.5.0-bin-hadoop2.6") #provide the correct path where spark downloaded folder is kept for SPARK_HOME .libPaths(c(file.path(Sys.getenv("SPARK_HOME"),""R",""lib"),".libPaths())) Once this is done, start RStudio and enter the following commands to start using SparkR: >library(SparkR) >sc ← sparkR.init(master="local") As mentioned, as of the latest version 1.5 when this chapter is in writing, SparkR supports limited functionalities of R. This mainly includes data slicing and dicing and summary stat functions. The current version does not support the use of contributed R packages; however, it is planned for a future release. On machine learning, currently SparkR supports the glm( ) function. We will do an example in the next section. Linear regression using SparkR In the following example, we will illustrate how to use SparkR for machine learning. >library(SparkR) >sc ← sparkR.init(master="local") >sqlContext ← sparkRSQL.init(sc) #Importing data >df ← read.csv("/Users/harikoduvely/Projects/Book/Data /ENB2012_data.csv",header = T) >#Excluding variable Y2,X6,X8 and removing records from 768 containing mainly null values >df ← df[1:768,c(1,2,3,4,5,7,9)] >#Converting to a Spark R Dataframe >dfsr ← createDataFrame(sqlContext,df) >model ← glm(Y1 ~ X1 + X2 + X3 + X4 + X5 + X7,data = dfsr,family = "gaussian") > summary(model) Summary In this article we have seen examples of SparkR and linear regression using SparkR. For more information on Spark you can refer to: https://www.packtpub.com/big-data-and-business-intelligence/spark-python-developers https://www.packtpub.com/big-data-and-business-intelligence/spark-beginners Resources for Article: Further resources on this subject: Data Analysis Using R[article] Introducing Bayesian Inference[article] Bayesian Network Fundamentals[article]

Google Web Toolkit 2 Application Development Cookbook Over 70 simple but incredibly effective practical recipes to develop web applications using GWT with JPA , MySQL and i Report Create impressive, complex browser-based web applications with GWT 2 Learn the most effective ways to create reports with parameters, variables, and subreports using iReport Create Swing-like web-based GUIs using the Ext GWT class library Develop applications using browser quirks, Javascript,HTML scriplets from scratch Part of Packt's Cookbook series: Each recipe is a carefully organized sequence of instructions to complete the task as efficiently as possible   The layout will be as shown in the diagram below: Creating the home page layout class This recipe creates a panel to place the menu bar, banner, sidebars, footer, and the main application layout. Ext GWT provides several options to define the top-level layout of the application. We will use the BorderLayout function. We will add the actual widgets after the layout is fully defined. The other recipes add the menu bar, banner, sidebars, and footers each, one-by-one. Getting ready Open the Sales Processing System project. How to do it... Let's list the steps required to complete the task. Go to File | New File. Select Java from Categories, and Java Class from File Types. Click on Next. Enter HomePage as the Class Name, and com.packtpub.client as Package. Click on Finish. Inherit the class ContentPanel. Press Ctrl + Shift + I to import the package automatically. Add a default constructor: package com.packtpub.client; import com.extjs.gxt.ui.client.widget.ContentPanel; public class HomePage extends ContentPanel { public HomePage() { } } Write the code of the following steps in this constructor. Set the size in pixels for the content panel: setSize(980,630); Hide the header: setHeaderVisible(false); Create a BorderLayout instance and set it for the content panel: BorderLayout layout = new BorderLayout(); setLayout(layout); Create a BorderLayoutData instance and configure it to be used for the menu bar and toolbar: BorderLayoutData menuBarToolBarLayoutData= new BorderLayoutData(LayoutRegion.NORTH, 55); menuBarToolBarLayoutData.setMargins(new Margins(5)); Create a BorderLayoutData instance and configure it to be used for the left-hand sidebar: BorderLayoutData leftSidebarLayoutData = new BorderLayoutData(LayoutRegion.WEST, 150); leftSidebarLayoutData.setSplit(true); leftSidebarLayoutData.setCollapsible(true); leftSidebarLayoutData.setMargins(new Margins(0, 5, 0, 5)); Create a BorderLayoutData instance and configure it to be used for the main contents, at the center: BorderLayoutData mainContentsLayoutData = new BorderLayoutData(LayoutRegion.CENTER); mainContentsLayoutData.setMargins(new Margins(0)); Create a BorderLayoutData instance and configure it to be used for the right-hand sidebar: BorderLayoutData rightSidebarLayoutData = new BorderLayoutData(LayoutRegion.EAST, 150); rightSidebarLayoutData.setSplit(true); rightSidebarLayoutData.setCollapsible(true); rightSidebarLayoutData.setMargins(new Margins(0, 5, 0, 5)); Create a BorderLayoutData instance and configure it to be used for the footer: BorderLayoutData footerLayoutData = new BorderLayoutData(LayoutRegion.SOUTH, 20); footerLayoutData.setMargins(new Margins(5)); How it works... Let's now learn how these steps allow us to complete the task of designing the application for the home page layout. The full page (home page) is actually a "content panel" that covers the entire area of the host page. The content panel is a container having top and bottom components along with separate header, footer, and body sections. Therefore, the content panel is a perfect building block for application-oriented user interfaces. In this example, we will place the banner at the top of the content panel. The body section of the content panel is further subdivided into five regions in order to place these—the menu bar and toolbar at the top, two sidebars on each side, a footer at the bottom, and a large area at the center to place the contents like forms, reports, and so on. A BorderLayout instance lays out the container into five regions, namely, north, south, east, west, and center. By using BorderLayout as the layout of the content panel, we will get five places to add five components. BorderLayoutData is used to specify layout parameters of each region of the container that has BorderLayout as the layout. We have created five instances of BorderLayoutData, to be used in the five regions of the container. There's more... Now, let's talk about some general information that is relevant to this recipe. Setting the size of the panel The setSize method is used to set the size for a panel. Any one of the two overloaded setSize methods can be used. A method has two int parameters, namely, width and height. The other one takes the same arguments as string. Showing or hiding header in the content panel Each content panel has built-in headers, which are visible by default. To hide the header, we can invoke the setHeaderVisible method, giving false as the argument, as shown in the preceding example. BorderLayoutData BorderLayoutData is used to set the layout parameters, such as margin, size, maximum size, minimum size, collapsibility, floatability, split bar, and so on for a region in a border panel. Consider the following line of code in the example we just saw: BorderLayoutData leftSidebarLayoutData = new BorderLayoutData(LayoutRegion.WEST, 150) It creates a variable leftSidebarLayoutData, where the size is 150 pixels and the region is the west of the border panel. rightSidebarLayoutData.setSplit(true) sets a split bar between this region and its neighbors. The split bar allows the user to resize the region. leftSidebarLayoutData.setCollapsible(true) makes the component collapsible, that is, the user will be able to collapse and expand the region. leftSidebarLayoutData.setMargins(new Margins(0, 5, 0, 5)) sets a margin where 0, 5, 0, and 5 are the top, right, bottom, and left margins, respectively. Classes and packages In the preceding example, some classes are used from Ext GWT library, as shown in the following table: ClassPackageBorderLayoutcom.extjs.gxt.ui.client.widget.layoutBorderLayoutDatacom.extjs.gxt.ui.client.widget.layoutContentPanelcom.extjs.gxt.ui.client.widgetMarginscom.extjs.gxt.ui.client.utilStylecom.extjs.gxt.ui.client See also The Adding the banner recipe The Adding menus recipe The Creating the left-hand sidebar recipe The Creating the right-hand sidebar recipe The Creating main content panel recipe The Creating the footer recipe The Using HomePage instance in EntryPoint recipe Adding the banner This recipe will create a method that we will use to add a banner in the content panel. Getting ready Place the banner image banner.png at the location webresourcesimages. You can use your own image or get it from the code sample provided on the Packt Publishing website (www.packtpub.com). How to do it... Create the method getBanner: public ContentPanel getBanner() { ContentPanel bannerPanel = new ContentPanel(); bannerPanel.setHeaderVisible(false); bannerPanel.add(new Image("resources/images/banner.png")); Image("resources/images/banner.png")); return bannerPanel; } Call the method setTopComponent of the ContentPanel class in the following constructor: setTopComponent(getBanner()); How it works... The method getBanner() creates an instance bannerPanel of type ContentPanel. The bannerPanel will just show the image from the location resources/images/banner.png. That's why, the header is made invisible by invoking setHeaderVisible(false). Instance of the com.google.gwt.user.client.ui.Image class, which represents the banner image, is added in the bannerPanel. In the default constructor of the HomePage class, the method setTopComponent(getBanner()) is called to set the image as the top component of the content panel. See also The Creating the home page layout class recipe The Adding menus recipe The Creating the left-hand sidebar recipe The Creating the right-hand sidebar recipe The Creating main content panel recipe The Creating the footer recipe The Using HomePage instance in EntryPoint recipe  

Basic DOJO 123 accordion In my earlier article I had used the version of the Toolkit which had the accordion in the Widgets. In the latest version which I am using, the accordion is found in digit/layout. The code is similar to that in the earlier article. Basically you create a accordion container and then place the accordion panes inside the container. In referencing the Dojo library I am using part of the references from the Dojo Toolkit 123 installed on my local IIS and part of the reference from the AOL site (uses the 1.0.0 script). Listing 1: AccordionOrig.htm: A basic accordion with three panes [DOJO 123] <head> <meta http-equiv="Content-Type" content="text/html; charset=UTF-8"> <title>Accordion Pane with jQueries</title> <style type="text/css"> @import "http://localhost/Dojo123/dojo123/dijit/themes/tundra/tundra.css"; @import "http://localhost/Dojo123/dojo123/dojo/resources/dojo.css" </style> <script type="text/javascript" src="http://o.aolcdn.com/dojo/1.0.0/dojo/dojo.xd.js" djConfig="parseOnLoad: true"></script> <script type="text/javascript"> dojo.require("dojo.parser"); dojo.require("dijit.layout.AccordionContainer"); </script> </head> <body class="tundra"> <div dojoType="dijit.layout.AccordionContainer" duration="200" style="margin-right: 30px; width: 400px; height: 400px; overflow: scroll"> <!--Pane 1 --> <div dojoType="dijit.layout.AccordionPane" selected="true" title="Page 1" style="color:red;overflow: scroll; background-color:#FFFF80;"> <!--Pane 1 content--> <p >Test 1</a></p > </div> <!--Pane 2 --> <div dojoType="dijit.layout.AccordionPane" title="Page 2" style="overflow: scroll;background-color:#FFFF80;"> <!--Pane 2 content--> <p >Test 2</p > </div> <!-- Pane 3--> <div dojoType="dijit.layout.AccordionPane" title="Page 3" style="color:magenta;overflow: scroll;background-color:#FFFF80;"> <!--Pane 3 content--> <p >Test 3</a></p > </div> </div> </body> </html> This page when browsed to, will display the accordion as shown in Figure 1. This was cross-browser compatible in the following browsers: IE 6.0, Opera 9.1, Firefox 3.0.5, and Safari 3.2.1. The page did not render correctly (all panes completely open) in Google Chrome 1.0.154.43. Figure 1 jQuery API Components used in the article jQuery 1.3 downloaded from this site is used as a source for the script. From the API reference only two simple components were chosen to be embedded in the panes - the Selector and the Effects. The slideUp() effect where in, when you click on the code sensitive area the region of the area on the web page slides up. H1 Selector styled using jQuery Using jQuery you can selectively apply style to tags, ids, etc. In the example shown in the code that follows the H1 tag is styled using jQuery. Listing 2: H1SelectorJQry.htm: Tag styling with jQuery <html> <head></head> <body> <script language="JavaScript" src="http://localhost/JayQuery/jquery-1.3.min.js"> </script> <h1>Jquery inside a DOJO Accordion Pane</h1> <script type="text/JavaScript"> $(document).ready(function(){ $("h1").css("color", "magenta");}); </script> </body> <html> In the above, the jQuery code (inside the script tags) renders the h1 tag in the color shown as in Figure 2. Figure 2 jQuery Effect: slideUp() The htm page with the listing 3 when browsed to, displays a pale green 300 x 300 square corresponding to the styling of the p tag as shown in Figure 4. When clicked anywhere inside this square, the square slides up and disappears. This is the slideUp() effect. Listing 3: p_slideUp.htm: Jquery Effect <html> <head></head> <body> <script language="JavaScript" src="http://localhost/JayQuery/jquery-1.3.min.js"> </script> <div><p style="width:300; height:300; background-color:palegreen; color:darkgreen;">Test</p></div> <script type="text/JavaScript"> $("p").click(function () { $(this).slideUp(); }); </script> </body> <html> This page gets displayed as shown in Figure 3. When you click anywhere in the pale green area the "P" region slides up. Figure 3