Search icon CANCEL
Subscription
0
Cart icon
Your Cart (0 item)
Close icon
You have no products in your basket yet
Save more on your purchases! discount-offer-chevron-icon
Savings automatically calculated. No voucher code required.
Arrow left icon
Explore Products
Best Sellers
New Releases
Books
Videos
Audiobooks
Learning Hub
Newsletter Hub
Free Learning
Arrow right icon
timer SALE ENDS IN
0 Days
:
00 Hours
:
00 Minutes
:
00 Seconds

How-To Tutorials - Web Development

1802 Articles
article-image-getting-started-emberjspart2
Daniel Ochoa
11 Jan 2016
5 min read
Save for later

Getting started with Ember.js – Part 2

Daniel Ochoa
11 Jan 2016
5 min read
In Part 1 of this blog, we got started with Ember.js by examining how to set up your development environment from beginning to end with Ember.js using ember-cli – Ember’s build tool. Ember-cli minifies and concatenates your JavaScript, giving you a strong conventional project structure and a powerful add-on system for extensions. In this Part 2 post, I’ll guide you through the setting up of a very basic todo-like Ember.js application to get your feet wet with actual Ember.js development. Setting up a more detailed overview for the posts Feel free to change the title of our app header (see Part 1). Go to ‘app/templates/application.hbs’ and change the wording inside the h2 tag to something like ‘Funny posts’ or anything you’d like. Let’s change our app so when a user clicks on the title of a post, it will take them to a different route based on the id of the post, for example, /posts/1bbe3 . By doing so, we are telling ember to display a different route and template. Next, let's run the following on the terminal: ember generate route post This will modify our app/router.js file by creating a route file for our post and a template. Let’s go ahead and open the app/router.js file to make sure it looks like the following: import Ember from 'ember'; import config from './config/environment'; var Router = Ember.Router.extend({ location: config.locationType }); Router.map(function() { this.resource('posts'); this.route('post', {path: '/posts/:post_id'}); }); export default Router; In the router file, we make sure the new ‘post’ route has a specific path by passing it a second argument with an object that contains a key called path and a value of ‘/posts/:post_id’. The colon in that path means the second parth of the path after /posts/ is a dynamic URL. In this URL, we will be passing the id of the post so we can determine what specific post to load on our post route. (So far, we have posts and post routes, so don’t get confused). Now, let's go to app/templates/posts.hbs and make sure we only have the following: <ul> {{#each model as |post|}} {{#link-to 'post' post tagName='li'}} {{post.title}} {{/link-to}} {{/each}} </ul> As you can see, we replaced our <li> element with an ember helper called ‘link-to’. What link-to does is that it generates for you the link for our single post route. The first argument is the name of the route, ‘post’, the second argument is the actual post itself and in the last part of the helper, we are telling Handlebars to render the link to as a <li> element by providing the tagName property. Ember is smart enough to know that if you link to a route and pass it an object, your intent is to set the model on that route to a single post. Now open ‘app/templates/post.hbs’ and replace the contents with just the following: {{model.title }} Now if you refresh the app from ‘/posts’ and click on a post title, you’ll be taken to a different route and you’ll see only the title of the post. What happens if you refresh the page at this URL? You’ll see errors on the console and nothing will be displayed. This is because you arrived at this URL from the previous post's route where you passed a single post as the argument to be the model for the current post route. When you hit refresh you lose this step so no model is set for the current route. You can fix that by adding the following to ‘app/routes/post.js’ : import Ember from 'ember'; export default Ember.Route.extend({ model(params) { return Ember.$.getJSON('https://www.reddit.com/tb/' + params.post_id + '.json?jsonp=?').then(result => { return result[0].data.children[0].data; }); } }); Now, whenever you refresh on a single post page, Ember will see that you don’t have a model so the model hook will be triggered on the route. In this case, it will grab the id of the post from the dynamic URL, which is passed as an argument to the query hook and it will make a request to reddit for the relevant post. In this case, notice that we are also returned the request promise and then filtering the results to only return the single post object we need. Change the app/templates/post.hbs template to the following: <div class="title"> <h1>{{model.title}}</h1> </div> <div class="image"> <img src="{{model.preview.images.firstObject.source.url}}" height="400"/> </div> <div class="author"> submitted by: {{model.author}} </div> Now, if you look at an individual post, you’ll get the title, image, and author for the post. Congratulations, you’ve built your first Ember.js application with dynamic data and routes. Hopefully, you now have a better grasp and understanding of some basic concepts for building more ambitious web applications using Ember. About the Author: Daniel Ochoa is a senior software engineer at Frog with a passion for crafting beautiful web and mobile experiences. His current interests are Node.js, Ember.js, Ruby on Rails, iOS development with Swift, and the Haskell language. He can be found on Twitter @DanyOchoaOzz.
Read more
  • 0
  • 0
  • 20386

article-image-migrating-wordpress-blog-middleman-and-deploying-amazon-s3
Mike Ball
07 Nov 2014
11 min read
Save for later

Migrating a WordPress Blog to Middleman and Deploying to Amazon S3

Mike Ball
07 Nov 2014
11 min read
Part 1: Getting up and running with Middleman Many of today’s most prominent web frameworks, such as Ruby on Rails, Django, Wordpress, Drupal, Express, and Spring MVC, rely on a server-side language to process HTTP requests, query data at runtime, and serve back dynamically constructed HTML. These platforms are great, yet developers of dynamic web applications often face complex performance challenges under heavy user traffic, independent of the underlying technology. High traffic, and frequent requests, may exploit processing-intensive code or network latency, in effect yielding a poor user experience or production outage. Static site generators such as Middleman, Jeckyll, and Wintersmith offer developers an elegant, highly scalable alternative to complex, dynamic web applications. Such tools perform dynamic processing and HTML construction during build time rather than runtime. These tools produce a directory of static HTML, CSS, and JavaScript files that can be deployed directly to a web server such as Nginx or Apache. This architecture reduces complexity and encourages a sensible separation of concerns; if necessary, user-specific customization can be handled via client-side interaction with third-party satellite services. In this three part series, we'll walk-through how to get started in developing a Middleman site, some basics of Middleman blogging, how to migrate content from an existing WordPress blog, and how to deploy a Middleman blog to production. We will also learn how to create automated tests, continuous integration, and automated deployments. In this part, we’ll cover the following: Creating a basic Middleman project Middleman configuration basics A quick overview of the Middleman template system Creating a basic Middleman blog Why should you use middleman? Middleman is a mature, full-featured static site generator. It supports a strong templating system, numerous Ruby-based HTML templating tools such as ERb and HAML, as well as a Sprockets-based asset pipeline used to manage CSS, JavaScript, and third-party client-side code. Middleman also integrates well with CoffeeScript, SASS, and Compass. Environment For this tutorial, I’m using an RVM-installed Ruby 2.1.2. I’m on Mac OSX 10.9.4. Installing middleman Install middleman via bundler: $ gem install middleman Create a basic middleman project called middleman-demo: $ middleman init middleman-demo This results in a middleman-demo directory with the following layout: ├── Gemfile ├── Gemfile.lock ├── config.rb └── source    ├── images    │   ├── background.png    │   └── middleman.png    ├── index.html.erb    ├── javascripts    │   └── all.js    ├── layouts    │   └── layout.erb    └── stylesheets        ├── all.css        └── normalize.css[SB4]  There are 5 directories and 10 files. A quick tour Here are a few notes on the middleman-demo layout: The Ruby Gemfile  cites Ruby gem dependencies; Gemfile.lock cites the full dependency chain, including  middleman-demo’s dependencies’ dependencies The config.rb  houses middleman-demo’s configuration The source directory houses middleman-demo ’s source code–the templates, style sheets, images, JavaScript, and other source files required by the  middleman-demo [SB7] site While a Middleman production build is simply a directory of static HTML, CSS, JavaScript, and image files, Middleman sites can be run via a simple web server in development. Run the middleman-demo development server: $ middleman Now, the middleman-demo site can be viewed in your web browser at  http://localhost:4567. Set up live-reloading Middleman comes with the middleman-livereload gem. The gem detects source code changes and automatically reloads the Middleman app. Activate middleman-livereload  by uncommenting the following code in config.rb: # Reload the browser automatically whenever files change configure :development do activate :livereload end Restart the middleman server to allow the configuration change to take effect. Now, middleman-demo should automatically reload on change to config.rb and your web browser should automatically refresh when you edit the source/* code. Customize the site’s appearance Middleman offers a mature HTML templating system. The source/layouts directory contains layouts, the common HTML surrounding individual pages and shared across your site. middleman-demo uses ERb as its template language, though Middleman supports other options such as HAML and Slim. Also note that Middleman supports the ability embed metadata within templates via frontmatter. Frontmatter allows page-specific variables to be embedded via YAML or JSON. These variables are available in a current_page.data namespace. For example, source/index.html.erb contains the following frontmatter specifying a title; it’s available to ERb templates as current_page.data.title: --- title: Welcome to Middleman --- Currently, middleman-demo is a default Middleman installation. Let’s customize things a bit. First, remove all the contents of source/stylesheets/all.css  to remove the default Middleman styles. Next, edit source/index.html.erb to be the following: --- title: Welcome to Middleman Demo --- <h1>Middleman Demo</h1> When viewing middleman-demo at http://localhost:4567, you’ll now see a largely unstyled HTML document with a single Middleman Demo heading. Install the middleman-blog plugin The middleman-blog plugin offers blog functionality to middleman applications. We’ll use middleman-blog in middleman-demo. Add the middleman-blog version 3.5.3 gem dependency to middleman-demo by adding the following to the Gemfile: gem "middleman-blog", "3.5.3 Re-install the middleman-demo gem dependencies, which now include middleman-blog: $ bundle install Activate middleman-blog and specify a URL pattern at which to serve blog posts by adding the following to config.rb: activate :blog do |blog| blog.prefix = "blog" blog.permalink = "{year}/{month}/{day}/{title}.html" end Write a quick blog post Now that all has been configured, let’s write a quick blog post to confirm that middleman-blog works. First, create a directory to house the blog posts: $ mkdir source/blog The source/blog directory will house markdown files containing blog post content and any necessary metadata. These markdown files highlight a key feature of middleman: rather than query a relational database within which content is stored, a middleman application typically reads data from flat files, simple text files–usually markdown–stored within the site’s source code repository. Create a markdown file for middleman-demo ’s first post: $ touch source/blog/2014-08-20-new-blog.markdown Next, add the required frontmatter and content to source/blog/2014-08-20-new-blog.markdown: --- title: New Blog date: 2014/08/20 tags: middleman, blog --- Hello world from Middleman! Features Rich templating system Built-in helpers Easy configuration Asset pipeline Lots more  Note that the content is authored in markdown, a plain text syntax, which is evaluated by Middleman as HTML. You can also embed HTML directly in the markdown post files. GitHub’s documentation provides a good overview of markdown. Next, add the following ERb template code to source/index.html.erb [SB37] to display a list of blog posts on middleman-demo ’s home page: <ul> <% blog.articles.each do |article| %> <li> <%= link_to article.title, article.path %> </li> <% end %> </ul> Now, when running middleman-demo and visiting http://localhost:4567, a link to the new blog post is listed on middleman-demo ’s home page. Clicking the link renders the permalink for the New Blog blog post at blog/2014-08-20/new-blog.html, as is specified in the blog configuration in config.rb. A few notes on the template code Note the use of a link_to method. This is a built-in middleman template helper. Middleman provides template helpers to simplify many common template tasks, such as rendering an anchor tag. In this case, we pass the link_to method two arguments, the intended anchor tag text and the intended href value. In turn, link_to generates the necessary HTML. Also note the use of a blog variable, within which an article’s method houses an array of all blog posts. Where did this come from?  middleman-demo is an instance of  Middleman::Application;  a blog  method on this instance. To explore other Middleman::Application methods, open middleman-demo via the built-in Middleman console by entering the following in your terminal: $ middleman console To view all the methods on the blog, including the aforementioned articles method, enter the following within the console: 2.1.2 :001 > blog.methods To view all the additional methods, beyond the blog, available to the Middleman::Application instance, enter the following within the console: 2.1.2 :001 > self.methods More can be read about all these methods on Middleman::Application’s rdoc.info class documentation. Cleaner URLs Note that the current new blog URL ends in .html. Let’s customize middleman-demo to omit .html from URLs. Add the following config.rb: activate :directory_indexes Now, rather than generating files such as /blog/2014-08-20/new-blog.html,  middleman-demo generates files such as /blog/2014-08-20/new-blog/index.html, thus enabling the page to be served by most web servers at a /blog/2014-08-20/new-blog/ path. Adjusting the templates Let’s adjust our the middleman-demo ERb templates a bit. First, note that <h1>Middleman Demo</h1> only displays on the home page; let’s make it render on all of the site’s pages. Move <h1>Middleman Demo</h1> from  source/index.html.erb  to source/layouts/layout.erb. Put it just inside the <body> tag: <body class="<%= page_classes %>"> <h1>Middleman Demo</h1> <%= yield %> </body> Next, let’s create a custom blog post template. Create the template file: $ touch source/layout/post.erb Add the following to extend the site-wide functionality of source/layouts/layout.erb to  source/layouts/post.erb: <% wrap_layout :layout do %> <h2><%= current_article.title %></h2> <p>Posted <%= current_article.date.strftime('%B %e, %Y') %></p> <%= yield %> <ul> <% current_article.tags.each do |tag| %> <li><a href="/blog/tags/<%= tag %>/"><%= tag %></a></li> <% end %> </ul> <% end %> Note the use of the wrap_layout  ERb helper.  The wrap_layout ERb helper takes two arguments. The first is the name of the layout to wrap, in this case :layout. The second argument is a Ruby block; the contents of the block are evaluated within the <%= yield %> call of source/layouts/layout.erb. Next, instruct  middleman-demo  to use  source/layouts/post.erb  in serving blog posts by adding the necessary configuration to  config.rb : page "blog/*", :layout => :post Now, when restarting the Middleman server and visiting  http://localhost:4567/blog/2014/08/20/new-blog/,  middleman-demo renders a more comprehensive blog template that includes the post’s title, date published, and tags. Let’s add a simple template to render a tags page that lists relevant tagged content. First, create the template: $ touch source/tag.html.erb And add the necessary ERb to list the relevant posts assigned a given tag: <h2>Posts tagged <%= tagname %></h2> <ul> <% page_articles.each do |post| %> <li> <a href="<%= post.url %>"><%= post.title %></a> </li> <% end %> </ul> Specify the blog’s tag template by editing the blog configuration in config.rb: activate :blog do |blog| blog.prefix = 'blog' blog.permalink = "{year}/{month}/{day}/{title}.html" # tag template: blog.tag_template = "tag.html" end Edit config.rb to configure middleman-demo’s tag template to use source/layout.erb rather than source/post.erb: page "blog/tags/*", :layout => :layout Now, when visiting http://localhost:4567/2014/08/20/new-blog/, you should see a linked list of New Blog’s tags. Clicking a tag should correctly render the tags page. Part 1 recap Thus far, middleman-demo serves as a basic Middleman-based blog example. It demonstrates Middleman templating, how to set up the middleman-blog  plugin, and how to make author markdown-based blog posts in Middleman. In part 2, we’ll cover migrating content from an existing Wordpress blog. We’ll also step through establishing an Amazon S3 bucket, building middleman-demo, and deploying to production. In part 3, we’ll cover how to create automated tests, continuous integration, and automated deployments. About this author Mike Ball is a Philadelphia-based software developer specializing in Ruby on Rails and JavaScript. He works for Comcast Interactive Media where he helps build web-based TV and video consumption applications.
Read more
  • 0
  • 0
  • 20173

article-image-using-classes
Packt
28 Nov 2014
26 min read
Save for later

Using Classes

Packt
28 Nov 2014
26 min read
In this article by Meir Bar-Tal and Jonathon Lee Wright, authors of Advanced UFT 12 for Test Engineers Cookbook, we will cover the following recipes: Implementing a class Implementing a simple search class Implementing a generic Login class Implementing function pointers (For more resources related to this topic, see here.) Introduction This article describes how to use classes in VBScript, along with some very useful and illustrative implementation examples. Classes are a fundamental feature of object-oriented programming languages such as C++, C#, and Java. Classes enable us to encapsulate data fields with the methods and properties that process them, in contrast to global variables and functions scattered in function libraries. UFT already uses classes, such as with reserved objects, and Test Objects are also instances of classes. Although elementary object-oriented features such as inheritance and polymorphism are not supported by VBScript, using classes can be an excellent choice to make your code more structured, better organized, and more efficient and reusable. Implementing a class In this recipe, you will learn the following: The basic concepts and the syntax required by VBScript to implement a class The different components of a class and interoperation How to implement a type of generic constructor function for VBScript classes How to use a class during runtime Getting ready From the File menu, navigate to New | Function Library…, or use the Alt + Shift + N shortcut. Name the new function library cls.MyFirstClass.vbs and associate it with your test. How to do it... We will build our MyFirstClass class from the ground up. There are several steps one must follow to implement a class; they are follows: Define the class as follows: Class MyFirstClass Next, we define the class fields. Fields are like regular variables, but encapsulated within the namespace defined by the class. The fields can be private or public. A private field can be accessed only by class members. A public field can be accessed from any block of code. The code is as follows: Class MyFirstClass Private m_sMyPrivateString Private m_oMyPrivateObject Public m_iMyPublicInteger End Class It is a matter of convention to use the prefix m_ for class member fields; and str for string, int for integer, obj for Object, flt for Float, bln for Boolean, chr for Character, lng for Long, and dbl for Double, to distinguish between fields of different data types. For examples of other prefixes to represent additional data types, please refer to sites such as https://en.wikipedia.org/wiki/Hungarian_notation. Hence, the private fields' m_sMyPrivateString and m_oMyPrivateObject will be accessible only from within the class methods, properties, and subroutines. The public field m_iMyPublicInteger will be accessible from any part of the code that will have a reference to an instance of the MyFirstClass class; and it can also allow partial or full access to private fields, by implementing public properties. By default, within a script file, VBScript treats as public identifiers such as function and subroutines and any constant or variable defined with Const and Dim respectively,, even if not explicitly defined. When associating function libraries to UFT, one can limit access to specific globally defined identifiers, by preceding them with the keyword Private. The same applies to members of a class, function, sub, and property. Class fields must be preceded either by Public or Private; the public scope is not assumed by VBScript, and failing to precede a field identifier with its access scope will result in a syntax error. Remember that, by default, VBScript creates a new variable if the explicit option is used at the script level to force explicit declaration of all variables in that script level. Next, we define the class properties. A property is a code structure used to selectively provide access to a class' private member fields. Hence, a property is often referred to as a getter (to allow for data retrieval) or setter (to allow for data change). A property is a special case in VBScript; it is the only code structure that allows for a duplicate identifier. That is, one can have a Property Get and a Property Let procedure (or Property Set, to be used when the member field actually is meant to store a reference to an instance of another class) with the same identifier. Note that Property Let and Property Set accept a mandatory argument. For example: Class MyFirstClass Private m_sMyPrivateString Private m_oMyPrivateObject    Public m_iMyPublicInteger  Property Get MyPrivateString()    MyPrivateString = m_sMyPrivateString End Property Property Let MyPrivateString(ByVal str)    m_sMyPrivateString = str End Property Property Get MyPrivateObject()    Set MyPrivateObject = m_oMyPrivateObject End Property Private Property Set MyPrivateObject(ByRef obj)    Set m_oMyPrivateObject = obj End Property End Class The public field m_iMyPublicInteger can be accessed from any code block, so defining a getter and setter (as properties are often referred to) for such a field is optional. However, it is a good practice to define fields as private and explicitly provide access through public properties. For fields that are for exclusive use of the class members, one can define the properties as private. In such a case, usually, the setter (Property Let or Property Set) would be defined as private, while the getter (Property Get) would be defined as public. This way, one can prevent other code components from making changes to the internal fields of the class to ensure data integrity and validity. Define the class methods and subroutines. A method is a function, which is a member of a class. Like fields and properties, methods (as well as subroutines) can be Private or Public. For example: Class MyFirstClass '… Continued Private Function MyPrivateFunction(ByVal str)    MsgBox TypeName(me) & " – Private Func: " & str    MyPrivateFunction = 0 End Function Function MyPublicFunction(ByVal str)    MsgBox TypeName(me) & " – Public Func: " & str    MyPublicFunction = 0 End Function Sub MyPublicSub(ByVal str)    MsgBox TypeName(me) & " – Public Sub: " & str End Sub End Class Keep in mind that subroutines do not return a value. Functions by design should not return a value, but they can be implemented as a subroutine. A better way is to, in any case, have a function return a value that tells the caller if it executed properly or not (usually zero (0) for no errors and one (1) for any fault). Recall that a function that is not explicitly assigned a value function and is not explicitly assigned a value, will return empty, which may cause problems if the caller attempts to evaluate the returned value. Now, we define how to initialize the class when a VBScript object is instantiated: Set obj = New MyFirstClass The Initialize event takes place at the time the object is created. It is possible to add code that we wish to execute every time an object is created. So, now define the standard private subroutine Class_Initialize, sometimes referred to (albeit only by analogy) as the constructor of the class. If implemented, the code will automatically be executed during the Initialize event. For example, if we add the following code to our class: Private Sub Class_Initialize MsgBox TypeName(me) & " started" End Sub Now, every time the Set obj = New MyFirstClass statement is executed, the following message will be displayed: Define how to finalize the class. We finalize a class when a VBScript object is disposed of (as follows), or when the script exits the current scope (such as when a local object is disposed when a function returns control to the caller), or a global object is disposed (when UFT ends its run session): Set obj = Nothing The Finalize event takes place at the time when the object is removed from memory. It is possible to add code that we wish to execute, every time an object is disposed of. If so, then define the standard private subroutine Class_Terminate, sometimes referred to (albeit only by analogy) as the destructor of the class. If implemented, the code will automatically be executed during the Finalize event. For example, if we add the following code to our class: Private Sub Class_Terminate MsgBox TypeName(me) & " ended" End Sub Now, every time the Set obj = Nothing statement is executed, the following message will be displayed: Invoking (calling) a class method or property is done as follows: 'Declare variables Dim obj, var 'Calling MyPublicFunction obj.MyPublicFunction("Hello") 'Retrieving the value of m_sMyPrivateString var = obj.MyPrivateString 'Setting the value of m_sMyPrivateString obj.MyPrivateString = "My String" Note that the usage of the public members is done by using the syntax obj.<method or property name>, where obj is the variable holding the reference to the object of class. The dot operator (.) after the variable identifier provides access to the public members of the class. Private members can be called only by other members of the class, and this is done like any other regular function call. VBScript supports classes with a default behavior. To utilize this feature, we need to define a single default method or property that will be invoked every time an object of the class is referred to, without specifying which method or property to call. For example, if we define the public method MyPublicFunction as default: Public Default Function MyPublicFunction(ByVal str) MsgBox TypeName(me) & " – Public Func: " & str MyPublicFunction = 0 End Function Now, the following statements would invoke the MyPublicFunction method implicitly: Set obj = New MyFirstClass obj("Hello") This is exactly the same as if we called the MyPublicFunction method explicitly: Set obj = New MyFirstClass obj.MyPublicFunction("Hello") Contrary to the usual standard for such functions, a default method or property must be explicitly defined as public. Now, we will see how to add a constructor-like function. When using classes stored in function libraries, UFT (know as QTP in previous versions), cannot create an object using the New operator inside a test Action. In general, the reason is linked to the fact that UFT uses a wrapper on top of WSH, which actually executes the VBScript (VBS 5.6) code. Therefore, in order to create instances of such a custom class, we need to use a kind of constructor function that will perform the New operation from the proper memory namespace. Add the following generic constructor to your function library: Function Constructor(ByVal sClass) Dim obj On Error Resume Next 'Get instance of sClass Execute "Set obj = New [" & sClass & "]" If Err.Number <> 0 Then    Set obj = Nothing      Reporter.ReportEvent micFail, "Constructor", "Failed      to create an instance of class '" & sClass & "'." End If Set Constructor = obj End Function We will then instantiate the object from the UFT Action, as follows: Set obj = Constructor("MyFirstClass") Consequently, use the object reference in the same fashion as seen in the previous line of code: obj.MyPublicFunction("Hello") How it works... As mentioned earlier, using the internal public fields, methods, subroutines, and properties is done using a variable followed by the dot operator and the relevant identifier (for example, the function name). As to the constructor, it accepts a string with the name of a class as an argument, and attempts to create an instance of the given class. By using the Execute command (which performs any string containing valid VBScript syntax), it tries to set the variable obj with a new reference to an instance of sClass. Hence, we can handle any custom class with this function. If the class cannot be instantiated (for instance, because the string passed to the function is faulty, the function library is not associated to the test, or there is a syntax error in the function library), then an error would arise, which is gracefully handled by using the error-handling mechanism, leading to the function returning nothing. Otherwise, the function will return a valid reference to the newly created object. See also The following articles at www.advancedqtp.com are part of a wider collection, which also discuss classes and code design in depth: An article by Yaron Assa at http://www.advancedqtp.com/introduction-to-classes An article by Yaron Assa at http://www.advancedqtp.com/introduction-to-code-design An article by Yaron Assa at http://www.advancedqtp.com/introduction-to-design-patterns Implementing a simple search class In this recipe, we will see how to create a class that can be used to execute a search on Google. Getting ready From the File menu, navigate to New | Test, and name the new test SimpleSearch. Then create a new function library by navigating to New | Function Library, or use the Alt + Shift + N shortcut. Name the new function library cls.Google.vbs and associate it with your test. How to do it... Proceed with the following steps: Define an environment variable as OPEN_URL. Insert the following code in the new library: Class GoogleSearch Public Function DoSearch(ByVal sQuery)    With me.Page_      .WebEdit("name:=q").Set sQuery      .WebButton("html id:=gbqfba").Click    End With    me.Browser_.Sync     If me.Results.WaitProperty("visible", 1, 10000) Then      DoSearch = GetNumResults()    Else      DoSearch = 0      Reporter.ReportEvent micFail, TypeName(Me),        "Search did not retrieve results until timeout"    End If End Function Public Function GetNumResults()    Dim tmpStr    tmpStr = me.Results.GetROProperty("innertext")    tmpStr = Split(tmpStr, " ")    GetNumResults = CLng(tmpStr(1)) 'Assumes the number      is always in the second entry End Function Public Property Get Browser_()    Set Browser_ = Browser(me.Title) End Property Public Property Get Page_()    Set Page_ = me.Browser_.Page(me.Title) End Property Public Property Get Results()    Set Results = me.Page_.WebElement(me.ResultsId) End Property Public Property Get ResultsId()    ResultsId = "html id:=resultStats" End Property Public Property Get Title()    Title = "title:=.*Google.*" End Property Private Sub Class_Initialize    If Not me.Browser_.Exist(0) Then      SystemUtil.Run "iexplore.exe",        Environment("OPEN_URL")      Reporter.Filter = rfEnableErrorsOnly      While Not Browser_.Exist(0)        Wait 0, 50      Wend      Reporter.Filter = rfEnableAll      Reporter.ReportEvent micDone, TypeName(Me),        "Opened browser"    Else      Reporter.ReportEvent micDone, TypeName(Me),        "Browser was already open"    End If End Sub Private Sub Class_Terminate    If me.Browser_.Exist(0) Then      me.Browser_.Close      Reporter.Filter = rfEnableErrorsOnly      While me.Browser_.Exist(0)        wait 0, 50      Wend      Reporter.Filter = rfEnableAll      Reporter.ReportEvent micDone, TypeName(Me),        "Closed browser"    End If End Sub End Class In Action, write the following code: Dim oGoogleSearch Dim oListResults Dim oDicSearches Dim iNumResults Dim sMaxResults Dim iMaxResults '--- Create these objects only in the first iteration If Not LCase(TypeName(oListResults)) = "arraylist" Then Set oListResults =    CreateObject("System.Collections.ArrayList") End If If Not LCase(TypeName(oDicSearches)) = "Dictionary" Then Set oDicSearches = CreateObject("Scripting.Dictionary") End If '--- Get a fresh instance of GoogleSearch Set oGoogleSearch = GetGoogleSearch() '--- Get search term from the DataTable for each action iteration sToSearch = DataTable("Query", dtLocalSheet) iNumResults = oGoogleSearch.DoSearch(sToSearch) '--- Store the results of the current iteration '--- Store the number of results oListResults.Add iNumResults '--- Store the search term attached to the number of results as key (if not exists) If Not oDicSearches.Exists(iNumResults) Then oDicSearches.Add iNumResults, sToSearch End If 'Last iteration (assuming we always run on all rows), so perform the comparison between the different searches If CInt(Environment("ActionIteration")) = DataTable.LocalSheet.GetRowCount Then 'Sort the results ascending oListResults.Sort 'Get the last item which is the largest iMaxResults = oListResults.item(oListResults.Count-1) 'Print to the Output pane for debugging Print iMaxResults 'Get the search text which got the most results sMaxResults = oDicSearches(iMaxResults) 'Report result Reporter.ReportEvent micDone, "Max search", sMaxResults    & " got " & iMaxResults 'Dispose of the objects used Set oListResults = Nothing Set oDicSearches = Nothing Set oGoogleSearch = Nothing End If In the local datasheet, create a parameter named Query and enter several values to be used in the test as search terms. Next, from the UFT home page navigate to View | Test Flow, and then right-click with the mouse on the Action component in the graphic display, then select Action Call Properties and set the Action to run on all rows. How it works... The Action takes care to preserve the data collected through the iterations in the array list oListResults and the dictionary oDicSearches. It checks if it reaches the last iteration after each search is done. Upon reaching the last iteration, it analyses the data to decide which term yielded the most results. A more detailed description of the workings of the code can be seen as follows. First, we create an instance of the GoogleSearch class, and the Class_Initialize subroutine automatically checks if the browser is not already open. If not open, Class_Initialize opens it with the SystemUtil.Run command and waits until it is open at the web address defined in Environment("OPEN_URL"). The Title property always returns the value of the Descriptive Programming (DP) value required to identify the Google browser and page. The Browser_, Page_, and Results properties always return a reference to the Google browser, page, and WebElement respectively, which hold the text with the search results. After the browser is open, we retrieve the search term from the local DataTable parameter Query and call the GoogleSearch DoSearch method with the search term string as parameter. The DoSearch method returns a value with the number of results, which are given by the internal method GetNumResults. In the Action, we store the number itself and add to the dictionary, an entry with this number as the key and the search term as the value. When the last iteration is reached, an analysis of the results is automatically done by invoking the Sort method of oListResults ArrayList, getting the last item (the greatest), and then retrieving the search term associated with this number from the dictionary; it reports the result. At last, we dispose off all the objects used, and then the Class_Terminate subroutine automatically checks if the browser is open. If open, then the Class_Terminate subroutine closes the browser. Implementing a generic Login class In this recipe, we will see how to implement a generic Login class. The class captures both, the GUI structure and the processes that are common to all applications with regard to their user access module. It is agnostic to the particular object classes, their technologies, and other identification properties. The class shown here implements the command wrapper design pattern, as it encapsulates a process (Login) with the main default method (Run). Getting ready You can use the same function library cls.Google.vbs as in the previous recipe Implementing a simple search class, or create a new one (for instance, cls.Login.vbs) and associate it with your test. How to do it... In the function library, we will write the following code to define the class Login: Class Login Private m_wndContainer 'Such as a Browser, Window,    SwfWindow Private m_wndLoginForm 'Such as a Page, Dialog,    SwfWindow Private m_txtUsername 'Such as a WebEdit, WinEdit,    SwfEdit Private m_txtIdField 'Such as a WebEdit, WinEdit,    SwfEdit Private m_txtPassword 'Such as a WebEdit, WinEdit,    SwfEdit Private m_chkRemember 'Such as a WebCheckbox,    WinCheckbox, SwfCheckbox Private m_btnLogin   'Such as a WebEdit, WinEdit,    SwfEdit End Class These fields define the test objects, which are required for any Login class, and the following fields are used to keep runtime data for the report: Public Status 'As Integer Public Info 'As String The Run function is defined as a Default method that accepts a Dictionary as argument. This way, we can pass a set of named arguments, some of which are optional, such as timeout. Public Default Function Run(ByVal ArgsDic)    'Check if the timeout parameter was passed, if not      assign it 10 seconds    If Not ArgsDic.Exists("timeout") Then ArgsDic.Add      "timeout", 10    'Check if the client window exists    If Not me.Container.Exist(ArgsDic("timeout")) Then      me.Status = micFail      me.Info   = "Failed to detect login        browser/dialog/window."      Exit Function    End If    'Set the Username    me.Username.Set ArgsDic("Username")    'If the login form has an additional mandatory field    If me.IdField.Exist(ArgsDic("timeout")) And      ArgsDic.Exists("IdField") Then      me.IdField.Set ArgsDic("IdField")    End If    'Set the password    me.Password.SetSecure ArgsDic("Password")    'It is a common practice that Login forms have a      checkbox to keep the user logged-in if set ON    If me.Remember.Exist(ArgsDic("timeout")) And      ArgsDic.Exists("Remember") Then      me.Remember.Set ArgsDic("Remember")    End If    me.LoginButton.Click End Function The Run method actually performs the login procedure, setting the username and password, as well as checking or unchecking the Remember Me or Keep me Logged In checkbox according to the argument passed with the ArgsDic dictionary. The Initialize method accepts Dictionary just like the Run method. However, in this case, we pass the actual TOs with which we wish to perform the login procedure. This way, we can actually utilize the class for any Login form, whatever the technology used to develop it. We can say that the class is technology agnostic. The parent client dialog/browser/window of the objects is retrieved using the GetTOProperty("parent") statement: Function Initialize(ByVal ArgsDic)    Set m_txtUsername = ArgsDic("Username")    Set m_txtIdField = ArgsDic("IdField")    Set m_txtPassword = ArgsDic("Password")    Set m_btnLogin   = ArgsDic("LoginButton")    Set m_chkRemember = ArgsDic("Remember")    'Get Parents    Set m_wndLoginForm =      me.Username.GetTOProperty("parent")    Set m_wndContainer =      me.LoginForm.GetTOProperty("parent") End Function In addition, here you can see the following properties used in the class for better readability: Property Get Container()    Set Container = m_wndContainer End Property Property Get LoginForm()    Set LoginForm = m_wndLoginForm End Property Property Get Username()    Set Username = m_txtUsername End Property Property Get IdField()    Set IdField = m_txtIdField End Property Property Get Password()    Set Password = m_txtPassword End Property Property Get Remember()    Set Remember = m_chkRemember End Property Property Get LoginButton()  Set LoginButton = m_btnLogin End Property Private Sub Class_Initialize()    'TODO: Additional initialization code here End Sub Private Sub Class_Terminate()    'TODO: Additional finalization code here End Sub We will also add a custom function to override the WinEdit and WinEditor Type methods: Function WinEditSet(ByRef obj, ByVal str) obj.Type str End Function This way, no matter which technology the textbox belongs to, the Set method will work seamlessly. To actually test the Login class, write the following code in the Test Action (this time we assume that the Login form was already opened by another procedure): Dim ArgsDic, oLogin 'Register the set method for the WinEdit and WinEditor RegisterUserFunc "WinEdit", "WinEditSet", "Set" RegisterUserFunc "WinEditor", "WinEditSet", "Set" 'Create a Dictionary object Set ArgsDic = CreateObject("Scripting.Dictionary") 'Create a Login object Set oLogin = New Login 'Add the test objects to the Dictionary With ArgsDic .Add "Username",    Browser("Gmail").Page("Gmail").WebEdit("txtUsername") .Add "Password",    Browser("Gmail").Page("Gmail").WebEdit("txtPassword") .Add "Remember",    Browser("Gmail").Page("Gmail")    .WebCheckbox("chkRemember") .Add "LoginButton",    Browser("Gmail").Page("Gmail").WebButton("btnLogin") End With 'Initialize the Login class oLogin.Initialize(ArgsDic) 'Initialize the dictionary to pass the arguments to the login ArgsDic.RemoveAll With ArgsDic .Add "Username", "myuser" .Add "Password", "myencriptedpassword" .Add "Remember", "OFF" End With 'Login oLogin.Run(ArgsDic) 'or: oLogin(ArgsDic) 'Report result Reporter.ReportEvent oLogin.Status, "Login", "Ended with " & GetStatusText(oLogin.Status) & "." & vbNewLine & oStatus.Info 'Dispose of the objects Set oLogin = Nothing Set ArgsDic = Nothing How it works... Here we will not delve into the parts of the code already explained in the Implementing a simple search class recipe. Let's see what we did in this recipe: We registered the custom function WinEditSet to the WinEdit and WinEditor TO classes using RegisterUserFunc. As discussed previously, this will make every call to the method set to be rerouted to our custom function, resulting in applying the correct method to the Standard Windows text fields. Next, we created the objects we need, a Dictionary object and a Login object. Then, we added the required test objects to Dictionary, and then invoked its Initialize method, passing the Dictionary as the argument. We cleared Dictionary and then added to it the values needed for actually executing the login (Username, Password, and the whether to remember the user or keep logged in checkboxes usually used in Login forms). We called the Run method for the Login class with the newly populated Dictionary. Later, we reported the result by taking the Status and Info public fields from the oLogin object. At the end of the script, we unregistered the custom function from all classes in the environment (StdWin in this case). Implementing function pointers What is a function pointer? A function pointer is a variable that stores the memory address of a block of code that is programmed to fulfill a specific function. Function pointers are useful to avoid complex switch case structures. Instead, they support direct access in runtime to previously loaded functions or class methods. This enables the construction of callback functions. A callback is, in essence, an executable code that is passed as an argument to a function. This enables more generic coding, by having lower-level modules calling higher-level functions or subroutines. This recipe will describe how to implement function pointers in VBScript, a scripting language that does not natively support the usage of pointers. Getting ready Create a new function library (for instance, cls.FunctionPointers.vbs) and associate it with your test. How to do it... Write the following code in the function library: Class WebEditSet    Public Default Function Run(ByRef obj, ByVal sText)        On Error Resume Next         Run = 1 'micFail (pessimistic initialization)        Select Case True            Case   obj.Exist(0) And _                    obj.GetROProperty("visible") And _                    obj.GetROProperty("enabled")                              'Perform the set operation                    obj.Set(sText)                              Case Else                Reporter.ReportEvent micWarning,                  TypeName(me), "Object not available."              Exit Function        End Select        If Err.Number = 0 Then            Run = 0 'micPass        End If    End Function End Class Write the following code in Action: Dim pFunctiontion Set pFunctiontion = New WebEditSet Reporter.ReportEvent pFunctiontion(Browser("Google").Page("Google") .WebEdit("q"), "UFT"), "Set the Google Search WebEdit", "Done" How it works... The WebEditSet class actually implements the command wrapper design pattern (refer to also the Implementing a generic Login class recipe). This recipe also demonstrates an alternative way of overriding any native UFT TO method without recurring to the RegisterUserFunc method. First, we create an instance of the WebEditSet class and set the reference to our pFunctiontion variable. Note that the Run method of WebEditSet is declared as a default function, so we can invoke its execution by merely referring to the object reference, as is done with the statement pFunctiontion in the last line of code in the How to do it… section. This way, pFunctiontion actually functions as if it were a function pointer. Let us take a close look at the following line of code, beginning with Reporter.ReportEvent: Reporter.ReportEvent pFunc(Browser("Google").Page("Google").WebEdit("q"), "UFT"), "Set the Google Search WebEdit", "Done" We call the ReportEvent method of Reporter, and as its first parameter, instead of a status constant such as micPass or micFail, we pass pFunctiontion and the arguments accepted by the Run method (the target TO and its parameter, a string). This way of using the function pointer actually implements a kind of callback. The value returned by the Run method of WebEditSet will determine whether UFT will report a success or failure in regard to the Set operation. It will return through the call invoked by accessing the function pointer. See also The following articles are part of a wider collection at www.advancedqtp.com, which also discusses function pointers in depth: An article by Meir Bar-Tal at http://www.advancedqtp.com/ function-pointers-in-vb-script-revised An article by Meir Bar-Tal at http://www.advancedqtp.com/using-to-custom-property-as-function-pointer Summary In this article, we learned how to implement a general class; basic concepts and the syntax required by VBScript to implement a class. Then we saw how to implement a simple class that can be used to execute a search on Google and a generic Login class. We also saw how to implement function pointers in VBScript along with various links to the articles that discusses function pointers. Resources for Article: Further resources on this subject: DOM and QTP [Article] Getting Started with Selenium Grid [Article] Quick Start into Selenium Tests [Article]
Read more
  • 0
  • 0
  • 20092

article-image-html-php-and-content-posting-drupal-6
Packt
15 Oct 2009
9 min read
Save for later

HTML, PHP, and Content Posting in Drupal 6

Packt
15 Oct 2009
9 min read
Input Formats and Filters It is necessary to stipulate the type of content we will be posting, in any given post. This is done through the use of the Input format setting that is displayed when posting content to the site—assuming the user in question has sufficient permissions to post different types of content. In order to control what is and is not allowed, head on over to the Input formats link under Site configuration. This will bring up a list of the currently defined input formats, like this: At the moment, you might be wondering why we need to go to all this trouble to decide whether people can add certain HTML tags to their content. The answer to this is that because both HTML and PHP are so powerful, it is not hard to subvert even fairly simple abilities for malicious purposes. For example, you might decide to allow users the ability to link to their homepages from their blogs. Using the ability to add a hyperlink to their postings, a malicious user could create a Trojan, virus or some other harmful content, and link to it from an innocuous and friendly looking piece of HTML like this: <p>Hi Friends! My <a href="link_to_trojan.exe">homepage</a> is a great place to meet and learn about my interests and hobbies. </p> This snippet writes out a short paragraph with a link, supposedly to the author's homepage. In reality, the hyperlink reference attribute points to a trojan, link_to_trojan.exe. That's just HTML! PHP can do a lot more damage—to the extent that if you don't have proper security or disaster-recovery policies in place, then it is possible that your site can be rendered useless or destroyed entirely. Security is the main reason why, as you may have noticed from the previous screenshot, anything other than Filtered HTML is unavailable for use by anyone except the administrator. By default, PHP is not even present, let alone disabled. When thinking about what permissions to allow, it is important to re-iterate the tenet: Never allow users more permissions than they require to complete their intended tasks! As they stand, you might not find the input formats to your liking, and so Drupal provides some functionality to modify them. Click on the configure link adjacent to the Filtered HTML option, and this will bring up the following page: The Edit tab provides the option to alter the Name property of the input format; the Roles section in this case cannot be changed, but as you will see when we come around to creating our own input format, roles can be assigned however you wish to allow certain users to make use of an input format, or not. The final section provides a checklist of the types of Filters to apply when using this input format. In this previous screenshot, all have been selected, and this causes the input format to apply the: HTML corrector – corrects any broken HTML within postings to prevent undesirable results in the rest of your page. HTML filter – determines whether or not to strip or remove unwanted HTML. Line break converter – Turns standard typed line breaks (i.e. whenever a poster clicks Enter) into standard HTML. URL filter – allows recognized links and email addresses to be clickable without having to write the HTML tags, manually. The line break converter is particularly useful for users because it means that they do not have to explicitly enter <br> or <p> HTML tags in order to display new lines or paragraph breaks—this can get tedious by the time you are writing your 400th blog entry. If this is disabled, unless the user has the ability to add the relevant HTML tags, the content may end up looking like this: Click on the Configure tab, at the top of the page, in order to begin working with the HTML filter. You should be presented with something like this: The URL filter option is really there to help protect the formatting and layout of your site. It is possible to have quite long URLs these days, and because URLs do not contain spaces, there is nowhere to naturally split them up. As a result, a browser might do some strange things to cater for the long string and whatever it is; this will make your site look odd. Decide how many characters the longest string should be and enter that number in the space provided. Remember that some content may appear in the sidebars, so you can't let it get too long if they is supposed to be a fixed width. The HTML filter section lets you specify whether to Strip disallowed tags, or escape them (Escape all tags causes any tags that are present in the post to be displayed as written). Remember that if all the tags are stripped from the content, you should enable the Line break converter so that users can at least paragraph their content properly. Which tags are to be stripped is decided in the Allowed HTML tags section, where a list of all the tags that are to be allowed can be entered—anything else gets handled appropriately. Selecting Display HTML help forces Drupal to provide HTML help for users posting content—try enabling and disabling this option and browsing to this relative URL in each case to see the difference: filter/tips. There is quite a bit of helpful information on HTML in the long filter tips; so take a moment to read over those. The filter tips can be reached whenever a user expands the Input format section of the content post and clicks on More information about formatting options at the bottom of that section. Finally, the Spam link deterrent is a useful tool if the site is being used to bombard members with links to unsanctioned (and often unsavory) products. Spammers will use anonymous accounts to post junk (assuming anonymous users are allowed to post content) and enabling this for anonymous posts is an effective way of breaking them. This is not the end of the story, because we also need to be able to create input formats in the event we require something that the default options can't cater for. For our example, there are several ways in which this can be done, but there are three main criteria that need to be satisfied before we can consider creating the page. We need to be able to: Upload image files and attach them to the post. Insert and display the image files within the body of the post. Use PHP in order to dynamically generate some of the content (this option is really only necessary to demonstrate how to embed PHP in a posting for future reference). There are several methods for displaying image files within posts. The one we will discuss here, does not require us to download and install any contribution modules, such as Img_assist. Instead, we will use HTML directly to achieve this, specifically, we use the <img> tag. Take a look at the previous screenshot that shows the configure page of the Filtered HTML input format. Notice that the <img> tag is not available for use. Let's create our own input format to cater for this, instead of modifying this default format. Before we do, first enable the PHP Filter module under Modules in Site building so that it can easily be used when the time comes. With that change saved, you will find that there is now an extra option to the Filters section of each input format configuration page: It's not a good idea to enable the PHP evaluator for either of the default options, but adding it to one of our own input formats will be ok to play with. Head on back to the main input formats page under Site configuration (notice that there is an additional input format available, called PHP code) and click on Add input format. This will bring up the same configuration type page we looked at earlier. It is easy to implement whatever new settings you want, based on how the input format is to be used. For our example, we need the ability to post images and make use of PHP scripts, so make the new input format as follows: As we will need to make use of some PHP code a bit later on, we have enabled the PHP evaluator option, as well as prevented the use of this format for anyone but ourselves—normally, you would create a format for a group of users who require the modified posting abilities, but in this case, we are simply demonstrating how to create a new input format; so this is fine for now. PHP should not be enabled for anyone other than yourself, or a highly trusted administrator who needs it to complete his or her work. Click Save configuration to add this new format to the list, and then click on the Configure tab to work on the HTML filter. The only change required between this input format and the default Filtered HTML, in terms of HTML, is the addition of the <img> and <div> tags, separated by a space in the Allowed HTML tags list, as follows: As things stand at the moment, you may run into problems with adding PHP code to any content postings. This is because some filters affect the function of others, and to be on the safe side, click on the Rearrange tab and set the PHP evaluator to execute first: Since the PHP evaluator's weight is the lowest, it is treated first, with all the others following suit. It's a safe bet that if you are getting unexpected results when using a certain type of filter, you need to come to this page and change the settings. We'll see a bit more about this, in a moment. Now, the PHP evaluator gets dibs on the content and can properly process any PHP. For the purposes of adding images and PHP to posts (as the primary user), this is all that is needed for now. Once satisfied with the settings save the changes. Before building the new page, it is probably most useful to have a short discourse on HTML, because it is a requirement if you are to attempt more complex postings.  
Read more
  • 0
  • 1
  • 20066

article-image-freeradius-working-authentication-methods
Packt
08 Sep 2011
6 min read
Save for later

FreeRADIUS: Working with Authentication Methods

Packt
08 Sep 2011
6 min read
Authentication is a process where we establish if someone is who he or she claims to be. The most common way is by a unique username and password. This article by Dirk van der Walt, author of FreeRADIUS Beginner's Guide, teaches authentication methods and how they work. Extensible Authentication Protocol (EAP) is covered later in a dedicated article. In this article we shall: Discuss PAP, CHAP, and MS-CHAP authentication protocols See when and how authentication is done in FreeRADIUS Explore ways to store passwords Look at other authentication methods (For more resources on this subject, see here.) Authentication protocols This section will give you background on three common authentication protocols. These protocols involve the supply of a username and password. The radtest program uses the Password Authentication Protocol (PAP) by default when testing authentication. PAP is not the only authentication protocol but probably the most generic and widely used. Authentication protocols you should know about are PAP, CHAP, and MS-CHAP. Each of these protocols involves a username and password. The next article on Extensible Authentication Protocol (EAP) protocol will introduce us to more authentication protocols. An authentication protocol is typically used on the data link layer that connects the client with the NAS. The network layer will only be established after the authentication is successful. The NAS acts as a broker to forward the requests from the user to the RADIUS server. The data link layer and network layer are layers inside the Open Systems Interconnect model (OSI model). The discussion of this model is almost guaranteed to be found in any book on networking:http://en.wikipedia.org/wiki/OSI_model PAP PAP was one of the first protocols used to facilitate the supply of a username and password when making point-to-point connections. With PAP the NAS takes the PAP ID and password and sends them in an Access-Request packet as the User-Name and User-Password. PAP is simpler compared to CHAP and MS-CHAP because the NAS simply hands the RADIUS server a username and password, which are then checked. This username and password come directly from the user through the NAS to the server in a single action. Although PAP transmits passwords in clear text, using it should not always be frowned upon. This password is only in clear text between the user and the NAS. The user's password will be encrypted when the NAS forwards the request to the RADIUS server. If PAP is used inside a secure tunnel it is as secure as the tunnel. This is similar to when your credit card details are tunnelled inside an HTTPS connection and delivered to a secure web server. HTTPS stands for Hypertext Transfer Protocol Secure and is a web standard that uses Secure Socket Layer/Transport Layer Security (SSL/TLS) to create a secure channel over an insecure network. Once this secure channel is established, we can transfer sensitive data, like credit card details, through it. HTTPS is used daily to secure many millions of transactions over the Internet. See the following schematic of a typical captive portal configuration. The following table shows the RADIUS AVPs involved in a PAP request: As you can see the value of User-Password is encrypted between the NAS and the RADIUS server. Transporting the user's password from the user to the NAS may be a security risk if it can be captured by a third party. CHAP CHAP stands for Challenge-Handshake Authentication Protocol and was designed as an improvement to PAP. It prevents you from transmitting a cleartext password. CHAP was created in the days when dial-up modems were popular and the concern about PAP's cleartext passwords was high. After a link is established to the NAS, the NAS generates a random challenge and sends it to the user. The user then responds to this challenge by returning a one-way hash calculated on an identifier (sent along with the challenge), the challenge, and the user's password. The user's response is then used by the NAS to create an Access-Request packet, which is sent to the RADIUS server. Depending on the reply from the RADIUS server, the NAS will return CHAP Success or CHAP Failure to the user. The NAS can also request at random intervals that the authentication process be repeated by sending a new challenge to the user. This is another reason why it is considered more secure than PAP. One major drawback of CHAP is that although the password is transmitted encrypted, the password source has to be in clear text for FreeRADIUS to perform password verification. The FreeRADIUS FAQ discuss the dangers of transmitting a cleartext password compared to storing all the passwords in clear text on the server. The following table shows the RADIUS AVPs involved in a CHAP request: MS-CHAP MS-CHAP is a challenge-handshake authentication protocol created by Microsoft. There are two versions, MS-CHAP version 1 and MS-CHAP version 2. The challenge sent by the NAS is identical in format to the standard CHAP challenge packet. This includes an identifier and arbitrary challenge. The response from the user is also identical in format to the standard CHAP response packet. The only difference is the format of the Value field. The Value field is sub-formatted to contain MS-CHAP-specific fields. One of the fields (NT-Response) contains the username and password in a very specific encrypted format. The reply from the user will be used by the NAS to create an Access-Request packet, which is sent to the RADIUS server. Depending on the reply from the RADIUS server, the NAS will return Success Packet or Failure Packet to the user. The RADIUS server is not involved with the sending out of the challenge. If you sniff the RADIUS traffic between an NAS and a RADIUS server you can confirm that there is only an Access-Request followed by an Access-Accept or Access-Reject. The sending out of a challenge to the user and receiving a response from her or him is between the NAS and the user. MS-CHAP also has some enhancements that are not part of CHAP, like the user's ability to change his or her password or inclusion of more descriptive error messages. The protocol is tightly integrated with the LAN Manager and NT Password hashes. FreeRADIUS will convert a user's cleartext password to an LM-Password and an NT-Password in order to determine if the password hash that came out of the MS-CHAP request is correct. Although there are known weaknesses with MS-CHAP, it remains widely used and very popular. Never say never. If your current requirement for the RADIUS deployment does not include the use of MS-CHAP, rather cater for the possibility that one day you may use it. The most popular EAP protocol makes use of MS-CHAP. EAP is crucial in Wi-Fi authentication. Because MS-CHAP is vendor specific, VSAs instead of AVPs are part of the Access-Request between the NAS and RADIUS server. This is used together with the User-Name AVP. Now that we know more about the authentication protocols, let's see how FreeRADIUS handles them.
Read more
  • 0
  • 0
  • 20016

article-image-tips-and-tricks-to-optimize-your-responsive-web-design
Sugandha Lahoti
31 May 2018
11 min read
Save for later

Tips and tricks to optimize your responsive web design

Sugandha Lahoti
31 May 2018
11 min read
Loosely put, website optimization refers to the activities and processes that improve your website's user experience and visibility while reducing the costs associated with hosting your website. In this article, we will learn tips and techniques for client-side optimization. This article is an excerpt from Mastering Bootstrap 4 - Second Edition by Benjamin Jakobus, and Jason Marah. In this book, you will learn to build a customized Bootstrap website from scratch, optimize your website and integrate it with third-party frameworks. CSS optimization Before we even consider compression, minification, and file concatenation, we should think about the ways in which we can simplify and optimize our existing style sheet without using third-party tools. Of course, we should have striven for an optimal style sheet, to begin with, and in many aspects we did. However, our style sheet still leaves room for improvement. Inline styles are bad After reading this article, if you only remember one thing, then let it be that inline styles are bad. Period. Avoid using them whenever possible. Why? That's because not only will they make your website impossible to maintain as the website grows, they also take up precious bytes as they force you to repeat the same rules over and over. Consider the following code piece: <div class="carousel-inner" role="listbox"> <div style="height: 400px" class="carousel-item active"> <img class="d-block img-fluid" src="images/brazil.png" data-modal-picture="#carousel-modal"> <div class="carousel-caption"> Brazil </div> </div> <div style="height: 400px" class="carousel-item"> <img class="d-block img-fluid" src="images/datsun.png" data-modal-picture="#carousel-modal"> <div class="carousel-caption"> Datsun 260Z </div> </div> <div style="height: 400px" class="carousel-item"> <img class="d-block img-fluid" src="images/skydive.png" data-modal-picture="#carousel-modal"> <div class="carousel-caption"> Skydive </div> </div> </div> Note how the rule for defining an item's height, style="height: 400px", is repeated three times, once for each of the three items. That's an additional 21 characters (or 21 bytes, assuming that our document is UTF-8) for each additional image. Multiplying 3*21 gives us 63 bytes, and 21 more bytes for every new image that you want to add. Not to mention that if you ever want to update the height of the images, you will need to manually update the style attribute for every single image. The solution is, of course, to replace the inline styles with an appropriate class. Let's go ahead and define an img class that can be applied to any carousel image: .carousel-item { height: 400px; } Now let's go ahead and remove the style rules: <div class="carousel-inner" role="listbox"> <div style="height: 400px" class="carousel-item active"> <img class="d-block img-fluid" src="images/brazil.png" data-modal-picture="#carousel-modal"> <div class="carousel-caption"> Brazil </div> </div> <div style="height: 400px" class="carousel-item"> <img class="d-block img-fluid" src="images/datsun.png" data-modal-picture="#carousel-modal"> <div class="carousel-caption"> Datsun 260Z </div> </div> <div style="height: 400px" class="carousel-item"> <img class="d-block img-fluid" src="images/skydive.png" data-modal-picture="#carousel-modal"> <div class="carousel-caption"> Skydive </div> </div> </div> That's great! Not only is our CSS now easier to maintain, but we also shaved 29 bytes off our website (the original inline styles required 63 bytes; our new class definition, however, requires only 34 bytes). Yes, this does not seem like much, especially in the world of high-speed broadband, but remember that your website will grow and every byte adds up. Avoid Long identifiers and class names The longer your strings, the larger your files. It's a no-brainer. As such, long identifier and class names naturally increase the size of your web page. Of course, extremely short class or identifier names tend to lack meaning and therefore will make it more difficult (if not impossible) to maintain your page. As such, one should strive for an ideal balance between length and expressiveness. Of course, even better than shortening identifiers is removing them altogether. One handy technique of removing these is to use hierarchical selection. Have a look at an events pagination code piece. For example, we are using the services-events-content identifier within our pagination logic, as follows: $('#services-events-pagination').bootpag({ total: 10 }).on("page", function(event, num){ $('#services-events-content div').hide(); var current_page = '#page-' + num; $(current_page).show(); }); To denote the services content, we broke the name of our identifier into three parts, namely, services, events, and content. Our markup is as follows: <div id="services-events-content"> <div id="page-1"> <h3>My Sample Event #1</h3> ... </div> </div> Let's try and get rid of this identifier altogether by observing two characteristics of our Events section: The services-events-content is an indirect descendent of a div with the id services-events. We cannot remove this id as it is required for the menu to work. The element with the id services-events-content is itself a div. If we were to remove its id, we could also remove the entire div. As such, we do not need a second identifier to select the pages that we wish to hide. Instead, all that we need to do is select the div within the div that is within the div that is assigned the id services-events. How do we express this as a CSS selector? It's easy—use #services-events div div div. Also, as such, our pagination logic is updated as follows: $('#services-events-pagination').bootpag({ total: 10 }).on("page", function(event, num){ $('#services-events div div div').hide(); var current_page = '#page-' + num; $(current_page).show(); }); Now, save and refresh. What's that? As you clicked on a page, the pagination control disappeared; that's because we are now hiding all div elements that are two div elements down from the element with the id services-events. Move the pagination control div outside its parent element. Our markup should now look as follows: <div role="tabpanel" class="tab-pane active" id="services-events"> <div class="container"> <div class="row"> <div id="page-1"> <h3>My Sample Event #1</h3> <h3>My Sample Event #2</h3> </div> <div id="page-2"> <h3>My Sample Event #3</h3> </div> </div> <div id="services-events-pagination"></div> </div> </div> Now save and refresh. That's better! Last but not least, let's update the css. Take the following code into consideration: #services-events-content div { display: none; } #services-events-content div img { margin-top: 0.5em; margin-right: 1em; } #services-events-content { height: 15em; overflow-y: scroll; } Replace this code with the following: #services-events div div div { display: none; } #services-events div div div img { margin-top: 0.5em; margin-right: 1em; } #services-events div div div { height: 15em; overflow-y: scroll; } That's it, we have simplified our style sheet and saved some bytes in the process! However, we have not really improved the performance of our selector. jQuery executes selectors from right to left, hence executing the last selector first. In this example, jQuery will first scan the complete DOM to discover all div elements (last selector executed first) and then apply a filter to return only those elements that are div, with a div parent, and then select only the ones with ID services-events as parent. While we can't really improve the performance of the selector in this case, we can still simplify our code further by adding a class to each page: <div id="page-1" class="page">...</div> <div id="page-2" class="page">...</div> <div id="page-3" class="page">...</div> Then, all we need to do is select by the given class: $('#services-events div.page').hide();. Alternatively, knowing that this is equal to the DOM element within the .on callback, we can do the following in order to prevent us from iterating through the whole DOM: $(this).parents('#services-vents').find('.page').hide(); The final code will look as follows: $('#services-events-pagination').bootpag({ total: 10 }).on("page", function(event, num) { $(this).parents('#services-events').find('.page').hide(); $('#page-' + num).show(); }); Note a micro-optimization in the preceding code—there was no need for us to create that var in memory. Hence, the last line changes to $('#page-' + num).show();. Use Shorthand rules when possible According to the Mozilla Developer Network (shorthand properties, Mozilla Developer Network, https://developer.mozilla.org/en-US/docs/Web/CSS/Shorthand_properties, accessed November 2015), shorthand properties are: "CSS properties that let you set the values of several other CSS properties simultaneously. Using a shorthand property, a Web developer can write more concise and often more readable style sheets, saving time and energy."                                                                                    – Mozilla Developer Network, 2015 Unless strictly necessary, we should never be using longhand rules. When possible, shorthand rules are always the preferred option. Besides the obvious advantage of saving precious bytes, shorthand rules also help increase your style sheet's maintainability. For example, border: 20px dotted #FFF is equivalent to three separate rules: border-style: dotted; border-width: 20px; border-color: #FFF; Group selectors Organizing selectors into groups will arguably also save some bytes. .navbar-myphoto .dropdown-menu > a:hover { color: gray; background-color: #504747; } .navbar-myphoto .dropdown-menu > a:focus { color: gray; background-color: #504747; } .navbar-myphoto .dropdown-menu > .active > a:focus { color: gray; background-color: #504747; } Note how each of the three selectors contains the same declarations, that is, the color and background-color properties are set to the exact same values for each selector. To prevent us from repeating these declarations, we should simply group them (reducing the code from 274 characters to 181 characters): .navbar-myphoto .dropdown-menu > a:hover, .navbar-myphoto .dropdown-menu > a:focus, .navbar-myphoto .dropdown-menu > .active > a:focus { color: gray; background-color: #504747; } Voilà! We just saved 93 bytes! (assuming UTF-8 encoding). Rendering times When optimizing your style rules, the number of bytes should not be your only concern. In fact, it comes secondary to the rendering time of your web page. CSS rules affect the amount of work that is required by the browser to render your page. As such, some rules are more expensive than others. For example, changing the color of an element is cheaper than changing its margin. The reason for this is that a change in color only requires your browser to draw the new pixels. While drawing itself is by no means a cheap operation, changing the margin of an element requires much more effort. Your browser needs to both recalculate the page layout and also draw the changes. Optimizing your page's rendering times is a complex topic, and as such is beyond the scope of this post. However, we recommend that you take a look at http://csstriggers.com/. This site provides a concise overview of the costs involved when updating a given CSS property. Minifying CSS and JavaScript Now it is time to look into minification. Minification is the process of removing redundant characters from a file without altering the actual information contained within it. In other words, minifying the css file will reduce its overall size, while leaving the actual CSS style rules intact. This is achieved by stripping out any whitespace characters within our file. Stripping out whitespace characters has the obvious result that our CSS is now practically unreadable and impossible to maintain. As such, minified style sheets should only be used when serving a page (that is, during production), and not during development. Clearly, minifying your style sheet manually would be an incredibly time-consuming (and hence pointless) task. Therefore, there exist many tools that will do the job for us. One such tool is npm minifier. Visit https://www.npmjs.com/package/minifier for more. Let's go ahead and install it: sudo npm install -g minifier Once installed, we can minify our style sheet by typing the following command: minify path-to-myphoto.css Here, path-to-myphoto.css represents the path to the MyPhoto style sheet. Go ahead and execute the command. Once minification is complete, you should see the Minification complete message. A new CSS file (myphoto.min.css) will have been created inside the directory containing the myphoto.css file. The new file should be 2,465 bytes. Our original myphoto.css file is 3,073 bytes. Minifying our style sheet just reduced the number of bytes to send by roughly 19%! We touched upon the basics of website optimization and testing. In the follow-up article, we will see how to use the build tool Grunt to automate the more common and mundane optimization tasks. To build responsive, dynamic, and mobile-first applications on the web with Bootstrap 4, check out this book  Mastering Bootstrap 4 - Second Edition. Get ready for Bootstrap v4.1; Web developers to strap up their boots How to use Bootstrap grid system for responsive website design? Bootstrap 4 Objects, Components, Flexbox, and Layout
Read more
  • 0
  • 0
  • 19518
Unlock access to the largest independent learning library in Tech for FREE!
Get unlimited access to 7500+ expert-authored eBooks and video courses covering every tech area you can think of.
Renews at $19.99/month. Cancel anytime
article-image-converting-tables-graphs-advanced
Packt
06 May 2013
7 min read
Save for later

Converting tables into graphs (Advanced)

Packt
06 May 2013
7 min read
(For more resources related to this topic, see here.) Getting ready We maintained the same structure for our table, however this time we do not use this example and load it via AJAX. So the markup looks as follows: <table id="dynamicTable" class="table"> <thead> <tr> <th>Reviews</th> <th>Top</th> <th>Rolling Stones</th> <th>Rock Hard</th> <th>Kerrang</th> </tr> </thead> <tbody> <tr> <th>Ac/Dc</th> <td>10</td> <td>9</td> <td>8</td> <td>9</td> </tr> <tr> <th>Queen</th> <td>9</td> <td>6</td> <td>8</td> <td>5</td> </tr> <tr> <th>Whitesnake</th> <td>8</td> <td>9</td> <td>8</td> <td>6</td> </tr> <tr> <th>Deep Purple</th> <td>10</td> <td>6</td> <td>9</td> <td>8</td> </tr> <tr> <th>Black Sabbath</th> <td>10</td> <td>5</td> <td>7</td> <td>8</td> </tr> </tbody> </table> How to do it... Let's see what we need to do: Add a div right on the top of our table with an ID called graph: <div id="graph"></div> We will use a jQuery Plugin called Highcharts, which can be downloaded for free from http://www.highcharts.com/products/highcharts. Add the following script to the bottom of our document: <script src = "highcharts.js"></script> Add a simple script to initialize the graph as follows: var chart; Highcharts.visualize = function(table, options) { // the data series options.series = []; var l= options.series.length; options.series[l] = { name: $('thead th:eq('+(l+1)+')', table).text(), data: [] }; $('tbody tr', table).each( function(i) { var tr = this; var th = $('th', tr).text(); var td = parseFloat($('td', tr).text()); options.series[0].data.push({name:th,y:td}); }); chart = new Highcharts.Chart(options); } // On document ready, call visualize on the datatable. $(document).ready(function() { var table = document.getElementById('dynamicTable'), options = { chart: { renderTo: 'graph', defaultSeriesType: 'pie' }, title: { text: 'Review Notes from Metal Mags' }, plotOptions: { pie: { allowPointSelect: true, cursor: 'pointer', dataLabels: { enabled: false }, showInLegend: true } }, tooltip: { pointFormat: 'Total: <b>{point.percentage}%</ b>', percentageDecimals: 1 } }; Highcharts.visualize(table, options); }); Many people choose to hide the div with the table in smaller devices and show only the graph. Once they've optimized our table and depending on the amount of data, there is no problem. It also shows that the choice is yours. Now when we look at the browser, we can view both the table and the graph as shown in the following screenshot: Browser screenshot at 320px. Highcharts plugins have an excellent quality in all browsers and works with SVG, they are compatible with iPad, iPhone, and IE 6. How it works... The plugin can generate the table using only a single data array, but by our intuition and step-by-step description of its uses, we have created the following code to generate the graph starting from a table previously created. We create the graph using the id#= dynamicTable function, where we read its contents through the following function: $('tbody tr', table).each( function(i) { var tr = this; var th = $('th', tr).text(); var td = parseFloat($('td', tr).text()); options.series[0].data.push({name:th,y:td}); }); In the plugin settings, we set the div graph to receive the graph after it is rendered by the script. We also add a pie type and a title for our graph. options = { chart: { renderTo: 'graph', defaultSeriesType: 'pie' }, title: { text: 'Review Notes from Metal Mags' }, There's more... We can hide the table using media query so that only the graph appears. Remember that it just hides the fact and does not prevent it from being loaded by the browser; however we still need it to build the graph. For this, just apply display none to the table inside the breakpoint: @media only screen and (max-width: 768px) { .table { display: none; } } Browser screenshot at 320px, without the table Merging data – numbers and text (Advanced) We introduce an alternative based on CSS3 for dealing with tables containing text and numbers. Getting ready Tables are used for different purposes, we will see an example where our data is not a data table. (Code Example: Chapter07_Codes_1 ) Browser screenshot at 1024px Although our table did not have many columns, showing it on a small screen is not easy. Hence we will progressively show the change in the table by subtracting the width of the screen. How to do it... Note that we have removed the .table class so this time apply the style directly in the table tags, see the following steps: Let's use a simple table structure as we saw before. Add some CSS3 to make some magic with our selectors. Set our breakpoints to two sizes. <table> <thead> <tr> <th>CONTACT</th> <th scope="col">Manager</th> <th scope="col">Label</th> <th scope="col">Phone</th> </tr> </thead> <tbody> <tr> <th scope="row">Black Sabbath</th> <td>Richard Both</td> <td>Atlantic</td> <td>+1 (408) 257-1500 </td> </tr> <tr> <th scope="row">Ac/DC</th> <td>Paolla Matazo</td> <td>Sony</td> <td>+1 (302) 236-0800</td> </tr> <tr> <th scope="row">Queen</th> <td>Suzane Yeld</td> <td>Warner</td> <td>+1 (103) 222-6754</td> </tr> <tr> <th scope="row">Whitesnake</th> <td>Paul S. Senne</td> <td>Vertigo</td> <td>+1 (456) 233-1243</td> </tr> <tr> <th scope="row">Deep Purple</th> <td>Ian Friedman</td> <td>CosaNostra</td> <td>+1 (200) 255-0066</td> </tr> </tbody> </table> Applying the style as follows: table { width: 100%; background-color: transparent; border-collapse: collapse; border-spacing: 0; background-color: #fff } th { text-align: left; } td:last-child, th:last-child { text-align:right; } td, th { padding: 6px 12px; } tr:nth-child(odd), tr:nth-child(odd) { background: #f3f3f3; } tr:nth-child(even) { background: #ebebeb; } thead tr:first-child, thead tr:first-child { background: #000; color:white; } table td:empty { background:white; } We use CSS3 pseudo-selectors here again to help in the transformation of the table. And the most important part, the Media Queries breakpoints: @media (max-width: 768px) { tr :nth-child(3) {display: none;} } @media (max-width: 480px) { thead tr:first-child {display: none;} th {display: block} td {display: inline!important;} } When the resolution is set to 768px, we note that the penultimate column is removed. This way we keep the most relevant information on the screen. We have hidden the information less relevant to the subject. And when we decrease further, we have the data distributed as a block. Summary In this article, we saw an alternative solution combining the previous recipes with another plugin for rendering graphics. Resources for Article : Further resources on this subject: MySQL 5.1 Plugin: HTML Storage Engine—Reads and Writes [Article] Creating Accessible Tables in Joomla! [Article] HTML5: Generic Containers [Article]
Read more
  • 0
  • 0
  • 19420

article-image-how-configure-msdtc-and-firewall-distributed-wcf-service
Packt
21 Jun 2010
4 min read
Save for later

How to configure MSDTC and the firewall for the distributed WCF service

Packt
21 Jun 2010
4 min read
Understanding the distributed transaction support of a WCF service As we have seen, distributed transaction support of a WCF service depends on the binding of the service, the operation contract attribute, the operation implementation behavior, and the client applications. The following table shows some possible combinations of the WCF-distributed transaction support: Binding permits transaction flow Client flows transaction Service contract opts in transaction Service operation requires transaction scope Possible result True Yes Allowed or Mandatory True Service executes under the flowed in transaction True or False No Allowed True Service creates and executes within a new transaction True Yes or No Allowed False Service executes without a transaction True or False No Mandatory True or False SOAP exception True Yes NotAllowed True or False SOAP exception Testing the distributed transaction support of the WCF service Now that we have changed the service to support distributed transaction and let the client propagate the transaction to the service, we will test this. We will propagate a transaction from the client to the service, test the multiple database support of the WCF service, and discuss the Distributed Transaction Coordinator and Firewall settings for the distributed transaction support of the WCF service. Configuring the Distributed Transaction Coordinator In a subsequent section, we will call two services to update two databases on two different computers. As these two updates are wrapped within one distributed transaction, Microsoft Distributed Transaction Coordinator (MSDTC) will be activated to manage this distributed transaction. If MSDTC is not started or configured properly the distributed transaction will not be successful. In this section, we will explain how to configure MSDTC on both machines. You can follow these steps to configure MSDTC on your local and remote machines: Open Component Services from Control Panel | Administrative Tools. In the Component Services window, expand Component Services, then Computers, and then right-click on My Computer. Select Properties from the context menu. On the My Computer Properties window, click on the MSDTC tab. If this machine is running Windows XP, click on the Security Configuration button. If this machine is running Windows 7, verify that Use local coordinator is checked and then close the My Computer Properties window. Expand Distributed Transaction Coordinator under My Computer node, right-click on Local DTC, select Properties from the context menu, and then from the Local DTC Properties window, click on the Security tab. You should now see the Security Configuration for DTC on this machine.Set it as in the following screenshot. Remember you have to make these changes for both your local and remote machines. You have to restart the MSDTC service after you have changed your MSDTC settings, for the changes to take effect.Also, to simplify our example, we have chosen the No Authentication Required option. You should be aware that not needing authentication is a serious security issue in production. For more information about WCF security, you can go to the MSDN WCF security website at this address:MSDN Library. Configuring the firewall Even though Distributed Transaction Coordinator has been enabled the distributed transaction may still fail if the firewall is turned on and hasn't been set up properly for MSDTC. To set up the firewall for MSTC, follow these steps: Open the Windows Firewall window from the Control Panel. If the firewall is not turned on you can skip this section. Go to the Allow a program or feature through Windows Firewall window(for Windows XP, you need to allow exceptions and go to the Exceptions tab on the Windows Firewall window). Add Distributed Transaction Coordinator to the program list (windowssystem32msdtc.exe) if it is not already on the list. Make sure the checkbox before this item is checked. Again you need to change your firewall setting for both your local and remote machines. Now the firewall will allow msdtc.exe to go through so our next test won't fail due to the firewall restrictions. You may have to restart IIS after you have changed your firewall settings. In some cases you may also have to stop and then restart your fi rewall for the changes to take effect.
Read more
  • 0
  • 0
  • 19402

article-image-getting-started-with-django-restful-web-services
Sugandha Lahoti
27 Mar 2018
19 min read
Save for later

Getting started with Django RESTful Web Services

Sugandha Lahoti
27 Mar 2018
19 min read
In this article, we will kick start with learning RESTful Web Service and subsequently learn how to create models and perform migration, serialization and deserialization in Django. Here’s a quick glance of what to expect in this article: Defining the requirements for RESTful Web Service Analyzing and understanding Django tables and the database Controlling, serialization and deserialization in Django Defining the requirements for RESTful Web Service Imagine a team of developers working on a mobile app for iOS and Android and requires a RESTful Web Service to perform CRUD operations with toys. We definitely don't want to use a mock web service and we don't want to spend time choosing and configuring an ORM (short for Object-Relational Mapping). We want to quickly build a RESTful Web Service and have it ready as soon as possible to start interacting with it in the mobile app. We really want the toys to persist in a database but we don't need it to be production-ready. Therefore, we can use the simplest possible relational database, as long as we don't have to spend time performing complex installations or configurations. Django REST framework, also known as DRF, will allow us to easily accomplish this task and start making HTTP requests to the first version of our RESTful Web Service. In this case, we will work with a very simple SQLite database, the default database for a new Django REST framework project. First, we must specify the requirements for our main resource: a toy. We need the following attributes or fields for a toy entity: An integer identifier A name An optional description A toy category description, such as action figures, dolls, or playsets A release date A bool value indicating whether the toy has been on the online store's homepage at least once In addition, we want to have a timestamp with the date and time of the toy's addition to the database table, which will be generated to persist toys. In a RESTful Web Service, each resource has its own unique URL. In our web service, each toy will have its own unique URL. The following table shows the HTTP verbs, the scope, and the semantics of the methods that our first version of the web service must support. Each method is composed of an HTTP verb and a scope. All the methods have a well-defined meaning for toys and collections: HTTP verb Scope Semantics GET Toy Retrieve a single toy GET Collection of toys Retrieve all the stored toys in the collection, sorted by their name in ascending order POST Collection of toys Create a new toy in the collection PUT Toy Update an existing toy DELETE Toy Delete an existing toy In the previous table, the GET HTTP verb appears twice but with two different scopes: toys and collection of toys. The first row shows a GET HTTP verb applied to a toy, that is, to a single resource. The second row shows a GET HTTP verb applied to a collection of toys, that is, to a collection of resources. We want our web service to be able to differentiate collections from a single resource of the collection in the URLs. When we refer to a collection, we will use a slash (/) as the last character for the URL, as in http://localhost:8000/toys. When we refer to a single resource of the collection we won't use a slash (/) as the last character for the URL, as in http://localhost:8000/toys/5. Let's consider that http://localhost:8000/toys/ is the URL for the collection of toys. If we add a number to the previous URL, we identify a specific toy with an ID or primary key equal to the specified numeric value. For example, http://localhost:8000/toys/42 identifies the toy with an ID equal to 42. We have to compose and send an HTTP request with the POST HTTP verb and http://localhost:8000/toys/ request URL to create a new toy and add it to the toys collection. In this example, our RESTful Web Service will work with JSON (short for JavaScript Object Notation), and therefore we have to provide the JSON key-value pairs with the field names and the values to create the new toy. As a result of the request, the server will validate the provided values for the fields, make sure that it is a valid toy, and persist it in the database. The server will insert a new row with the new toy in the appropriate table and it will return a 201 Created status code and a JSON body with the recently added toy serialized to JSON, including the assigned ID that was automatically generated by the database and assigned to the toy object: POST http://localhost:8000/toys/ We have to compose and send an HTTP request with the GET HTTP verb and http://localhost:8000/toys/{id} request URL to retrieve the toy whose ID matches the specified numeric value in {id}. For example, if we use the request URL http://localhost:8000/toys/25, the server will retrieve the toy whose ID matches 25. As a result of the request, the server will retrieve a toy with the specified ID from the database and create the appropriate toy object in Python. If a toy is found, the server will serialize the toy object into JSON, return a 200 OK status code, and return a JSON body with the serialized toy object. If no toy matches the specified ID, the server will return only a 404 Not Found status: GET http://localhost:8000/toys/{id} We have to compose and send an HTTP request with the PUT HTTP verb and request URL http://localhost:8000/toys/{id} to retrieve the toy whose ID matches the value in {id} and replace it with a toy created with the provided data. In addition, we have to provide the JSON key-value pairs with the field names and the values to create the new toy that will replace the existing one. As a result of the request, the server will validate the provided values for the fields, make sure that it is a valid toy, and replace the one that matches the specified ID with the new one in the database. The ID for the toy will be the same after the update operation. The server will update the existing row in the appropriate table and it will return a 200 OK status code and a JSON body with the recently updated toy serialized to JSON. If we don't provide all the necessary data for the new toy, the server will return a 400 Bad Request status code. If the server doesn't find a toy with the specified ID, the server will only return a 404 Not Found status: PUT http://localhost:8000/toys/{id} We have to compose and send an HTTP request with the DELETE HTTP verb and request URL http://localhost:8000/toys/{id} to remove the toy whose ID matches the specified numeric value in {id}. For example, if we use the request URL http://localhost:8000/toys/34, the server will delete the toy whose ID matches 34. As a result of the request, the server will retrieve a toy with the specified ID from the database and create the appropriate toy object in Python. If a toy is found, the server will request the ORM delete the toy row associated with this toy object and the server will return a 204 No Content status code. If no toy matches the specified ID, the server will return only a 404 Not Found status: DELETE http://localhost:8000/toys/{id} Creating first Django model Now, we will create a simple Toy model in Django, which we will use to represent and persist toys. Open the toys/models.py file. The following lines show the initial code for this file with just one import statement and a comment that indicates we should create the models: from django.db import models #Create your models here. The following lines show the new code that creates a Toy class, specifically, a Toy model in the toys/models.py file. The code file for the sample is included in the hillar_django_restful_02_01 folder in the restful01/toys/models.py file: from django.db import models class Toy(models.Model): created = models.DateTimeField(auto_now_add=True) name = models.CharField(max_length=150, blank=False, default='') description = models.CharField(max_length=250, blank=True, default='') toy_category = models.CharField(max_length=200, blank=False, default='') release_date = models.DateTimeField() was_included_in_home = models.BooleanField(default=False) class Meta: ordering = ('name',) The Toy class is a subclass of the django.db.models.Model class and defines the following attributes: created, name, description, toy_category, release_date, and was_included_in_home. Each of these attributes represents a database column or field. We specified the field types, maximum lengths, and defaults for many attributes. The class declares a Meta inner class that declares an ordering attribute and sets its value to a tuple of string whose first value is the 'name' string. This way, the inner class indicates to Django that, by default, we want the results ordered by the name attribute in ascending order. Running initial migration Now, it is necessary to create the initial migration for the new Toy model we recently coded. We will also synchronize the SQLite database for the first time. By default, Django uses the popular self-contained and embedded SQLite database, and therefore we don't need to make changes in the initial ORM configuration. In this example, we will be working with this default configuration. Of course, we will upgrade to another database after we have a sample web service built with Django. We will only use SQLite for this example. We just need to run the following Python script in the virtual environment that we activated in the previous chapter. Make sure you are in the restful01 folder within the main folder for the virtual environment when you run the following command: python manage.py makemigrations toys The following lines show the output generated after running the previous command: Migrations for 'toys': toys/migrations/0001_initial.py: - Create model Toy The output indicates that the restful01/toys/migrations/0001_initial.py file includes the code to create the Toy model. The following lines show the code for this file that was automatically generated by Django. The code file for the sample is included in the hillar_django_restful_02_01 folder in the restful01/toys/migrations/0001_initial.py file: # -*- coding: utf-8 -*- # Generated by Django 1.11.5 on 2017-10-08 05:19 from   future    import unicode_literals from django.db import migrations, models class Migration(migrations.Migration): initial = True dependencies = [ ] operations = [ migrations.CreateModel( name='Toy', fields=[ ('id', models.AutoField(auto_created=True, primary_key=True, serialize=False, verbose_name='ID')), ('created', models.DateTimeField(auto_now_add=True)), ('name', models.CharField(default='', max_length=150)), ('description', models.CharField(blank=True, default='', max_length=250)), ('toy_category', models.CharField(default='', max_length=200)), ('release_date', models.DateTimeField()), ('was_included_in_home', models.BooleanField(default=False)), ], options={ 'ordering': ('name',), }, ), ] Understanding migrations The automatically generated code defines a subclass of the django.db.migrations.Migration class named Migration, which defines an operation that creates the Toy model's table and includes it in the operations attribute. The call to the migrations.CreateModel method specifies the model's name, the fields, and the options to instruct the ORM to create a table that will allow the underlying database to persist the model. The fields argument is a list of tuples that includes information about the field name, the field type, and additional attributes based on the data we provided in our model, that is, in the Toy class. Now, run the following Python script to apply all the generated migrations. Make sure you are in the restful01 folder within the main folder for the virtual environment when you run the following command: python manage.py migrate The following lines show the output generated after running the previous command: Operations to perform: Apply all migrations: admin, auth, contenttypes, sessions, toys Running migrations: Applying contenttypes.0001_initial... OK Applying auth.0001_initial... OK Applying admin.0001_initial... OK Applying admin.0002_logentry_remove_auto_add... OK Applying contenttypes.0002_remove_content_type_name... OK Applying auth.0002_alter_permission_name_max_length... OK Applying auth.0003_alter_user_email_max_length... OK Applying auth.0004_alter_user_username_opts... OK Applying auth.0005_alter_user_last_login_null... OK Applying auth.0006_require_contenttypes_0002... OK Applying auth.0007_alter_validators_add_error_messages... OK Applying auth.0008_alter_user_username_max_length... OK Applying sessions.0001_initial... OK Applying toys.0001_initial... OK After we run the previous command, we will notice that the root folder for our restful01 project now has a db.sqlite3 file that contains the SQLite database. We can use the SQLite command line or any other application that allows us to easily check the contents of the SQLite database to check the tables that Django generated. The first migration will generate many tables required by Django and its installed apps before running the code that creates the table for the Toys model. These tables provide support for user authentication, permissions, groups, logs, and migration management. We will work with the models related to these tables after we add more features and security to our web services. After the migration process creates all these Django tables in the underlying database, the first migration runs the Python code that creates the table required to persist our model. Thus, the last line of the running migrations section displays Applying toys.0001_initial. Analyzing the database In most modern Linux distributions and macOS, SQLite is already installed, and therefore you can run the sqlite3 command-line utility. In Windows, if you want to work with the sqlite3.exe command-line utility, you have to download the bundle of command-line tools for managing SQLite database files from the downloads section of the SQLite webpage at http://www.sqlite.org/download.html. For example, the ZIP file that includes the command-line tools for version 3.20.1 is sqlite- tools-win32-x8 6-3200100.zip. The name for the file changes with the SQLite version. You just need to make sure that you download the bundle of command-line tools and not the ZIP file that provides the SQLite DLLs. After you unzip the file, you can include the folder that includes the command-line tools in the PATH environment variable, or you can access the sqlite3.exe command-line utility by specifying the full path to it. Run the following command to list the generated tables. The first argument, db.sqlite3, specifies the file that contains that SQLite database and the second argument indicates the command that we want the sqlite3 command-line utility to run against the specified database: sqlite3 db.sqlite3 ".tables" The following lines show the output for the previous command with the list of tables that Django generated in the SQLite database: auth_group                django_admin_log auth_group_permissions                          django_content_type auth_permission                         django_migrations auth_user                 django_session auth_user_groups                      toys_toy auth_user_user_permissions The following command will allow you to check the contents of the toys_toy table after we compose and send HTTP requests to the RESTful Web Service and the web service makes CRUD operations to the toys_toy table: sqlite3 db.sqlite3 "SELECT * FROM toys_toy ORDER BY name;" Instead of working with the SQLite command-line utility, you can use a GUI tool to check the contents of the SQLite database. DB Browser for SQLite is a useful, free, multiplatform GUI tool that allows us to easily check the database contents of an SQLite database in Linux, macOS, and Windows. You can read more information about this tool and download its different versions from http://sqlitebrowser.org. Once you have installed the tool, you just need to open the db.sqlite3 file and you can check the database structure and browse the data for the different tables. After we start working with the first version of our web service, you need to check the contents of the toys_toy table with this tool The SQLite database engine and the database file name are specified in the restful01/settings.py Python file. The following lines show the declaration of the DATABASES dictionary, which contains the settings for all the databases that Django uses. The nested dictionary maps the database named default with the django.db.backends.sqlite3 database engine and the db.sqlite3 database file located in the BASE_DIR folder (restful01): DATABASES = { 'default': { 'ENGINE': 'django.db.backends.sqlite3', 'NAME': os.path.join(BASE_DIR, 'db.sqlite3'), } } After we execute the migrations, the SQLite database will have the following tables. Django uses prefixes to identify the modules and applications that each table belongs to. The tables that start with the auth_ prefix belong to the Django authentication module. The table that starts with the toys_ prefix belongs to our toys application. If we add more models to our toys application, Django will create new tables with the toys_ prefix: auth_group: Stores authentication groups auth_group_permissions: Stores permissions for authentication groups auth_permission: Stores permissions for authentication auth_user: Stores authentication users auth_user_groups: Stores authentication user groups auth_user_groups_permissions: Stores permissions for authentication user groups django_admin_log: Stores the Django administrator log django_content_type: Stores Django content types django_migrations: Stores the scripts generated by Django migrations and the date and time at which they were applied django_session: Stores Django sessions toys_toy: Persists the Toys model sqlite_sequence: Stores sequences for SQLite primary keys with autoincrement fields Understanding the table generated by Django The toys_toy table persists in the database the Toy class we recently created, specifically, the Toy model. Django's integrated ORM generated the toys_toy table based on our Toy model. Run the following command to retrieve the SQL used to create the toys_toy table: sqlite3 db.sqlite3 ".schema toys_toy" The following lines show the output for the previous command together with the SQL that the migrations process executed, to create the toys_toy table that persists the Toy model. The next lines are formatted to make it easier to understand the SQL code. Notice that the output from the command is formatted in a different way: CREATE TABLE IF NOT EXISTS "toys_toy" ( "id" integer NOT NULL PRIMARY KEY AUTOINCREMENT, "created" datetime NOT NULL, "name" varchar(150) NOT NULL, "description" varchar(250) NOT NULL, "toy_category" varchar(200) NOT NULL, "release_date" datetime NOT NULL, "was_included_in_home" bool NOT NULL ); The toys_toy table has the following columns (also known as fields) with their SQLite types, all of them not nullable: id: The integer primary key, an autoincrement row created: DateTime name: varchar(150) description: varchar(250) toy_category: varchar(200) release_date: DateTime was_included_in_home: bool Controlling, serialization and deserialization Our RESTful Web Service has to be able to serialize and deserialize the Toy instances into JSON representations. In Django REST framework, we just need to create a serializer class for the Toy instances to manage serialization to JSON and deserialization from JSON. Now, we will dive deep into the serialization and deserialization process in Django REST framework. It is very important to understand how it works because it is one of the most important components for all the RESTful Web Services we will build. Django REST framework uses a two-phase process for serialization. The serializers are mediators between the model instances and Python primitives. Parser and renderers handle as mediators between Python primitives and HTTP requests and responses. We will configure our mediator between the Toy model instances and Python primitives by creating a subclass of the rest_framework.serializers.Serializer class to declare the fields and the necessary methods to manage serialization and deserialization. We will repeat some of the information about the fields that we have included in the Toy model so that we understand all the things that we can configure in a subclass of the Serializer class. However, we will work with shortcuts, which will reduce boilerplate code later in the following examples. We will write less code in the following examples by using the ModelSerializer class. Now, go to the restful01/toys folder and create a new Python code file named serializers.py. The following lines show the code that declares the new ToySerializer class. The code file for the sample is included in the hillar_django_restful_02_01 folder in the restful01/toys/serializers.py file: from rest_framework import serializers from toys.models import Toy class ToySerializer(serializers.Serializer): pk = serializers.IntegerField(read_only=True) name = serializers.CharField(max_length=150) description = serializers.CharField(max_length=250) release_date = serializers.DateTimeField() toy_category = serializers.CharField(max_length=200) was_included_in_home = serializers.BooleanField(required=False) def create(self, validated_data): return Toy.objects.create(**validated_data) def update(self, instance, validated_data): instance.name = validated_data.get('name', instance.name) instance.description = validated_data.get('description', instance.description) instance.release_date = validated_data.get('release_date', instance.release_date) instance.toy_category = validated_data.get('toy_category', instance.toy_category) instance.was_included_in_home = validated_data.get('was_included_in_home', instance.was_included_in_home) instance.save() return instance The ToySerializer class declares the attributes that represent the fields that we want to be serialized. Notice that we have omitted the created attribute that was present in the Toy model. When there is a call to the save method that ToySerializer inherits from the serializers.Serializer superclass, the overridden create and update methods define how to create a new instance or update an existing instance. In fact, these methods must be implemented in our class because they only raise a NotImplementedError exception in their base declaration in the serializers.Serializer superclass. The create method receives the validated data in the validated_data argument. The code creates and returns a new Toy instance based on the received validated data. The update method receives an existing Toy instance that is being updated and the new validated data in the instance and validated_data arguments. The code updates the values for the attributes of the instance with the updated attribute values retrieved from the validated data. Finally, the code calls the save method for the updated Toy instance and returns the updated and saved instance. We designed a RESTful Web Service to interact with a simple SQLite database and perform CRUD operations with toys. We defined the requirements for our web service and understood the tasks performed by each HTTP method and different scope. You read an excerpt from the book, Django RESTful Web Services, written by Gaston C. Hillar. This book helps developers build complex RESTful APIs from scratch with Django and the Django REST Framework. The code bundle for the article is hosted on GitHub.  
Read more
  • 0
  • 0
  • 19255

article-image-manipulating-jquery-tables
Packt
24 Oct 2009
20 min read
Save for later

Manipulating jQuery tables

Packt
24 Oct 2009
20 min read
In this article by Karl Swedberg and Jonathan Chaffer, we will use an online bookstore as our model website, but the techniques we cook up can be applied to a wide variety of other sites as well, from weblogs to portfolios, from market-facing business sites to corporate intranets. In this article, we will use jQuery to apply techniques for increasing the readability, usability, and visual appeal of tables, though we are not dealing with tables used for layout and design. In fact, as the web standards movement has become more pervasive in the last few years, table-based layout has increasingly been abandoned in favor of CSS‑based designs. Although tables were often employed as a somewhat necessary stopgap measure in the 1990s to create multi-column and other complex layouts, they were never intended to be used in that way, whereas CSS is a technology expressly created for presentation. But this is not the place for an extended discussion on the proper role of tables. Suffice it to say that in this article we will explore ways to display and interact with tables used as semantically marked up containers of tabular data. For a closer look at applying semantic, accessible HTML to tables, a good place to start is Roger Johansson's blog entry, Bring on the Tables at www.456bereastreet.com/archive/200410/bring_on_the_tables/. Some of the techniques we apply to tables in this article can be found in plug‑ins such as Christian Bach's Table Sorter. For more information, visit the jQuery Plug‑in Repository at http://jQuery.com/plugins. Sorting One of the most common tasks performed with tabular data is sorting. In a large table, being able to rearrange the information that we're looking for is invaluable. Unfortunately, this helpful operation is one of the trickiest to put into action. We can achieve the goal of sorting in two ways, namely Server-Side Sorting and JavaScript Sorting. Server-Side Sorting A common solution for data sorting is to perform it on the server side. Data in tables often comes from a database, which means that the code that pulls it out of the database can request it in a given sort order (using, for example, the SQL language's ORDER BY clause). If we have server-side code at our disposal, it is straightforward to begin with a reasonable default sort order. Sorting is most useful when the user can determine the sort order. A common idiom is to make the headers of sortable columns into links. These links can go to the current page, but with a query string appended indicating the column to sort by: <table id="my-data">   <tr>     <th class="name"><a href="index.php?sort=name">Name</a></th>     <th class="date"><a href="index.php?sort=date">Date</a></th>   </tr>   ... </table> The server can react to the query string parameter by returning the database contents in a different order. Preventing Page Refreshes This setup is simple, but requires a page refresh for each sort operation. As we have seen, jQuery allows us to eliminate such page refreshes by using AJAX methods. If we have the column headers set up as links as before, we can add jQuery code to change those links into AJAX requests: $(document).ready(function() {   $('#my-data .name a').click(function() {     $('#my-data').load('index.php?sort=name&type=ajax');     return false;   });   $('#my-data .date a').click(function() {     $('#my-data').load('index.php?sort=date&type=ajax');     return false;   }); }); Now when the anchors are clicked, jQuery sends an AJAX request to the server for the same page. We add an additional parameter to the query string so that the server can determine that an AJAX request is being made. The server code can be written to send back only the table itself, and not the surrounding page, when this parameter is present. This way we can take the response and insert it in place of the table. This is an example of progressiveenhancement. The page works perfectly well without any JavaScript at all, as the links for server-side sorting are still present. When JavaScript is present, however, the AJAX hijacks the page request and allows the sort to occur without a full page load. JavaScript Sorting There are times, though, when we either don't want to wait for server responses when sorting, or don't have a server-side scripting language available to us. A viable alternative in this case is to perform the sorting entirely on the browser using JavaScript client-side scripting. For example, suppose we have a table listing books, along with their authors, release dates, and prices: <table class="sortable">   <thead>     <tr>       <th></th>       <th>Title</th>       <th>Author(s)</th>       <th>Publish&nbsp;Date</th>       <th>Price</th>     </tr>   </thead>   <tbody>     <tr>       <td>         <img src="../covers/small/1847192386.png" width="49"              height="61" alt="Building Websites with                                                 Joomla! 1.5 Beta 1" />       </td>       <td>Building Websites with Joomla! 1.5 Beta 1</td>       <td>Hagen Graf</td>       <td>Feb 2007</td>       <td>$40.49</td>     </tr>     <tr>       <td><img src="../covers/small/1904811620.png" width="49"                height="61" alt="Learning Mambo: A Step-by-Step                Tutorial to Building Your Website" /></td>       <td>Learning Mambo: A Step-by-Step Tutorial to Building Your           Website</td>       <td>Douglas Paterson</td>       <td>Dec 2006</td>       <td>$40.49</td>     </tr>     ...   </tbody> </table> We'd like to turn the table headers into buttons that sort by their respective columns. Let us look into ways of doing this.   Row Grouping Tags Note our use of the <thead> and <tbody> tags to segment the data into row groupings. Many HTML authors omit these implied tags, but they can prove useful in supplying us with more convenient CSS selectors to use. For example, suppose we wish to apply typical even/odd row striping to this table, but only to the body of the table: $(document).ready(function() {   $('table.sortable tbody tr:odd').addClass('odd');   $('table.sortable tbody tr:even').addClass('even'); }); This will add alternating colors to the table, but leave the header untouched: Basic Alphabetical Sorting Now let's perform a sort on the Titlecolumn of the table. We'll need a class on the table header cell so that we can select it properly: <thead>   <tr>     <th></th>    <th class="sort-alpha">Title</th>     <th>Author(s)</th>     <th>Publish&nbsp;Date</th>     <th>Price</th>   </tr> </thead> To perform the actual sort, we can use JavaScript's built in .sort()method. It does an in‑place sort on an array, and can take a function as an argument. This function compares two items in the array and should return a positive or negative number depending on the result. Our initial sort routine looks like this: $(document).ready(function() {   $('table.sortable').each(function() {     var $table = $(this);     $('th', $table).each(function(column) {       if ($(this).is('.sort-alpha')) {         $(this).addClass('clickable').hover(function() {           $(this).addClass('hover');         }, function() {           $(this).removeClass('hover');         }).click(function() {           var rows = $table.find('tbody > tr').get();           rows.sort(function(a, b) {             var keyA = $(a).children('td').eq(column).text()                                                       .toUpperCase();             var keyB = $(b).children('td').eq(column).text()                                                       .toUpperCase();             if (keyA < keyB) return -1;             if (keyA > keyB) return 1;             return 0;           });           $.each(rows, function(index, row) {             $table.children('tbody').append(row);           });         });       }     });   }); }); The first thing to note is our use of the .each() method to make iteration explicit. Even though we could bind a click handler to all headers with the sort-alpha class just by calling $('table.sortable th.sort-alpha').click(), this wouldn't allow us to easily capture a crucial bit of information—the column index of the clicked header. Because .each() passes the iteration index into its callback function, we can use it to find the relevant cell in each row of the data later. Once we have found the header cell, we retrieve an array of all of the data rows. This is a great example of how .get()is useful in transforming a jQuery object into an array of DOM nodes; even though jQuery objects act like arrays in many respects, they don't have any of the native array methods available, such as .sort(). With .sort() at our disposal, the rest is fairly straightforward. The rows are sorted by comparing the textual contexts of the relevant table cell. We know which cell to look at because we captured the column index in the enclosing .each() call. We convert the text to uppercase because string comparisons in JavaScript are case-sensitive and we wish our sort to be case-insensitive. Finally, with the array sorted, we loop through the rows and reinsert them into the table. Since .append() does not clone nodes, this moves them rather than copying them. Our table is now sorted. This is an example of progressive enhancement's counterpart, gracefuldegradation. Unlike with the AJAX solution discussed earlier, we cannot make the sort work without JavaScript, as we are assuming the server has no scripting language available to it in this case. The JavaScript is required for the sort to work, so by adding the "clickable" class only through code, we make sure not to indicate with the interface that sorting is even possible unless the script can run. The page degrades into one that is still functional, albeit without sorting available. We have moved the actual rows around, hence our alternating row colors are now out of whack: We need to reapply the row colors after the sort is performed. We can do this by pulling the coloring code out into a function that we call when needed: $(document).ready(function() {   var alternateRowColors = function($table) {     $('tbody tr:odd', $table).removeClass('even').addClass('odd');     $('tbody tr:even', $table).removeClass('odd').addClass('even');   };     $('table.sortable').each(function() {     var $table = $(this);     alternateRowColors($table);     $('th', $table).each(function(column) {       if ($(this).is('.sort-alpha')) {         $(this).addClass('clickable').hover(function() {           $(this).addClass('hover');         }, function() {           $(this).removeClass('hover');         }).click(function() {           var rows = $table.find('tbody > tr').get();           rows.sort(function(a, b) {             var keyA = $(a).children('td').eq(column).text()                                                       .toUpperCase();             var keyB = $(b).children('td').eq(column).text()                                                       .toUpperCase();             if (keyA < keyB) return -1;             if (keyA > keyB) return 1;             return 0;           });           $.each(rows, function(index, row) {             $table.children('tbody').append(row);           });           alternateRowColors($table);         });       }     });   }); }); This corrects the row coloring after the fact, fixing our issue:   The Power of Plug-ins The alternateRowColors()function that we wrote is a perfect candidate to become a jQuery plug-in. In fact, any operation that we wish to apply to a set of DOM elements can easily be expressed as a plug-in. In this case, we only need to modify our existing function a little bit: jQuery.fn.alternateRowColors = function() {   $('tbody tr:odd', this).removeClass('even').addClass('odd');   $('tbody tr:even', this).removeClass('odd').addClass('even');   return this; }; We have made three important changes to the function. It is defined as a new property of jQuery.fn rather than as a standalone function. This registers the function as a plug-in method. We use the keyword this as a replacement for our $table parameter. Within a plug-in method, thisrefers to the jQuery object that is being acted upon. Finally, we return this at the end of the function. The return value makes our new method chainable. More information on writing jQuery plug-ins can be found in Chapter 10 of our book Learning jQuery. There we will discuss making a plug-in ready for public consumption, as opposed to the small example here that is only to be used by our own code. With our new plug-in defined, we can call $table.alternateRowColors(), which is a more natural jQuery syntax, intead of alternateRowColors($table). Performance Concerns Our code works, but is quite slow. The culprit is the comparator function, which is performing a fair amount of work. This comparator will be called many times during the course of a sort, which means that every extra moment it spends on processing will be magnified. The actual sort algorithm used by JavaScript is not defined by the standard. It may be a simple sort like a bubble sort (worst case of Θ(n2) in computational complexity terms) or a more sophisticated approach like quick sort (which is Θ(n log n) on average). In either case doubling the number of items increases the number of times the comparator function is called by more than double. The remedy for our slow comparator is to pre-compute the keys for the comparison. We begin with the slow sort function: rows.sort(function(a, b) {   keyA = $(a).children('td').eq(column).text().toUpperCase();   keyB = $(b).children('td').eq(column).text().toUpperCase();   if (keyA < keyB) return -1;   if (keyA > keyB) return 1;   return 0; }); $.each(rows, function(index, row) {   $table.children('tbody').append(row); }); We can pull out the key computation and do that in a separate loop: $.each(rows, function(index, row) {   row.sortKey = $(row).children('td').eq(column).text().toUpperCase(); }); rows.sort(function(a, b) {   if (a.sortKey < b.sortKey) return -1;   if (a.sortKey > b.sortKey) return 1;   return 0; }); $.each(rows, function(index, row) {   $table.children('tbody').append(row);   row.sortKey = null; }); In the new loop, we are doing all of the expensive work and storing the result in a new property. This kind of property, attached to a DOM element but not a normal DOM attribute, is called an expando.This is a convenient place to store the key since we need one per table row element. Now we can examine this attribute within the comparator function, and our sort is markedly faster.  We set the expando property to null after we're done with it to clean up after ourselves. This is not necessary in this case, but is a good habit to establish because expando properties left lying around can be the cause of memory leaks. For more information, see Appendix C.   Finessing the Sort Keys Now we want to apply the same kind of sorting behavior to the Author(s) column of our table. By adding the sort-alpha class to its table header cell, the Author(s)column can be sorted with our existing code. But ideally authors should be sorted by last name, not first. Since some books have multiple authors, and some authors have middle names or initials listed, we need outside guidance to determine what part of the text to use as our sort key. We can supply this guidance by wrapping the relevant part of the cell in a tag: <tr>   <td>     <img src="../covers/small/1847192386.png" width="49" height="61"             alt="Building Websites with Joomla! 1.5 Beta 1" /></td>   <td>Building Websites with Joomla! 1.5 Beta 1</td>   <td>Hagen <span class="sort-key">Graf</span></td>   <td>Feb 2007</td>   <td>$40.49</td> </tr> <tr>   <td>     <img src="../covers/small/1904811620.png" width="49" height="61"          alt="Learning Mambo: A Step-by-Step Tutorial to Building                                                 Your Website" /></td>   <td>     Learning Mambo: A Step-by-Step Tutorial to Building Your Website   </td>   <td>Douglas <span class="sort-key">Paterson</span></td>   <td>Dec 2006</td>   <td>$40.49</td> </tr> <tr>   <td>     <img src="../covers/small/1904811299.png" width="49" height="61"                   alt="Moodle E-Learning Course Development" /></td>   <td>Moodle E-Learning Course Development</td>   <td>William <span class="sort-key">Rice</span></td>   <td>May 2006</td>   <td>$35.99</td> </tr> Now we have to modify our sorting code to take this tag into account, without disturbing the existing behavior for the Titlecolumn, which is working well. By prepending the marked sort key to the key we have previously calculated, we can sort first on the last name if it is called out, but on the whole string as a fallback: $.each(rows, function(index, row) {   var $cell = $(row).children('td').eq(column);   row.sortKey = $cell.find('.sort-key').text().toUpperCase()                                   + ' ' + $cell.text().toUpperCase(); }); Sorting by the Author(s)column now uses the last name:     If two last names are identical, the sort uses the entire string as a tiebreaker for positioning.
Read more
  • 0
  • 0
  • 19215
article-image-introduction-using-nodejs-hadoops-mapreduce-jobs
Harri Siirak
25 Sep 2015
5 min read
Save for later

Using Node.js and Hadoop to store distributed data

Harri Siirak
25 Sep 2015
5 min read
Hadoop is a well-known open-source software framework for distributed storage and distributed processing of very large data sets on computer clusters built from commodity hardware. It's designed with a fundamental assumption that hardware failures can (and will) happen and thus should be automatically handled in software by the framework. Under the hood it's using HDFS (Hadoop Distributed File System) for the data storage. HDFS can store large files across multiple machines and it achieves reliability by replicating the data across multiple hosts (default replication factor is 3 and can be configured to be higher when needed). Although it's designed for mostly immutable files and may not be suitable for systems requiring concurrent write-operations. Its target usage is not only restricted to MapReduce jobs, but it also can be used for cost effective and reliable data storage. In the following examples, I am going to give you an overview of how to establish connections to HDFS storage (namenode) and how to perform basic operations on the data. As you can probably guess, I'm using Node.js to build these examples. Node.js is a platform built on Chrome's JavaScript runtime for easily building fast, scalable network applications. Node.js uses an event-driven, non-blocking I/O model that makes it lightweight and efficient, perfect for data-intensive real-time applications that run across distributed devices. So it's really ideal for what I want to show you next. Two popular libraries for acccessing HDFS in Node.js are node-hdfs and webhdfs. The first one uses Hadoop's native libhdfs library and protocol to communicate with Hadoop namenode, albeit it seems to be not maintained anymore and doesn't support Stream API. Another one is using WebHDFS, which defines a public HTTP REST API, directly built into Hadoop's core (namenodes and datanodes both) and which permits clients to access Hadoop from multiple languages without installing Hadoop, and supports all HDFS user operations including reading files, writing to files, making directories, changing permissions and renaming. More details about WebHDFS REST API and about its implementation details and response codes/types can be found from here. At this point I'm assuming that you have Hadoop cluster up and running. There are plenty of good tutorials out there showing how to setup and run Hadoop cluster (single and multi node). Installing and using the webhdfs library webhdfs implements most of the REST API calls, albeit it's not yet supporting Hadoop delegation tokens. It's also Stream API compatible what makes its usage pretty straightforward and easy. Detailed examples and use cases for another supported calls can be found from here. Install webhdfs from npm: npm install wehbhdfs Create a new script named webhdfs-client.js: // Include webhdfs module var WebHDFS = require('webhdfs'); // Create a new var hdfs = WebHDFS.createClient({ user: 'hduser', // Hadoop user host: 'localhost', // Namenode host port: 50070 // Namenode port }); module.exports = hdfs; Here we initialized new webhdfs client with options, including namenode's host and port where we are connecting to. Let's proceed with a more detailed example. Storing file data in HDFS Create a new script named webhdfs-write-test.js and add the code below. // Include created client var hdfs = require('./webhdfs-client'); // Include fs module for local file system operations var fs = require('fs'); // Initialize readable stream from local file // Change this to real path in your file system var localFileStream = fs.createReadStream('/path/to/local/file'); // Initialize writable stream to HDFS target var remoteFileStream = hdfs.createWriteStream('/path/to/remote/file'); // Pipe data to HDFS localFileStream.pipe(remoteFileStream); // Handle errors remoteFileStream.on('error', function onError (err) { // Do something with the error }); // Handle finish event remoteFileStream.on('finish', function onFinish () { // Upload is done }); Basically what we are doing here is that we're initializing readable file stream from a local filesystem and piping its contents seamlessly into remote HDFS target. Optionally webhdfs exposes error and finish. Reading file data from HDFS Let's retrieve the data what we just stored in HDFS storage. Create a new script named webhdfs-read-test.js and add code below. var hdfs = require('./webhdfs-client'); var fs = require('fs'); // Initialize readable stream from HDFS source var remoteFileStream = hdfs.createReadStream('/path/to/remote/file'); // Variable for storing data var data = new Buffer(); remoteFileStream.on('error', function onError (err) { // Do something with the error }); remoteFileStream.on('data', function onChunk (chunk) { // Concat received data chunk data = Buffer.concat([ data, chunk ]); }); remoteFileStream.on('finish', function onFinish () { // Upload is done // Print received data console.log(data.toString()); }); What's next? Now when we have data in Hadoop cluster, we can start processing it by spawning some MapReduce jobs, and when it's processed we can retrieve the output data. In the second part of this article, I'm going to give you an overview of how Node.js can be used as part of MapReduce jobs. About the author Harri is a senior Node.js/Javascript developer among a talented team of full-stack developers who specialize in building scalable and secure Node.js based solutions. He can be found on Github at harrisiirak.
Read more
  • 0
  • 2
  • 19157

article-image-building-games-html5-and-dart
Packt
21 Sep 2015
19 min read
Save for later

Building Games with HTML5 and Dart

Packt
21 Sep 2015
19 min read
In this article written by Ivo Balbaert, author of the book Learning Dart - Second Edition, you will learn to create a well-known memory game. Also, you will design a model first and work up your way from a modest beginning to a completely functional game, step by step. You will also learn how to enhance the attractiveness of web games with audio and video techniques. The following topics will be covered in this article: The model for the memory game Spiral 1—drawing the board Spiral 2—drawing cells Spiral 3—coloring the cells Spiral 4—implementing the rules Spiral 5—game logic (bringing in the time element) Spiral 6—some finishing touches Spiral 7—using images (For more resources related to this topic, see here.) The model for the memory game When started, the game presents a board with square cells. Every cell hides an image that can be seen by clicking on the cell, but this disappears quickly. You must remember where the images are, because they come in pairs. If you quickly click on two cells that hide the same picture, the cells will "flip over" and the pictures will stay visible. The objective of the game is to turn over all the pairs of matching images in a very short time. After some thinking we came up with the following model, which describes the data handled by the application. In our game, we have a number of pictures, which could belong to a Catalog. For example, a travel catalog with a collection of photos from our trips or something similar. Furthermore, we have a collection of cells and each cell is hiding a picture. Also, we have a structure that we will call memory, and this contains the cells in a grid of rows and columns. We could draw it up as shown in the following figure. You can import the model from the game_memory_json.txt file that contains its JSON representation: A conceptual model of the memory game The Catalog ID is its name, which is mandatory, but the description is optional. The Picture ID consists of the sequence number within the Catalog. The imageUri field stores the location of the image file. width and height are optional properties, since they may be derived from the image file. The size may be small, medium, or large to help select an image. The ID of a Memory is its name within the Catalog, the collection of cells is determined by the memory length, for example, 4 cells per side. Each cell is of the same length cellLength, which is a property of the memory. A memory is recalled when all the image pairs are discovered. Some statistics must be kept, such as recall count, the best recall time in seconds, and the number of cell clicks to recover the whole image (minTryCount). The Cell has the row and column coordinates and also the coordinates of its twin with the same image. Once the model is discussed and improved, model views may be created: a Board would be a view of the Memory concept and a Box would be a view of the Cell concept. The application would be based on the Catalog concept. If there is no need to browse photos of a catalog and display them within a page, there would not be a corresponding view. Now, we can start developing this game from scratch. Spiral 1 – drawing the board The app starts with main() in educ_memory_game.dart: library memory; import 'dart:html'; part 'board.dart'; void main() { // Get a reference to the canvas. CanvasElement canvas = querySelector('#canvas'); (1) new Board(canvas); (2) } We'll draw a board on a canvas element. So, we need a reference that is given in line (1). The Board view is represented in code as its own Board class in the board.dart file. Since everything happens on this board, we construct its object with canvas as an argument (line (2)). Our game board will be periodically drawn as a rectangle in line (4) by using the animationFrame method from the Window class in line (3): part of memory; class Board { CanvasElement canvas; CanvasRenderingContext2D context; num width, height; Board(this.canvas) { context = canvas.getContext('2d'); width = canvas.width; height = canvas.height; window.animationFrame.then(gameLoop); (3) } void gameLoop(num delta) { draw(); window.animationFrame.then(gameLoop); } void draw() { clear(); border(); } void clear() { context.clearRect(0, 0, width, height); } void border() { context..rect(0, 0, width, height)..stroke(); (4) } } This is our first result: The game board Spiral 2 – drawing cells In this spiral, we will give our app code some structure: Board is a view, so board.dart is moved to the view folder. We will also introduce here the Memory class from our model in its own code memory.dart file in the model folder. So, we will have to change the part statements to the following: part 'model/memory.dart'; part 'view/board.dart'; The Board view needs to know about Memory. So, we will include it in the Board class and make its object in the Board constructor: new Board(canvas, new Memory(4)); The Memory class is still very rudimentary with only its length property: class Memory { num length; Memory(this.length); } Our Board class now also needs a method to draw the lines, which we decided to make private because it is specific to Board, as well as the clear() and border()methods: void draw() { _clear(); _border(); _lines(); } The lines method is quite straightforward; first draw it on a piece of paper and translate it to code using moveTo and lineTo. Remember that x goes from top-left to right and y goes from top-left to bottom: void _lines() { var gap = height / memory.length; var x, y; for (var i = 1; i < memory.length; i++) { x = gap * i; y = x; context ..moveTo(x, 0) ..lineTo(x, height) ..moveTo(0, y) ..lineTo(width, y); } } The result is a nice grid: Board with cells Spiral 3 – coloring the cells To simplify, we will start using colors instead of pictures to be shown in the grid. Up until now, we didn't implement the cell from the model. Let's do that in modelcell.dart. We start simple by saying that the Cell class has the row, column, and color properties, and it belongs to a Memory object passed in its constructor: class Cell { int row, column; String color; Memory memory; Cell(this.memory, this.row, this.column); } Because we need a collection of cells, it is a good idea to make a Cells class, which contains List. We give it an add method and also an iterator so that we are able to use a for…in statement to loop over the collection: class Cells { List _list; Cells() { _list = new List(); } void add(Cell cell) { _list.add(cell); } Iterator get iterator => _list.iterator; } We will need colors that are randomly assigned to the cells. We will also need some utility variables and methods that do not specifically belong to the model and don't need a class. Hence, we will code them in a folder called util. To specify the colors for the cells, we will use two utility variables: a List variable of colors (colorList), which has the name colors, and a colorMap variable that maps the names to their RGB values. Refer to utilcolor.dart; later on, we can choose some fancier colors: var colorList = ['black', 'blue', //other colors ]; var colorMap = {'black': '#000000', 'blue': '#0000ff', //... }; To generate (pseudo) random values (ints, doubles, or Booleans), Dart has the Random class from dart:math. We will use the nextInt method, which takes an integer (the maximum value) and returns a positive random integer in the range from 0 (inclusive) to max (exclusive). We will build upon this in utilrandom.dart to make methods that give us a random color: int randomInt(int max) => new Random().nextInt(max); randomListElement(List list) => list[randomInt(list.length - 1)]; String randomColor() => randomListElement(colorList); String randomColorCode() => colorMap[randomColor()]; Our Memory class now contains an instance of the Cells class: Cells cells; We build this in the Memory constructor in a nested for loop, where each cell is successively instantiated with a row and column, given a random color, and added to cells: Memory(this.length) { cells = new Cells(); var cell; for (var x = 0; x < length; x++) { for (var y = 0; y < length; y++) { cell = new Cell(this, x, y); cell.color = randomColor(); cells.add(cell); } } } We can draw a rectangle and fill it with a color at the same time. So, we realize that we don't need to draw lines as we did in the previous spiral! The _boxes method is called from the draw animation: with a for…in statement, we loop over the collection of cells and call the _colorBox method that will draw and color the cell for each cell: void _boxes() { for (Cell cell in memory.cells) { _colorBox(cell); } } void _colorBox(Cell cell) { var gap = height / memory.length; var x = cell.row * gap; var y = cell.column * gap; context ..beginPath() ..fillStyle = colorMap[cell.color] ..rect(x, y, gap, gap) ..fill() ..stroke() ..closePath(); } Spiral 4 – implementing the rules However, wait! Our game can only work if the same color appears in only two cells: a cell and its twin cell. Moreover, a cell can be hidden or not: the color can be seen or not? To take care of this, the Cell class gets two new attributes: Cell twin; bool hidden = true; The _colorBox method in the Board class can now show the color of the cell when hidden is false (line (2)); when hidden = true (the default state), a neutral gray color will be used for the cell (line (1)): static const String COLOR_CODE = '#f0f0f0'; We also gave the gap variable a better name, boxSize: void _colorBox(Cell cell) { var x = cell.column * boxSize; var y = cell.row * boxSize; context.beginPath(); if (cell.hidden) { context.fillStyle = COLOR_CODE; (1) } else { context.fillStyle = colorMap[cell.color]; (2) } // same code as in Spiral 3 } The lines (1) and (2) can also be stated more succinctly with the ? ternary operator. Remember that the drawing changes because the _colorBox method is called via draw at 60 frames per second and the board can react to a mouse click. In this spiral, we will show a cell when it is clicked together with its twin cell and then they will stay visible. Attaching an event handler for this is easy. We add the following line to the Board constructor: querySelector('#canvas').onMouseDown.listen(onMouseDown); The onMouseDown event handler has to know on which cell the click occurred. The mouse event e contains the coordinates of the click in its e.offset.x and e.offset.y properties (lines (3) and (4)). We will obtain the cell's row and column by using a truncating division ~/ operator dividing the x (which gives the column) and y (which gives the row) values by boxSize: void onMouseDown(MouseEvent e) { int row = e.offset.y ~/ boxSize; (3) int column = e.offset.x ~/ boxSize; (4) Cell cell = memory.getCell(row, column); (5) cell.hidden = false; (6) cell.twin.hidden = false; (7) } Memory has a collection of cells. To get the cell with a specified row and column value, we will add a getCell method to memory and call it in line (5). When we have the cell, we will set its hidden property and that of its twin cell to false (lines (6) to (7)). The getCell method must return the cell at the given row and column. It loops through all the cells in line (8) and checks each cell, whether it is positioned at that row and column (line (9)). If yes, it will return that cell: Cell getCell(int row, int column) { for (Cell cell in cells) { (8) if (cell.intersects(row, column)) { (9) return cell; } } } For this purpose, we will add an intersects method to the Cell class. This checks whether its row and column match the given row and column for the current cell (see line (10)): bool intersects(int row, int column) { if (this.row == row && this.column == column) { (10) return true; } return false; } Now, we have already added a lot of functionality, but the drawing of the board will need some more thinking: How to give a cell (and its twin cell) a random color that is not yet used? How to attach a cell randomly to a twin cell that is not yet used? To end this, we will have to make the constructor of Memory a lot more intelligent: Memory(this.length) { if (length.isOdd) { (1) throw new Exception( 'Memory length must be an even integer: $length.'); } cells = new Cells(); var cell, twinCell; for (var x = 0; x < length; x++) { for (var y = 0; y < length; y++) { cell = getCell(y, x); (2) if (cell == null) { (3) cell = new Cell(this, y, x); cell.color = _getFreeRandomColor(); (4) cells.add(cell); twinCell = _getFreeRandomCell(); (5) cell.twin = twinCell; (6) twinCell.twin = cell; twinCell.color = cell.color; cells.add(twinCell); } } } } The number of pairs given by ((length * length) / 2) must be even. This is only true if the length parameter of Memory itself is even, so we checked it in line (1). Again, we coded a nested loop and got the cell at that row and column. However, as the cell at that position has not yet been made (line (3)), we continued to construct it and assign its color and twin. In line (4), we called _getFreeRandomColor to get a color that is not yet used: String _getFreeRandomColor() { var color; do { color = randomColor(); } while (usedColors.any((c) => c == color)); (7) usedColors.add(color); (8) return color; } The do…while loop continues as long as the color is already in a list of usedColors. On exiting from the loop, we found an unused color, which is added to usedColors in line (8) and also returned. We then had to set everything for the twin cell. We searched for a free one with the _getFreeRandomCell method in line (5). Here, the do…while loop continues until a (row, column) position is found where cell == null is, meaning that we haven't yet created a cell there (line (9)). We will promptly do this in line (10): Cell _getFreeRandomCell() { var row, column; Cell cell; do { row = randomInt(length); column = randomInt(length); cell = getCell(row, column); } while (cell != null); (9) return new Cell(this, row, column); (10) } From line (6) onwards, the properties of the twin cell are set and added to the list. This is all we need to produce the following result: Paired colored cells Spiral 5 – game logic (bringing in the time element) Our app isn't playable yet: When a cell is clicked, its color must only show for a short period of time (say one second) When a cell and its twin cell are clicked within a certain time interval, they must remain visible All of this is coded in the mouseDown event handler and we also need a lastCellClicked variable of the Cell type in the Board class. Of course, this is exactly the cell we get in the mouseDown event handler. So, we will set it in line (5) in the following code snippet: void onMouseDown(MouseEvent e) { // same code as in Spiral 4 - if (cell.twin == lastCellClicked && lastCellClicked.shown) { (1) lastCellClicked.hidden = false; (2) if (memory.recalled) memory.hide(); (3) } else { new Timer(const Duration(milliseconds: 1000), () => cell.hidden = true); (4) } lastCellClicked = cell; (5) } In line (1), we checked whether the last clicked cell was the twin cell and whether this is still shown. Then, we made sure in (2) that it stays visible. shown is a new getter in the Cell class to make the code more readable: bool get shown => !hidden;. If at that moment all the cells were shown (the memory is recalled), we again hid them in line (3). If the last clicked cell was not the twin cell, we hid the current cell after one second in line (4). recalled is a simple getter (read-only property) in the Memory class and it makes use of a Boolean variable in Memory that is initialized to false (_recalled = false;): bool get recalled { if (!_recalled) { if (cells.every((c) => c.shown)) { (6) _recalled = true; } } return _recalled; } In line (6), we tested that if every cell is shown, then this variable is set to true (the game is over). every is a new method in the Cells List and a nice functional way to write this is given as follows: bool every(Function f) => list.every(f); The hide method is straightforward: hide every cell and reset the _recalled variable to false: hide() { for (final cell in cells) cell.hidden = true; _recalled = false; } This is it, our game works! Spiral 6 – some finishing touches A working program always gives its developer a sense of joy, and rightfully so. However, this doesn't that mean you can leave the code as it is. On the contrary, carefully review your code for some time to see whether there is room for improvement or optimization. For example, are the names you used clear enough? The color of a hidden cell is now named simply COLOR_CODE in board.dart, renaming it to HIDDEN_CELL_COLOR_CODE makes its meaning explicit. The List object used in the Cells class can indicate that it is List<Cell>, by applying the fact that Dart lists are generic. The parameter of the every method in the Cell class is more precise—it is a function that accepts a cell and returns bool. Our onMouseDown event handler contains our game logic, so it is very important to tune it if possible. After some thought, we see that the code from the previous spiral can be improved; in the following line, the second condition after && is, in fact, unnecessary: if (cell.twin == lastCellClicked && lastCellClicked.shown) {...} When the player has guessed everything correctly, showing the completed screen for a few seconds will be more satisfactory (line (2)). So, this portion of our event handler code will change to: if (cell.twin == lastCellClicked) { (1) lastCellClicked.hidden = false; if (memory.recalled) { // game over new Timer(const Duration(milliseconds: 5000), () => memory.hide()); (2) } } else if (cell.twin.hidden) { new Timer(const Duration(milliseconds: 800), () => cell.hidden = true); } Why don’t we show a "YOU HAVE WON!" banner. We will do this by drawing the text on the canvas (line (3)), so we must do it in the draw() method (otherwise, it would disappear after INTERVAL milliseconds): void draw() { _clear(); _boxes(); if (memory.recalled) { // game over context.font = "bold 25px sans-serif"; context.fillStyle = "red"; context.fillText("YOU HAVE WON !", boxSize, boxSize * 2); (3) } } Then, the same game with the same configuration can be played again. We could make it more obvious that a cell is hidden by decorating it with a small circle in the _colorBox method (line (4)): if (cell.hidden) { context.fillStyle = HIDDEN_CELL_COLOR_CODE; var centerX = cell.column * boxSize + boxSize / 2; var centerY = cell.row * boxSize + boxSize / 2; var radius = 4; context.arc(centerX, centerY, radius, 0, 2 * PI, false); (4) } We do want to give our player a chance to start over by supplying a Play again button. The easiest way will be to simply refresh the screen (line (5)) by adding this code to the startup script: void main() { canvas = querySelector('#canvas'); ButtonElement play = querySelector('#play'); play.onClick.listen(playAgain); new Board(canvas, new Memory(4)); } playAgain(Event e) { window.location.reload(); (5) } Spiral 7 – using images One improvement that certainly comes to mind is the use of pictures instead of colors as shown in the Using images screenshot. How difficult would that be? It turns out that this is surprisingly easy, because we already have the game logic firmly in place! In the images folder, we supply a number of game pictures. Instead of the color property, we give the cell a String property (image), which will contain the name of the picture file. We then replace utilcolor.dart with utilimages.dart, which contains a imageList variable with the image filenames. In utilrandom.dart, we will replace the color methods with the following code: String randomImage() => randomListElement(imageList); The changes to memory.dart are also straightforward: replace the usedColor list with List usedImages = []; and the _getFreeRandomColor method with _getFreeRandomImage, which will use the new list and method: List usedImages = []; String _getFreeRandomImage() { var image; do { image = randomImage(); } while (usedImages.any((i) => i == image)); usedImages.add(image); return image; } In board.dart, we replace _colorBox(cell) with _imageBox(cell). The only new thing is how to draw the image on canvas. For this, we need ImageElement objects. Here, we have to be careful to create these objects only once and not over and over again in every draw cycle, because this produces a flickering screen. We will store the ImageElements object in a Map: var imageMap = new Map<String, ImageElement>(); Then, we populate this in the Board constructor with a for…in loop over memory.cells: for (var cell in memory.cells) { ImageElement image = new Element.tag('img'); (1) image.src = 'images/${cell.image}'; (2) imageMap[cell.image] = image; (3) } We create a new ImageElement object in line (1), giving it the complete file path to the image file as a src property in line (2) and store it in imageMap in line (3). The image file will then be loaded into memory only once. We don't do any unnecessary network access to effectively cache the images. In the draw cycle, we will load the image from imageMap and draw it in the current cell with the drawImage method in line (4): if (cell.hidden) { // see previous code } else { ImageElement image = imageMap[cell.image]; context.drawImage(image, x, y); // resize to cell size (4) } Perhaps, you can think of other improvements? Why not let the player specify the game difficulty by asking the number of boxes. It is 16 now. Check whether the input is a square of an even number. Do you have enough colors to choose from? Perhaps, dynamically building a list with enough random colors would be a better idea. Calculating and storing the statistics discussed in the model would also make the game more attractive. Another enhancement from the model is to support different catalogs of pictures. Go ahead and exercise your Dart skills! Summary By thoroughly investigating two games applying all of Dart we have already covered, your Dart star begins to shine. For other Dart games, visit http://www.builtwithdart.com/projects/games/. You can find more information at http://www.dartgamedevs.org/ on building games. Resources for Article: Further resources on this subject: Slideshow Presentations [article] Dart with JavaScript [article] Practical Dart [article]
Read more
  • 0
  • 0
  • 19049

article-image-clojure-domain-specific-languages-design-concepts-clojure
Packt
13 Dec 2013
3 min read
Save for later

Clojure for Domain-specific Languages - Design Concepts with Clojure

Packt
13 Dec 2013
3 min read
(For more resources related to this topic, see here.) Every function is a little program When I first started getting deep into Clojure development, my friend Tom Marble taught me a very good lesson with a single sentence. I'm not sure if he's the originator of this idea, but he told me to think of writing functions as though "every function is a small program". I'm not really sure what I thought about functions before I heard this, but it all made sense the very moment he told me this. Why write a function as if it were its own program? Because both a function and a program are created to handle a specific set of problems, and this method of thinking allows us to break down our problems into a simpler group of problems. Each set of problems might only need a very limited collection of functions to solve them, so to make a function that fits only a single problem isn't really any different from writing a small program to get the very same result. Some might even call this the Unix philosophy, in the sense that you're trying to build small, extendable, simple, and modular code. A pure function What are the benefits of a program-like function? There are many benefits to this approach of development, but the two clear advantages are that the debugging process can be simplified with the decoupling of task, and this approach can make our code more modular. This approach also allows us to better build pure functions. A pure function isn't dependent on any variable outside the function. Anything other than the arguments passed to the function can't be realized by a pure function. Because our program will cause side effects as a result of execution, not all of our functions can be truly pure. This doesn't mean we should forget about trying to develop program-like functions. Our code inherently becomes more modular because pure functions can survive on their own. This is key when needing to build flexible, extendable, and reusable code components. Floor to roof development It is also known as bottom-up development and is the concept of building basic low- level pieces of a program and then combining them to build the whole program. This approach leads to more reusable code that can be more easily tested because each part of the program acts as an individual building block and doesn't require a large portion of the program to be completed to run a test. Each function only does one thing When a function is written to perform a specific task, that function shouldn't do anything unrelated to the original problem it's needed to solve. For example, if you were to write a function named parse-xml, the function should be able to act as a program that can only parse XML data. If the example function does anything else other than parse lines of XML input, it is probably badly designed and will cause confusion when trying to debug errors in our programs. This practice will help us keep our functions to a more reasonable size and can also help simplify the debugging process.
Read more
  • 0
  • 0
  • 18828
article-image-using-socketio-and-express-together
Packt
23 Sep 2014
16 min read
Save for later

Using Socket.IO and Express together

Packt
23 Sep 2014
16 min read
In this article by Joshua Johanan, the author of the book Building Scalable Apps with Redis and Node.js, tells us that Express application is just the foundation. We are going to add features until it is a fully usable app. We currently can serve web pages and respond to HTTP, but now we want to add real-time communication. It's very fortunate that we just spent most of this article learning about Socket.IO; it does just that! Let's see how we are going to integrate Socket.IO with an Express application. (For more resources related to this topic, see here.) We are going to use Express and Socket.IO side by side. Socket.IO does not use HTTP like a web application. It is event based, not request based. This means that Socket.IO will not interfere with Express routes that we have set up, and that's a good thing. The bad thing is that we will not have access to all the middleware that we set up for Express in Socket.IO. There are some frameworks that combine these two, but it still has to convert the request from Express into something that Socket.IO can use. I am not trying to knock down these frameworks. They simplify a complex problem and most importantly, they do it well (Sails is a great example of this). Our app, though, is going to keep Socket.IO and Express separated as much as possible with the least number of dependencies. We know that Socket.IO does not need Express, as all our examples have not used Express in any way. This has an added benefit in that we can break off our Socket.IO module and run it as its own application at a future point in time. The other great benefit is that we learn how to do it ourselves. We need to go into the directory where our Express application is. Make sure that our pacakage.json has all the additional packages for this article and run npm.install. The first thing we need to do is add our configuration settings. Adding Socket.IO to the config We will use the same config file that we created for our Express app. Open up config.js and change the file to what I have done in the following code: var config = {port: 3000,secret: 'secret',redisPort: 6379,redisHost: 'localhost',routes: {   login: '/account/login',   logout: '/account/logout'}};module.exports = config; We are adding two new attributes, redisPort and redisHost. This is because of how the redis package configures its clients. We also are removing the redisUrl attribute. We can configure all our clients with just these two Redis config options. Next, create a directory under the root of our project named socket.io. Then, create a file called index.js. This will be where we initialize Socket.IO and wire up all our event listeners and emitters. We are just going to use one namespace for our application. If we were to add multiple namespaces, I would just add them as files underneath the socket.io directory. Open up app.js and change the following lines in it: //variable declarations at the topVar io = require('./socket.io');//after all the middleware and routesvar server = app.listen(config.port);io.startIo(server); We will define the startIo function shortly, but let's talk about our app.listen change. Previously, we had the app.listen execute, and we did not capture it in a variable; now we are. Socket.IO listens using Node's http.createServer. It does this automatically if you pass in a number into its listen function. When Express executes app.listen, it returns an instance of the HTTP server. We capture that, and now we can pass the http server to Socket.IO's listen function. Let's create that startIo function. Open up index.js present in the socket.io location and add the following lines of code to it: var io = require('socket.io');var config = require('../config');var socketConnection = function socketConnection(socket){socket.emit('message', {message: 'Hey!'});};exports.startIo = function startIo(server){io = io.listen(server);var packtchat = io.of('/packtchat');packtchat.on('connection', socketConnection);return io;}; We are exporting the startIo function that expects a server object that goes right into Socket.IO's listen function. This should start Socket.IO serving. Next, we get a reference to our namespace and listen on the connection event, sending a message event back to the client. We also are loading our configuration settings. Let's add some code to the layout and see whether our application has real-time communication. We will need the Socket.IO client library, so link to it from node_modules like you have been doing, and put it in our static directory under a newly created js directory. Open layout.ejs present in the packtchatviews location and add the following lines to it: <!-- put these right before the body end tag --><script type="text/javascript" src="/js/socket.io.js"></script><script>var socket = io.connect("http://localhost:3000/packtchat");socket.on('message', function(d){console.log(d);});</script> We just listen for a message event and log it to the console. Fire up the node and load your application, http://localhost:3000. Check to see whether you get a message in your console. You should see your message logged to the console, as seen in the following screenshot: Success! Our application now has real-time communication. We are not done though. We still have to wire up all the events for our app. Who are you? There is one glaring issue. How do we know who is making the requests? Express has middleware that parses the session to see if someone has logged in. Socket.IO does not even know about a session. Socket.IO lets anyone connect that knows the URL. We do not want anonymous connections that can listen to all our events and send events to the server. We only want authenticated users to be able to create a WebSocket. We need to get Socket.IO access to our sessions. Authorization in Socket.IO We haven't discussed it yet, but Socket.IO has middleware. Before the connection event gets fired, we can execute a function and either allow the connection or deny it. This is exactly what we need. Using the authorization handler Authorization can happen at two places, on the default namespace or on a named namespace connection. Both authorizations happen through the handshake. The function's signature is the same either way. It will pass in the socket server, which has some stuff we need such as the connection's headers, for example. For now, we will add a simple authorization function to see how it works with Socket.IO. Open up index.js, present at the packtchatsocket.io location, and add a new function that will sit next to the socketConnection function, as seen in the following code: var io = require('socket.io');var socketAuth = function socketAuth(socket, next){return next();return next(new Error('Nothing Defined'));};var socketConnection = function socketConnection(socket){socket.emit('message', {message: 'Hey!'});};exports.startIo = function startIo(server){io = io.listen(server);var packtchat = io.of('/packtchat');packtchat.use(socketAuth);packtchat.on('connection', socketConnection);return io;}; I know that there are two returns in this function. We are going to comment one out, load the site, and then switch the lines that are commented out. The socket server that is passed in will have a reference to the handshake data that we will use shortly. The next function works just like it does in Express. If we execute it without anything, the middleware chain will continue. If it is executed with an error, it will stop the chain. Let's load up our site and test both by switching which return gets executed. We can allow or deny connections as we please now, but how do we know who is trying to connect? Cookies and sessions We will do it the same way Express does. We will look at the cookies that are passed and see if there is a session. If there is a session, then we will load it up and see what is in it. At this point, we should have the same knowledge about the Socket.IO connection that Express does about a request. The first thing we need to do is get a cookie parser. We will use a very aptly named package called cookie. This should already be installed if you updated your package.json and installed all the packages. Add a reference to this at the top of index.js present in the packtchatsocket.io location with all the other variable declarations: Var cookie = require('cookie'); And now we can parse our cookies. Socket.IO passes in the cookie with the socket object in our middleware. Here is how we parse it. Add the following code in the socketAuth function: var handshakeData = socket.request;var parsedCookie = cookie.parse(handshakeData.headers.cookie); At this point, we will have an object that has our connect.sid in it. Remember that this is a signed value. We cannot use it as it is right now to get the session ID. We will need to parse this signed cookie. This is where cookie-parser comes in. We will now create a reference to it, as follows: var cookieParser = require('cookie-parser'); We can now parse the signed connect.sid cookie to get our session ID. Add the following code right after our parsing code: var sid = cookieParser.signedCookie (parsedCookie['connect.sid'], config.secret); This will take the value from our parsedCookie and using our secret passphrase, will return the unsigned value. We will do a quick check to make sure this was a valid signed cookie by comparing the unsigned value to the original. We will do this in the following way: if (parsedCookie['connect.sid'] === sid)   return next(new Error('Not Authenticated')); This check will make sure we are only using valid signed session IDs. The following screenshot will show you the values of an example Socket.IO authorization with a cookie: Getting the session We now have a session ID so we can query Redis and get the session out. The default session store object of Express is extended by connect-redis. To use connect-redis, we use the same session package as we did with Express, express-session. The following code is used to create all this in index.js, present at packtchatsocket.io: //at the top with the other variable declarationsvar expressSession = require('express-session');var ConnectRedis = require('connect-redis')(expressSession);var redisSession = new ConnectRedis({host: config.redisHost, port: config.redisPort}); The final line is creating the object that will connect to Redis and get our session. This is the same command used with Express when setting the store option for the session. We can now get the session from Redis and see what's inside of it. What follows is the entire socketAuth function along with all our variable declarations: var io = require('socket.io'),connect = require('connect'),cookie = require('cookie'),expressSession = require('express-session'),ConnectRedis = require('connect-redis')(expressSession),redis = require('redis'),config = require('../config'),redisSession = new ConnectRedis({host: config.redisHost, port: config.redisPort});var socketAuth = function socketAuth(socket, next){var handshakeData = socket.request;var parsedCookie = cookie.parse(handshakeData.headers.cookie);var sid = connect.utils.parseSignedCookie(parsedCookie['connect.sid'], config.secret);if (parsedCookie['connect.sid'] === sid) return next(new Error('Not Authenticated'));redisSession.get(sid, function(err, session){   if (session.isAuthenticated)   {     socket.user = session.user;     socket.sid = sid;     return next();   }   else     return next(new Error('Not Authenticated'));});}; We can use redisSession and sid to get the session out of Redis and check its attributes. As far as our packages are concerned, we are just another Express app getting session data. Once we have the session data, we check the isAuthenticated attribute. If it's true, we know the user is logged in. If not, we do not let them connect yet. We are adding properties to the socket object to store information from the session. Later on, after a connection is made, we can get this information. As an example, we are going to change our socketConnection function to send the user object to the client. The following should be our socketConnection function: var socketConnection = function socketConnection(socket){socket.emit('message', {message: 'Hey!'});socket.emit('message', socket.user);}; Now, let's load up our browser and go to http://localhost:3000. Log in and then check the browser's console. The following screenshot will show that the client is receiving the messages: Adding application-specific events The next thing to do is to build out all the real-time events that Socket.IO is going to listen for and respond to. We are just going to create the skeleton for each of these listeners. Open up index.js, present in packtchatsocket.io, and change the entire socketConnection function to the following code: var socketConnection = function socketConnection(socket){socket.on('GetMe', function(){});socket.on('GetUser', function(room){});socket.on('GetChat', function(data){});socket.on('AddChat', function(chat){});socket.on('GetRoom', function(){});socket.on('AddRoom', function(r){});socket.on('disconnect', function(){});}; Most of our emit events will happen in response to a listener. Using Redis as the store for Socket.IO The final thing we are going to add is to switch Socket.IO's internal store to Redis. By default, Socket.IO uses a memory store to save any data you attach to a socket. As we know now, we cannot have an application state that is stored only on one server. We need to store it in Redis. Therefore, we add it to index.js, present in packtchatsocket.io. Add the following code to the variable declarations: Var redisAdapter = require('socket.io-redis'); An application state is a flexible idea. We can store the application state locally. This is done when the state does not need to be shared. A simple example is keeping the path to a local temp file. When the data will be needed by multiple connections, then it must be put into a shared space. Anything with a user's session will need to be shared, for example. The next thing we need to do is add some code to our startIo function. The following code is what our startIo function should look like: exports.startIo = function startIo(server){io = io.listen(server);io.adapter(redisAdapter({host: config.redisHost, port: config.redisPort}));var packtchat = io.of('/packtchat');packtchat.use(socketAuth);packtchat.on('connection', socketConnection);return io;}; The first thing is to start the server listening. Next, we will call io.set, which allows us to set configuration options. We create a new redisStore and set all the Redis attributes (redisPub, redisSub, and redisClient) to a new Redis client connection. The Redis client takes a port and the hostname. Socket.IO inner workings We are not going to completely dive into everything that Socket.IO does, but we will discuss a few topics. WebSockets This is what makes Socket.IO work. All web servers serve HTTP, that is, what makes them web servers. This works great when all you want to do is serve pages. These pages are served based on requests. The browser must ask for information before receiving it. If you want to have real-time connections, though, it is difficult and requires some workaround. HTTP was not designed to have the server initiate the request. This is where WebSockets come in. WebSockets allow the server and client to create a connection and keep it open. Inside of this connection, either side can send messages back and forth. This is what Socket.IO (technically, Engine.io) leverages to create real-time communication. Socket.IO even has fallbacks if you are using a browser that does not support WebSockets. The browsers that do support WebSockets at the time of writing include the latest versions of Chrome, Firefox, Safari, Safari on iOS, Opera, and IE 11. This means the browsers that do not support WebSockets are all the older versions of IE. Socket.IO will use different techniques to simulate a WebSocket connection. This involves creating an Ajax request and keeping the connection open for a long time. If data needs to be sent, it will send it in an Ajax request. Eventually, that request will close and the client will immediately create another request. Socket.IO even has an Adobe Flash implementation if you have to support really old browsers (IE 6, for example). It is not enabled by default. WebSockets also are a little different when scaling our application. Because each WebSocket creates a persistent connection, we may need more servers to handle Socket.IO traffic then regular HTTP. For example, when someone connects and chats for an hour, there will have only been one or two HTTP requests. In contrast, a WebSocket will have to be open for the entire hour. The way our code base is written, we can easily scale up more Socket.IO servers by themselves. Ideas to take away from this article The first takeaway is that for every emit, there needs to be an on. This is true whether the sender is the server or the client. It is always best to sit down and map out each event and which direction it is going. The next idea is that of note, which entails building our app out of loosely coupled modules. Our app.js kicks everything that deals with Express off. Then, it fires the startIo function. While it does pass over an object, we could easily create one and use that. Socket.IO just wants a basic HTTP server. In fact, you can just pass the port, which is what we used in our first couple of Socket.IO applications (Ping-Pong). If we wanted to create an application layer of Socket.IO servers, we could refactor this code out and have all the Socket.IO servers run on separate servers other than Express. Summary At this point, we should feel comfortable about using real-time events in Socket.IO. We should also know how to namespace our io server and create groups of users. We also learned how to authorize socket connections to only allow logged-in users to connect. Resources for Article: Further resources on this subject: Exploring streams [article] Working with Data Access and File Formats Using Node.js [article] So, what is Node.js? [article]
Read more
  • 0
  • 0
  • 18817

article-image-eav-model
Packt
10 Aug 2015
11 min read
Save for later

EAV model

Packt
10 Aug 2015
11 min read
In this article by Allan MacGregor, author of the book Magento PHP Developer's Guide - Second Edition, we cover details about EAV models, its usefulness in retrieving data, and the advantages it provides to the merchants and developers. EAV stands for entity, attribute, and value and is probably the most difficult concept for new Magento developers to grasp. While the EAV concept is not unique to Magento, it is rarely implemented on modern systems. Additionally, a Magento implementation is not a simple one. (For more resources related to this topic, see here.) What is EAV? In order to understand what EAV is and what its role within Magento is, we need to break down parts of the EAV model: Entity: This represents the data items (objects) inside Magento products, customers, categories, and orders. Each entity is stored in the database with a unique ID. Attribute: These are our object properties. Instead of having one column per attribute on the product table, attributes are stored on separate sets of tables. Value: As the name implies, it is simply the value link to a particular attribute. This data model is the secret behind Magento's flexibility and power, allowing entities to add and remove new properties without having to make any changes to the code, templates, or the database schema. This model can be seen as a vertical way of growing our database (new attributes and more rows), while the traditional model involves a horizontal growth pattern (new attributes and more columns), which would result in a schema redesign every time new attributes are added. The EAV model not only allows for the fast evolution of our database, but is also more effective because it only works with non-empty attributes, avoiding the need to reserve additional space in the database for null values. If you are interested in exploring and learning more about the Magento database structure, I highly recommend visiting www.magereverse.com. Adding a new product attribute is as simple going to the Magento backend and specifying the new attribute type, be it color, size, brand, or anything else. The opposite is true as well and we can get rid of unused attributes on our products or customer models. For more information on managing attributes, visit http://www.magentocommerce.com/knowledge-base/entry/how-do-attributes-work-in-magento. The Magento community edition currently has eight different types of EAV objects: Customer Customer Address Products Product Categories Orders Invoices Credit Memos Shipments The Magento Enterprise Edition has one additional type called RMA item, which is part of the Return Merchandise Authorization (RMA) system. All this flexibility and power is not free; there is a price to pay. Implementing the EAV model results in having our entity data distributed on a large number of tables. For example, just the Product Model is distributed to around 40 different tables. The following diagram only shows a few of the tables involved in saving the information of Magento products: Other major downsides of EAV are the loss of performance while retrieving large collections of EAV objects and an increase in the database query complexity. As the data is more fragmented (stored in more tables), selecting a single record involves several joins. One way Magento works around this downside of EAV is by making use of indexes and flat tables. For example, Magento can save all the product information into the flat_catalog table for easier and faster access. Let's continue using Magento products as our example and manually build the query to retrieve a single product. If you have phpmyadmin or MySQL Workbench installed on your development environment, you can experiment with the following queries. Each can be downloaded on the PHPMyAdmin website at http://www.phpmyadmin.net/ and the MySQL Workbench website at http://www.mysql.com/products/workbench/. The first table that we need to use is the catalog_product_entity table. We canconsider this our main product EAV table since it contains the main entity records for our products: Let's query the table by running the following SQL query: SELECT FROM `catalog_product_entity`; The table contains the following fields: entity_id: This is our product unique identifier that is used internally by Magento. entity_type_id: Magento has several different types of EAV models. Products, customers, and orders are just some of them. Identifying each of these by type allows Magento to retrieve the attributes and values from the appropriate tables. attribute_set_id: Product attributes can be grouped locally into attribute sets. Attribute sets allow even further flexibility on the product structure as products are not forced to use all available attributes. type_id: There are several different types of products in Magento: simple, configurable, bundled, downloadable, and grouped products; each with unique settings and functionality. sku: This stands for Stock Keeping Unit and is a number or code used to identify each unique product or item for sale in a store. This is a user-defined value. has_options: This is used to identify if a product has custom options. required_options: This is used to identify if any of the custom options that are required. created_at: This is the row creation date. updated_at: This is the last time the row was modified. Now we have a basic understanding of the product entity table. Each record represents a single product in our Magento store, but we don't have much information about that product beyond the SKU and the product type. So, where are the attributes stored? And how does Magento know the difference between a product attribute and a customer attribute? For this, we need to take a look into the eav_attribute table by running the following SQL query: SELECT FROM `eav_attribute`; As a result, we will not only see the product attributes, but also the attributes corresponding to the customer model, order model, and so on. Fortunately, we already have a key to filter the attributes from this table. Let's run the following query: SELECT FROM `eav_attribute` WHERE entity_type_id = 4; This query tells the database to only retrieve the attributes where the entity_type_id column is equal to the product entity_type_id(4). Before moving, let's analyze the most important fields inside the eav_attribute table: attribute_id: This is the unique identifier for each attribute and primary key of the table. entity_type_id: This relates each attribute to a specific eav model type. attribute_code: This is the name or key of our attribute and is used to generate the getters and setters for our magic methods. backend_model: These manage loading and storing data into the database. backend_type: This specifies the type of value stored in the backend (database). backend_table: This is used to specify if the attribute should be stored on a special table instead of the default EAV table. frontend_model: These handle the rendering of the attribute element into a web browser. frontend_input: Similar to the frontend model, the frontend input specifies the type of input field the web browser should render. frontend_label: This is the label/name of the attribute as it should be rendered by the browser. source_model: These are used to populate an attribute with possible values. Magento comes with several predefined source models for countries, yes or no values, regions, and so on. Retrieving the data At this point, we have successfully retrieved a product entity and the specific attributes that apply to that entity. Now it's time to start retrieving the actual values. In order to simplify the example (and the query) a little, we will only try to retrieve the name attribute of our products. How do we know which table our attribute values are stored on? Well, thankfully, Magento follows a naming convention to name the tables. If we inspect our database structure, we will notice that there are several tables using the catalog_product_entity prefix: catalog_product_entity catalog_product_entity_datetime catalog_product_entity_decimal catalog_product_entity_int catalog_product_entity_text catalog_product_entity_varchar catalog_product_entity_gallery catalog_product_entity_media_gallery catalog_product_entity_tier_price Wait! How do we know which is the right table to query for our name attribute values? If you were paying attention, I already gave you the answer. Remember that the eav_attribute table had a column called backend_type? Magento EAV stores each attribute on a different table based on the backend type of that attribute. If we want to confirm the backend type of our name attribute, we can do so by running the following code: SELECT FROM `eav_attribute` WHERE `entity_type_id` =4 AND `attribute_code` = 'name'; As a result, we should see that the backend type is varchar and that the values for this attribute are stored in the catalog_product_entity_varchar table. Let's inspect this table: The catalog_product_entity_varchar table is formed by only 6 columns: value_id: This is the attribute value unique identifier and primary key entity_type_id: This is the entity type ID to which this value belongs attribute_id: This is the foreign key that relates the value to our eav_entity table store_id: This is the foreign key matching an attribute value with a storeview entity_id: This is the foreign key relating to the corresponding entity table, in this case, catalog_product_entity value: This is the actual value that we want to retrieve Depending on the attribute configuration, we can have it as a global value, meaning, it applies across all store views or a value per storeview. Now that we finally have all the tables that we need to retrieve the product information, we can build our query: SELECT p.entity_id AS product_id, var.value AS product_name, p.sku AS product_sku FROM catalog_product_entity p, eav_attribute eav, catalog_product_entity_varchar var WHERE p.entity_type_id = eav.entity_type_id AND var.entity_id = p.entity_id    AND eav.attribute_code = 'name'    AND eav.attribute_id = var.attribute_id From our query, we should see a result set with three columns, product_id, product_name, and product_sku. So let's step back for a second in order to get product names with SKUs with raw SQL. We had to write a five-line SQL query, and we only retrieved two values from our products, from one single EAV value table if we want to retrieve a numeric field such as price or a text-value-like product. If we didn't have an ORM in place, maintaining Magento would be almost impossible. Fortunately, we do have an ORM in place, and most likely, you will never need to deal with raw SQL to work with Magento. That said, let's see how we can retrieve the same product information by using the Magento ORM: Our first step is going to be to instantiate a product collection: $collection = Mage::getModel('catalog/product')->getCollection(); Then we will specifically tell Magento to select the name attribute: $collection->addAttributeToSelect('name'); Then, we will ask it to sort the collection by name: $collection->setOrder('name', 'asc'); Finally, we will tell Magento to load the collection: $collection->load(); The end result is a collection of all products in the store sorted by name. We can inspect the actual SQL query by running the following code: echo $collection->getSelect()->__toString(); In just three lines of code, we are telling Magento to grab all the products in the store, to specifically select the name, and finally order the products by name. The last line $collection->getSelect()->__toString(); allows to see the actual query that Magento is executing in our behalf. The actual query being generated by Magento is as follows: SELECT `e`.. IF( at_name.value_id >0, at_name.value, at_name_default.value ) AS `name` FROM `catalog_product_entity` AS `e` LEFT JOIN `catalog_product_entity_varchar` AS `at_name_default` ON (`at_name_default`.`entity_id` = `e`.`entity_id`) AND (`at_name_default`.`attribute_id` = '65') AND `at_name_default`.`store_id` =0 LEFT JOIN `catalog_product_entity_varchar` AS `at_name` ON ( `at_name`.`entity_id` = `e`.`entity_id` ) AND (`at_name`.`attribute_id` = '65') AND (`at_name`.`store_id` =1) ORDER BY `name` ASC As we can see, the ORM and the EAV models are wonderful tools that not only put a lot of power and flexibility in the hands of the developers, but they also do it in a way that is comprehensive and easy to use. Summary In this article, we learned about EAV models and how they are structured to provide Magento with data flexibility and extensibility that both merchants and developers can take advantage of. Resources for Article: Further resources on this subject: Creating a Shipping Module [article] Preparing and Configuring Your Magento Website [article] Optimizing Magento Performance — Using HHVM [article]
Read more
  • 0
  • 0
  • 18756
Modal Close icon
Modal Close icon