Search icon CANCEL
Subscription
0
Cart icon
Your Cart (0 item)
Close icon
You have no products in your basket yet
Save more on your purchases! discount-offer-chevron-icon
Savings automatically calculated. No voucher code required.
Arrow left icon
Explore Products
Best Sellers
New Releases
Books
Videos
Audiobooks
Learning Hub
Newsletter Hub
Free Learning
Arrow right icon
timer SALE ENDS IN
0 Days
:
00 Hours
:
00 Minutes
:
00 Seconds

How-To Tutorials

7010 Articles
article-image-making-better-faq-page
Packt
24 Jul 2014
17 min read
Save for later

Making a Better FAQ Page

Packt
24 Jul 2014
17 min read
(For more resources related to this topic, see here.) Marking up the FAQ page We'll get started by taking some extra care and attention with the way we mark up our FAQ list. As with most things that deal with web development, there's no right way of doing anything, so don't assume this approach is the only correct one. Any markup that makes sense semantically and makes it easy to enhance your list with CSS and JavaScript is perfectly acceptable. Time for action – setting up the HTML file Perform the following steps to get the HTML file set up for our FAQ page: We'll get started with our sample HTML file, the jQuery file, the scripts.js file, and the styles.css file. In this case, our HTML page will contain a definition list with the questions inside the <dt> tags and the answers wrapped in the <dd> tags. By default, most browsers will indent the <dd> tags, which means the questions hang into the left margin, making them easy to scan. Inside the <body> tag of your HTML document, add a heading and a definition list as shown in the following code: <h1>Frequently Asked Questions</h1> <dl> <dt>What is jQuery?</dt> <dd> <p>jQuery is an awesome JavaScript library</p> </dd> <dt>Why should I use jQuery?</dt> <dd> <p>Because it's awesome and it makes writing JavaScript faster and easier</p> </dd> <dt>Why would I want to hide the answers to my questions?</dt> <dd> <p>To make it easier to peruse the list of available questions - then you simply click to see the answer you're interested in reading.</p> </dd> <dt>What if my answers were a lot longer and more complicated than these examples?</dt> <dd> <p>The great thing about the &lt;dd&gt; element is that it's a block level element that can contain lots of other elements.</p> <p>That means your answer could contain:</p> <ul> <li>Unordered</li> <li>Lists</li> <li>with lots</li> <li>of items</li> <li>(or ordered lists or even another definition list)</li> </ul> <p>Or it might contain text with lots of <strong>special</strong> <em>formatting</em>.</p> <h2>Other things</h2> <p>It can even contain headings. Your answers could take up an entire screen or more all on their own - it doesn't matter since the answer will be hidden until the user wants to see it.</p> </dd> <dt>What if a user doesn't have JavaScript enabled?</dt> <dd> <p>You have two options for users with JavaScript disabled - which you choose might depend on the content of your page.</p> <p>You might just leave the page as it is - and make sure the &lt;dt&gt; tags are styled in a way that makes them stand out and easy to pick up when you're scanning down through the page. This would be a great solution if your answers are relatively short.</p> <p>If your FAQ page has long answers, it might be helpful to put a table of contents list of links to individual questions at the top of the page so users can click it to jump directly to the question and answer they're interested in.This is similar to what we did in the tabbed example, but in this case, we'd usejQuery to hide the table of contents when the page loaded since users with JavaScript wouldn't need to see the table of contents.</p> </dd> </dl> You can adjust the style of the page however you'd like by adding in some CSS styles. The following screenshot shows how the page is styled: For users with JavaScript disabled, this page works fine as is. The questions hang into the left margin and are bolder and larger than the rest of the text on the page, making them easy to scan. What just happened? We set up a basic definition list to hold our questions and answers. The default style of the definition list lends itself nicely to making the list of questions scannable for site visitors without JavaScript. We can enhance that further with our own custom CSS code to make the style of our list match our site. As this simple collapse-and-show (or accordion) action is such a common one, two new elements have been proposed for HTML5: <summary> and <details> that will enable us to build accordions in HTML without the need for JavaScript interactivity. However, at the time of writing this, the new elements are only supported in Webkit browsers, which require some finagling to get them styled with CSS, and are also not accessible. Do keep an eye on these new elements to see if more widespread support for them develops. You can read about the elements in the HTML5 specs (http://www.whatwg.org/specs/web-apps/current-work/multipage/interactive-elements.html). If you'd like to understand the elements better, the HTML5 Doctor has a great tutorial that explains their use and styling at http://html5doctor.com/the-details-and-summary-elements/. Time for action – moving around an HTML document Perform the following steps to move from one element to another in JavaScript: We're going to keep working with the files we set up in the previously. Open up the scripts.js file that's inside your scripts folder. Add a document ready statement, then write a new empty function called dynamicFAQ, as follows: $(document).ready(function(){ }); function dynamicFAQ() { // Our function will go here } Let's think through how we'd like this page to behave. We'd like to have all the answers to our questions hidden when the page is loaded. Then, when a user finds the question they're looking for, we'd like to show the associated answer when they click on the question. This means the first thing we'll need to do is hide all the answers when the page loads. Get started by adding a class jsOff to the <body> tag, as follows: <body class="jsOff"> Now, inside the document ready statement in scripts.js, add the line of code that removes the jsOff class and adds a class selector of jsOn: $(document).ready(function(){ $('body').removeClass('jsOff').addClass('jsOn'); }); Finally, in the styles.css file, add this bit of CSS to hide the answers for the site visitors who have JavaScript enabled: .jsOn dd { display: none; } Now if you refresh the page in the browser, you'll see that the <dd> elements and the content they contain are no longer visible (see the following screenshot): Now, we need to show the answer when the site visitor clicks on a question. To do that, we need to tell jQuery to do something whenever someone clicks on one of the questions or the <dt> tags. Inside the dynamicFAQ function, add a line of code to add a click event handler to the <dt> elements, as shown in the following code: function dynamicFAQ() { $('dt').on('click', function(){ //Show function will go here }); } When the site visitor clicks on a question, we want to get the answer to that question and show it because our FAQ list is set up as follows: <dl> <dt>Question 1</dt> <dd>Answer to Question 1</dd> <dt>Question 2</dt> <dd>Answer to Question 2</dd> ... </dl> We know that the answer is the next node or element in the DOM after our question. We'll start from the question. When a site visitor clicks on a question, we can get the current question by using jQuery's $(this) selector. The user has just clicked on a question, and we say $(this) to mean the question they just clicked on. Inside the new click function, add $(this) so that we can refer to the clicked question, as follows: $('dt').on('click', function(){ $(this); }); Now that we have the question that was just clicked, we need to get the next thing, or the answer to that question so that we can show it. This is called traversing the DOM in JavaScript. It just means that we're moving to a different element in the document. jQuery gives us the next method to move to the next node in the DOM. We'll select our answer by inserting the following code: $('dt').on('click', function(){ $(this).next(); }); Now, we've moved from the question to the answer. Now all that's left to do is show the answer. To do so, add a line of code as follows: $('dt').on('click', function(){ $(this).next().show(); }); If you refresh the page in the browser, you might be disappointed to see that nothing happens when we click the questions. Don't worry—that's easy to fix. We wrote a dynamicFAQ() function, but we didn't call it. Functions don't work until they're called. Inside the document ready statement, call the function as follows: $(document).ready(function(){ $('body').removeClass('jsOff').addClass('jsOn'); dynamicFAQ(); }); Now, if we load the page in the browser, you can see that all of our answers are hidden until we click on the question. This is nice and useful, but it would be even nicer if the site visitor could hide the answer again when they're done reading it to get it out of their way. Luckily, this is such a common task, jQuery makes this very easy for us. All we have to do is replace our call to the show method with a call to the toggle method as follows: $('dt').on('click', function(){ $(this).next().toggle(); }); Now when you refresh the page in the browser, you'll see that clicking on the question once shows the answer and clicking on the question a second time hides the answer again. What just happened? We learned how to traverse the DOM—how to get from one element to another. Toggling the display of elements on a page is a common JavaScript task, so jQuery already has built-in methods to handle it and make it simple and straightforward to get this up and running on our page. That was pretty easy—just a few lines of code. Sprucing up our FAQ page That was so easy, in fact, that we have plenty of time left over to enhance our FAQ page to make it even better. This is where the power of jQuery becomes apparent—you can not only create a show/hide FAQ page, but you can make it a fancy one and still meet your deadline. How's that for impressing a client or your boss? Time for action – making it fancy Perform the following steps to add some fancy new features to the FAQ page: Let's start with a lit le CSS code to change the cursor to a pointer and add a little hover effect to our questions to make it obvious to site visitors that the questions are clickable. Open up the styles.css file that's inside the styles folder and add the following bit of CSS code: .jsOn dt { cursor: pointer; } .jsOn dt:hover { color: #ac92ec; } We're only applying these styles for those site visitors that have JavaScript enabled. These styles definitely help to communicate to the site visitor that the questions are clickable. You might also choose to change something other than the font color for the hover effect. Feel free to style your FAQ list however you'd like. Have a look at the following screenshot: Now that we've made it clear that our <dt> elements can be interacted with, let's take a look at how to show the answers in a nicer way. When we click on a question to see the answer, the change isn't communicated to the site visitor very well; the jump in the page is a little disconcerting and it takes a moment to realize what just happened. It would be nicer and easier to understand if the questions were to slide into view. The site visitor could literally see the question appearing and would understand immediately what change just happened on the screen. jQuery makes that easy for us. We just have to replace our call to the toggle method with a call to the slideToggle method: $('dt').on('click', function(){ $(this).next().slideToggle(); }); Now if you view the page in your browser, you can see that the questions slide smoothly in and out of view when the question is clicked. It's easy to understand what's happening when the page changes, and the animation is a nice touch. Now, there's just one lit le detail we've still got to take care of. Depending on how you've styled your FAQ list, you might see a lit le jump in the answer at the end of the animation. This is caused by some extra margins around the <p> tags inside the <dd> element. They don't normally cause any issues in HTML, and browsers can figure how to display them correctly. However, when we start working with animation, sometimes this becomes a problem. It's easy to fix. Just remove the top margin from the <p> tags inside the FAQ list as follows: .content dd p { margin-top: 0; } If you refresh the page in the browser, you'll see that the little jump is now gone and our animation smoothly shows and hides the answers to our questions. What just happened? We replaced our toggle method with the slideToggle method to animate the showing and hiding of the answers. This makes it easier for the site visitor to understand the change that's taking place on the page. We also added some CSS to make the questions appear to be clickable to communicate the abilities of our page to our site visitors. We're almost there! jQuery made animating that show and hide so easy that we've still got time left over to enhance our FAQ page even more. It would be nice to add some sort of indicator to our questions to show that they're collapsed and can be expanded, and to add some sort of special style to our questions once they're opened to show that they can be collapsed again. Time for action – adding some final touches Perform the following steps to add some finishing touches to our FAQ list: Let's start with some simple CSS code to add a small arrow icon to the left side of our questions. Head back into style.css and modify the styles a bit to add an arrow as follows: .jsOn dt:before { border: 0.5em solid; border-color: transparent transparent transparent #f2eeef; content: ''; display: inline-block; height: 0; margin-right: 0.5em; vertical-align: middle; width: 0; } .jsOn dt:hover:before { border-left-color: #ac92ec; } You might be wondering about this sort of odd bit of CSS. This is a technique to create triangles in pure CSS without having to use any images. If you're not familiar with this technique, I recommend checking out appendTo's blog post that explains pure CSS triangles at http://appendto.com/2013/03/pure-css-triangles-explained/. We've also included a hover style so that the triangle will match the text color when the site visitor hovers his/her mouse over the question. Note that we're using the jsOn class so that arrows don't get added to the page unless the site visitors have JavaScript enabled. See the triangles created in the following screenshot: Next, we'll change the arrow to a different orientation when the question is opened. We'll create a new CSS class open and use it to de fine some new styles for our CSS arrow using the following code: .jsOn dt.open:before { border-color: #f2eeef transparent transparent transparent; border-bottom-width: 0; } .jsOn dt.open:hover:before { border-left-color: transparent; border-top-color: #ac92ec; } Just make sure you add these new classes after the other CSS we're using to style our <dt> tags. This will ensure that the CSS cascades the way we intended. So we have our CSS code to change the arrows and show our questions are open, but how do we actually use that new class? We'll use jQuery to add the class to our question when it is opened and to remove the class when it's closed. jQuery provides some nice methods to work with CSS classes. The addClass method will add a class to a jQuery object and the removeClass method will remove a class. However, we want to toggle our class just like we're toggling the show and hide phenomenon of our questions. jQuery's got us covered for that too. We want the class to change when we click on the question, so we'll add a line of code inside our dynamicFAQ function that we're calling each time a <dt> tag is clicked as follows: $('dt').on('click', function(){ $(this).toggleClass('open'); $(this).next().slideToggle(); }); Now when you view the page, you'll see your open styles being applied to the <dt> tags when they're open and removed again when they're closed. To see this, have a look at the following screenshot: However, we can actually crunch our code to be a little bit smaller. Remember how we chain methods in jQuery? We can take advantage of chaining again. We have a bit of redundancy in our code because we're starting two different lines with $(this). We can remove this extra $(this) and just add our toggleClass method to the chain we've already started as follows: $(this).toggleClass('open').next().slideToggle(); This helps keep our code short and concise, and just look at what we're accomplishing in one line of code! What just happened? We created the CSS styles to style the open and closed states of our questions, and then we added a bit of code to our JavaScript to change the CSS class of the question to use our new styles. jQuery provides a few different methods to update CSS classes, which is o t en a quick and easy way to update the display of our document in response to input from the site visitor. In this case, since we wanted to add and remove a class, we used the toggleClass method. It saved us from having to figure out on our own whether we needed to add or remove the open class. We also took advantage of chaining to simply add this new functionality to our existing line of code, making the animated show and hide phenomenon of the answer and the change of CSS class of our question happen all in just one line of code. How's that for impressive power in a small amount of code? Summary You learned how to set up a basic FAQ page that hides the answers to the questions until the site visitor needs to see them. Because jQuery made this so simple, we had plenty of time left over to enhance our FAQ page even more, adding animations to our show and hide phenomenon for the answers, and taking advantage of CSS to style our questions with special open and closed classes to communicate to our site visitors how our page works. And we did all of that with just a few lines of code! Resources for Article: Further resources on this subject: Calendars in jQuery 1.3 with PHP using jQuery Week Calendar Plugin: Part 1 [article] Using jQuery and jQuery Animation: Tips and Tricks [article] Using jQuery and jQueryUI Widget Factory plugins with RequireJS [article]
Read more
  • 0
  • 0
  • 9102

article-image-article-creating-an-application-using-aspnetmvcangularjsservicestack
Packt
23 Jul 2014
8 min read
Save for later

Creating an Application using ASP.NET MVC, AngularJS and ServiceStack

Packt
23 Jul 2014
8 min read
(For more resources related to this topic, see here.) Routing considerations for ASP.NET MVC and AngularJS In the previous example, we had to make changes to the ASP.NET MVC routing so it ignores the requests handled by the ServiceStack framework. Since the AngularJS application currently uses hashbang URLs, we don't need to make any other changes to the ASP.NET MVC routing. Changing an AngularJS application to use the HTML5 History API instead of hashbang URLs requires a lot more work as it will conflict directly with the ASP.NET MVC routing. You need to set up IIS URL rewriting and use the URL Rewrite module for IIS 7 and higher, which is available at www.iis.net/downloads/microsoft/url-rewrite. AngularJS application routes have to be mapped using this module to the ASP.NET MVC view that hosts the client-side application. We also need to ensure that web service request paths are excluded from URL rewriting. You can explore some changes required for the HTML5 navigation mode in the project found in the Example2 folder from the source code for this article. The HTML5 History API is not supported in Internet Explorer 8 and 9. Using ASP.NET bundling and minification features for AngularJS files So far, we have referenced and included JavaScript and CSS files directly in the _Layout.cshtml file. This makes it difficult to reuse script references between different views, and the assets are not concatenated and minified when deployed to a production environment. Microsoft provides a NuGet package called Microsoft.AspNet.Web.Optimization that contains this essential functionality. When you create a new ASP.NET MVC project, it gets installed and configured with default options. First, we need to add a new BundleConfig.cs file, which will define collections of scripts and style sheets under a virtual path, such as ~/bundles/app, that does not match a physical file. This file will contain the following code: bundles.Add(new ScriptBundle("~/bundles/app").Include( "~/scripts/app/app.js", "~/scripts/app/services/*.js", "~/scripts/app/controllers/*.js")); You can explore these changes in the project found in the Example3 folder from the source code for this article. If you take a look at the BundleConfig.cs file, you will see three script bundles and one style sheet bundle defined. Nothing is stopping you from defining only one script bundle instead, to reduce the resource requests further. We can now reference the bundles in the _Layout.cshtml file and replace the previous scripts with the following code: @Scripts.Render("~/bundles/basejs") @Scripts.Render("~/bundles/angular") @Scripts.Render("~/bundles/app") Each time we add a new file to a location like ~/scripts/app/services/ it will automatically be included in its bundle. If we add the following line of code to the BundleConfig.RegisterBundles method, when we run the application, the scripts or style sheets defined in a bundle will be minified (all of the whitespace, line separators, and comments will be removed) and concatenated in a single file: BundleTable.EnableOptimizations = true; If we take a look at the page source, the script section now looks like the following code: <script src ="/bundles/basejs?v=bWXds_q0E1qezGAjF9o48iD8-hlMNv7nlAONwLLM0Wo1"></script> <script src ="/bundles/angular?v=k-PtTeaKyBiBwT4gVnEq9YTPNruD0u7n13IOEzGTvfw1"></script> <script src ="/bundles/app?v=OKa5fFQWjXSQCNcBuWm9FJLcPFS8hGM6uq1SIdZNXWc1"></script> Using this process, the previous separate requests for each script or style sheet file will be reduced to a request to one or more bundles that are much reduced in content due to concatenation and minification. For convenience, there is a new EnableOptimizations value in web.config that will enable or disable the concatenation and minification of the asset bundles. Securing the AngularJS application We previously discussed that we need to ensure that all browser requests are secured and validated on the server for specific scenarios. Any browser request can be manipulated and changed even unintentionally, so we cannot rely on client-side validation alone. When discussing securing an AngularJS application, there are a couple of alternatives available, of which I'll mention the following: You can use client-side authentication and employ a web service call to authenticate the current user. You can create a time-limited authentication token that will be passed with each data request. This approach involves additional code in the AngularJS application to handle authentication. You can rely on server-side authentication and use an ASP.NET MVC view that will handle any unauthenticated request. This view will redirect to the view that hosts the AngularJS application only when the authentication is successful. The AngularJS application will implicitly use an authentication cookie that is set on the server side, and it does not need any additional code to handle authentication. I prefer server-side authentication as it can be reused with other server-side views and reduces the code required to implement it on both the client side and server side. We can implement server-side authentication in at least two ways, as follows: We can use the ASP.NET Identity system or the older ASP.NET Membership system for scenarios where we need to integrate with an existing application We can use built-in ServiceStack authentication features, which have a wide range of options with support for many authentication providers. This approach has the benefit that we can add a set of web service methods that can be used for authentication outside of the ASP.NET MVC context. The last approach ensures the best integration between ASP.NET MVC and ServiceStack, and it allows us to introduce a ServiceStack NuGet package that provides new productivity benefits for our sample application. Using the ServiceStack.Mvc library ServiceStack has a library that allows deeper integration with ASP.NET MVC through the ServiceStack.Mvc NuGet package. This library provides access to the ServiceStack dependency injection system for ASP.NET MVC applications. It also introduces a new base controller class called ServiceStackController; this can be used by ASP.NET MVC controllers to gain access to the ServiceStack caching, session, and authentication infrastructures. To install this package, you need to run the following command in the NuGet Package Manager Console: Install-Package ServiceStack.Mvc -Version 3.9.71 The following line needs to be added to the AppHost.Configure method, and it will register a ServiceStack controller factory class for ASP.NET MVC: ControllerBuilder.Current.SetControllerFactory(new FunqControllerFactory(container)); The ControllerBuilder.Current.SetControllerFactory method is an ASP.NET MVC extension point that allows the replacement of its DefaultControllerFactory class with a custom one. This class is tasked with matching requests with controllers, among other responsibilities. The FunqControllerFactory class provided in the new NuGet package inherits the DefaultControllerFactory class and ensures that all controllers that have dependencies managed by the ServiceStack dependency injection system will be resolved at application runtime. To exemplify this, the BicycleRepository class is now referenced in the HomeController class, as shown in the following code: public class HomeController : Controller { public BicycleRepository BicycleRepository { get; set; } // // GET: /Home/ public ActionResult Index() { ViewBag.BicyclesCount = BicycleRepository.GetAll().Count(); return View(); } } The application menu now displays the current number of bicycles as initialized in the BicycleRepository class. If we add a new bicycle and refresh the browser page, the menu bicycle count is updated. This highlights the fact that the ASP.NET MVC application uses the same BicycleRepository instance as ServiceStack web services. You can explore this example in the project found in the Example4 folder from the source code for this article. Using the ServiceStack.Mvc library, we have reached a new milestone by bridging ASP.NET MVC controllers with ServiceStack services. In the next section, we will effectively transition to a single server-side application with unified caching, session, and authentication infrastructures. The building blocks of the ServiceStack security infrastructure ServiceStack has built-in, optional authentication and authorization provided by its AuthFeature plugin, which builds on two other important components as follows: Caching: Every service or controller powered by ServiceStack has optional access to an ICacheClient interface instance that provides cache-related methods. The interface needs to be registered as an instance of one of the many caching providers available: an in-memory cache, a relational database cache, a cache based on a key value data store using Redis, a memcached-based cache, a Microsoft Azure cache, and even a cache based on Amazon DynamoDB. Sessions: These are enabled by the SessionFeature ServiceStack plugin and rely on the caching component when the AuthFeature plugin is not enabled. Every service or controller powered by ServiceStack has an ISession property that provides read and write access to the session data. Each ServiceStack request automatically has two cookies set: an ss-id cookie, which is a regular session cookie, and an ss-pid cookie, which is a permanent cookie with an expiry date set far in the future. You can also gain access to a typed session as part of the AuthFeature plugin that will be explored next.
Read more
  • 0
  • 0
  • 4135

article-image-importance-securing-web-services
Packt
23 Jul 2014
10 min read
Save for later

The Importance of Securing Web Services

Packt
23 Jul 2014
10 min read
(For more resources related to this topic, see here.) In the upcoming sections of this article we are going to briefly explain several concepts about the importance of securing web services. The importance of security The management of securities is one of the main aspects to consider when designing applications. No matter what, neither the functionality nor the information of organizations can be exposed to all users without any kind of restriction. Suppose the case of a human resource management application that allows you to consult wages of their employees, for example, if the company manager needs to know the salary of one of their employees, it is not something of great importance. But in the same context, imagine that one of the employees wants to know the salary of their colleagues, if access to this information is completely open, it could generate problems among employees with varied salaries. Security management options Java provides some options for security management. Right now we will explain some of them and demonstrate how to implement them. All authentication methods are practically based on credentials delivery from the client to the server. In order to perform this, there are several methods: BASIC authentication DIGEST authentication CLIENT CERT authentication Using API keys The Security Management in applications built with Java including those ones with RESTful web services, always rely on JAAS. Basic authentication by providing user credentials Possibly one of the most used techniques in all kind of applications. The user, before gaining functionality over the application is requested to enter a username and password both are validated in order to verify if credentials are correct (belongs to an application user). We are 99 percent sure you have performed this technique at least once, maybe through a customized mechanism, or if you used JEE platform, probably through JAAS. This kind of control is known as basic authentication. In order to have a working example, let’s start our application server JBoss AS 7, then go to bin directory and execute the file add-user.bat (.sh file for UNIX users). Finally, we will create a new user as follows: As a result, we will have a new user in JBOSS_HOME/standalone/configuration/application - users.properties file. JBoss is already set with a default security domain called other; the same one uses the information stored in the file we mentioned earlier in order to authenticate. Right now we are going to configure the application to use this security domain, inside the folder WEB-INF from resteasy-examples project, let's create a file named jboss-web.xml with the following content: <?xml version="1.0" encoding="UTF-8"?> <jboss-web> <security-domain>other</security-domain> </jboss-web> Alright, let's configure the file web.xml in order to aggregate the securities constraints. In the following block of code, you will see on bold what you should add: <?xml version="1.0" encoding="UTF-8"?> <web-app version="3.0" xsi_schemaLocation="http://java.sun.com/xml/ns/javaee http://java.sun.com/xml/ns/javaee/web-app_3_0.xsd"> <!-- Roles --> <security-role> <description>Any rol </description> <role-name>*</role-name> </security-role> <!-- Resource / Role Mapping --> <security-constraint> <display-name>Area secured</display-name> <web-resource-collection> <web-resource-name>protected_resources</web-resource-name> <url-pattern>/services/*</url-pattern> <http-method>GET</http-method> <http-method>POST</http-method> </web-resource-collection> <auth-constraint> <description>User with any role</description> <role-name>*</role-name> </auth-constraint> </security-constraint> <login-config> <auth-method>BASIC</auth-method> </login-config> </web-app> From a terminal let's go to the home folder of the resteasy-examples project and execute mvn jboss-as:redeploy. Now we are going to test our web service as we did earlier by using SoapUI. We are going to perform request using the POST method to the URL SOAP UI shows us the HTTP 401 error; this means that the request wasn't authorized. This is because we performed the request without delivering the credentials to the server. Digest access authentication This authentication method makes use of a hash function to encrypt the password entered by the user before sending it to the server. This makes it obviously much safer than the BASIC authentication method, in which the user’s password travels in plain text that can be easily read by whoever intercepts. To overcome such drawbacks, digest MD5 authentication applies a function on the combination of the values of the username, realm of application security, and password. As a result we obtain an encrypted string that can hardly be interpreted by an intruder. Now, in order to perform what we explained before, we need to generate a password for our example user. And we have to generate it using the parameters we talked about earlier; username, realm, and password. Let’s go into the directory of JBOSS_HOME/modules/org/picketbox/main/ from a terminal and type the following: java -cp picketbox-4.0.7.Final.jar org.jboss. security.auth.callback.RFC2617Digest username MyRealmName password We will obtain the following result: RFC2617 A1 hash: 8355c2bc1aab3025c8522bd53639c168 Through this process we obtain the encrypted password, and use it in our password storage file (the JBOSS_HOME/standalone/configuration/application-users.properties). We must replace the password in the file and it will be used for the user username. We have to replace it because the old password doesn't contain the realm name information of the application. Next, We have to modify the web.xml file in the tag auth-method and change the value FORM to DIGEST, and we should set the application realm name this way: <login-config> <auth-method>DIGEST</auth-method> <realm-name>MyRealmName</realm-name> </login-config> Now, let's create a new security domain in JBoss, so we can manage the authentication mechanism DIGEST. In the file JBOSS_HOME/standalone/configuration/standalone.xml, on the section <security-domains>, let's add the following entry: <security-domain name="domainDigest" cache-type ="default"> <authentication> <login-module code="UsersRoles" flag="required"> <module-option name="usersProperties" value="${jboss.server.config.dir} /application-users.properties"/> <module-option name="rolesProperties" value="${jboss.server.config.dir}/ application-roles.properties"/> <module-option name="hashAlgorithm" value="MD5"/> <module-option name= "hashEncoding" value="RFC2617"/> <module-option name="hashUserPassword" value="false"/> <module-option name="hashStorePassword" value="true"/> <module-option name="passwordIsA1Hash" value="true"/> <module-option name="storeDigestCallback" value=" org.jboss.security.auth.callback.RFC2617Digest"/> </login-module> </authentication> </security-domain> Finally, in the application, change the security domain name in the file jboss-web.xml as shown in the following snippet: <?xml version="1.0" encoding="UTF-8"?> <jboss-web> <security-domain>java:/jaas/domainDigest</security-domain> </jboss-web> We are going to change the authentication method from BASIC to DIGEST in the web.xml file. Also we enter the name of the security realm; all these changes must be applied in the tag login-config, this way: <login-config> <auth-method>DIGEST</auth-method> <realm-name>MyRealmName</realm-name </login-config> Now, restart the application server and then redeploy the application on JBoss. To do this, execute the next command on the terminal: mvn jboss-as:redeploy Authentication through certificates It is a mechanism in which a trust agreement is established between the server and the client through certificates. They must be signed by an agency established to ensure that the certificate presented for authentication is legitimate, it is known as CA. This security mechanism needs that our application server uses HTTPS as communication protocol. So we must enable HTTPS. Let's add a connector in the standalone.xml file; look for the following line: <connector name="http" Add the following block of code: <connector name="https" protocol="HTTP/1.1" scheme="https" socket-binding="https" secure="true"> <ssl password="changeit" certificate-key-file="${jboss.server.config.dir}/server.keystore" verify-client="want" ca-certificate-file="${jboss.server.config.dir}/server.truststore"/> </connector> Next we add the security domain: <security-domain name="RequireCertificateDomain"> <authentication> <login-module code="CertificateRoles" flag="required"> <module-option name="securityDomain" value="RequireCertificateDomain"/> <module-option name="verifier" value=" org.jboss.security.auth.certs.AnyCertVerifier"/> <module-option name="usersProperties" value= "${jboss.server.config.dir}/my-users.properties"/> <module-option name="rolesProperties" value= "${jboss.server.config.dir}/my-roles.properties"/> </login-module> </authentication> <jsse keystore-password="changeit" keystore-url= "file:${jboss.server.config.dir}/server.keystore" truststore-password="changeit" truststore-url ="file:${jboss.server.config.dir}/server.truststore"/> </security-domain> As you can see, we need two files: my-users.properties and my-roles.properties, both are empty and located in the JBOSS_HOME/standalone/configuration path. We are going to add the <user-data-constraint> tag in the web.xml in this way: <security-constraint> ...<user-data-constraint> <transport-guarantee>CONFIDENTIAL</transport-guarantee> </user-data-constraint> </security-constraint> Then, change the authentication method to CLIENT-CERT: <login-config> <auth-method>CLIENT-CERT</auth-method> </login-config> And finally change the security domain in the jboss-web.xml file in the following way: <?xml version="1.0" encoding="UTF-8"?> <jboss-web> <security-domain>RequireCertificateDomain</security-domain> </jboss-web> Now, restart the application server, and redeploy the application with Maven: mvn jboss-as:redeploy API keys With the advent of cloud computing, it is not difficult to think of applications that integrate with many others available in the cloud. Right now, it's easy to see how applications interact with Flickr, Facebook, Twitter, Tumblr, and so on through APIKeys usage. This authentication method is used primarily when we need to authenticate from another application but we do not want to access the private user data hosted in another application, on the contrary, if you want to access this information, you must use OAuth. Today it is very easy to get an API key. Simply log into one of the many cloud providers and obtain credentials, consisting of a KEY and a SECRET, the same that are needed to interact with the authenticating service providers. Keep in mind that when creating an API Key, accept the terms of the supplier, which clearly states what we can and cannot do, protecting against abusive users trying to affect their services. The following chart shows how this authentication mechanism works: Summary In this article, we went through some models of authentication. We can apply them to any web service functionality we created. As you realize, it is important to choose the correct security management, otherwise information is exposed and can easily be intercepted and used by third parties. Therefore, tread carefully. Resources for Article: Further resources on this subject: RESTful Java Web Services Design [Article] Debugging REST Web Services [Article] RESTful Services JAX-RS 2.0 [Article]
Read more
  • 0
  • 0
  • 3403

article-image-indexes
Packt
23 Jul 2014
8 min read
Save for later

Indexes

Packt
23 Jul 2014
8 min read
(For more resources related to this topic, see here.) As a database administrator (DBA) or developer, one of your most important goals is to ensure that the query times are consistent with the service-level agreement (SLA) and are meeting user expectations. Along with other performance enhancement techniques, creating indexes for your queries on underlying tables is one of the most effective and common ways to achieve this objective. The indexes of underlying relational tables are very similar in purpose to an index section at the back of a book. For example, instead of flipping through each page of the book, you use the index section at the back of the book to quickly find the particular information or topic within the book. In the same way, instead of scanning each individual row on the data page, SQL Server uses indexes to quickly find the data for the qualifying query. Therefore, by indexing an underlying relational table, you can significantly enhance the performance of your database. Indexing affects the processing speed for both OLTP and OLAP and helps you achieve optimum query performance and response time. The cost associated with indexes SQL Server uses indexes to optimize overall query performance. However, there is also a cost associated with indexes; that is, indexes slow down insert, update, and delete operations. Therefore, it is important to consider the cost and benefits associated with indexes when you plan your indexing strategy. How SQL Server uses indexes A table that doesn't have a clustered index is stored in a set of data pages called a heap. Initially, the data in the heaps is stored in the order in which the rows are inserted into the table. However, SQL Server Database Engine moves the data around the heap to store the rows efficiently. Therefore, you cannot predict the order of the rows for heaps because data pages are not sequenced in any particular order. The only way to guarantee the order of the rows from a heap is to use the SELECT statement with the ORDER BY clause. Access without an index When you access the data, SQL Server first determines whether there is a suitable index available for the submitted SELECT statement. If no suitable index is found for the submitted SELECT statement, SQL Server retrieves the data by scanning the entire table. The database engine begins scanning at the physical beginning of the table and scans through the full table page by page and row by row to look for qualifying data that is specified in the submitted SELECT statement. Then, it extracts and returns the rows that meet the criteria in the format specified in the submitted SELECT statement. Access with an index The process is improved when indexes are present. If the appropriate index is available, SQL Server uses it to locate the data. An index improves the search process by sorting data on key columns. The database engine begins scanning from the first page of the index and only scans those pages that potentially contain qualifying data based on the index structure and key columns. Finally, it retrieves the data rows or pointers that contain the locations of the data rows to allow direct row retrieval. The structure of indexes In SQL Server, all indexes—except full-text, XML, in-memory optimized, and columnstore indexes—are organized as a balanced tree (B-tree). This is because full-text indexes use their own engine to manage and query full-text catalogs, XML indexes are stored as internal SQL Server tables, in-memory optimized indexes use the Bw-tree structure, and columnstore indexes utilize SQL Server in-memory technology. In the B-tree structure, each page is called a node. The top page of the B-tree structure is called the root node. Non-leaf nodes, also referred to as intermediate levels, are hierarchical tree nodes that comprise the index sort order. Non-leaf nodes point to other non-leaf nodes that are one step below in the B-tree hierarchy, until reaching the leaf nodes. Leaf nodes are at the bottom of the B-tree hierarchy. The following diagram illustrates the typical B-tree structure: Index types In SQL Server 2014, you can create several types of indexes. They are explored in the next sections. Clustered indexes A clustered index sorts table or view rows in the order based on clustered index key column values. In short, a leaf node of a clustered index contains data pages, and scanning them will return the actual data rows. Therefore, a table can have only one clustered index. Unless explicitly specified as nonclustered, SQL Server automatically creates the clustered index when you define a PRIMARY KEY constraint on a table. When should you have a clustered index on a table? Although it is not mandatory to have a clustered index per table, according to the TechNet article, Clustered Index Design Guidelines, with few exceptions, every table should have a clustered index defined on the column or columns that used as follows: The table is large and does not have a nonclustered index. The presence of a clustered index improves performance because without it, all rows of the table will have to be read if any row needs to be found. A column or columns are frequently queried, and data is returned in a sorted order. The presence of a clustered index on the sorting column or columns prevents the sorting operation from being started and returns the data in the sorted order. A column or columns are frequently queried, and data is grouped together. As data must be sorted before it is grouped, the presence of a clustered index on the sorting column or columns prevents the sorting operation from being started. A column or columns data that are frequently used in queries to search data ranges from the table. The presence of clustered indexes on the range column will help avoid the sorting of the entire table data. Nonclustered indexes Nonclustered indexes do not sort or store the data of the underlying table. This is because the leaf nodes of the nonclustered indexes are index pages that contain pointers to data rows. SQL Server automatically creates nonclustered indexes when you define a UNIQUE KEY constraint on a table. A table can have up to 999 nonclustered indexes. You can use the CREATE INDEX statement to create clustered and nonclustered indexes. A detailed discussion on the CREATE INDEX statement and its parameters is beyond the scope of this article. For help with this, refer to the CREATE INDEX (Transact-SQL) article at http://msdn.microsoft.com/en-us/library/ms188783.aspx. SQL Server 2014 also supports new inline index creation syntax for standard, disk-based database tables, temp tables, and table variables. For more information, refer to the CREATE TABLE (SQL Server) article at http://msdn.microsoft.com/en-us/library/ms174979.aspx. Single-column indexes As the name implies, single-column indexes are based on a single-key column. You can define it as either clustered or nonclustered. You cannot drop the index key column or change the data type of the underlying table column without dropping the index first. Single-column indexes are useful for queries that search data based on a single column value. Composite indexes Composite indexes include two or more columns from the same table. You can define composite indexes as either clustered or nonclustered. You can use composite indexes when you have two or more columns that need to be searched together. You typically place the most unique key (the key with the highest degree of selectivity) first in the key list. For example, examine the following query that returns a list of account numbers and names from the Purchasing.Vendor table, where the name and account number starts with the character A: USE [AdventureWorks2012]; SELECT [AccountNumber] , [Name] FROM [Purchasing].[Vendor] WHERE [AccountNumber] LIKE 'A%' AND [Name] LIKE 'A%'; GO If you look at the execution plan of this query without modifying the existing indexes of the table, you will notice that the SQL Server query optimizer uses the table's clustered index to retrieve the query result, as shown in the following screenshot: As our search is based on the Name and AccountNumber columns, the presence of the following composite index will improve the query execution time significantly: USE [AdventureWorks2012]; GO CREATE NONCLUSTERED INDEX [AK_Vendor _ AccountNumber_Name] ON [Purchasing].[Vendor] ([AccountNumber] ASC, [Name] ASC) ON [PRIMARY]; GO Now, examine the query execution plan of this query once again, after creating the previous composite index on the Purchasing.Vendor table, as shown in the following screenshot: As you can see, SQL Server performs a seek operation on this composite index to retrieve the qualifying data. Summary Thus we have learned what indexes are, how SQL Server uses indexes, structure of indexes, and some of the types of indexes. Resources for Article: Further resources on this subject: Easily Writing SQL Queries with Spring Python [article] Manage SQL Azure Databases with the Web Interface 'Houston' [article] VB.NET Application with SQL Anywhere 10 database: Part 1 [article]
Read more
  • 0
  • 0
  • 7666

article-image-introspecting-maya-python-and-pymel
Packt
23 Jul 2014
8 min read
Save for later

Introspecting Maya, Python, and PyMEL

Packt
23 Jul 2014
8 min read
(For more resources related to this topic, see here.) Maya and Python are both excellent and elegant tools that can together achieve amazing results. And while it may be tempting to dive in and start wielding this power, it is prudent to understand some basic things first. In this article, we will look at Python as a language, Maya as a program, and PyMEL as a framework. We will begin by briefly going over how to use the standard Python interpreter, the Maya Python interpreter, the Script Editor in Maya, and your Integrated Development Environment (IDE) or text editor in which you will do the majority of your development. Our goal for the article is to build a small library that can easily link us to documentation about Python and PyMEL objects. Building this library will illuminate how Maya, Python and PyMEL are designed, and demonstrate why PyMEL is superior to maya.cmds. We will use the powerful technique of type introspection to teach us more about Maya's node-based design than any Hypergraph or static documentation can. Creating your library There are generally three different modes you will be developing in while programming Python in Maya: using the mayapy interpreter to evaluate short bits of code and explore ideas, using your Integrated Development Environment to work on the bulk of the code, and using Maya's Script Editor to help iterate and test your work. In this section, we'll start learning how to use all three tools to create a very simple library. Using the interpreter The first thing we must do is find your mayapy interpreter. It should be next to your Maya executable, named mayapy or mayapy.exe. It is a Python interpreter that can run Python code as if it were being run in a normal Maya session. When you launch it, it will start up the interpreter in interactive mode, which means you enter commands and it gives you results, interactively. The >>> and ... characters in code blocks indicate something you should enter at the interactive prompt; the code listing in the article and your prompt should look basically the same. In later listings, long output lines will be elided with ... to save on space. Start a mayapy process by double clicking or calling it from the command line, and enter the following code: >>> print 'Hello, Maya!' Hello, Maya! >>> def hello(): ... return 'Hello, Maya!' ... >>> hello() 'Hello, Maya!' The first statement prints a string, which shows up under the prompting line. The second statement is a multiline function definition. The ... indicates the line is part of the preceding line. The blank line following the ... indicates the end of the function. For brevity, we will leave out empty ... lines in other code listings. After we define our hello function, we invoke it. It returns the string "Hello, Maya!", which is printed out beneath the invocation. Finding a place for our library Now, we need to find a place to put our library file. In order for Python to load the file as a module, it needs to be on some path where Python can find it. We can see all available paths by looking at the path list on the sys module. >>> import sys >>> for p in sys.path: ... print p C:Program FilesAutodeskMaya2013binpython26.zip C:Program FilesAutodeskMaya2013PythonDLLs C:Program FilesAutodeskMaya2013Pythonlib C:Program FilesAutodeskMaya2013Pythonlibplat-win C:Program FilesAutodeskMaya2013Pythonliblib-tk C:Program FilesAutodeskMaya2013bin C:Program FilesAutodeskMaya2013Python C:Program FilesAutodeskMaya2013Pythonlibsite-packages A number of paths will print out; I've replicated what's on my Windows system, but yours will almost definitely be different. Unfortunately, the default paths don't give us a place to put custom code. They are application installation directories, which we should not modify. Instead, we should be doing our coding outside of all the application installation directories. In fact, it's a good practice to avoid editing anything in the application installation directories entirely. Choosing a development root Let's decide where we will do our coding. To be concise, I'll choose C:mayapybookpylib to house all of our Python code, but it can be anywhere. You'll need to choose something appropriate if you are on OSX or Linux; we will use ~/mayapybook/pylib as our path on these systems, but I'll refer only to the Windows path except where more clarity is needed. Create the development root folder, and inside of it create an empty file named minspect.py. Now, we need to get C:mayapybookpylib onto Python's sys.path so it can be imported. The easiest way to do this is to use the PYTHONPATH environment variable. From a Windows command line you can run the following to add the path, and ensure it worked: > set PYTHONPATH=%PYTHONPATH%;C:mayapybookpylib > mayapy.exe >>> import sys >>> 'C:\mayapybook\pylib' in sys.path True >>> import minspect >>> minspect <module 'minspect' from '...minspect.py'> The following is the equivalent commands on OSX or Linux: $ export PYTHONPATH=$PYTHONPATH:~/mayapybook/pylib $ mayapy >>> import sys >>> '~/mayapybook/pylib' in sys.path True >>> import minspect >>> minspect <module 'minspect' from '.../minspect.py'> There are actually a number of ways to get your development root onto Maya's path. The option presented here (using environment variables before starting Maya or mayapy) is just one of the more straightforward choices, and it works for mayapy as well as normal Maya. Calling sys.path.append('C:\mayapybook\pylib') inside your userSetup.py file, for example, would work for Maya but not mayapy (you would need to use maya.standalone.initialize to register user paths, as we will do later). Using set or export to set environment variables only works for the current process and any new children. If you want it to work for unrelated processes, you may need to modify your global or user environment. Each OS is different, so you should refer to your operating system's documentation or a Google search. Some possibilities are setx from the Windows command line, editing /etc/environment in Linux, or editing /etc/launchd.conf on OS X. If you are in a studio environment and don't want to make changes to people's machines, you should consider an alternative such as using a script to launch Maya which will set up the PYTHONPATH, instead of launching the maya executable directly. Creating a function in your IDE Now it is time to use our IDE to do some programming. We'll start by turning the path printing code we wrote at the interactive prompt into a function in our file. Open C:mayapybookpylibminspect.py in your IDE and type the following code: import sys def syspath(): print 'sys.path:' for p in sys.path: print ' ' + p Save the file, and bring up your mayapy interpreter. If you've closed down the one from the last session, make sure C:mayapybookpylib (or whatever you are using as your development root) is present on your sys.path or the following code will not work! See the preceding section for making sure your development root is on your sys.path. >>> import minspect >>> reload(minspect) <module 'minspect' from '...minspect.py'> >>> minspect.syspath() C:Program FilesAutodeskMaya2013binpython26.zip C:Program FilesAutodeskMaya2013PythonDLLs C:Program FilesAutodeskMaya2013Pythonlib C:Program FilesAutodeskMaya2013Pythonlibplat-win C:Program FilesAutodeskMaya2013Pythonliblib-tk C:Program FilesAutodeskMaya2013bin C:Program FilesAutodeskMaya2013Python C:Program FilesAutodeskMaya2013Pythonlibsite-packages First, we import the minspect module. It may already be imported if this was an old mayapy session. That is fine, as importing an already-imported module is fast in Python and causes no side effects. We then use the reload function, which we will explore in the next section, to make sure the most up-to-date code is loaded. Finally, we call the syspath function, and its output is printed. Your actual paths will likely vary. Reloading code changes It is very common as you develop that you'll make changes to some code and want to immediately try out the changed code without restarting Maya or mayapy. You can do that with Python's built-in reload function. The reload function takes a module object and reloads it from disk so that the new code will be used. When we jump between our IDE and the interactive interpreter (or the Maya application) as we did earlier, we will usually reload the code to see the effect of our changes. I will usually write out the import and reload lines, but occasionally will only mention them in text preceding the code. Keep in mind that reload is not a magic bullet. When you are dealing with simple data and functions as we are here, it is usually fine. But as you start building class hierarchies, decorators, and other things that have dependencies or state, the situation can quickly get out of control. Always test your code in a fresh version of Maya before declaring it done to be sure it does not have some lingering defect hidden by reloading. Though once you are a master Pythonista you can ignore these warnings and figure out how to reload just about anything!
Read more
  • 0
  • 0
  • 3716

article-image-linking-dynamic-content-external-websites
Packt
22 Jul 2014
5 min read
Save for later

Linking Dynamic Content from External Websites

Packt
22 Jul 2014
5 min read
(For more resources related to this topic, see here.) Introduction to the YouTube API YouTube provides three different APIs for a client application to access. The following figure shows the three different APIs provided by YouTube: Configuring a YouTube API In the Google Developers Console, we need to create a client project. We will be creating a new project, called PacktYoutubeapi. The URL for the Google Developers Console is https://console.developers.google.com. The following screenshot shows the pop-up window that appears when you want to create a new client project in the Developers Console: After the successful creation of the new client project, it will be available in the Console's project list. The following screenshot shows our new client project listed in the Developers Console: There is an option available to enable access to the YouTube API for our application. The following screenshot shows the YouTube API listed in the Developers Console. By default, the status of this API is OFF for the application. To enable this API for our application, we need to toggle the STATUS button to ON. The following screenshot shows the status of the YouTube API, which is ON for our application: To access YouTube API methods, we need to create an API key for our client application. You can find the option to create a public API key in the APIs & auth section. The following screenshot shows the Credentials subsection where you can create an API key: In the preceding screenshot, you can see a button to create a new API key. After clicking on this button, it provides some choices to create an API key, and after the successful creation of an API key, the key will be listed in the Credentials section. The following screenshot shows the API key generated for our application: Searching for a YouTube video In this section, we will learn about integrating a YouTube-related search video. YouTube Data API Version 3.0 is the new API to access YouTube data. It requires the API key that has been created in the previous section. The main steps that we have to follow to do a YouTube search are: After adding the YouTube Search button, click on it to trigger the search process. The script reads the data-booktitle attribute to get the title. This will serve as a keyword for the search. Check the following screenshot for the HTML markup showing the data-booktitle attribute: Then, it creates an AJAX request to make an asynchronous call to the YouTube API, and returns a promise object. After the successful completion of the AJAX call, the promise object is resolved successfully. Once the data is available, we fetch the jQuery template for the search results and compile it with a script function. We then link it to the search data returned by the AJAX call and generate the HTML markup for rendering. The base URL for the YouTube search is through a secure HTTP protocol, https://www.googleapis.com/youtube/v3/search. It takes different parameters as input for the search and filter criteria. Some of the important parameters are field and part. The part parameter The part parameter is about accessing a resource from a YouTube API. It really helps the application to choose resource components that your application actually uses. The following figure shows some of the resource components: The fields parameter The fields parameter is used to filter out the exact fields that are needed by the client application. This is really helpful to reduce the size of the response. For example, fields = items(id, snippet(title)) will result in a small footprint of a response containing an ID and a title. The YouTube button markup We have added a button in our jQuery product template to display the search option in the product. The following code shows the updated template: <script id="aProductTemplate" type="text/x-jquery-tmpl"> <div class="ts-product panel panel-default"> <div class="panel-head"> <div class="fb-like" data-href="${url}" datalayout=" button_count" data-action="like" data-show-faces="true" datashare=" true"> </div> </div> <div class="panel-body"> <span class="glyphicon glyphicon-certificate ts-costicon"> <label>${cost}$</label> </span> <img class="img-responsive" src ="${url}"> <h5>${title}</h5> </div> <div class="panel-footer"> <button type="button" class="btn btn-danger btn-block packt-youtube-button" data-bookTitle="${title}">YouTube Search</ button> <button type="button" class="btn btn-info btn-block">Buy</ button> <button type="button" class="btn btn-info btn-block twitme" data-bookTitle="${title}" data-imgURI="${url}">Tweet</button> <div class="g-plus-button"> <div class="g-plusone" data-width="180" datahref="${ url}"></div> </div> </div> </div> </script> The following screenshot shows the updated product markup with a YouTube button added to the product template:
Read more
  • 0
  • 0
  • 11323
Unlock access to the largest independent learning library in Tech for FREE!
Get unlimited access to 7500+ expert-authored eBooks and video courses covering every tech area you can think of.
Renews at €18.99/month. Cancel anytime
article-image-wireless-and-mobile-hacks
Packt
22 Jul 2014
6 min read
Save for later

Wireless and Mobile Hacks

Packt
22 Jul 2014
6 min read
(For more resources related to this topic, see here.) So I don't think it's possible to go to a conference these days and not see a talk on mobile or wireless. (They tend to schedule the streams to have both mobile and wireless talks at the same time—the sneaky devils. There is no escaping the wireless knowledge!) So, it makes sense that we work out some ways of training people how to skill up on these technologies. We're going to touch on some older vulnerabilities that you don't see very often, but as always, when you do, it's good to know how to insta-win. Wireless environment setup This article is a bit of an odd one, because with Wi-Fi and mobile, it's much harder to create a safe environment for your testers to work in. For infrastructure and web app tests, you can simply say, "it's on the network, yo" and they'll get the picture. However, Wi-Fi and mobile devices are almost everywhere in places that require pen testing. It's far too easy for someone to get confused and attempt to pwn a random bystander. While this sounds hilarious, it is a serious issue if that occurs. So, adhere to the following guidelines for safer testing: Where possible, try and test away from other people and networks. If there is an underground location nearby, testing becomes simpler as floors are more effective than walls for blocking Wi-Fi signals (contrary to the firmly held beliefs of anyone who's tried to improve their home network signal). If you're an individual who works for a company, or you know, has the money to make a Faraday cage, then by all means do the setup in there. I'll just sit here and be jealous. Unless it's pertinent to the test scenario, provide testers with enough knowledge to identify the devices and networks they should be attacking. A good way to go is to provide the Mac address as they very rarely collide. (Mac randomizing tools be damned.) If an evil network has to be created, name it something obvious and reduce the access to ensure that it is visible to as few people as possible. The naming convention we use is Connectingtomewillresultin followed by pain, death, and suffering. While this steers away the majority of people, it does appear to attract the occasional fool, but that's natural selection for you. Once again, but it is worth repeating, don't use your home network. Especially in this case, using your home equipment could expose you to random passersby or evil neighbors. I'm pretty sure my neighbor doesn't know how to hack, but if he does, I'm in for a world of hurt. Software We'll be using Kali Linux as the base for this article as we'll be using the tools provided by Kali to set up our networks for attack. Everything you need is built into Kali, but if you happen to be using another build such as Ubuntu or Debian, you will need the following tools: Iwtools (apt-get install iw): This is the wireless equivalent of ifconfig that allows the alteration of wireless adapters, and provides a handy method to monitor them. Aircrack suite (apt-get install aircrack-ng): The basic tools of wireless attacking are available in the Aircrack suite. This selection of tools provides a wide range of services, including cracking encryption keys, monitoring probe requests, and hosting rogue networks. Hostapd (apt-get install hostapd): Airbase-ng doesn't support WPA-2 networks, so we need to bring in the serious programs for serious people. This can also be used to host WEP networks, but getting Aircrack suite practice is not to be sniffed at. Wireshark (apt-get install wireshark): Wireshark is one of the most widely used network analytics tools. It's not only used by pen testers, but also by people who have CISSP and other important letters after their names. This means that it's a tool that you should know about. dnschef (https://thesprawl.org/projects/dnschef/): Thanks to Duncan Winfrey, who pointed me in this direction. DNSchef is a fantastic resource for doing DNS spoofing. Other alternatives include DNS spoof and Metasploit's Fake DNS. Crunch (apt-get install crunch): Crunch generates strings in a specified order. While it seems very simple, it's incredibly useful. Use with care though; it has filled more than one unwitting user's hard drive. Hardware You want to host a dodgy network. The first question to ask yourself, after the question you already asked yourself about software, is: is your laptop/PC capable of hosting a network? If your adapter is compatible with injection drivers, you should be fine. A quick check is to boot up Kali Linux and run sudo airmon-ng start <interface>. This will put your adapter in promiscuous mode. If you don't have the correct drivers, it'll throw an error. Refer to a potted list of compatible adapters at http://www.aircrack-ng.org/doku.php?id=compatibility_drivers. However, if you don't have access to an adapter with the required drivers, fear not. It is still possible to set up some of the scenarios. There are two options. The first and most obvious is "buy an adapter." I can understand that you might not have a lot of cash kicking around, so my advice is to pick up an Edimax ew-7711-UAN—it's really cheap and pretty compact. It has a short range and is fairly low powered. It is also compatible with Raspberry Pi and BeagleBone, which is awesome but irrelevant. The second option is a limited solution. Most phones on the market can be used as wireless hotspots and so can be used to set up profiles for other devices for the phone-related scenarios in this article. Unfortunately, unless you have a rare and epic phone, it's unlikely to support WEP, so that's out of the question. There are solutions for rooted phones, but I wouldn't instruct you to root your phone, and I'm most certainly not providing a guide to do so. Realistically, in order to create spoofed networks effectively and set up these scenarios, a computer is required. Maybe I'm just not being imaginative enough.
Read more
  • 0
  • 0
  • 10385

article-image-tuning-solr-jvm-and-container
Packt
22 Jul 2014
6 min read
Save for later

Tuning Solr JVM and Container

Packt
22 Jul 2014
6 min read
(For more resources related to this topic, see here.) Some of these JVMs are commercially optimized for production usage; you may find comparison studies at http://dior.ics.muni.cz/~makub/java/speed.html. Some of the JVM implementations provide server versions, which would be more appropriate than normal ones. Since Solr runs in JVM, all the standard optimizations for applications are applicable to it. It starts with choosing the right heap size for your JVM. The heap size depends upon the following aspects: Use of facets and sorting options Size of the Solr index Update frequencies on Solr Solr cache Heap size for JVM can be controlled by the following parameters: Parameter Description -Xms This is the minimum heap size required during JVM initialization, that is, container -Xmx This is the maximum heap size up to which the JVM or J2EE container can consume Deciding heap size Heap in JVM contributes as a major factor while optimizing the performance of any system. JVM uses heap to store its objects, as well as its own content. Poor allocation of JVM heap results in Java heap space OutOfMemoryError thrown at runtime crashing the application. When the heap is allocated with less memory, the application takes a longer time to initialize, as well as slowing the execution speed of the Java process during runtime. Similarly, higher heap size may underutilize expensive memory, which otherwise could have been used by the other application. JVM starts with initial heap size, and as the demand grows, it tries to resize the heap to accommodate new space requirements. If a demand for memory crosses the maximum limit, JVM throws an Out of Memory exception. The objects that expire or are unused, unnecessarily consume memory in JVM. This memory can be taken back by releasing these objects by a process called garbage collection. Although it's tricky to find out whether you should increase or reduce the heap size, there are simple ways that can help you out. In a memory graph, typically, when you start the Solr server and run your first query, the memory usage increases, and based on subsequent queries and memory size, the memory graph may increase or remain constant. When garbage collection is run automatically by the JVM container, it sharply brings down its usage. If it's difficult to trace GC execution from the memory graph, you can run Solr with the following additional parameters: -Xloggc:<some file> -verbose:gc -XX:+PrintGCTimeStamps -XX:+PrintGCDetails If you are monitoring the heap usage continuously, you will find a graph that increases and decreases (sawtooth); the increase is due to the querying that is going on consistently demanding more memory by your Solr cache, and decrease is due to GC execution. In a running environment, the average heap size should not grow over time or the number of GC runs should be less than the number of queries executed on Solr. If that's not the case, you will need more memory. Features such as Solr faceting and sorting requires more memory on top of traditional search. If memory is unavailable, the operating system needs to perform hot swapping with the storage media, thereby increasing the response time; thus, users find huge latency while searching on large indexes. Many of the operating systems allow users to control swapping of programs. How can we optimize JVM? Whenever a facet query is run in Solr, memory is used to store each unique element in the index for each field. So, for example, a search over a small set of facet value (an year from 1980 to 2014) will consume less memory than a search with larger set of facet value, such as people's names (can vary from person to person). To reduce the memory usage, you may set the term index divisor to 2 (default is 4) by setting the following in solrconfig.xml: <indexReaderFactory name="IndexReaderFactory" class="solr.StandardIndexReaderFactory"> <int name="setTermIndexDivisor">2</int> </indexReaderFactory > From Solr 4.x onwards, the ability to set the min, max (term index divisor) block size ability is not available. This will reduce the memory usage for storing all the terms to half; however, it will double the seek time for terms and will impact a little on your search runtime. One of the causes of large heap is the size of index, so one solution is to introduce SolrCloud and the distributed large index into multiple shards. This will not reduce your memory requirement, but will spread it across the cluster. You can look at some of the optimized GC parameters described at http://wiki.apache.org/solr/ShawnHeisey#GC_Tuning page. Similarly, Oracle provides a GC tuning guide for advanced development stages, and it can be seen at http://www.oracle.com/technetwork/java/javase/gc-tuning-6-140523.html. Additionally, you can look at the Solr performance problems at http://wiki.apache.org/solr/SolrPerformanceProblems. Optimizing JVM container JVM containers allow users to have their requests served in threads. This in turn enables JVM to support concurrent sessions created for different users connecting at the same time. The concurrency can, however, be controlled to reduce the load on the search server. If you are using Apache Tomcat, you can modify the following entries in server.xml for changing the number of concurrent connections: Similarly, in Jetty, you can control the number of connections held by modifying jetty.xml: Similarly, for other containers, these files can change appropriately. Many containers provide a cache on top of the application to avoid server hits. This cache can be utilized for static pages such as the search page. Containers such as Weblogic provide a development versus production mode. Typically, a development mode runs with 15 threads and a limited JDBC pool size by default, whereas, for a production mode, this can be increased. For tuning containers, besides standard optimization, specific performance-tuning guidelines should be followed, as shown in the following table: Container Performance tuning guide Jetty http://wiki.eclipse.org/Jetty/Howto/High_Load Tomcat http://www.mulesoft.com/tcat/tomcat-performance and http://javamaster.wordpress.com/2013/03/13/apache-tomcat-tuning-guide/ JBoss https://access.redhat.com/site/documentation/en-US/JBoss_Enterprise_Application_Platform/5/pdf/Performance_Tuning_Guide/JBoss_Enterprise_Application_Platform-5-Performance_Tuning_Guide-en-US.pdf Weblogic http://docs.oracle.com/cd/E13222_01/wls/docs92/perform/WLSTuning.html Websphere http://www.ibm.com/developerworks/websphere/techjournal/0909_blythe/0909_blythe.html Apache Solr works better with the default container it ships with, Jetty, since it offers a small footprint compared to other containers such as JBoss and Tomcat for which the memory required is a little higher. Summary In this article, we have learned about about Apache Solr which runs on the underlying JVM in the J2EE container and tuning containers. Resources for Article: Further resources on this subject: Apache Solr: Spellchecker, Statistics, and Grouping Mechanism [Article] Getting Started with Apache Solr [Article] Apache Solr PHP Integration [Article]
Read more
  • 0
  • 0
  • 16951

article-image-customizing-skin-guiskin
Packt
22 Jul 2014
8 min read
Save for later

Customizing skin with GUISkin

Packt
22 Jul 2014
8 min read
(For more resources related to this topic, see here.) Prepare for lift off We will begin by creating a new project in Unity. Let's start our project by performing the following steps: First, create a new project and name it MenuInRPG. Click on the Create new Project button, as shown in the following screenshot: Next, import the assets package by going to Assets | Import Package | Custom Package...; choose Chapter2Package.unityPackage, which we just downloaded, and then click on the Import button in the pop-up window link, as shown in the following screenshot: Wait until it's done, and you will see the MenuInRPGGame and SimplePlatform folders in the Window view. Next, click on the arrow in front of the SimplePlatform folder to bring up the drop-down options and you will see the Scenes folder and the SimplePlatform_C# and SimplePlatform_JS scenes, as shown in the following screenshot: Next, double-click on the SimplePlatform_C# (for a C# user) and SimplePlatform_JS (for a Unity JavaScript user) scenes, as shown in the preceding screenshot, to open the scene that we will work on in this project. When you double-click on either of the SimplePlatform scenes, Unity will display a pop-up window asking whether you want to save the current scene or not. As we want to use the SimplePlatform scene, just click on the Don't Save button to open up the SimplePlatform scene, as shown in the following screenshot: Then, go to the MenuInRPGGame/Resources/UI folder and click on the first file to make sure that the Texture Type and Format fields are selected correctly, as shown in the following screenshot: Why do we set it up in this way? This is because we want to have a UI graphic to look as close to the source image as possible. However, we set the Format field to Truecolor, which will make the size of the image larger than Compress, but will show the right color of the UI graphics. Next, we need to set up the Layers and Tags configurations; for this, go to Edit | Project Settings | Tags and set them as follows: Tags Element 0 UI Element 1 Key Element 2 RestartButton Element 3 Floor Element 4 Wall Element 5 Background Element 6 Door Layers User Layer Background User Layer Level Use Layer UI At last, we will save this scene in the MenuInRPGGame/Scenes folder, and name it MenuInRPG by going to File | Save Scene as... and then save it. Engage thrusters Now we are ready to create a GUI skin; for this, perform the following steps: Let's create a new GUISkin object by going to Assets | Create | GUISkin, and we will see New GUISkin in our Project window. Name the GUISkin object as MenuSkin. Then, click on MenuSkin and go to its Inspector window. We will see something similar to the following screenshot: You will see many properties here, but don't be afraid, because this is the main key to creating custom graphics for our UI. Font is the base font for the GUI skin. From Box to Scroll View, each property is GUIStyle, which is used for creating our custom UI. The Custom Styles property is the array of GUIStyle that we can set up to apply extra styles. Settings are the setups for the entire GUI. Next, we will set up the new font style for our menu UI; go to the Font line in the Inspector view, click the circle icon, and select the Federation Kalin font. Now, you have set up the base font for GUISkin. Next, click on the arrow in front of the Box line to bring up a drop-down list. We will see all the properties, as shown in the following screenshot: For more information and to learn more about these properties, visit http://unity3d.com/support/documentation/Components/class-GUISkin.html. Name is basically the name of this style, which by default is box (the default style of GUI.Box). Next, we will be seeing our custom UI to this GUISkin; click on the arrow in front of Normal to bring up the drop-down list, and you will see two parameters—Background and Text Color. Click on the circle icon on the right-hand side of the Background line to bring up the Select Texture2D window and choose the boxNormal texture, or you can drag the boxNormal texture from the MenuInRPG/Resources/UI folder and drop it to the Background space. We can also use the search bar in the Select Texture2D window or the Project view to find our texture by typing boxNormal in the search bar, as shown in the following screenshot: Then, under the Text Color line, we leave the color as the default color—because we don't need any text to be shown in this style—and repeat the previous step with On Normal by using the boxNormal texture. Next, click on the arrow in front of Active under Background. Choose the boxActive texture, and repeat this step for On Active. Then, go to each property in the Box style and set the following parameters: Border: Left: 14, Right: 14, Top: 14, Bottom: 14 Padding: Left: 6, Right: 6, Top: 6, Bottom: 6 For other properties of this style, we will leave them as default. Next, we go to the following properties in the MenuSkin inspector and set them as follows: Label Normal | Text Color R 27, G: 95, B: 104, A: 255 Window Normal | Background myWindow On Normal | Background myWindow Border Left: 27, Right: 27, Top: 55, Bottom: 96 Padding Left: 30, Right: 30, Top: 60, Bottom: 30 Horizontal Scrollbar Normal | Background horScrollBar Border Left: 4, Right: 4, Top: 4, Bottom: 4 Horizontal Scrollbar Thumb Normal | Background horScrollBarThumbNormal Hover | Background horScrollBarThumbHover Border Left: 4, Right: 4, Top: 4, Bottom: 4 Horizontal Scrollbar Left Button Normal | Background arrowLNormal Hover | Background arrowLHover Fixed Width 14 Fixed Height 15 Horizontal Scrollbar Right Button Normal | Background arrowRNormal Hover | Background arrowRHover Fixed Width 14 Fixed Height 15 Vertical Scrollbar Normal | Background verScrollBar Border Left: 4, Right: 4, Top: 4, Bottom: 4 Padding Left: 0, Right: 0, Top: 0, Bottom: 0 Vertical Scrollbar Thumb Normal | Background verScrollBarThumbNormal Hover | Background verScrollBarThumbHover Border Left: 4, Right: 4, Top: 4, Bottom: 4 Vertical Scrollbar Up Button Normal | Background arrowUNormal Hover | Background arrowUHover Fixed Width 16 Fixed Height 14 Vertical Scrollbar Down Button Normal | Background arrowDNormal Hover | Background arrowDHover Fixed Width 16 Fixed Height 14 We have finished setting up of the default styles. Now we will go to the Custom Styles property and create our custom GUIStyle to use for this menu; go to Custom Styles and under Size, change the value to 6. Then, we will see Element 0 to Element 5. Next, we go to the first element or Element 0; under Name, type Tab Button, and we will see Element 0 change to Tab Button. Set it as follows: Tab Button (or Element 0) Name Tab Button Normal Background tabButtonNormal Text Color R: 27, G: 62, B: 67, A: 255 Hover Background tabButtonHover Text Color R: 211, G: 166, B: 9, A: 255 Active Background tabButtonActive Text Color R: 27, G: 62, B: 67, A: 255 On Normal Background tabButtonActive Text Color R: 27, G: 62, B: 67, A: 255 Border Left: 12, Right: 12, Top: 12, Bottom: 4 Padding Left: 6, Right: 6, Top: 6, Bottom: 4 Font Size 14 Alignment Middle Center Fixed Height 31 The settings are shown in the following screenshot: For the Text Color value, we can also use the eyedropper tool next to the color box to copy the same color, as we can see in the following screenshot: We have finished our first style, but we still have five styles left, so let's carry on with Element 1 with the following settings: Exit Button (or Element 1) Name Exit Button Normal | Background buttonCloseNormal Hover | Background buttonCloseHover Fixed Width 26 Fixed Height 22 The settings for Exit Button are showed in the following screenshot: The following styles will create a style for Element 2: Text Item (or Element 2) Name Text Item Normal | Text Color R: 27, G: 95, B: 104, A: 255 Alignment Middle Left Word Wrap Check The settings for Text Item are shown in the following screenshot: To set up the style for Element 3, the following settings should be done: Text Amount (or Element 3) Name Text Amount Normal | Text Color R: 27, G: 95, B: 104, A: 255 Alignment Middle Right Word Wrap Check The settings for Text Amount are shown in the following screenshot: The following settings should be done to create Selected Item: Selected Item (or Element 4) Name Selected Item Normal | Text Color R: 27, G: 95, B: 104, A: 255 Hover Background itemSelectHover Text Color R: 27, G: 95, B: 104, A: 255 Active Background itemSelectHover Text Color R: 27, G: 95, B: 104, A: 255 On Normal Background itemSelectActive Text Color R: 27, G: 95, B: 104, A: 255 Border Left: 6, Right: 6, Top: 6, Bottom: 6 Margin Left: 2, Right: 2, Top: 2, Bottom: 2 Padding Left: 4, Right: 4, Top: 4, Bottom: 4 Alignment Middle Center Word Wrap Check The settings are shown in the following screenshot: To create the Disabled Click style, the following settings should be done: Disabled Click (or Element 5) Name Disabled Click Normal Background itemSelectNormal Text Color R: 27, G: 95, B: 104, A: 255 Border Left: 6, Right: 6, Top: 6, Bottom: 6 Margin Left: 2, Right: 2, Top: 2, Bottom: 2 Padding Left: 4, Right: 4, Top: 4, Bottom: 4 Alignment Middle Center Word Wrap Check The settings for Disabled Click are shown in the following screenshot: And now, we have finished this step.
Read more
  • 0
  • 0
  • 13832

article-image-net-framework-primer
Packt
22 Jul 2014
17 min read
Save for later

The .NET Framework Primer

Packt
22 Jul 2014
17 min read
(For more resources related to this topic, see here.) An evaluation framework for .NET Framework APIs Understanding the .NET Framework in its entirety, including keeping track of the APIs that are available in various versions (for example, 3.5, 4, 4.5, 4.5.1, and so on, and platforms such as Windows 8, Windows Phone 8, and Silverlight 5) is a near impossible undertaking. What software developers and architects need is a high-level framework to logically partition the .NET Framework and identify the APIs that should be used to address a given requirement or category of requirements. API boundaries in the .NET Framework can be a little fuzzy. Some logical APIs span multiple assemblies and namespaces. Some are nicely contained within a neat hierarchy within a single root namespace. To confuse matters even further, single assemblies might contain portions of multiple APIs. The most practical way to distinguish an API is to use the API's root namespace or the namespace that contains the majority of the API's implementation. We will point out the cases where an API spans multiple namespaces or there are peculiarities in the namespaces of an API. Evaluation framework dimensions The dimensions for .NET Framework API evaluation framework are as follows: Maturity: This dimension indicates how long the API has been available, how long it has been part of the .NET Framework, and what the API's expected longevity is. It is also a measure of how relevant the API is or an indication that an API has been subsumed by newer and better APIs. Productivity: This dimension is an indication of how the use of the API will impact developer productivity. This dimension is measured by how easy the API is to learn and use, how well known the API is within the developer community, how simple or complex it is to use, the richness of the tool support (primarily in Visual Studio), and how abstract the API is, that is, whether it is declarative or imperative. Performance: This dimension indicates whether the API was designed with performance, resource utilization, user interface responsiveness, or scalability in mind; alternatively, it indicates whether convenience, ease of use, or code pithiness were the primary design criteria, which often comes at the expense of the former. Availability: This dimension indicates whether the API is available only on limited versions of the .NET Framework and Microsoft operating systems, or whether it is available everywhere that managed code is executed, including third-party implementations on non-Microsoft operating systems, for example, Mono on Linux. Evaluation framework ratings Each dimension of the API evaluation framework is given a four-level rating. Let's take a look at the ratings for each of the dimensions. The following table describes the ratings for Maturity: Rating Glyph Description Emerging This refers to a new API that was either added to the .NET Framework in the last release or is a candidate for addition in an upcoming release that has not gained widespread adoption yet. This also includes APIs that are not officially part of the .NET Framework. New and promising This is an API that has been in the .NET Framework for a couple of releases; it is already being used by the community in production systems, but it has yet to hit the mainstream. This rating may also include Microsoft APIs that are not officially part of .NET, but show a great deal of promise, or are being used extensively in production. Tried and tested This is an API that has been in the .NET Framework for multiple releases, has attained very broad adoption, has been refined and improved with each release, and is probably not going to be subsumed by a new API or deprecated in a later version of the Framework. Showing its age The API is no longer relevant or has been subsumed by a superior API, entirely deprecated in recent versions of .NET, or metamorphosedmerged into a new API. The following table describes the ratings for Productivity: Rating Glyph Description Decrease This is a complex API that is difficult to learn and use and not widely understood within the .NET developer community. Typically, these APIs are imperative, that is, they expose the underlying plumbing that needs to be understood to correctly use the API, and there is little or no tooling provided in Visual Studio. Using this API results in lowered developer productivity. No or little impact This API is fairly well known and used by the .NET developer community, but its use will have little effect on productivity, either because of its complexity, steep learning curve, and lack of tool support, or because there is simply no alternative API Increase This API is well known and used by the .NET developer community, is easy to learn and use, has good tool support, and is typically declarative; that is, the API allows the developer to express the behavior they want without requiring an understanding of the underlying plumbing, and in minimal lines of code too. Significant increase This API is very well known and used in the .NET developer community, is very easy to learn, has excellent tool support, and is declarative and pithy. Its use will significantly improve developer productivity. Performance and Scalability Rating Glyph Description Decrease The API was designed for developer productivity or convenience and will more than likely result in the slower execution of code and the increased usage of system resources (when compared to the use of other .NET APIs that provide the same or similar capabilities). Do not use this API if performance is a concern. No or little impact The API strikes a good balance between performance and developer productivity. Using it should not significantly impact the performance or scalability of your application. If performance is a concern, you can use the API, but do so with caution and make sure you measure its impact. Increase The API has been optimized for performance or scalability, and it generally results in faster, more scalable code that uses fewer system resources. It is safe to use in performance-sensitive code paths if best practices are followed. Significant increase The API was designed and written from the ground up with performance and scalability in mind. The use of this API will result in a significant improvement of performance and scalability over other APIs. The following table describes the ratings for Availability: Rating Glyph Description Rare The API is available in limited versions of the .NET Framework and on limited operating systems. Avoid this API if you are writing code that needs to be portable across all platforms. Limited This API is available on most versions of the .NET Framework and Microsoft operating systems. It is generally safe to use, unless you are targeting very old versions of .NET or Microsoft operating systems. Microsoft Only This API is available on all versions of the .NET Framework and all Microsoft operating systems. It is safe to use if you are on the Microsoft platform and are not targeting third-party CLI implementations, such as Mono.   Universal The API is available on all versions of .NET, including those from third parties, and it is available on all operating systems, including non-Microsoft operating systems. It is always safe to use this API. The .NET Framework The rest of this article will highlight some of the more commonly used APIs within the .NET Framework and rate each of these APIs using the Evaluation framework described previously. The Base Class Library The Base Class Library (BCL) is the heart of the .NET Framework. It contains base types, collections, and APIs to work with events and attributes; console, file, and network I/O; and text, XML, threads, application domains, security, debugging, tracing, serialization, interoperation with native COM and Win32 APIs, and the other core capabilities that most .NET applications need. The BCL is contained within the mscorlib.dll, System.dll, and System.Core.dll assemblies The mscorlib.dll assembly is loaded during the CLR Bootstrap(not by the CLR Loader), contains all non optional APIs and types, and is universally available in every .NET process, such as Silverlight, Windows Phone, and ASP.NET. Optional BCL APIs and types are available in System.dll and System.Core.dll, which are loaded on demand by the CLR Loader, as with all other managed assemblies. It would be a rare exception, however, when a .NET application does not use either of these two aforementioned assemblies since they contain some very useful APIs. When creating any project type in Visual Studio, these assemblies will be referenced by default. For the purpose of this framework, we will treat all of the BCL as a logical unit and not differentiate the nonoptional APIs (that is, the ones contained within mscorlib.dll), from the optional ones. Despite being a significant subset of the .NET Framework, the BCL contains a significant number of namespaces and APIs. The next sections describe a partial list of some of the more notable namespaces/APIs within the BCL, with an evaluation for each: System namespace System.Text namespace System.IO namespace System.Net namespace System.Collections namespace System.Collections.Generic namespace System.Collections.Concurrent namespace System.Linq namespace System.Xml namespace System.Xml.Linq namespace System.Security.Cryptography namespace System.Threading namespace System.Threading.Tasks namespace System.ServiceProcess namespace System.ComponentModel.Composition namespace System.ComponentModel.DataAnnotations namespace ADO.NET Most computer programs are meaningless without appropriate data to operate over. Accessing this data in an efficient way has become one of the greatest challenges modern developers face as the datasets have grown in size, from megabytes, to gigabytes, to terabytes, and now petabytes, in the most extreme cases, for example, Google's search database is around a petabyte. Though relational databases no longer hold the scalability high ground, a significant percentage of the world's data still resides in them and will probably continue to do so for the foreseeable future. ADO.NET contains a number of APIs to work with relational data and data provider APIs to access Microsoft SQL Server, Oracle Database, OLEDB, ODBC, and SQL Server Compact Edition. System.Data namespace System.Data.Entity namespace System.Data.Linq namespace System.Data.Services namespace Windows Forms Windows Forms (WinForms) was the original API for developing the user interface (UI) of Windows desktop applications with the .NET Framework. It was released in the first version of .NET and every version since then. System.Windows.Forms namespace The WinForms API is contained within the System.Windows.Forms namespace. Though WinForms is a managed API, it is actually a fairly thin façade over earlier, unmanaged APIs, primarily Win32 and User32, and any advanced use of WinForms requires a good understanding of these underlying APIs. The advanced customizations of WinForms controls often require the use of the System.Drawing API, which is also just a managed shim over the unmanaged GDI+ API. Many new applications are still developed using WinForms, despite its age and the alternative .NET user interface APIs that are available. It is a very well understood API, is very stable, and has been optimized for performance (though it is not GPU-accelerated like WPF or WinRT). There are a significant number of vendors who produce feature-rich, high-quality, third-party WinForms controls, and WinForms is available in every version of .NET and on most platforms, including Mono. WinForms is clearly showing its age, particularly when its capabilities are compared to those of WPF and WinRT, but it is still a viable API for applications that exclusively target the desktop and where a sophisticated modern UI is not necessary. The following table shows the evaluation of the System.Windows.Forms namespace: Maturity Productivity Performance Availability Windows Presentation Foundation Windows Presentation Foundation (WPF) is an API, introduced in .NET 3.0, for developing rich user interfaces for .NET applications, with no dependencies on legacy Windows APIs and with support for GPU-accelerated 3D rendering, animation, and media playback. If you want to play a video on a clickable button control on the surface of an animated, 3D rotating cube and the only C# code you want to write is the button click event handler, then WPF is the API for the job. See the WPFSample code for a demonstration. System.Windows namespace The System.Windows namespace contains the Windows Presentation Foundation API. WPF includes many of the "standard" controls that are in WinForms, for example, Button, Label, CheckBox, ComboBox, and so on. However, it also includes APIs to create, animate, and render 3D graphics; render multimedia; draw bitmap and vector graphics; and perform animation. WPF addresses many of the limitations of Windows Forms, but this power comes at a price. WPF introduces a number of novel concepts that developers will need to master, including a new, declarative UI markup called Extensible Application Markup Language (XAML), new event handling, data binding and control theming mechanisms, and a variant of the Model-view-controller (MVC) pattern called Model View ViewModel (MVVM); that said, the use of this pattern is optional but highly recommended. WPF has significantly more moving parts than WinForms, if you ignore the underlying native Windows APIs that WinForm abstracts. Microsoft, though, has gone to some lengths to make the WPF development experience easier for both UI designers and developers. Developers using WPF can choose to design and develop user interfaces using XAML, any of the .NET languages, or most often a combination of the two. Visual Studio and Expression Blend provide rich WYSIWYG designers to create WPF controls and interfaces and hide the complexities of the underlying XAML. Direct tweaking of the XAML is sometimes required for precise adjustments. WPF is now a mature, stable API that has been highly optimized for performance—all of its APIs are GPU accelerated. Though it is probably not as well known as WinForms, it has become relatively well known within the developer community, particularly because Silverlight, which is Microsoft's platform for developing rich web and mobile applications, uses a subset of WPF. Many of the third-party control vendors who produce WinForm controls now also produce equivalent WPF controls. The tools for creating WPF applications, predominantly Visual Studio and Expression Blend, are particularly good, and there are also a number of good third-party and open source tools to work with XAML. The introduction of WinRT and the increasingly powerful capabilities of web browser technologies, including HTML5, CSS3, JavaScript, WebGL, and GPU-acceleration, raise valid questions about the long-term future of WPF and Silverlight. Microsoft seems to be continuing to promote the use of WPF, and even WinRT supports a variant of the XAML markup language, so it should remain a viable API for a while. The following table shows the evaluation of the System.Windows namespace: Maturity Productivity Performance Availability ASP.NET The .NET Framework was originally designed to be Microsoft's first web development platform, and it included APIs to build both web applications and web services. These APIs were, and still are, part of the ASP.NET web development framework that lives in the System.Web namespace. ASP.NET has come a very long way since the first release of .NET, and it is the second most widely used and popular web framework in the world today (see http://trends.builtwith.com/framework). The ASP.NET platform provides a number of complimentary APIs that can be used to develop web applications, including Web Forms, web services, MVC, web pages, Web API, and SignalR. System.Web.Forms namespace System.Web.Mvc namespace System.Web.WebPages namespace System.Web.Services namespace Microsoft.AspNet.SignalR namespace Windows Communication Foundation One of the major selling points of the first release of .NET was that the platform had support for web services baked in, in the form of ASP.NET Web Services. Web Services have come a very long way since SOAP was invented in 1998 and the first release of .NET, and WCF has subsumed the limited capabilities of ASP.NET Web Services with a far richer platform. WCF has also subsumed the original .NET Remoting (System.Runtime.Remoting), MSMQ (System.Messaging), and Enterprise Services (System.EnterpriseServices) APIs. System.ServiceModel namespace The root namespace for WCF is System.ServiceModel. This API includes support for most of the WS-* web services standards and non-HTTP or XML-based services, including MSMQ and TCP services that use binary or Message Transmission Optimization Mechanism (MTOM) message encoding. Address, Binding, and Contract (ABC) of WCF are very well understood by the majority of the developer community, though deep technical knowledge of WCF's inner workings is rarer. The use of attributes to declare service and data contracts and a configuration-over-code approach makes the WCF API highly declarative, and creating sophisticated services that use advanced WS-* capabilities is relatively easy. WCF is very stable and can be used to create high-performance distributed applications. WCF is available on all recent versions of .NET, though not all platforms include the server components of WCF. Partial support for WCF is also available on third-party CLI implementations, such as Mono. REST-based web services, that serve relatively simple XML or JSON, have become very popular, and though WCF fairly recently added support for REST, these capabilities have now evolved into the ASP.NET Web API. The following table shows the evaluation of the System.ServiceModel namespace: Maturity Productivity Performance Availability Windows Workflow Foundation Windows Workflow Foundation (WF) is a workflow framework that was introduced in .NET 3.0, and that brings the power and flexibility of declarative workflow or business process design and execution to .NET applications. System.Activities namespace The System.Activities namespace contains the Windows Workflow Foundation API. WF includes a workflow runtime, a hosting API, a number of basic workflow activities, APIs to create custom activities, and a workflow designer control, which was originally a WinForms control but is now a WPF control as of .NET 4.0. WF also uses a variant of the same XAML markup, which WPF and WinRT use, to represent workflows; that said, an excellent designer, hosted by default in Visual Studio, should mean that you never have to directly modify the XAML. The adoption of the first few versions of the WF API was limited, but WF was completely rewritten for .NET 4.0, and many of the shortcomings of the original version were entirely addressed. WF is now a mature, stable, best-of-breed workflow API, with a proven track record. The previous implementation of WF is still available in current versions of the .NET Framework, for migration and interoperation purposes, and is in the System.Workflow namespace. WF is used by SharePoint Server, Windows Server AppFabric, Windows Azure AppFabric, Office 365, Visual Studio Team Foundation Server (MSBuild), and a number of other Microsoft products and services. Windows Server AppFabric and Windows Azure AppFabric enable a new class of scalable SOA server and cloud application called a Workflow Service, which is a combination of the capabilities of WCF and WF. WF has a relatively small but strong following within the .NET developer community. There are also a number of third-party and open source WF activity libraries and tools available. Though applications composed using workflows typically have poorer performance than those that are implemented entirely in code, the flexibility and significantly increased developer productivity (particularly when it comes to modifying existing processes) that workflows give you are often worth the performance price. That said, Microsoft has made significant investments in optimizing the performance of WF, and it should be more than adequate for most enterprise application scenarios. Though versions of WF are available on other CLI platforms, the availability of WF 4.x is limited to Microsoft platforms and .NET 4.0 and higher. The evaluation of the System.Workflow namespace shown in the following table is for the most recent version of WF (the use of versions of WF prior to 4.0 is not recommended for new applications): Maturity Productivity Performance Availability Summary There is more to the .NET Framework than has been articulated in this primer; it includes many useful APIs that have not even been mentioned here, for example, System.Media, System.Speech, and the Windows Identity Framework. There are also a number of very powerful APIs developed by Microsoft (and Microsoft Research) that are not (yet) officially part of the .NET Framework; for example, Reactive Extensions, Microsoft Solver Foundation, Windows Azure APIs, and the new .NET for Windows Store Apps APIs are worth looking into. Resources for Article:   Further resources on this subject: Content Based Routing on Microsoft Platform [article] Building the Content Based Routing Solution on Microsoft Platform [article] Debatching Bulk Data on Microsoft Platform [article]
Read more
  • 0
  • 0
  • 2018
article-image-configuration
Packt
21 Jul 2014
12 min read
Save for later

Configuration

Packt
21 Jul 2014
12 min read
(For more resources related to this topic, see here.) Configuration targets In this section, we look at the different layers that can be configured. The layers are: SYSTEM: This layer is system-wide and found in /etc/gitconfig GLOBAL: This layer is global for the user and found in ~/.gitconfig LOCAL: This layer is local to the current repository and found in .git/config Getting ready We will use the jgit repository for this example, as shown in the following command: $ git clone https://git.eclipse.org/r/jgit/jgit $ cd jgit How to do it... In the previous example, we saw how we could use the command git config --list to list configuration entries. This list is actually made from three different levels of configuration that Git offers: system-wide configuration, SYSTEM; global configuration for the user, GLOBAL; and local repository configuration, LOCAL. For each of these configuration layers, we can query the existing configuration. On a Windows box with a default installation of the Git extensions, the different configuration layers will look approximately like the following: $ git config --list --system core.symlinks=false core.autocrlf=true color.diff=auto color.status=auto color.branch=auto color.interactive=true pack.packsizelimit=2g help.format=html http.sslcainfo=/bin/curl-ca-bundle.crt sendemail.smtpserver=/bin/msmtp.exe diff.astextplain.textconv=astextplain rebase.autosquash=true $ git config --list --global merge.tool=kdiff3 mergetool.kdiff3.path=C:/Program Files (x86)/KDiff3/kdiff3.exe diff.guitool=kdiff3 difftool.kdiff3.path=C:/Program Files (x86)/KDiff3/kdiff3.exe core.editor="C:/Program Files (x86)/GitExtensions/GitExtensions.exe" fileeditor core.autocrlf=true credential.helper=!"C:/Program Files (x86)/GitExtensions/GitCredentialWinStore/git-credential-winst ore.exe" user.name=Aske Olsson user.email=aske.olsson@switch-gears.dk $ git config --list --local core.repositoryformatversion=0 core.filemode=false core.bare=false core.logallrefupdates=true core.symlinks=false core.ignorecase=true core.hidedotfiles=dotGitOnly remote.origin.url=https://git.eclipse.org/r/jgit/jgit remote.origin.fetch=+refs/heads/*:refs/remotes/origin/* branch.master.remote=origin branch.master.merge=refs/heads/master We can also query a single key and limit the scope to one of the three layers, by using the following command: $ git config --global user.email aske.olsson@switch-gears.dk We can set the e-mail address of the user to a different one for the current repository: $ git config --local user.email aske@switch-gears.dk Now, listing the GLOBAL layer user.email will return aske.olsson@switch-gears.dk, listing LOCAL gives aske@switch-gears.dk, and listing user.email without specifying the layer gives the effective value that is used in the operations on this repository, in this case, the LOCAL value aske@switch-gears.dk. The effective value is the value, which takes precedence when needed. When two or more values are specified for the same key, but on different layers, the lowest layer takes precedence. When a configuration value is needed, Git will first look in the LOCAL configuration. If not found here, the GLOBAL configuration is queried. If it is not found in the GLOBAL configuration, the SYSTEM configuration is used. If none of this works, the default value in Git is used. In the previous example, user.email is specified in both the GLOBAL and LOCAL layers. Hence, the LOCAL layer will be used. How it works... Querying the three layers of configuration simply returns the content of the configuration files: /etc/gitconfig for system-wide configuration, ~/.gitconfig for user-specific configuration, and .git/config for repository-specific configuration. When not specifying the configuration layer, the returned value will be the effective value. There's more... Instead of setting all the configuration values on the command line by the key value, it is possible to set them by just editing the configuration file directly. Open the configuration file in your favorite editor and set the configuration you need, or use the built-in git config -e repository to edit the configuration directly in the Git-configured editor. You can set the editor to the editor of your choice either by changing the $EDITOR environment variable or with the core.editor configuration target, for example: $ git config --global core.editor vim Querying the existing configuration In this example, we will look at how we can query the existing configuration and set the configuration values. Getting ready We'll use jgit again by using the following command: $ cd jgit How to do it... To view all the effective configurations for the current Git repository, run the following command: $ git config --list user.name=Aske Olsson user.email=askeolsson@switch-gears.dk core.repositoryformatversion=0 core.filemode=false core.bare=false core.logallrefupdates=true remote.origin.url=https://git.eclipse.org/r/jgit/jgit remote.origin.fetch=+refs/heads/*:refs/remotes/origin/* branch.master.remote=origin branch.master.merge=refs/heads/master The previous output will of course reflect the user running the command. Instead of Aske Olsson as the name and the e-mail, the output should reflect your settings. If we are just interested in a single configuration item, we can just query it by its section.key or section.subsection.key: $ git config user.name Aske Olsson $ git config remote.origin.url https://git.eclipse.org/r/jgit/jgit How it works... Git's configuration is stored in plaintext files, and works like a key-value storage. You can set/query by key and get the value back. An example of the text-based configuration file is shown as follows (from the jgit repository): $ cat .git/config [core] repositoryformatversion = 0 filemode = false bare = false logallrefupdates = true [remote "origin"] url = https://git.eclipse.org/r/jgit/jgit fetch = +refs/heads/*:refs/remotes/origin/* [branch "master"] remote = origin merge = refs/heads/master There's more... It is also easy to set configuration values. Just use the same syntax as when querying the configuration except add an argument to the value. To set a new e-mail address on the LOCAL layer, we can execute the following command line: git config user.email askeolsson@example.com The LOCAL layer is the default if nothing else is specified. If you require whitespaces in the value, you can enclose the string in quotation marks, as you would do when configuring your name: git config user.name "Aske Olsson" You can even set your own configuration, which does not have any effect on the core Git, but can be useful for scripting/builds and so on: $ git config my.own.config "Whatever I need" List the value $ git config my.own.config Whatever I need It is also very easy to delete/unset configuration entries: $ git config --unset my.own.config List the value $ git config my.own.config Templates In this example, we will see how to create a template commit message that will be displayed in the editor when creating a commit. The template is only for the local user and not distributed with the repository in general. Getting ready In this example, we will use the example repository: $ git clone https://github.com/dvaske/data-model.git $ cd data-model We'll use the following code as a commit message template for commit messages: Short description of commit Longer explanation of the motivation for the change Fixes-Bug: Enter bug-id or delete line Implements-Requirement: Enter requirement-id or delete line Save the commit message template in $HOME/.gitcommitmsg.txt. The filename isn't fixed and you can choose a filename of your liking. How to do it... To let Git know about our new commit message template, we can set the configuration variable commit.template to point at the file we just created with that template; we'll do it globally so it is applicable to all our repositories: $ git config --global commit.template $HOME/.gitcommitmsg.txt Now, we can try to change a file, add it, and create a commit. This will bring up our preferred editor with the commit message template preloaded: $ git commit Short description of commit Longer explanation of the motivation for the change Fixes-Bug: Enter bug-id or delete line Implements-Requirement: Enter requirement-id or delete line # Please enter the commit message for your changes. Lines starting # with '#' will be ignored, and an empty message aborts the commit. # On branch master # Changes to be committed: # (use "git reset HEAD <file>..." to unstage) # # modified: another-file.txt # ~ ~ ".git/COMMIT_EDITMSG" 13 lines, 396 characters We can now edit the message according to our commit and save to complete the commit. How it works... When commit.template is set, Git simply uses the content of the template file as a starting point for all commit messages. This is quite convenient if you have a commit-message policy as it greatly increases the chances of the policy being followed. You can even have different templates tied to different repositories, since you can just set the configuration at the local level. A .git directory template Sometimes, having a global configuration isn't enough. You will also need to trigger the execution of scripts (aka Git hooks), exclude files, and so on. It is possible to achieve this with the template option set to git init. It can be given as a command-line option to git clone and git init, or as the $GIT_TEMPLATE_DIR environment variable, or as the configuration option init.templatedir. It defaults to /usr/share/git-core/templates. The template option works by copying files in the template directory to the .git ($GIT_DIR) folder after it has been created. The default directory contains sample hooks and some suggested exclude patterns. In the following example, we'll see how we can set up a new template directory, and add a commit message hook and exclude file. Getting ready First, we will create the template directory. We can use any name we want, and we'll use ~/.git_template, as shown in the following command: $ mkdir ~/.git_template Now, we need to populate the directory with some template files. This could be a hook or an exclude file. We will create one hook file and an exclude file. The hook file is located in .git/hooks/name-of-hook and the exclude file in .git/info/exclude. Create the two directories needed hooks and info, as shown in the following command: $ mkdir ~/.git_template/{hooks,info} To keep the sample hooks provided by the default template directory (the Git installation), we copy the files in the default template directory to the new one. When we use our newly created template directory, we'll override the default one. So, copying the default files to our template directory will make sure that except for our specific changes the template directory is similar to the default one, as shown in the following command: $ cd ~/.git_template/hooks $ cp /usr/share/git-core/templates/hooks/* . We'll use the commit-msg hook as the example hook: #!/bin/sh MSG_FILE="$1" echo "nHi from the template commit-msg hook" >> $MSG_FILE The hook is very simple and will just add Hi from the template commit-msg hook to the end of the commit message. Save it as commit-msg in the ~/.git_template/hooks directory and make it executable by using the following command: chmod +x ~/.git_template/hooks/commit-msg Now that the commit message hook is done, let's also add an exclude file to the example. The exclude file works like the .gitignore file, but is not tracked in the repository. We'll create an exclude file that excludes all the *.txt files, as follows: $ echo *.txt > ~/.git_template/info/exclude Now, our template directory is ready for use. How to do it... Our template directory is ready and we can use it, as described earlier, as a command-line option, an environment variable or, as in this example, to be set as a configuration: $ git config --global init.templatedir ~/.git_template Now, all Git repositories we create using init or clone will have the default files of the template directory. We can test if it works by creating a new repository as follows: $ git init template-example $ cd template-example Let's try to create a .txt file and see what git status tells us. It should be ignored by the exclude file from the template directory: $ echo "this is the readme file" > README.txt $ git status The exclude file worked! You can put in the file endings yourself or just leave it blank and keep to the .gitignore files. To test if the commit-msg hook also works, let us try to create a commit. First, we need a file to commit. So, let's create that and commit it as follows: $ echo "something to commit" > somefile $ git add somefile $ git commit –m "Committed something" We can now check the history with git log: $ git log -1 commit 1f7d63d7e08e96dda3da63eadc17f35132d24064 Author: Aske Olsson <aske.olsson@switch-gears.dk> Date: Mon Jan 6 20:14:21 2014 +0100 Committed something Hi from the template commit-msg hook How it works... When Git creates a new repository, either via init or clone, it will copy the files from the template directory to the new repository when creating the directory structure. The template directory can be defined either by a command-line argument, environment variable, or configuration option. If nothing is specified, the default template directory will be used (distributed with the Git installation). By setting the configuration as a --global option, the template directory defined will apply to all of the user's (new) repositories. This is a very nice way to distribute the same hooks across repositories, but it also has some drawbacks. As the files in the template directory are only copied to the Git repositories, updates to the template directory do not affect the existing repositories. This can be solved by running git init in each existing repository to reinitialize the repository, but this can be quite cumbersome. Also, the template directory can enforce hooks on some repositories where you don't want them. This is quite easily solved by simply deleting the hook files in .git/hooks of that repository.
Read more
  • 0
  • 0
  • 1929

article-image-hardware-configuration
Packt
21 Jul 2014
2 min read
Save for later

Hardware configuration

Packt
21 Jul 2014
2 min read
The hardware configuration of this project is not really complex. For each motion sensor module you want to build, you'll need to do the following steps. The first one is to plug an XBee module on the XBee shield. Then, you need to plug the shield into your Arduino board, as shown in the following image: Now, you can connect the motion sensor. It has three pins: VCC (for the positive power supply), GND (which corresponds to the reference voltage level), and SIG (which will turn to a digital HIGH state in case any motion is detected). Connect VCC to the Arduino 5V pin, GND to Arduino GND, and SIG to Arduino pin number 8 (the example code uses pin 8, but you could also use any digital pin). You should end up with something similar to this image: You will also need to set a jumper correctly on the board so we can upload a sketch. On the XBee shield, you have a little switch close to the XBee module to choose between the XBee module being connected directly to the Arduino board serial interface (which means you can't upload any sketches anymore) or leaving it disconnected. As we need to upload the Arduino sketch first, you need to put this switch to DLINE, as shown in this image: You will also need to connect the XBee explorer board to your computer at this point. Simply insert one XBee module to the board as shown in the following image: Now that this is done, you can power up everything by connecting the Arduino board and explorer module to your computer via USB cables. If you want to use several XBee motion sensors, you will need to repeat the beginning of the procedure for each of them: assemble one Arduino board with an XBee shield, one XBee module, and one motion sensor. However, you only need one USB XBee module connected to your computer if you have many sensors. Summary In this article, we learned about the hardware configuration required to build wireless XBee motion detectors. We looked at Arduino R3 board, XBee module, and XBee shield and the other important hardware configuration. Resources for Article: Further resources on this subject: Playing with Max 6 Framework [Article] Our First Project – A Basic Thermometer [Article] Sending Data to Google Docs [Article]
Read more
  • 0
  • 0
  • 24366

article-image-article-design-patterns
Packt
21 Jul 2014
5 min read
Save for later

Design patterns

Packt
21 Jul 2014
5 min read
(For more resources related to this topic, see here.) Design patterns are ways to solve a problem and the way to get your intended result in the best possible manner. So, design patterns are not only ways to create a large and robust system, but they also provide great architectures in a friendly manner. In software engineering, a design pattern is a general repeatable and optimized solution to a commonly occurring problem within a given context in software design. It is a description or template for how to solve a problem, and the solution can be used in different instances. The following are some of the benefits of using design patterns: Maintenance Documentation Readability Ease in finding appropriate objects Ease in determining object granularity Ease in specifying object interfaces Ease in implementing even for large software projects Implements the code reusability concept If you are not familiar with design patterns, the best way to begin understanding is observing the solutions we use for commonly occurring, everyday life problems. Let's take a look at the following image: Many different types of power plugs exist in the world. So, we need a solution that is reusable, optimized, and cheaper than buying a new device for different power plug types. In simple words, we need an adapter. Have a look at the following image of an adapter: In this case, an adapter is the best solution that's reusable, optimized, and cheap. But an adapter does not provide us with a solution when our car's wheel blows out. In object-oriented languages, we the programmers use the objects to do whatever we want to have the outcome we desire. Hence, we have many types of objects, situations, and problems. That means we need more than just one approach to solving different kinds of problems. Elements of design patterns The following are the elements of design patterns: Name: This is a handle we can use to describe the problem Problem: This describes when to apply the pattern Solution: This describes the elements, relationships, responsibilities, and collaborations, in a way that we follow to solve a problem Consequences: This details the results and trade-offs of applying the pattern Classification of design patterns Design patterns are generally divided into three fundamental groups: Creational patterns Structural patterns Behavioral patterns Let's examine these in the following subsections. Creational patterns Creational patterns are a subset of design patterns in the field of software development; they serve to create objects. They decouple the design of an object from its representation. Object creation is encapsulated and outsourced (for example, in a factory) to keep the context of object creation independent from concrete implementation. This is in accordance with the rule: "Program on the interface, not the implementation." Some of the features of creational patterns are as follows: Generic instantiation: This allows objects to be created in a system without having to identify a specific class type in code (Abstract Factory and Factory pattern) Simplicity: Some of the patterns make object creation easier, so callers will not have to write large, complex code to instantiate an object (Builder (Manager) and Prototype pattern) Creation constraints: Creational patterns can put bounds on who can create objects, how they are created, and when they are created The following patterns are called creational patterns: The Abstract Factory pattern The Factory pattern The Builder (Manager) pattern The Prototype pattern The Singleton pattern Structural patterns In software engineering, design patterns structure patterns facilitate easy ways for communications between various entities. Some of the examples of structures of the samples are as follows: Composition: This composes objects into a tree structure (whole hierarchies). Composition allows customers to be uniformly treated as individual objects according to their composition. Decorator: This dynamically adds options to an object. A Decorator is a flexible alternative embodiment to extend functionality. Flies: This is a share of small objects (objects without conditions) that prevent overproduction. Adapter: This converts the interface of a class into another interface that the clients expect. Adapter lets those classes work together that would normally not be able to because of the different interfaces. Facade: This provides a unified interface meeting the various interfaces of a subsystem. Facade defines a higher-level interface to the subsystem, which is easier to use. Proxy: This implements the replacement (surrogate) of another object that controls access to the original object. Bridge: This separates an abstraction from its implementation, which can then be independently altered. Behavioral patterns Behavioral patterns are all about a class' objects' communication. Behavioral patterns are those patterns that are most specifically concerned with communication between objects. The following is a list of the behavioral patterns: Chain of Responsibility pattern Command pattern Interpreter pattern Iterator pattern Mediator pattern Memento pattern Observer pattern State pattern Strategy pattern Template pattern Visitor pattern If you want to check out the usage of some patterns in the Laravel core, have a look at the following list: The Builder (Manager) pattern: IlluminateAuthAuthManager and IlluminateSessionSessionManager The Factory pattern: IlluminateDatabaseDatabaseManager and IlluminateValidationFactory The Repository pattern: IlluminateConfigRepository and IlluminateCacheRepository The Strategy pattern: IIlluminateCacheStoreInterface and IlluminateConfigLoaderInterface The Provider pattern: IIlluminateAuthAuthServiceProvider and IlluminateHashHashServiceProvider Summary In this article, we have explained the fundamentals of design patterns. We've also introduced some design patterns that are used in the Laravel Framework. Resources for Article: Further resources on this subject: Laravel 4 - Creating a Simple CRUD Application in Hours [article] Your First Application [article] Creating and Using Composer Packages [article]
Read more
  • 0
  • 0
  • 15869
article-image-sharding-action
Packt
21 Jul 2014
9 min read
Save for later

Sharding in Action

Packt
21 Jul 2014
9 min read
(For more resources related to this topic, see here.) Preparing the environment Before jumping into configuring and setting up the cluster network, we have to check some parameters and prepare the environment. To enable sharding for a database or collection, we have to configure some configuration servers that hold the cluster network metadata and shards information. Other parts of the cluster network use these configuration servers to get information about other shards. In production, it's recommended to have exactly three configuration servers in different machines. The reason for establishing each shard on a different server is to improve the safety of data and nodes. If one of the machines crashes, the whole cluster won't be unavailable. For the testing and developing environment, you can host all the configuration servers on a single server.Besides, we have two more parts for our cluster network, shards and mongos, or query routers. Query routers are the interface for all clients. All read/write requests are routed to this module, and the query router or mongos instance, using configuration servers, route the request to the corresponding shard. The following diagram shows the cluster network, modules, and the relation between them: It's important that all modules and parts have network access and are able to connect to each other. If you have any firewall, you should configure it correctly and give proper access to all cluster modules. Each configuration server has an address that routes to the target machine. We have exactly three configuration servers in our example, and the following list shows the hostnames: cfg1.sharding.com cfg2.sharding.com cfg3.sharding.com In our example, because we are going to set up a demo of sharding feature, we deploy all configuration servers on a single machine with different ports. This means all configuration servers addresses point to the same server, but we use different ports to establish the configuration server. For production use, all things will be the same, except you need to host the configuration servers on separate machines. In the next section, we will implement all parts and finally connect all of them together to start the sharding server and run the cluster network. Implementing configuration servers Now it's time to start the first part of our sharding. Establishing a configuration server is as easy as running a mongod instance using the --configsvr parameter. The following scheme shows the structure of the command: mongod --configsvr --dbpath <path> --port <port> If you don't pass the dbpath or port parameters, the configuration server uses /data/configdb as the path to store data and port 27019 to execute the instance. However, you can override the default values using the preceding command. If this is the first time that you have run the configuration server, you might be faced with some issues due to the existence of dbpath. Before running the configuration server, make sure that you have created the path; otherwise, you will see an error as shown in the following screenshot: You can simply create the directory using the mkdir command as shown in the following line of command: mkdir /data/configdb Also, make sure that you are executing the instance with sufficient permission level; otherwise, you will get an error as shown in the following screenshot: The problem is that the mongod instance can't create the lock file because of the lack of permission. To address this issue, you should simply execute the command using a root or administrator permission level. After executing the command using the proper permission level, you should see a result like the following screenshot: As you can see now, we have a configuration server for the hostname cfg1.sharding.com with port 27019 and with dbpath as /data/configdb. Also, there is a web console to watch and control the configuration server running on port 28019. By pointing the web browser to the address http://localhost:28019/, you can see the console. The following screenshot shows a part of this web console: Now, we have the first configuration server up and running. With the same method, you can launch other instances, that is, using /data/configdb2 with port 27020 for the second configuration server, and /data/configdb3 with port 27021 for the third configuration server. Configuring mongos instance After configuring the configuration servers, we should bind them to the core module of clustering. The mongos instance is responsible to bind all modules and parts together to make a complete sharding core. This module is simple and lightweight, and we can host it on the same machine that hosts other modules, such as configuration servers. It doesn't need a separate directory to store data. The mongos process uses port 27017 by default, but you can change the port using the configuration parameters. To define the configuration servers, you can use the configuration file or command-line parameters. Create a new file using your text editor in the /etc/ directory and add the following configuring settings: configdb = cfg1.sharding.com:27019, cfg2.sharding.com:27020 cfg3.sharding.com:27021 To execute and run the mongos instance, you can simply use the following command: mongos -f /etc/mongos.conf After executing the command, you should see an output like the following screenshot: Please note that if you have a configuration server that has been already used in a different sharding network, you can't use the existing data directory. You should create a new and empty data directory for the configuration server. Currently, we have mongos and all configuration servers that work together pretty well. In the next part, we will add shards to the mongos instance to complete the whole network. Managing mongos instance Now it's time to add shards and split whole dataset into smaller pieces. For production use, each shard should be a replica set network, but for the development and testing environment, you can simply add a single mongod instances to the cluster. To control and manage the mongos instance, we can simply use the mongo shell to connect to the mongos and execute commands. To connect to the mongos, you use the following command: mongo --host <mongos hostname> --port <mongos port> For instance, our mongos address is mongos1.sharding.com and the port is 27017. This is depicted in the following screenshot: After connecting to the mongos instance, we have a command environment, and we can use it to add, remove, or modify shards, or even get the status of the entire sharding network. Using the following command, you can get the status of the sharding network: sh.status() The following screenshot illustrates the output of this command: Because we haven't added any shards to sharding, you see an error that says there are no shards in the sharding network. Using the sh.help() command, you can see all commands as shown in the following screenshot: Using the sh.addShard() function, you can add shards to the network. Adding shards to mongos After connecting to the mongos, you can add shards to sharding. Basically, you can add two types of endpoints to the mongos as a shard; replica set or a standalone mongod instance. MongoDB has a sh namespace and a function called addShard(), which is used to add a new shard to an existing sharding network. Here is the example of a command to add a new shard. This is shown in the following screenshot: To add a replica set to mongos you should follow this scheme: setname/server:port For instance, if you have a replica set with the name of rs1, hostname mongod1.replicaset.com, and port number 27017, the command will be as follows: sh.addShard("rs1/mongod1.replicaset.com:27017") Using the same function, we can add standalone mongod instances. So, if we have a mongod instance with the hostname mongod1.sharding.com listening on port 27017, the command will be as follows: sh.addShard("mongod1.sharding.com:27017") You can use a secondary or primary hostname to add the replica set as a shard to the sharding network. MongoDB will detect the primary and use the primary node to interact with sharding. Now, we add the replica set network using the following command: sh.addShard("rs1/mongod1.replicaset.com:27017") If everything goes well, you won't see any output from the console, which means the adding process was successful. This is shown in the following screenshot: To see the status of sharding, you can use the sh.status() command. This is demonstrated in the following screenshot: Next, we will establish another standalone mongod instance and add it to sharding. The port number of mongod is 27016 and the hostname is mongod1.sharding.com. The following screenshot shows the output after starting the new mongod instance: Using the same approach, we will add the preceding node to sharding. This is shown in the following screenshot: It's time to see the sharding status using the sh.status() command: As you can see in the preceding screenshot, now we have two shards. The first one is a replica set with the name rs1, and the second shard is a standalone mongod instance on port 27016. If you create a new database on each shard, MongoDB syncs this new database with the mongos instance. Using the show dbs command, you can see all databases from all shards as shown in the following screenshot: The configuration database is an internal database that MongoDB uses to configure and manage the sharding network. Currently, we have all sharding modules working together. The last and final step is to enable sharding for a database and collection. Summary In this article, we prepared the environment for sharding of a database. We also learned about the implementation of a configuration server. Next, after configuring the configuration servers, we saw how to bind them to the core module of clustering. Resources for Article: Further resources on this subject: MongoDB data modeling [article] Ruby with MongoDB for Web Development [article] Dart Server with Dartling and MongoDB [article]
Read more
  • 0
  • 0
  • 4599

Packt
18 Jul 2014
17 min read
Save for later

C10K – A Non-blocking Web Server in Go

Packt
18 Jul 2014
17 min read
This article by Nathan Kozyra, author of Mastering Concurrency in Go, tackles one of the Internet's most famous and esteemed challenges and attempt to solve it with core Go packages. (For more resources related to this topic, see here.) We've built a few usable applications; things we can start with and leapfrog into real systems for everyday use. By doing so, we've been able to demonstrate the basic and intermediate-level patterns involved in Go's concurrent syntax and methodology. However, it's about time we take on a real-world problem—one that has vexed developers (and their managers and VPs) for a great deal of the early history of the Web. In addressing and, hopefully, solving this problem, we'll be able to develop a high-performance web server that can handle a very large volume of live, active traffic. For many years, the solution to this problem was solely to throw hardware or intrusive caching systems at the problem; so, alternately, solving it with programming methodology should excite any programmer. We'll be using every technique and language construct we've learned so far, but we'll do so in a more structured and deliberate way than we have up to now. Everything we've explored so far will come into play, including the following points: Creating a visual representation of our concurrent application Utilizing goroutines to handle requests in a way that will scale Building robust channels to manage communication between goroutines and the loop that will manage them Profiling and benchmarking tools (JMeter, ab) to examine the way our event loop actually works Timeouts and concurrency controls—when necessary—to ensure data and request consistency Attacking the C10K problem The genesis of the C10K problem is rooted in serial, blocking programming, which makes it ideal to demonstrate the strength of concurrent programming, especially in Go. When he asked this in 1999, for many server admins and engineers, serving 10,000 concurrent visitors was something that would be solved with hardware. The notion that a single server on common hardware could handle this type of CPU and network bandwidth without falling over seemed foreign to most. The crux of his proposed solutions relied on producing non-blocking code. Of course, in 1999, concurrency patterns and libraries were not widespread. C++ had some polling and queuing options available via some third-party libraries and the earliest predecessor to multithreaded syntaxes, later available through Boost and then C++11. Over the coming years, solutions to the problem began pouring in across various flavors of languages, programming design, and general approaches. Any performance and scalability problem will ultimately be bound to the underlying hardware, so as always, your mileage may vary. Squeezing 10,000 concurrent connections on a 486 processor with 500 MB of RAM will certainly be more challenging than doing so on a barebones Linux server stacked with memory and multiple cores. It's also worth noting that a simple echo server would obviously be able to assume more cores than a functional web server that returns larger amounts of data and accepts greater complexity in requests, sessions, and so on, as we'll be dealing with here. Failing of servers at 10,000 concurrent connections When the Web was born and the Internet commercialized, the level of interactivity was pretty minimal. If you're a graybeard, you may recall the transition from NNTP/IRC and the like and how extraordinarily rudimentary the Web was. To address the basic proposition of [page request] → [HTTP response], the requirements on a web server in the early 1990s were pretty lenient. Ignoring all of the error responses, header reading, and settings, and other essential (but unrelated to the in → out mechanism) functions, the essence of the early servers was shockingly simple, at least compared to the modern web servers. The first web server was developed by the father of the Web, Tim Berners-Lee. Developed at CERN (such as WWW/HTTP itself), CERN httpd handled many of the things you would expect in a web server today—hunting through the code, you'll find a lot of notation that will remind you that the very core of the HTTP protocol is largely unchanged. Unlike most technologies, HTTP has had an extraordinarily long shelf life. Written in C in 1990, it was unable to utilize a lot of concurrency strategies available in languages such as Erlang. Frankly, doing so was probably unnecessary—the majority of web traffic was a matter of basic file retrieval and protocol. The meat and potatoes of a web server were not dealing with traffic, but rather dealing with the rules surrounding the protocol itself. You can still access the original CERN httpd site and download the source code for yourself from http://www.w3.org/Daemon/. I highly recommend that you do so as both a history lesson and a way to look at the way the earliest web server addressed some of the earliest problems. However, the Web in 1990 and the Web when the C10K question was first posed were two very different environments. By 1999, most sites had some level of secondary or tertiary latency provided by third-party software, CGI, databases, and so on, all of which further complicated the matter. The notion of serving 10,000 flat files concurrently is a challenge in itself, but try doing so by running them on top of a Perl script that accesses a MySQL database without any caching layer; this challenge is immediately exacerbated. By the mid 1990s, the Apache web server had taken hold and largely controlled the market (by 2009, it had become the first server software to serve more than 100 million websites). Apache's approach was rooted heavily in the earliest days of the Internet. At its launch, connections were initially handled first in, first out. Soon, each connection was assigned a thread from the thread pool. There are two problems with the Apache server. They are as follows: Blocking connections can lead to a domino effect, wherein one or more slowly resolved connections could avalanche into inaccessibility Apache had hard limits on the number of threads/workers you could utilize, irrespective of hardware constraints It's easy to see the opportunity here, at least in retrospect. A concurrent server that utilizes actors (Erlang), agents (Clojure), or goroutines (Go) seems to fit the bill perfectly. Concurrency does not solve the C10k problem in itself, but it absolutely provides a methodology to facilitate it. The most notable and visible example of an approach to the C10K problem today is Nginx, which was developed using concurrency patterns, widely available in C by 2002 to address—and ultimately solve—the C10k problem. Nginx, today, represents either the #2 or #3 web server in the world, depending on the source. Using concurrency to attack C10K There are two primary approaches to handle a large volume of concurrent requests. The first involves allocating threads per connection. This is what Apache (and a few others) do. On the one hand, allocating a thread to a connection makes a lot of sense—it's isolated, controllable via the application's and kernel's context switching, and can scale with increased hardware. One problem for Linux servers—on which the majority of the Web lives—is that each allocated thread reserves 8 MB of memory for its stack by default. This can (and should) be redefined, but this imposes a largely unattainable amount of memory required for a single server. Even if you set the default stack size to 1 MB, we're dealing with a minimum of 10 GB of memory just to handle the overhead. This is an extreme example that's unlikely to be a real issue for a couple of reasons: first, because you can dictate the maximum amount of resources available to each thread, and second, because you can just as easily load balance across a few servers and instances rather than add 10 GB to 80 GB of RAM. Even in a threaded server environment, we're fundamentally bound to the issue that can lead to performance decreases (to the point of a crash). First, let's look at a server with connections bound to threads (as shown in the following diagram), and visualize how this can lead to logjams and, eventually, crashes: This is obviously what we want to avoid. Any I/O, network, or external process that can impose some slowdown can bring about that avalanche effect we talked about, such that our available threads are taken (or backlogged) and incoming requests begin to stack up. We can spawn more threads in this model, but as mentioned earlier, there are potential risks there too, and even this will fail to mitigate the underlying problem. Taking another approach In an attempt to create our web server that can handle 10,000 concurrent connections, we'll obviously leverage our goroutine/channel mechanism to put an event loop in front of our content delivery to keep new channels recycled or created constantly. For this example, we'll assume we're building a corporate website and infrastructure for a rapidly expanding company. To do this, we'll need to be able to serve both static and dynamic content. The reason we want to introduce dynamic content is not just for the purposes of demonstration—we want to challenge ourselves to show 10,000 true concurrent connections even when a secondary process gets in the way. As always, we'll attempt to map our concurrency strategy directly to goroutines and channels. In a lot of other languages and applications, this is directly analogous to an event loop, and we'll approach it as such. Within our loop, we'll manage the available goroutines, expire or reuse completed ones, and spawn new ones where necessary. In this example visualization, we show how an event loop (and corresponding goroutines) can allow us to scale our connections without employing too many hard resources such as CPU threads or RAM: The most important step for us here is to manage that event loop. We'll want to create an open, infinite loop to manage the creation and expiration of our goroutines and respective channels. As part of this, we will also want to do some internal logging of what's happening, both for benchmarking and debugging our application. Building our C10K web server Our web server will be responsible for taking requests, routing them, and serving either flat files or dynamic files with templates parsed against a few different data sources. As mentioned earlier, if we exclusively serve flat files and remove much of the processing and network latency, we'd have a much easier time with handling 10,000 concurrent connections. Our goal is to approach as much of a real-world scenario as we can—very little of the Web operates on a single server in a static fashion. Most websites and applications utilize databases, CDNs (Content Delivery Networks), dynamic and uncached template parsing, and so on. We need to replicate them whenever possible. For the sake of simplicity, we'll separate our content by type and filter them through URL routing, as follows: /static/[request]: This will serve request.html directly /template/[request]: This will serve request.tpl after its been parsed through Go /dynamic/[request][number]: This will also serve request.tpl and parse it against a database source's record By doing this, we should get a better mixture of possible HTTP request types that could impede the ability to serve large numbers of users simultaneously, especially in a blocking web server environment. You can find Go's exceptional library to generate safe data-driven templating at http://golang.org/pkg/html/template/. By safe, we're largely referring to the ability to accept data and move it directly into templates without worrying about the sort of injection issues that are behind a large amount of malware and cross-site scripting. For the database source, we'll use MySQL here, but feel free to experiment with other databases if you're more comfortable with them. Like the html/template package, we're not going to put a lot of time into outlining MySQL and/or its variants. Benchmarking against a blocking web server It's only fair to add some starting benchmarks against a blocking web server first so that we can measure the effect of concurrent versus nonconcurrent architecture. For our starting benchmarks, we'll eschew any framework, and we'll go with our old stalwart, Apache. For the sake of completeness here, we'll be using an Intel i5 3GHz machine with 8 GB of RAM. While we'll benchmark our final product on Ubuntu, Windows, and OS X here, we'll focus on Ubuntu for our example. Our localhost domain will have three plain HTML files in /static, each trimmed to 80 KB. As we're not using a framework, we don't need to worry about raw dynamic requests, but only about static and dynamic requests in addition to data source requests. For all examples, we'll use a MySQL database (named master) with a table called articles that will contain 10,000 duplicate entries. Our structure is as follows: CREATE TABLE articles ( article_id INT NOT NULL AUTO_INCREMENT, article_title VARCHAR(128) NOT NULL, article_text VARCHAR(128) NOT NULL, PRIMARY KEY (article_id) ) With ID indexes ranging sequentially from 0-10,000, we'll be able to generate random number requests, but for now, we just want to see what kind of basic response we can get out of Apache serving static pages with this machine. For this test, we'll use Apache's ab tool and then gnuplot to sequentially map the request time as the number of concurrent requests and pages; we'll do this for our final product as well, but we'll also go through a few other benchmarking tools for it to get some better details.   Apache's AB comes with the Apache web server itself. You can read more about it at http://httpd.apache.org/docs/2.2/programs/ab.html. You can download it for Linux, Windows, OS X, and more from http://httpd.apache.org/download.cgi. The gnuplot utility is available for the same operating systems at http://www.gnuplot.info/. So, let's see how we did it. Have a look at the following graph: Ouch! Not even close. There are things we can do to tune the connections available (and respective threads/workers) within Apache, but this is not really our goal. Mostly, we want to know what happens with an out-of-the-box Apache server. In these benchmarks, we start to drop or refuse connections at around 800 concurrent connections. More troubling is that as these requests start stacking up, we see some that exceed 20 seconds or more. When this happens in a blocking server, each request behind it is queued; requests behind that are similarly queued and the entire thing starts to fall apart. Even if we cannot hit 10,000 concurrent connections, there's a lot of room for improvement. While a single server of any capacity is no longer the way we expect a web server environment to be designed, being able to squeeze as much performance as possible out of that server, ostensibly with our concurrent, event-driven approach, should be our goal. Handling requests The Gorilla toolkit certainly makes this easier, but we should also know how to intercept the functionality to impose our own custom handler. Here is a simple web router wherein we handle and direct requests using a custom http.Server struct, as shown in the following code: var routes []string type customRouter struct { } func (customRouter) ServeHTTP(rw http.ResponseWriter, r *http.Request) { fmt.Println(r.URL.Path); } func main() { var cr customRouter; server := &http.Server { Addr: ":9000", Handler:cr, ReadTimeout: 10 * time.Second, WriteTimeout: 10 * time.Second, MaxHeaderBytes: 1 << 20, } server.ListenAndServe() } Here, instead of using a built-in URL routing muxer and dispatcher, we're creating a custom server and custom handler type to accept URLs and route requests. This allows us to be a little more robust with our URL handling. In this case, we created a basic, empty struct called customRouter and passed it to our custom server creation call. We can add more elements to our customRouter type, but we really don't need to for this simple example. All we need to do is to be able to access the URLs and pass them along to a handler function. We'll have three: one for static content, one for dynamic content, and one for dynamic content from a database. Before we go so far though, we should probably see what our absolute barebones, HTTP server written in Go, does when presented with the same traffic that we sent Apache's way. By old school, we mean that the server will simply accept a request and pass along a static, flat file. You could do this using a custom router as we did earlier, taking requests, opening files, and then serving them, but Go provides a much simpler mode to handle this basic task in the http.FileServer method. So, to get some benchmarks for the most basic of Go servers against Apache, we'll utilize a simple FileServer and test it against a test.html page (which contains the same 80 KB file that we had with Apache). As our goal with this test is to improve our performance in serving flat and dynamic pages, the actual specs for the test suite are somewhat immaterial. We'd expect that while the metrics will not match from environment to environment, we should see a similar trajectory. That said, it's only fair we supply the environment used for these tests; in this case, we used a MacBook Air with a 1.4 GHz i5 processor and 4 GB of memory. First, we'll do this with our absolute best performance out of the box with Apache, which had 850 concurrent connections and 900 total requests. The results are certainly encouraging as compared to Apache. Neither of our test systems were tweaked much (Apache as installed and basic FileServer in Go), but Go's FileServer handles 1,000 concurrent connections without so much as a blip, with the slowest clocking in at 411 ms. Apache has made a great number of strides pertaining to concurrency and performance options in the last five years, but to get there does require a bit of tuning and testing. The intent of this experiment is not intended to denigrate Apache, which is well tested and established. Instead, it's to compare the out-of-the-box performance of the world's number 1 web server against what we can do with Go. To really get a baseline of what we can achieve in Go, let's see if Go's FileServer can hit 10,000 connections on a single, modest machine out of the box: ab -n 10500 -c 10000 -g test.csv http://localhost:8080/a.html We will get the following output: Success! Go's FileServer by itself will easily handle 10,000 concurrent connections, serving flat, static content. Of course, this is not the goal of this particular project—we'll be implementing real-world obstacles such as template parsing and database access, but this alone should show you the kind of starting point that Go provides for anyone who needs a responsive server that can handle a large quantity of basic web traffic. Routing requests So, let's take a step back and look again at routing our traffic through a traditional web server to include not only our static content, but also the dynamic content. We'll want to create three functions that will route traffic from our customRouter:serveStatic():: read function and serve a flat file serveRendered():, parse a template to display serveDynamic():, connect to MySQL, apply data to a struct, and parse a template. To take our requests and reroute, we'll change the ServeHTTP method for our customRouter struct to handle three regular expressions. For the sake of brevity and clarity, we'll only be returning data on our three possible requests. Anything else will be ignored. In a real-world scenario, we can take this approach to aggressively and proactively reject connections for requests we think are invalid. This would include spiders and nefarious bots and processes, which offer no real value as nonusers.
Read more
  • 0
  • 0
  • 9801
Modal Close icon
Modal Close icon