How-To Tutorials

(For more resources related to this topic, see here.) Markdown is a lightweight markup language that simplifies the workflow of web writers. It was created in 2004 by John Gruber with contributions and feedback from Aaron Swartz. Markdown was described by John Gruber as: "A text-to-HTML conversion tool for web writers. Markdown allows you to write using an easy-to-read, easy-to-write plain text format, then convert it to structurally valid XHTML (or HTML)." Markdown is two different things: A simple syntax to create documents in plain text A software tool written in Perl that converts the plain text formatting to HTML Markdown's formatting syntax was designed with simplicity and readability as a design goal. We add rich formatting to plain text without considering that we are writing using a markup language. The main features of Markdown Markdown is: Easy to use: Markdown has an extremely simple syntax that you can learn quickly Fast: Writing is much faster than with HTML, we can dramatically reduce the time we spend crafting HTML tags Clean: We can clearly read and write documents that are always translated into HTML without mistakes or errors Flexible: It is suitable for many things such as writing on the Internet, e-mails, creating presentations Portable: Documents are just plain text; we can edit Markdown with any basic text editor in any operating system Made for writers: Writers can focus on distraction-free writing Here, we can see a quick comparison of the same document between HTML and Markdown. This is the final result that we achieve in both cases: The following code is written in HTML: <h1>Markdown</h1><p>This a <strong>simple</strong> example of Markdown.</p><h2>Features:</h2><ul><li>Simple</li><li>Fast</li><li>Portable</li></ul><p>Check the <a href="http://daringfireball.net/projects/markdown/">official website</a>.</p> The following code is an equivalent document written in Markdown: # Markdown This a **simple** example of Markdown. ## Features: - Simple - Fast - Portable Check the [official website]. [official website]:http://daringfireball.net/projects/markdown/ summary In this article, we learned the basics of Markdown and got to know its features. We also saw how convenient Markdown is, thus proving the fact that it's made for writers. Resources for Article: Further resources on this subject: Generating Reports in Notebooks in RStudio [Article] Database, Active Record, and Model Tricks [Article] Formatting and Enhancing Your Moodle Materials: Part 1 [Article]

(For more resources related to this topic, see here.) Backbone.js is a lightweight JavaScript framework that is based on the Model-View-Controller (MVC) pattern and allows developers to create single-page web applications. With Backbone, it is possible to update a web page quickly using the REST approach with a minimal amount of data transferred between a client and a server. Backbone.js is becoming more popular day by day and is being used on a large scale for web applications and IT startups; some of them are as follows: Groupon Now!: The team decided that their first product would be AJAX-heavy but should still be linkable and shareable. Though they were completely new to Backbone, they found that its learning curve was incredibly quick, so they were able to deliver the working product in just two weeks. Foursquare: This used the Backbone.js library to create model classes for the entities in foursquare (for example, venues, check-ins, and users). They found that Backbone's model classes provide a simple and light-weight mechanism to capture an object's data and state, complete with the semantics of a classical inheritance. LinkedIn mobile: This used Backbone.js to create its next-generation HTML5 mobile web app. Backbone made it easy to keep the app modular, organized, and extensible, so it was possible to program the complexities of LinkedIn's user experience. Moreover, they are using the same code base in their mobile applications for iOS and Android platforms. WordPress.com: This is a SaaS version of Wordpress and uses Backbone.js models, collections, and views in its notification system, and is integrating Backbone.js into the Stats tab and into other features throughout the home page. Airbnb: This is a community marketplace for users to list, discover, and book unique spaces around the world. Its development team has used Backbone in many latest products. Recently, they rebuilt a mobile website with Backbone.js and Node.js tied together with a library named Rendr. You can visit the following links to get acquainted with other usage examples of Backbone.js: http://backbonejs.org/#examples Backbone.js was started by Jeremy Ashkenas from DocumentCloud in 2010 and is now being used and improved by lots of developers all over the world using Git, the distributed version control system. In this article, we are going to provide some practical examples of how to use Backbone.js, and we will structure a design for a program named Billing Application by following the MVC and Backbone pattern. Reading this article is especially useful if you are new to developing with Backbone.js. Designing an application with the MVC pattern MVC is a design pattern that is widely used in user-facing software, such as web applications. It is intended for splitting data and representing it in a way that makes it convenient for user interaction. To understand what it does, understand the following: Model: This contains data and provides business logic used to run the application View: This presents the model to the user Controller: This reacts to user input by updating the model and the view There could be some differences in the MVC implementation, but in general it conforms to the following scheme: Worldwide practice shows that the use of the MVC pattern provides various benefits to the developer: Following the separation of the concerned paradigm, which splits an application into independent parts, it is easier to modify or replace It achieves code reusability by rendering a model in different views without the need to implement model functionality in each view It requires less training and has a quicker startup time for the new developers within an organization To have a better understanding of the MVC pattern, we are going to design a Billing Application. We will refer to this design throughout the book when we are learning specific topics. Our Billing Application will allow users to generate invoices, manage them, and send them to clients. According to the worldwide practice, the invoice should contain a reference number, date, information about the buyer and seller, bank account details, a list of provided products or services, and an invoice sum. Let's have a look at the following screenshot to understand how an invoice appears: How to do it... Let's follow the ensuing steps to design an MVC structure for the Billing Application: Let's write down a list of functional requirements for this application. We assume that the end user may want to be able to do the following: Generate an invoice E-mail the invoice to the buyer Print the invoice See a list of existing invoices Manage invoices (create, read, update, and delete) Update an invoice status (draft, issued, paid, and canceled) View a yearly income graph and other reports To simplify the process of creating multiple invoices, the user may want to manage information about buyers and his personal details in the specific part of the application before he/she creates an invoice. So, our application should provide additional functionalities to the end user, such as the following: The ability to see a list of buyers and use it when generating an invoice The ability to manage buyers (create, read, update, and delete) The ability to see a list of bank accounts and use it when generating an invoice The ability to manage his/her own bank accounts (create, read, update, and delete) The ability to edit personal details and use them when generating an invoice Of course, we may want to have more functions, but this is enough for demonstrating how to design an application using the MVC pattern. Next, we architect an application using the MVC pattern. After we have defined the features of our application, we need to understand what is more related to the model (business logic) and what is more related to the view (presentation). Let's split the functionality into several parts. Then, we learn how to define models. Models present data and provide data-specific business logic. Models can be related to each other. In our case, they are as follows: InvoiceModel InvoiceItemModel BuyerModel SellerModel BankAccountModel Then, will define collections of models. Our application allows users to operate on a number of models, so they need to be organized into a special iterable object named Collection. We need the following collections: InvoiceCollection InvoiceItemCollection BuyerCollection BankAccountCollection Next, we define views. Views present a model or a collection to the application user. A single model or collection can be rendered to be used by multiple views. The views that we need in our application are as follows: EditInvoiceFormView InvoicePageView InvoiceListView PrintInvoicePageView EmailInvoiceFormView YearlyIncomeGraphView EditBuyerFormView BuyerPageView BuyerListView EditBankAccountFormView BankAccountPageView BankAccountListView EditSellerInfoFormView ViewSellectInfoPageView ConfirmationDialogView Finally, we define a controller. A controller allows users to interact with an application. In MVC, each view can have a different controller that is used to do following: Map a URL to a specific view Fetch models from a server Show and hide views Handle user input Defining business logic with models and collections Now, it is time to design business logic for the Billing Application using the MVC and OOP approaches. In this recipe, we are going to define an internal structure for our application with model and collection objects. Although a model represents a single object, a collection is a set of models that can be iterated, filtered, and sorted. Relations between models and collections in the Billing Application conform to the following scheme: How to do it... For each model, we are going to create two tables: one for properties and another for methods: We define BuyerModel properties. Name Type Required Unique id Integer Yes Yes name Text Yes   address Text Yes   phoneNumber Text No   Then, we define SellerModel properties. Name Type Required Unique id Integer Yes Yes name Text Yes   address Text Yes   phoneNumber Text No   taxDetails Text Yes   After this, we define BankAccountModel properties. Name Type Required Unique id Integer Yes Yes beneficiary Text Yes   beneficiaryAccount Text Yes   bank Text No   SWIFT Text Yes   specialInstructions Text No   We define InvoiceItemModel properties. Name Arguments Return Type Unique calculateAmount - Decimal   Next, we define InvoiceItemModel methods. We don't need to store the item amount in the model, because it always depends on the price and the quantity, so it can be calculated. Name Type Required Unique id Integer Yes Yes deliveryDate Date Yes   description Text Yes   price Decimal Yes   quantity Decimal Yes   Now, we define InvoiceModel properties. Name Type Required Unique id Integer Yes Yes referenceNumber Text Yes   date Date Yes   bankAccount Reference Yes   items Collection Yes   comments Text No   status Integer Yes   We define InvoiceModel methods. The invoice amount can easily be calculated as the sum of invoice item amounts. Name Arguments Return Type Unique calculateAmount   Decimal   Finally, we define collections. In our case, they are InvoiceCollection, InvoiceItemCollection, BuyerCollection, and BankAccountCollection. They are used to store models of an appropriate type and provide some methods to add/remove models to/from the collections. How it works... Models in Backbone.js are implemented by extending Backbone.Model, and collections are made by extending Backbone.Collection. To implement relations between models and collections, we can use special Backbone extensions. To learn more about object properties, methods, and OOP programming in JavaScript, you can refer to the following resource: https://developer.mozilla.org/en-US/docs/JavaScript/Introduction_to_Object-Oriented_JavaScript Modeling an application's behavior with views and a router Unlike traditional MVC frameworks, Backbone does not provide any distinct object that implements controller functionality. Instead, the controller is diffused between Backbone.Router and Backbone. View and the following is done: A router handles URL changes and delegates application flow to a view. Typically, the router fetches a model from the storage asynchronously. When the model is fetched, it triggers a view update. A view listens to DOM events and either updates a model or navigates an application through a router. The following diagram shows a typical workflow in a Backbone application: How to do it... Let's follow the ensuing steps to understand how to define basic views and a router in our application: First, we need to create wireframes for an application. Let's draw a couple of wireframes in this recipe: The Edit Invoice page allows users to select a buyer, to select the seller's bank account from the lists, to enter the invoice's date and a reference number, and to build a table of shipped products and services. The Preview Invoice page shows how the final invoice will be seen by a buyer. This display should render all the information we have entered in the Edit Invoice form. Buyer and seller information can be looked up in the application storage. The user has the option to either go back to the Edit display or save this invoice. Then, we will define view objects. According to the previous wireframes, we need to have two main views: EditInvoiceFormView and PreviewInvoicePageView. These views will operate with InvoiceModel; it refers to other objects, such as BankAccountModel and InvoiceItemCollection. Now, we will split views into subviews. For each item in the Products or Services table, we may want to recalculate the Amount field depending on what the user enters in the Price and Quantity fields. The first way to do this is to re-render the entire view when the user changes the value in the table; however, it is not an efficient way, and it takes a significant amount of computer power to do this. We don't need to re-render the entire view if we want to update a small part of it. It is better to split the big view into different, independent pieces, such as subviews, that are able to render only a specific part of the big view. In our case, we can have the following views: As we can see, EditInvoiceItemTableView and PreviewInvoiceItemTableView render InvoiceItemCollection with the help of the additional views EditInvoiceItemView and PreviewInvoiceItemView that render InvoiceItemModel. Such separation allows us to re-render an item inside a collection when it is changed. Finally, we will define URL paths that will be associated with a corresponding view. In our case, we can have several URLs to show different views, for example: /invoice/add /invoice/:id/edit /invoice/:id/preview Here, we assume that the Edit Invoice view can be used for either creating a new invoice or editing an existing one. In the router implementation, we can load this view and show it on specific URLs. How it works... The Backbone.View object can be extended to create our own view that will render model data. In a view, we can define handlers to user actions, such as data input and keyboard or mouse events. In the application, we can have a single Backbone.Router object that allows users to navigate through an application by changing the URL in the address bar of the browser. The router object contains a list of available URLs and callbacks. In a callback function, we can trigger the rendering of a specific view associated with a URL. If we want a user to be able to jump from one view to another, we may want him/her to either click on regular HTML links associated with a view or navigate to an application programmatically.

(For more resources related to this topic, see here.) You can find this instruction in the OGlobalConfiguration.java file in the autoConfig() method. Furthermore, you can enable/disable level 1 cache, level 2 cache, or both. You can also set the number of records that will be stored in each level as follows: cache.level1.size: This sets the number of records to be stored in the level 1 caches (default -1, no limit) cache.level2.size: This sets the number of records to be stored in the level 2 cache (default -1, no limit) cache.level1.enabled: This is a boolean value, it enables/disables the level 1 cache (default, true) cache.level2.enabled: This is a boolean value, it enables/disables the level 2 cache (default, true) Mapping files OrientDB uses NIO to map data files in memory. However, you can change the way this mapping is performed. This is achieved by modifying the file access strategy. Mode 0: It uses the memory mapping for all the operations. Mode 1 (default): It uses the memory mapping, but new reads are performed only if there is enough memory, otherwise the regular Java NIO file read/write is used. Mode 2: It uses the memory mapping only if the data has been previously loaded. Mode 3: It uses memory mapping until there is space available, then use regular JAVA NIO file read/write. Mode 4: It disables all the memory mapping techniques. To set the strategy mode, you must use the file.mmap.strategy configuration property. Connections When you have to connect with a remote database you have some options to improve your application performance. You can use the connection pools, and define the timeout value to acquire a new connection. The pool has two attributes: minPool: It is the minimum number of opened connections maxPool: It is the maximum number of opened connections When the first connection is requested to the pool, a number of connections corresponding to the minPool attribute are opened against the server. If a thread requires a new connection, the requests are satisfied by using a connection from the pool. If all the connections are busy, a new one is created until the value of maxPool is reached. Then the thread will wait, so that a connection is freed. Minimum and maximum connections are defined by using the client.channel.minPool (default value 1) and client.channel.maxPool (default value 5) properties. However, you can override these values in the client code by using the setProperty() method of the connection class. For example: database = new ODatabaseDocumentTx("remote:localhost/demo");database.setProperty("minPool", 10);database.setProperty("maxPool", 50);database.open("admin", "admin"); You can also change the connection timeout values. In fact, you may experience some problem, if there are network latencies or if some server-side operations require more time to be performed. Generally these kinds of problems are shown in the logfile with warnings: WARNING: Connection re-acquired transparently after XXXms and Yretries: no errors will be thrown at application level You can try to change the network.lockTimeout and the network.socketTimeout values. The first one indicates the timeout in milliseconds to acquire a lock against a channel (default is 15000), the second one indicates the TCP/IP socket timeout in milliseconds (default is 10000). There are some other properties you can try to modify to resolve network issues. These are as follows: network.socketBufferSize: This is the TCP/IP socket buffer size in bytes (default 32 KB) network.retry: This indicates the number of retries a client should do to establish a connection against a server (default is 5) network.retryDelay: This indicates the number of milliseconds a client will wait before retrying to establish a new connection (default is 500) Transactions If your primary objective is the performance, avoid using transactions. However, if it is very important for you to have transactions to group operations, you can increase overall performance by disabling the transaction log. To do so just set the tx.useLog property to false. If you disable the transaction log, OrientDB cannot rollback operations in case JVM crashes. Other transaction parameters are as follows: tx.log.synch: It is a Boolean value. If set, OrientDB executes a synch against the filesystem for each transaction log entry. This slows down the transactions, but provides reliability on non- reliable devices. Default value is false. tx.commit.synch: It is a Boolean value. If set, it performs a storage synch after a commit. Default value is true. Massive insertions If you want to do a massive insertion, there are some tricks to speed up the operation. First of all, do it via Java API. This is the fastest way to communicate with OrientDB. Second, instruct the server about your intention: db.declareIntent( new OIntentMassiveInsert() );//your code here....db.declareIntent( null ); Here db is an opened database connection. By declaring the OIntentMassiveInsert() intent, you are instructing OrientDB to reconfigure itself (that is, it applies a set of preconfigured configuration values) because a massive insert operation will begin. During the massive insert, avoid creating a new ODocument instance for each record to insert. On the contrary, just create an instance the first time, and then clean it using the reset() method: ODocument doc = new ODocument();for(int i=0; i< 9999999; i++){doc.reset(); //here you will reset the ODocument instancedoc.setClassName("Author");doc.field("id", i);doc.field("name", "John");doc.save();} This trick works only in a non-transactional context. Finally, avoid transactions if you can. If you are using a graph database and you have to perform a massive insertion of vertices, you can still reset just one vertex: ODocument doc = db.createVertex();...doc.reset();... Moreover, since a graph database caches the most used elements, you may disable this: db.setRetainObjects(false); Datafile fragmentation Each time a record is updated or deleted, a hole is created in the datafiles structure. OrientDB tracks these holes and tries to reuse them. However, many updates and deletes can cause a fragmentation of datafiles, just like in a filesystem. To limit this problem, it is suggested to set the oversize attribute of the classes you create. The oversize attribute is used to allocate more space for records once they are created, so as to avoid defragmentation upon updates. The oversize attribute is a multiplying factor where 1.0 or 0 means no oversize. The default values are 0 for document, and 2 for vertices. OrientDB has a defrag algorithm that starts automatically when certain conditions are verified. You can set some of these conditions by using the following configuration parameter: file.defrag.holeMaxDistance: It defines the maximum distance in bytes between two holes that triggers the defrag procedure. The default is 32 KB, -1 means dynamic size. The dynamic size is computed in the ODataLocal class in the getCloserHole() method, as Math.max(32768 * (int) (size / 10000000), 32768), where size is the current size of the file. The profiler OrientDB has an embedded profiler that you can use to analyze the behavior of the server. The configuration parameters that act on the profiler are as follows: profiler.enabled: This is a boolean value (enable/disable the profiler), the default value is false. profiler.autoDump.interval: It is the number of seconds between profiler dump. The default value is 0, which means no dump. profiler.autoDump.reset: This is a boolean value, reset the profile at every dump. The default is true. The dump is a JSON string structured in sections. The first one is a huge collection of information gathered at runtime related to the configuration and resources used by each object in the database. The keys are structured as follows: db.<db-name>: They are database-related metrics db.<db-name>.cache: They are metrics about databases' caching db.<db-name>.data: They are metrics about databases' datafiles, mainly data holes db.<db-name>.index: They are metrics about databases' indexes system.disk: They are filesystem-related metrics system.memory: They are RAM-related metrics system.config.cpus: They are the number of the cores process.network: They are network metrics process.runtime: They provide process runtime information and metrics server.connections.actives: They are number of active connections The second part of the dump is a collection of chronos. A chrono is a log of an operation, for example, a create operation, an update operation, and so on. Each chrono has the following attributes: last: It is the last time recorded min: It is the minimum time recorded max: It is the maximum time recorded average: It is the average time recorded total: It is the total time recorded entries: It is the number of times the specific metric has been recorded Finally, there are sections about many counters. Query tips In the following paragraphs some useful information on how to optimize the queries execution is given. The explain command You can see how OrientDB accesses the data by using the explain command in the console. To use this command simply write explain followed by the select statement: orientdb> explain select from Posts A set of key-value pairs are returned. Keys mean the following: resultType: It is the type of the returned resultset. It can be collection, document, or number. resultSize: It is the number of records retrieved if the resultType is collection. recordReads: It is the number of records read from datafiles. involvedIndexes: They are the indices involved in the query. indexReads: It is the number of records read from the indices. documentReads: They are the documents read from the datafiles. This number could be different from recordReads, because in a scanned cluster there can be different kinds of records. documentAnalyzedCompatibleClass: They are the documents analyzed belonging to the class requested by the query. This number could be different from documentReads, because a cluster may contain several different classes. elapsed: This time is measured in nanoseconds, it is the time elapsed to execute the statement. As you can see, OrientDB can use indices to speed up the reads. Indexes You can define indexes as we do in a relational database using the create index statement or via Java API using the createIndex() method of the OClass class: create index <class>.<property> [unique|notunique|fulltext] [field type] Or for composite index (an index on more than one property): create index <index_name> on <class> (<field1>,<field2>)[unique|notunique|fulltext] If you create a composite index, OrientDB will use it also when in a where clause you don't specify a criteria against all the indexed fields. So you can avoid this to build an index for each field you use in the queries if you have already built a composite one. This is the case of a partial match search and further information about it can be found in the OrientDB wiki at https://github.com/nuvolabase/orientdb/wiki/Indexes#partial-match-search. Generally, the indexes don't work with the like operator. If you want to perform the following query: select from Authors where name like 'j%' And you want use an index, you must define on the field name a FULLTEXT index. FULLTEXT indices permit to index string fields. However keep in mind that indices slow down the insert, update, and delete operations. Summary In this article we have seen some strategies that try to optimize both the OrientDB server installation and queries. Resources for Article: Further resources on this subject: Comparative Study of NoSQL Products [Article] Connecting to Microsoft SQL Server Compact 3.5 with Visual Studio [Article] Microsoft SQL Azure Tools [Article]

(For more resources related to this topic, see here.) Speeding account entry with account aliases As organizations grow, the chart of accounts tends to grow larger and more complex as well. Companies want to segment their business by departments, locations, or divisions; all of this means that more and more accounts get added to the chart and, as the chart of accounts grows, it becomes more difficult to select the right account. Dynamics GP provides the account alias feature as a way to quickly select the right account. Account aliases provide a way to create shortcuts to specific accounts that can dramatically speed up the process of selecting the correct account. We'll look at how that works in this recipe. As organizations grow, the chart of accounts tends to grow larger and more complex as well. Companies want to segment their business by departments, locations, or divisions; all of this means that more and more accounts get added to the chart and, as the chart of accounts grows, it becomes more difficult to select the right account. Dynamics GP provides the account alias feature as a way to quickly select the right account. Account aliases provide a way to create shortcuts to specific accounts that can dramatically speed up the process of selecting the correct account. We'll look at how that works in this recipe. Getting ready Setting up account aliases requires a user with access to the Account Maintenance window. To get to this window, perform the following steps: Select Financial from the navigation pane on the left. Click on Accounts on the Financial area page under Cards. This will open the Account Maintenance window. Click on the Lookup button (magnifying glass) next to the Account Number or use the keyboard shortcut Ctrl + Q. Find and select account 000-2100-00. In the middle of the Account Maintenance window you can find the Account Alias field. Enter AP in the Alias field. This associates the letters AP with the account's selected payable account. This means that the user now only has to enter AP instead of the full account number to use the account's payable account. How to do it... Once aliases have been set up, let's see how the user can quickly select an account using an alias. To demonstrate how this works, click on Financial in the navigation pane on the left. Select Transaction Entry from the Financial area page under Transactions. In the Transaction Entry window, select the top line in the grid area on the lower half of the window. Click on the expansion button (represented by a blue arrow) next to the Account heading to open the Account Entry window. In the Alias field, type AP and press Enter. The Account Alias window will close and the account represented by the alias will appear in the Transaction Entry window. How it works... Account aliases provide quick shortcuts for account entry. Keeping them short and obvious makes them easy to use. Aliases are less useful if users have to think about them. Limiting them to the most commonly used accounts makes them more useful. Most users don't mind occasionally looking up the odd account, but they shouldn't have to memorize long account strings for regularly used account numbers. It's counter-productive to put an alias on every account since that would make finding the right alias as difficult as finding the right account number. The setup process should be performed on the most commonly used accounts to provide easy access. Cleaning account lookups by removing accounts from lookups A consequence of company growth is that the chart of accounts grows and the account lookups can get clogged up by the number of accounts in the system. While the General Ledger will stop showing an account in a lookup when the account is made inactive, other modules will continue to show these inactive codes. However, Dynamics GP does contain a feature that can be used to remove inactive codes from lookups; this same feature can also be used to remove codes from lookups in series where the code should not be used, such as a sales code in the purchasing or inventory series. How to do it... Here we will see how to remove inactive accounts from lookups: Open Financial from the navigation pane on the left. In the main area page, under Cards , select Account. Enter, or do a lookup for, the account to be made inactive and removed from the lookups. Check the Inactive checkbox. Press and hold the Ctrl key and click on each of the lines in the Include in Lookup list. Click on Save to commit the changes. Next time a lookup is done in any of the now deselected modules, the account will not be included in the list. If the account is to be included in lookups in some modules but not in others, simply leave selected the modules in which the account should be included. How it works... Accounts will only be shown in lookups when the series is selected in the Include in Lookup list. For series other than General Ledger, simply marking an account inactive is not enough to remove it from the lookup although the code can't be used when the account is inactive. Gaining visibility by using horizontal scroll arrows A consequence of company growth is that not only does the chart of accounts grow larger and less intuitive, but also the actual lengths of account numbers tend to grow longer as well. Companies want to be able to report by account, department, location, and so on, which results in a proliferation of segments added to the main account number and can create very long accounts. Dynamics GP can accommodate an account number as long as 66 characters. The longest I've seen used in practice was of 27 characters and even that was unwieldy. Most users only need a portion of that length for their day-to-day work. This presents a problem because very long account numbers won't fit into the account number field on most screens. For this recipe, we'll look at how Dynamics GP provides a solution to this in the User Preferences area. How to do it... Here we'll see how to increase the visibility of long account numbers. In the navigation pane on the left, select Home. Click on User Preferences on the shortcut bar. Check the box for Horizontal Scroll Arrows. This turns on the functionality that allows users to scroll horizontally within the Account field, allowing them to see the full account number. How it works... Once horizontal scroll arrows are activated, small arrows appear at the left and right side of the Account field, letting users scroll right and left to see the full account number. There's more... Horizontal scroll arrows are implemented on a per user basis, meaning each user has to turn this on individually. Administrators can make this active for all users with a SQL script. Additionally, for companies using alphanumeric characters in their chart of accounts, wide letters such as M or W are often difficult to see. There is also an option to increase the visible width of a particular segment. Activating horizontal scroll arrows for all users Horizontal scroll arrows are activated by the user. However, an administrator can turn this feature on for all users in all companies by running the following SQL script against the Dynamics database: Update SY01400 Set HSCRLARW=1 Widening segments for better visibility When companies use alphanumeric characters in their chart of accounts, wide letters, such as M or W, are often cut off. Horizontal scroll arrows don't help because the problem is that the segment field is too narrow, not the entire account field. To resolve this problem, Dynamics GP provides an option to widen the segment fields as well. In the navigation pane, click on Administration and select Account Format. For each segment that needs to be wider, select the Display Width column and change it from Standard to Expansion 1 , Expansion 2, or Expansion 3 to widen the field; Expansion 3 represents the widest option. Companies using only numbers in their chart of accounts won't need to widen the segment field, but firms that include letters as part of their charts will need to increase the width. The following is a list of the expansion options and the letters they are designed to accommodate: Expansion 1: A, B, E, K, P, S, V, X, Y Expansion 2: C, D, G, H, M, N, O, Q, R, U Expansion 3: W Streamlining payables processing by prioritizing vendors Management of vendor payments is a critical activity for any firm; it's even more critical in difficult economic times. Companies need to understand and control payments and a key component of this is prioritizing vendors. Every firm has both critical and expendable vendors. Paying critical vendors on time is a key business driver. For example, a newspaper that doesn't pay its newsprint supplier won't be in business long. However, they can safely delay payments to their janitorial vendor without worrying about going under. Dynamics GP provides a mechanism to prioritize vendors and apply those priorities when selecting which checks to print. That is the focus of this recipe. Getting ready Setting this up first requires that the company figures out who the priority vendors are. That part is beyond the scope of this book. The Vendor Priority field in Dynamics GP is a three-character field, but users shouldn't be seduced by the possibilities of three characters. A best practice is to keep the priorities simple by using 1, 2, 3 or A, B, C. Anything more complicated than that tends to confuse users and actually makes it harder to prioritize vendors. Once the vendor priorities have been determined, the priority needs to be set in Dynamics GP. Attaching a priority to a vendor is the first step. To do that, perform the following steps: Select Purchasing from the navigation pane. In the Purchasing area page, under Cards, click on Vendor Maintenance. Once the Vendor Maintenance window opens, click on the Lookup button (magnifying glass) next to Vendor ID. Select a vendor and click on OK. Once the vendor information is populated, click on the Options button. This opens the Vendor Maintenance Options screen. In the center left we find the Payment Priority field. Enter 1 in Payment Priority and click on Save. How to do it... Now that a vendor has been set up with a priority, let's see how to apply that information when selecting checks to print. To use vendor priorities to select invoices for payment, click on Select Checks from the Purchasing area page. In the Select Payables Checks window, enter CHEQUES to name the check batch. Press Tab to move off of the Batch ID field and click on Add to add the batch. Pick a Checkbook ID and click on Save to save the batch. In the Select By field, click on the drop-down box and select Payment Priority. Enter 1 in both the From and To boxes. Click on the Insert button to lock in Payment Priority as an option. Click on Build Batch at the top. If there are any transactions where the vendor is set to a priority of 1, this will populate a batch of checks based on the vendor priority. How it works... Since priority is one of the built-in options for selecting checks, it's easy to ensure that high priority vendors get selected to be paid first. All of this is easily accomplished with basic Dynamics GP functionality that most people miss. Getting clarity with user-defined fields Throughout Dynamics GP, maintenance cards typically include at least two user-defined fields. User-defined fields can be renamed in the setup screen for the related module. This provides a great mechanism to add in special information. We'll take a look at a typical use of a user-defined field in this recipe. How to do it... For our example, we'll look at using a user-defined field to rename the User-Defined 1 field to Region in Customer Maintenance . From the navigation pane, select Sales . In the Sales page, click on Setup and then click on Receivables and finally Options . In the User-Defined 1 field, type Region and click on OK to close each window. Back on the Sales page, click on Customer under the Cards area. On the bottom left above User-Defined 2 is the newly named Region field ready to be filled in. How it works... Changing the field name only changes the display field; it doesn't change the underlying field name in the database. SmartLists are smart enough to show the new name. In our example, the description for Region would appear in a SmartList, not User-Defined 1 . User-defined fields like this are present for customers, vendors, accounts, sales orders, fixed assets, inventory items, and purchase receipts among others. Each of these can be renamed in their respective setup screens. There's more... All user-defined fields are not the same; some have special features. Special User-Defined 1 features User-Defined 1 has special features inside of Dynamics GP. Most of the -n reports inside the Dynamics GP allow sorting and selection by the User-Defined 1 field. These options aren't provided for User-Defined 2 . Consequently, administrators should carefully consider what information belongs in User-Defined 1 before changing its name since the effects of this selection will be felt throughout the system. Company Setup user-defined fields In the Company Setup window, there are two user-defined fields at the top right and there is no option in Dynamics GP to rename these fields. The Company Setup window is accessed by clicking on Administration in the navigation pane, and then clicking on Company under the Setup and Company headers. Expanded user-defined fields Certain areas such as fixed assets, inventory items, and purchase receipts have more complex types of user-defined fields that can include dates, list selections, and currency. Developing connections with Internet user-defined fields Dynamics GP provides a built-in set of Internet fields for users to enter information such as web pages, e-mail addresses, and FTP sites. What many people don't know is that these are actually user-defined fields and can be changed by an administrator that allows firms to add a second e-mail address or remove the FTP link if they want to. In this recipe, we'll look at how to customize these fields. It is important to keep in mind when setting up Internet user-defined fields that these settings affect all the Internet user-defined field names attached to address ID's assigned to a company, customers, employees, items, salespeople, and vendors. How to do it... Customizing the Internet user-defined fields is easy, so let's take a look at how to do it. For our example, we'll add the social networking service Twitter as a new label. Select Administration from the navigation pane. Under the Setup and Company headers in the Administration area page, pick Company . Click on the Internet User Defined button and change the Image description next to Label 4 to Twitter. Click on OK . Back on the Company Setup screen, click on the blue italic letter "I" to the right of the Address ID to open the Internet Information window. In the Twitter field, type http://www.twitter.com/azurecurve. Click on the link associated with the Twitter field on the left. This opens a web browser and navigates to my Twitter account so that you can follow me. Click on Save to update the record. How it works... The secret to Internet user-defined fields is how the data is entered. Internet items use a prefix in the field to identify the type of Internet transaction to be used with the link: http:// is used for web pages, Mailto:// for e-mail, and ftp:// for FTP sites. These prefixes tell Dynamics GP what to do when a link is clicked, and if no prefix is entered, Dynamics GP will try to figure out what to do and may or may not succeed. If http://www.microsoft.com is entered in a field, the link to the left will start the default browser and open the Microsoft web page. If http:// is not included, but www is, GP figures out that it should open a web page. Just putting in Microsoft.com isn't enough for GP to understand that the link corresponds to a web page. Similarly, if a user enters <mailto://mpolino@gmail.com> and clicks the link, the default e-mail client opens up and is ready to send an e-mail to me. If no prefix is used with an e-mail address, GP will respond with a "File Not Found" error when the link is clicked. It's not smart enough to know that the @ symbol means that this is an e-mail account. Using a prefix in the Internet user-defined fields explicitly defines how this link should work and provides the most consistency to users. There's more... Some Internet user-defined fields look special but aren't and some really are special. Login and password By default, the Internet user-defined Label 5 field is named Login and the Label 6 field is named Password . These fields are supposed to represent the login and password for one of the associated web pages or FTP sites. However, these fields are not encrypted and there is limited security control, so it may not be appropriate to leave these fields named Login and Password if a company doesn't want users entering that information here. Label 7 and Label 8 Label 7 and Label 8 in Internet user defined fields are special fields that allow a user to look up and attach links to files located on the computer or the network. Clicking on the label name on the left opens the associated file. Other fields can accept files and these fields can still accept a prefix and link or free form text, but their special ability to look up file names mean that administrators should consider reserving them for file attachments.

(For more resources related to this topic, see here.) Getting ready We will work from the app.py file from the sched directory and the models.py file. How to do it... Flask provides a session object, which behaves like a Python dictionary, and persists automatically across requests. You can, in your Flask application code: from flask import session# ... in a request ...session['spam'] = 'eggs'# ... in another request ...spam = session.get('spam') # 'eggs' Flask-Login provides a simple means to track a user in Flask's session. Update requirements.txt: FlaskFlask-LoginFlask-ScriptFlask-SQLAlchemyWTForms Then: $ pip install -r requirements.txt We can then load Flask-Login into sched's request handling, in app.py: from flask.ext.login import LoginManager, current_userfrom flask.ext.login import login_user, logout_userfrom sched.models import User# Use Flask-Login to track current user in Flask's session.login_manager = LoginManager()login_manager.setup_app(app)login_manager.login_view = 'login'@login_manager.user_loaderdef load_user(user_id):"""Flask-Login hook to load a User instance from ID."""return db.session.query(User).get(user_id) Flask-Login requires four methods on the User object, inside class User in models.py: def get_id(self):return str(self.id)def is_active(self):return Truedef is_anonymous(self):return Falsedef is_authenticated(self):return True Flask-Login provides a UserMixin (flask.ext.login.UserMixin) if you prefer to use its default implementation. We then provide routes to log the user in when authenticated and log out. In app.py: @app.route('/login/', methods=['GET', 'POST']) def login(): if current_user.is_authenticated(): return redirect(url_for('appointment_list')) form = LoginForm(request.form) error = None if request.method == 'POST' and form.validate(): email = form.username.data.lower().strip() password = form.password.data.lower().strip() user, authenticated = User.authenticate(db.session.query, email, password) if authenticated: login_user(user) return redirect(url_for('appointment_list')) else: error = 'Incorrect username or password.' return render_template('user/login.html', form=form, error=error) @app.route('/logout/') def logout(): logout_user() return redirect(url_for('login')) We then decorate every view function that requires a valid user, in app.py: from flask.ext.login import login_required@app.route('/appointments/')@login_requireddef appointment_list():# ... How it works... On login_user, Flask-Login gets the user object's ID from User.get_id and stores it in Flask's session. Flask-Login then sets a before_request handler to load the user instance into the current_user object, using the load_user hook we provide. The logout_user function then removes the relevant bits from the session. If no user is logged in, then current_user will provide an anonymous user object which results in current_user.is_anonymous() returning True and current_user. is_authenticated() returning False, which allows application and template code to base logic on whether the user is valid. (Flask-Login puts current_user into all template contexts.) You can use User.is_active to make user accounts invalid without actually deleting them, by returning False as appropriate. View functions decorated with login_required will redirect the user to the login view if the current user is not authenticated, without calling the decorated function. There's more... Flask's session supports display of messages and protection against request forgery. Flashing messages When you want to display a simple message to indicate a successful operation or a failure quickly, you can use Flask's flash messaging, which loads the message into the session until it is retrieved. In application code, inside request handling code: from flask import flashflash('Sucessfully did that thing.', 'success') In template code, where you can use the 'success' category for conditional display: {% for cat, m in get_flashed_messages(with_categories=true) %}<div class="alert">{{ m }}</div>{% endfor %} Cross-site request forgery protection Malicious web code will attempt to forge data-altering requests for other web services. To protect against forgery, you can load a randomized token into the session and into the HTML form, and reject the request when the two do not match. This is provided in the Flask-SeaSurf extension, pythonhosted.org/Flask-SeaSurf/ or the Flask-WTF extension (which integrates WTForms), pythonhosted.org/Flask-ETF/. Summary This article explained how to keep users logged in for on-going requests after authentication. It shed light on how Flask provides a session object, which behaves like a Python dictionary, and persists automatically across requests. It also spoke about coding in Flask application. We got acquainted with flashing messages and cross-site request forgery protection. Resources for Article: Further resources on this subject: Python Testing: Installing the Robot Framework [Article] Getting Started with Spring Python [Article] Creating Skeleton Apps with Coily in Spring Python [Article]

(For more resources related to this topic, see here.) Understanding IT Service Management (ITSM) The success of ITSM lies in putting the customer first. ITSM suggests designing all processes to provide value to customers by facilitating the outcomes they want, without the ownership of specific costs and risks. This quality service is provided through a set of the organization's own resources and capabilities. The capabilities of an IT service organization generally lie with its people, process, or technology. While people and technology could be found in the market, the organizational processes need to be defined, developed, and often customized within the organization. The processes mature with the organization, and hence need to be given extra focus. Release Management, Incident Management, and so on, are some of the commonly heard ITSM processes. It's easy to confuse these with functions, which as per ITIL, has a different meaning associated with it. Many of us do not associate different meanings for many similar terms. Here are some examples: Incident Management versus Problem Management Change Management versus Release Management Service Level Agreement (SLA) versus Operational Level Agreement (OLA) Service Portfolio versus Service Catalog This book will strive to bring out the fine differences between such terms, as and when we formally introduce them. This should make the concepts clear while avoiding any confusion. So, let us first see the difference between a process and a function. Differentiating between process and function A process is simply a structured set of activity designed to accomplish a specific objective. It takes one or more defined inputs and turns them into defined outputs. Characteristics A process is measurable, aimed at specific results, delivers primary results to a customer, and responds to specific triggers. Whereas, a function is a team or a group of people and the tools they use to carry out the processes. Hence, while Release Management, Incident Management, and so on are processes. The IT Service Desk is a function, which might be responsible for carrying out these processes. Luckily, ServiceDesk Plus provides features for managing both processes and functions. Differentiating between Service Level Agreement (SLA) and Operational Level Agreement (OLA) Service Level Agreement, or SLA, is a widely used term and often has some misconceptions attached to it. Contrary to popular belief, SLA is not necessarily a legal contract, but should be written in simple language, which can be understood by all parties without any ambiguity. An SLA is simply an agreement between a service provider and the customer(s) and documents the service targets and responsibilities of all parties. There are three types of SLAs defined in ITIL: Service Based SLA: All customers get the same deal for a specific service Customer Based SLA: A customer gets the same deal for all services Multilevel SLA: This involves a combination of corporate level, service level, and customer level SLAs. An Operational Level Agreement, or OLA, on the other hand, is the agreement between the service provider and another part of the same organization. An OLA is generally a prerequisite to help meet the SLA. There might be legal contracts between the service provider and some external suppliers as well, to help meet the SLA(s). These third-party legal contracts are called Underpinning Contracts. As must be evident, management and monitoring of these agreements is of utmost importance for the service organization. Here is how to create SLA records easily and track them in ServiceDesk Plus: Agree SLA with the customers. Go to Admin tab. Click on Service Level Agreements in the Helpdesk block. Image All SLA-based mail escalations are enabled by default. These can be disabled by clicking on the Disable Escalation button. Four SLAs are set by default—High SLA, Medium SLA, Normal SLA, and Low SLA. More could be added, if needed. Click on any SLA Name to view/edit its details. SLAs for sites, if any, can be configured by the site admin from the Service Level Agreement for combo box. SLA Rules block, below SLA details, is used for setting the rules and criteria for the SLA. Once agreed with the customers, configuring SLAs in the tool is pretty easy and straightforward. Escalations are taken care of automatically, as per the defined rules. To monitor the SLAs for a continuous focus on customer satisfaction, several Flash Reports are available under the Reports tab, for use on the fly. Differentiating between Service Portfolio and Service Catalog This is another example of terms often used interchangeably. However, ITIL clarifies that the Service Catalog lists only live IT services but Service Portfolio is a bigger set including services in the pipeline and retired services as well. Service Catalog contains information about two types of IT services: Customer-facing services (referred to as Business Service Category ) and Supporting services, with the complexities hidden from the business (referred to as IT Service Category) ServiceDesk Plus plays a vital role in managing the ways in which these services are exposed to users. The software provides a simple and effective interface to browse through the services and monitor their status. Users can also request for availing these services from within the module. The Service Catalog can also be accessed from the Admin tab, by clicking on Service Catalog under the Helpdesk block. The page lists the configured service categories and can be used to Add Service Category , Manage the service items, and Add Service under each category. Deleting a Service Category Deletion of an existing Service Category should be done with care. Here are the steps: Select Service Categories from Manage dropdown. A window with Service Categories List will open. Select the check box next to the Service Category to be deleted and then press the Delete button on the interface. A confirmation box will appear and on confirmation, the Service Category will be processed for deletion. If the concerned Service Category is in use by a module, then it will be grayed out and the category will be unavailable for further usage. To bring it back into usage, click on the edit icon Image next to the category name and uncheck the box for Service not for further usage in the new window.   The following two options under the Manage dropdown provide additional features for the customization of service request forms: Additional Fields: This can be used to capture additional details about the service apart from the predefined fields Service Level Agreements : This can be used to configure Service Based SLAs Summary We now understand the ITSM concepts, the fine differences between some of the terms, and also why software like ServiceDesk Plus is modeled after ITIL framework. We've also seen how SLAs and Service Catalog could be configured and tracked using ServiceDesk Plus. Resources for Article :   Further resources on this subject: Introduction to vtiger CRM [Article] Overview of Microsoft Dynamics CRM 2011 [Article] Customizing PrestaShop Theme Part 1 [Article]

(For more resources related to this topic, see here.) If you haven't used jQuery in a while, that's okay, we'll get you up to speed very quickly. The first thing to realize is that the Document.Ready function is extremely important when using UI. Although page loading happens incredibly fast, we always want our DOM (the HTML content) to be loaded before our UI code gets applied. Otherwise we have nothing to apply it to! We want to place our code inside the Document.Ready function, and we will be writing it the shorthand way as we did previously. Please remove the previous UI checking code in your header that you had previously: $(function() {// Your code here is called only once the DOM is completelyloaded}); Easy enough. Let's refresh on some jQuery selectors. We'll be using these a lot in our examples so we can manipulate our page. I'll write out a few DOM elements next and how you can select them. I will apply hide() to them so we know what's been selected and hidden. Feel free to place the JavaScript portion in your header script tags and the HTML elements within your <body> tags as follows: Selecting elements (unchanging the HTML entities) as follows: $('p').hide();<p>This is a paragraph</p><p>And here is another</p><p>All paragraphs will go hidden!</p> Selecting classes as follows: $('.edit').hide();<p>This is an intro paragraph</p><p class="edit">But this will go hidden!</p><p>Another paragraph</p><p class="edit">This will also go hidden!</p> Selecting IDs as follows: <div id="box">Hide the Box </div><div id="house">Just a random divider</div> Those are the three basic selectors. We can get more advanced and use the CSS3 selectors as follows: $("input[type=submit]").hide();<form><input type="text" name="name" /><input type="submit" /></form> Lastly, you can chain your DOM tree to hide elements more specifically: $("table tr td.hidden").hide(); <table> <tbody> <tr> <td>Data</td> <td class="hidden">Hide Me</td> </tr> </tbody> </table> Step 3 – console.log is your best friend I brought up that developing with the console open is very helpful. When you need to know details about a JavaScript item you have, whether it be the typeof type or value, a friend of yours is the console.log() method. Notice that it is always in lowercase. This allows you to place things in the console rather than somewhere on your page. For example, if I were having trouble figuring out what a value was returning to me, I would simply do the following: function add(a, b) {return a + b;}var total = add(5, 20);console.log(total); This will give me the result I wanted to know quickly and easily. Internet Explorer does not support console logging, it will prevent your JavaScript from running once it hits a console.log method. Make sure to comment out or remove all the console logs before releasing a live project or else all the IE users will have a serious problem. Step 4 – creating the slider widget Let's get busy! Open your template file and let's create a DOM element to attach a slider widget to. And to make it more interesting, we are also going to add an additional DIV to show a text value. Here is what I placed in my <body> tag: <div id="slider"></div><div id="text"></div> It doesn't have to be a <div> tag, but it's a good generic block-level element to use. Next, to attach a slider element we place the following in our <script> tags (the empty ones): $(function() {var my_slider = $("#slider").slider();}); Refresh your page, and you will have a widget that can slide along a bar. If you don't see a slider, first check your browser's development tools console to see if there are any JavaScript errors. If you don't see any still, make sure you don't have a JavaScript blocker on! The reason we assign a variable to the slider is because, later on, we may want to reference the options, which you'll see next. You are not required to do this, but if you want to access the slider outside of its initial setup, you must give it a variable name. Our widget doesn't do much now, but it feels cool to finally make something, whatever it is! Let's break down a few things we can customize. There are three categories: Options: These are defined in a JavaScript object ({}) and will determine how you want your widget to behave when it's loaded, for example, you could set your slider to have minimum and maximum values. Events: These are always a function and they are triggered when a user does something to your item. Methods: You can use methods to destroy a widget, get and set values from outside of the widget, and even set different options from what you started with. To play with a few categories, the easiest start is to adjust the options. Let's do it by creating an empty object inside our slider: var my_slider = $("#slider").slider({}); Then we'll create a minimum and maximum value for our slider using the following code: var my_slider = $("#slider").slider({min: 1,max: 50}); Now our slider will accept and move along a bar with 50 values. There are many more options at UI API located at api.jquery.com under slider. You'll find many other options we won't have time to cover such as a step option to make the slider count every two digits, as follows: var my_slider = $("#slider").slider({min: 1,max: 50,step: 2}); If we want to attach this to a text field we created in the DOM, a good way to start is by assigning the minimum value in the DIV, as this way we only have to change it once: var min = my_slider.slider('option', 'min');$("#text").html(min); Next we want to update the text value every time the slider is moved, easy enough; this will introduce us to our first event. Let's add it: var my_slider = $("#slider").slider({min: 1,max: 50,step: 2,change: function(event, ui) {$("#text").html(ui.value);}}); Summary This article describes the basis for all widgets. Creating them, setting the options, events, and methods. That is the very simple pattern that handles everything for us. Resources for Article: Further resources on this subject: Using jQuery and jQueryUI Widget Factory plugins with RequireJS [Article] jQuery Animation: Tips and Tricks [Article] New Effects Added by jQuery UI [Article]

(For more resources related to this topic, see here.) From the perspective of functionality, a Hadoop cluster is composed of an HDFS cluster and a MapReduce cluster . The HDFS cluster consists of the default filesystem for Hadoop. It has one or more NameNodes to keep track of the filesystem metadata, while actual data blocks are stored on distributed slave nodes managed by DataNode. Similarly, a MapReduce cluster has one JobTracker daemon on the master node and a number of TaskTrackers on the slave nodes. The JobTracker manages the life cycle of MapReduce jobs. It splits jobs into smaller tasks and schedules the tasks to run by the TaskTrackers. A TaskTracker executes tasks assigned by the JobTracker in parallel by forking one or a number of JVM processes. As a Hadoop cluster administrator, you will be responsible for managing both the HDFS cluster and the MapReduce cluster. In general, system administrators should maintain the health and availability of the cluster. More specifically, for an HDFS cluster, it means the management of the NameNodes and DataNodes and the management of the JobTrackers and TaskTrackers for MapReduce. Other administrative tasks include the management of Hadoop jobs, for example configuring job scheduling policy with schedulers. Managing the HDFS cluster The health of HDFS is critical for a Hadoop-based Big Data platform. HDFS problems can negatively affect the efficiency of the cluster. Even worse, it can make the cluster not function properly. For example, DataNode's unavailability caused by network segmentation can lead to some under-replicated data blocks. When this happens, HDFS will automatically replicate those data blocks, which will bring a lot of overhead to the cluster and cause the cluster to be too unstable to be available for use. In this recipe, we will show commands to manage an HDFS cluster. Getting ready Before getting started, we assume that our Hadoop cluster has been properly configured and all the daemons are running without any problems. Log in to the master node from the administrator machine with the following command: ssh hduser@master How to do it... Use the following steps to check the status of an HDFS cluster with hadoop fsck: Check the status of the root filesystem with the following command: hadoop fsck / We will get an output similar to the following: FSCK started by hduser from /10.147.166.55 for path / at Thu Feb 28 17:14:11 EST 2013 .. /user/hduser/.staging/job_201302281211_0002/job.jar: Under replicated blk_-665238265064328579_1016. Target Replicas is 10 but found 5 replica(s). .................................Status: HEALTHY Total size: 14420321969 B Total dirs: 22 Total files: 35 Total blocks (validated): 241 (avg. block size 59835360 B) Minimally replicated blocks: 241 (100.0 %) Over-replicated blocks: 0 (0.0 %) Under-replicated blocks: 2 (0.8298755 %) Mis-replicated blocks: 0 (0.0 %) Default replication factor: 2 Average block replication: 2.0248964 Corrupt blocks: 0 Missing replicas: 10 (2.0491803 %) Number of data-nodes: 5 Number of racks: 1 FSCK ended at Thu Feb 28 17:14:11 EST 2013 in 28 milliseconds The filesystem under path '/' is HEALTHY The output shows that some percentage of data blocks is under-replicated. But because HDFS can automatically make duplication for those data blocks, the HDFS filesystem and the '/' directory are both HEALTHY. Check the status of all the files on HDFS with the following command: hadoop fsck / -files We will get an output similar to the following: FSCK started by hduser from /10.147.166.55 for path / at Thu Feb 28 17:40:35 EST 2013 / <dir> /home <dir> /home/hduser <dir> /home/hduser/hadoop <dir> /home/hduser/hadoop/tmp <dir> /home/hduser/hadoop/tmp/mapred <dir> /home/hduser/hadoop/tmp/mapred/system <dir> /home/hduser/hadoop/tmp/mapred/system/jobtracker.info 4 bytes, 1 block(s): OK /user <dir> /user/hduser <dir> /user/hduser/randtext <dir> /user/hduser/randtext/_SUCCESS 0 bytes, 0 block(s): OK /user/hduser/randtext/_logs <dir> /user/hduser/randtext/_logs/history <dir> /user/hduser/randtext/_logs/history/job_201302281451_0002_13620904 21087_hduser_random-text-writer 23995 bytes, 1 block(s): OK /user/hduser/randtext/_logs/history/job_201302281451_0002_conf.xml 22878 bytes, 1 block(s): OK /user/hduser/randtext/part-00001 1102231864 bytes, 17 block(s): OK Status: HEALTHY Hadoop will scan and list all the files in the cluster. This command scans all ? les on HDFS and prints the size and status. Check the locations of file blocks with the following command: hadoop fsck / -files -locations The output of this command will contain the following information: The first line tells us that file part-00000 has 17 blocks in total and each block has 2 replications (replication factor has been set to 2). The following lines list the location of each block on the DataNode. For example, block blk_6733127705602961004_1127 has been replicated on hosts 10.145.231.46 and 10.145.223.184. The number 50010 is the port number of the DataNode. Check the locations of file blocks containing rack information with the following command: hadoop fsck / -files -blocks -racks Delete corrupted files with the following command: hadoop fsck -delete Move corrupted files to /lost+found with the following command: hadoop fsck -move Use the following steps to check the status of an HDFS cluster with hadoop dfsadmin: Report the status of each slave node with the following command: hadoop dfsadmin -report The output will be similar to the following: Configured Capacity: 422797230080 (393.76 GB) Present Capacity: 399233617920 (371.82 GB) DFS Remaining: 388122796032 (361.47 GB) DFS Used: 11110821888 (10.35 GB) DFS Used%: 2.78% Under replicated blocks: 0 Blocks with corrupt replicas: 0 Missing blocks: 0 ------------------------------------------------- Datanodes available: 5 (5 total, 0 dead) Name: 10.145.223.184:50010 Decommission Status : Normal Configured Capacity: 84559446016 (78.75 GB) DFS Used: 2328719360 (2.17 GB) Non DFS Used: 4728565760 (4.4 GB) DFS Remaining: 77502160896(72.18 GB) DFS Used%: 2.75% DFS Remaining%: 91.65% Last contact: Thu Feb 28 20:30:11 EST 2013 ... The first section of the output shows the summary of the HDFS cluster, including the configured capacity, present capacity, remaining capacity, used space, number of under-replicated data blocks, number of data blocks with corrupted replicas, and number of missing blocks. The following sections of the output information show the status of each HDFS slave node, including the name (ip:port) of the DataNode machine, commission status, configured capacity, HDFS and non-HDFS used space amount, HDFS remaining space, and the time that the slave node contacted the master. Refresh all the DataNodes using the following command: hadoop dfsadmin -refreshNodes Check the status of the safe mode using the following command: hadoop dfsadmin -safemode get We will be able to get the following output: Safe mode is OFF The output tells us that the NameNode is not in safe mode. In this case, the filesystem is both readable and writable. If the NameNode is in safe mode, the filesystem will be read-only (write protected). Manually put the NameNode into safe mode using the following command: hadoop dfsadmin -safemode enter This command is useful for system maintenance. Make the NameNode to leave safe mode using the following command: hadoop dfsadmin -safemode leave If the NameNode has been in safe mode for a long time or it has been put into safe mode manually, we need to use this command to let the NameNode leave this mode. Wait until NameNode leaves safe mode using the following command: hadoop dfsadmin -safemode wait This command is useful when we want to wait until HDFS finishes data block replication or wait until a newly commissioned DataNode to be ready for service. Save the metadata of the HDFS filesystem with the following command: hadoop dfsadmin -metasave meta.log The meta.log file will be created under the directory $HADOOP_HOME/logs. Its content will be similar to the following: 21 files and directories, 88 blocks = 109 total Live Datanodes: 5 Dead Datanodes: 0 Metasave: Blocks waiting for replication: 0 Metasave: Blocks being replicated: 0 Metasave: Blocks 0 waiting deletion from 0 datanodes. Metasave: Number of datanodes: 5 10.145.223.184:50010 IN 84559446016(78.75 GB) 2328719360(2.17 GB) 2.75% 77502132224(72.18 GB) Thu Feb 28 21:43:52 EST 2013 10.152.166.137:50010 IN 84559446016(78.75 GB) 2357415936(2.2 GB) 2.79% 77492854784(72.17 GB) Thu Feb 28 21:43:52 EST 2013 10.145.231.46:50010 IN 84559446016(78.75 GB) 2048004096(1.91 GB) 2.42% 77802893312(72.46 GB) Thu Feb 28 21:43:54 EST 2013 10.152.161.43:50010 IN 84559446016(78.75 GB) 2250854400(2.1 GB) 2.66% 77600096256(72.27 GB) Thu Feb 28 21:43:52 EST 2013 10.152.175.122:50010 IN 84559446016(78.75 GB) 2125828096(1.98 GB) 2.51% 77724323840(72.39 GB) Thu Feb 28 21:43:53 EST 2013 21 files and directories, 88 blocks = 109 total ... How it works... The HDFS filesystem will be write protected when NameNode enters safe mode. When an HDFS cluster is started, it will enter safe mode first. The NameNode will check the replication factor for each data block. If the replica count of a data block is smaller than the configured value, which is 3 by default, the data block will be marked as under-replicated. Finally, an under-replication factor, which is the percentage of under-replicated data blocks, will be calculated. If the percentage number is larger than the threshold value, the NameNode will stay in safe mode until enough new replicas are created for the under-replicated data blocks so as to make the under-replication factor lower than the threshold. We can get the usage of the fsck command using: hadoop fsck The usage information will be similar to the following: Usage: DFSck <path> [-move | -delete | -openforwrite] [-files [-blocks [-locations | -racks]]] <path> start checking from this path -move move corrupted files to /lost+found -delete delete corrupted files -files print out files being checked -openforwrite print out files opened for write -blocks print out block report -locations print out locations for every block -racks print out network topology for data-node locations By default fsck ignores files opened for write, use -openforwrite to report such files. They are usually tagged CORRUPT or HEALTHY depending on their block allocation status.   We can get the usage of the dfsadmin command using: hadoop dfsadmin The output will be similar to the following: Usage: java DFSAdmin [-report] [-safemode enter | leave | get | wait] [-saveNamespace] [-refreshNodes] [-finalizeUpgrade] [-upgradeProgress status | details | force] [-metasave filename] [-refreshServiceAcl] [-refreshUserToGroupsMappings] [-refreshSuperUserGroupsConfiguration] [-setQuota <quota> <dirname>...<dirname>] [-clrQuota <dirname>...<dirname>] [-setSpaceQuota <quota> <dirname>...<dirname>] [-clrSpaceQuota <dirname>...<dirname>] [-setBalancerBandwidth <bandwidth in bytes per second>] [-help [cmd]] There's more… Besides using command line, we can use the web UI to check the status of an HDFS cluster. For example, we can get the status information of HDFS by opening the link http://master:50070/dfshealth.jsp. We will get a web page that shows the summary of the HDFS cluster such as the configured capacity and remaining space. For example, the web page will be similar to the following screenshot: By clicking on the Live Nodes link, we can check the status of each DataNode. We will get a web page similar to the following screenshot: By clicking on the link of each node, we can browse the directory of the HDFS filesystem. The web page will be similar to the following screenshot: The web page shows that file /user/hduser/randtext has been split into five partitions. We can browse the content of each partition by clicking on the part-0000x link. Configuring SecondaryNameNode Hadoop NameNode is a single point of failure. By configuring SecondaryNameNode, the filesystem image and edit log files can be backed up periodically. And in case of NameNode failure, the backup files can be used to recover the NameNode. In this recipe, we will outline steps to configure SecondaryNameNode. Getting ready We assume that Hadoop has been configured correctly. Log in to the master node from the cluster administration machine using the following command: ssh hduser@master How to do it... Perform the following steps to configure SecondaryNameNode: Stop the cluster using the following command: stop-all.sh Add or change the following into the file $HADOOP_HOME/conf/hdfs-site.xml: <property> <name>fs.checkpoint.dir</name> <value>/hadoop/dfs/namesecondary</value> </property> If this property is not set explicitly, the default checkpoint directory will be ${hadoop.tmp.dir}/dfs/namesecondary. Start the cluster using the following command: start-all.sh The tree structure of the NameNode data directory will be similar to the following: ${dfs.name.dir}/ ├── current │ ├── edits │ ├── fsimage │ ├── fstime │ └── VERSION ├── image │ └── fsimage ├── in_use.lock └── previous.checkpoint ├── edits ├── fsimage ├── fstime └── VERSION And the tree structure of the SecondaryNameNode data directory will be similar to the following: ${fs.checkpoint.dir}/ ├── current │ ├── edits │ ├── fsimage │ ├── fstime │ └── VERSION ├── image │ └── fsimage └── in_use.lock There's more... To increase redundancy, we can configure NameNode to write filesystem metadata on multiple locations. For example, we can add an NFS shared directory for backup by changing the following property in the file $HADOOP_HOME/conf/hdfs-site.xml: <property> <name>dfs.name.dir</name> <value>/hadoop/dfs/name,/nfs/name</value> </property> Managing the MapReduce cluster A typical MapReduce cluster is composed of one master node that runs the JobTracker and a number of slave nodes that run TaskTrackers. The task of managing a MapReduce cluster includes maintaining the health as well as the membership between TaskTrackers and the JobTracker. In this recipe, we will outline commands to manage a MapReduce cluster. Getting ready We assume that the Hadoop cluster has been properly configured and running. Log in to the master node from the cluster administration machine using the following command: ssh hduser@master How to do it... Perform the following steps to manage a MapReduce cluster: List all the active TaskTrackers using the following command: hadoop -job -list-active-trackers This command can help us check the registration status of the TaskTrackers in the cluster. Check the status of the JobTracker safe mode using the following command: hadoop mradmin -safemode get We will get the following output: Safe mode is OFF The output tells us that the JobTracker is not in safe mode. We can submit jobs to the cluster. If the JobTracker is in safe mode, no jobs can be submitted to the cluster. Manually let the JobTracker enter safe mode using the following command: hadoop mradmin -safemode enter This command is handy when we want to maintain the cluster. Let the JobTracker leave safe mode using the following command: hadoop mradmin -safemode leave When maintenance tasks are done, you need to run this command. If we want to wait for safe mode to exit, the following command can be used: hadoop mradmin -safemode wait Reload the MapReduce queue configuration using the following command: hadoop mradmin -refreshQueues Reload active TaskTrackers using the following command: hadoop mradmin -refreshNodes How it works... Get the usage of the mradmin command using the following: hadoop mradmin The usage information will be similar to the following: Usage: java MRAdmin [-refreshServiceAcl] [-refreshQueues] [-refreshUserToGroupsMappings] [-refreshSuperUserGroupsConfiguration] [-refreshNodes] [-safemode <enter | leave | get | wait>] [-help [cmd]] ... The meaning of the command options is listed in the following table: Option Description -refreshServiceAcl Force JobTracker to reload service ACL. -refreshQueues Force JobTracker to reload queue configurations. -refreshUserToGroupsMappings Force JobTracker to reload user group mappings. -refreshSuperUserGroupsConfiguration Force JobTracker to reload super user group mappings. -refreshNodes Force JobTracker to refresh the JobTracker hosts. -help [cmd] Show the help info for a command or all commands. Summary In this article, we learned Managing the HDFS cluster, configuring SecondaryNameNode, and managing the MapReduce cluster. As a Hadoop cluster administrator, as the system administrator is responsible for managing both the HDFS cluster and the MapReduce cluster, he/she must be aware of how to manage these in order to maintain the health and availability of the cluster. More specifically, for an HDFS cluster, it means the management of the NameNodes and DataNodes and the management of the JobTrackers and TaskTrackers for MapReduce, which is covered in this article. Resources for Article : Further resources on this subject: Analytics – Drawing a Frequency Distribution with MapReduce (Intermediate) [Article] Advanced Hadoop MapReduce Administration [Article] Understanding MapReduce [Article]

(For more resources related to this topic, see here.) One of its most important characteristics is that it is an in-place substitution of the .NET 4.0 and only runs on Windows Vista SP2 or later systems. .NET 4.5 breathes asynchronous features and makes writing async code even easier. It also provides us with the Task Parallel Library (TPL) Dataflow Library to help us create parallel and concurrent applications. Another very important addition is the portable libraries, which allow us to create managed assemblies that we can refer through different target applications and platforms, such as Windows 8, Windows Phone, Silverlight, and Xbox. We couldn't avoid mentioning Managed Extensibility Framework (MEF), which now has support for generic types, a convention-based programming model, and multiple scopes. Of course, this all comes together with a brand-new tooling, Visual Studio 2012, which you can find at http://msdn.microsoft.com/en-us/vstudio. Just be careful if you have projects in .NET 4.0 since it is an in-place install. For this article I'd like to give a special thanks to Layla Driscoll from the Microsoft .NET team who helped me summarize the topics, focus on what's essential, and showcase it to you, dear reader, in the most efficient way possible. Thanks, Layla. There are some features that we will not be able to explore through this article as they are just there and are part of the CLR but are worth explaining for better understanding: Support for arrays larger than 2 GB on 64-bit platforms, which can be enabled by an option in the app config file. Improved performance on the server's background garbage collection, which must be enabled in the <gcServer> element in the runtime configuration schema. Multicore JIT: Background JIT (Just In Time) compilation on multicore CPUs to improve app performance. This basically creates profiles and compiles methods that are likely to be executed on a separate thread. Improved performance for retrieving resources. The culture-sensitive string comparison (sorting, casing, normalization, and so on) is delegated to the operating system when running on Windows 8, which implements Unicode 6.0. On other platforms, the .NET framework will behave as in the previous versions, including its own string comparison data implementing Unicode 5.0. Next we will explore, in practice, some of these features to get a solid grasp on what .NET 4.5 has to offer and, believe me, we will have our hands full! Creating a portable library Most of us have often struggled and hacked our code to implement an assembly that we could use in different .NET target platforms. Portable libraries are here to help us to do exactly this. Now there is an easy way to develop a portable assembly that works without modification in .NET Framework, Windows Store apps style, Silverlight, Windows Phone, and XBOX 360 applications. The trick is that the Portable Class Library project supports a subset of assemblies from these platforms, providing us a Visual Studio template. This article will show you how to implement a basic application and help you get familiar with Visual Studio 2012. Getting ready In order to use this section you should have Visual Studio 2012 installed. Note that you will need a Visual Studio 2012 SKU higher than Visual Studio Express for it to fully support portable library projects. How to do it... Here we will create a portable library and see how it works: First, open Visual Studio 2012 and create a new project. We will select the Portable Class Library template from the Visual C# category. Now open the Properties dialog box of our newly created portable application and, in the library we will see a new section named Target frameworks. Note that, for this type of project, the dialog box will open as soon as the project is created, so opening it will only be necessary when modifying it afterwards. If we click on the Change button, we will see all the multitargeting possibilities for our class. We will see that we can target different versions of a framework. There is also a link to install additional frameworks. The one that we could install right now is XNA but we will click on Cancel and let the dialog box be as it is. Next, we will click on the show all files icon at the top of the Solution Explorer window (the icon with two papers and some dots behind them), right-click on the References folder, and click on Add Reference. We will observe on doing so that we are left with a .NET subset of assemblies that are compatible with the chosen target frameworks. We will add the following lines to test the portable assembly: using System;using System.Collections.Generic;using System.Linq;using System.Text;namespace pcl_myFirstPcl{public static class MyPortableClass {public static string GetSomething() {return "I am a portable class library"; } }} Build the project. Next, to try this portable assembly we could add, for example, a Silverlight project to the solution, together with an ASP.NET Web application project to wrap the Silverlight. We just need to add a reference to the portable library project and add a button to the MainPage.xaml page that calls the portable library static method we created. The code behind it should look as follows. Remember to add a using reference to our portable library namespace. using System.Windows.Documents;using System.Windows.Input;using System.Windows.Media;using System.Windows.Media.Animation;using System.Windows.Shapes;using pcl_myFirstPcl;namespace SilverlightApplication_testPCL{public partial class MainPage : UserControl {public MainPage() {InitializeComponent(); }private void Button_Click_1(object sender, RoutedEventArgs e) { String something = MyPortableClass.GetSomething();MessageBox.Show("Look! - I got this string from my portable class library: " + something); } }} We can execute the code and check if it works. In addition, we could add other types of projects, reference the Portable Library Class, and ensure that it works properly. How it works... We created a portable library from the Portable Class Library project template and selected the target frameworks. We saw the references; note that it reinforces the visibility of the assemblies that break the compatibility with the targeted platforms, helping us to avoid mistakes. Next we added some code, a target reference application that referenced the portable class, and used it. There's more... We should be aware that when deploying a .NET app that references a Portable Class Library assembly, we must specify its dependency to the correct version of the .NET Framework, ensuring that the required version is installed. A very common and interesting usage of the Portable Class Library would be to implement MVVM. For example, we could put the View Model and Model classes inside a portable library and share it with Windows Store apps, Silverlight, and Windows Phone applications. The architecture is described in the following diagram, which has been taken from MSDN (http://msdn.microsoft.com/en-us/library/hh563947%28v=vs.110%29.aspx): It is really interesting that the list of target frameworks is not limited and we even have a link to install additional frameworks, so I guess that the number of target frameworks will eventually grow. Controlling the timeout in regular expressions .NET 4.5 gives us improved control on the resolution of regular expressions so we can react when they don't resolve on time. This is extremely useful if we don't control the regular expressions/patterns, such as the ones provided by the users. A badly formed pattern can have bad performance due to excessive backtracking and this new feature is really a lifesaver. How to do it... Next we are going to control the timeout in the regular expression, where we will react if the operation takes more than 1 millisecond: Create a new Visual Studio project of type Console Application, named caRegexTimeout. Open the Program.cs file and add a using clause for using regular expressions: Using System.Text.RegularExpressions; Add the following method and call it from the Main function: private static void ExecuteRegexExpression() {bool RegExIsMatch = false;string testString = "One Tile to rule them all, One Tile to find them… ";string RegExPattern = @"([a-z ]+)*!";TimeSpantsRegexTimeout = TimeSpan.FromMilliseconds(1);try {RegExIsMatch = Regex.IsMatch(testString, RegExPattern, RegexOptions.None, tsRegexTimeout); }catch (RegexMatchTimeoutException ex) {Console.WriteLine("Timeout!!");Console.WriteLine("- Timeout specified: " + ex.MatchTimeout); }catch (ArgumentOutOfRangeException ex) {Console.WriteLine("ArgumentOutOfRangeException!!");Console.WriteLine(ex.Message); }Console.WriteLine("Finished succesfully: " + RegExIsMatch.ToString());Console.ReadLine();} If we execute it, we will see that it doesn't fi nish successfully, showing us some details in the console window. Next, we will change testString and RegExPattern to: String testString = "jose@brainsiders.com";String RegExPattern = @"^([w-.]+)@([w-.]+).[a-zA-Z]{2,4}$"; If we run it, we will now see that it runs and fi nishes successfully. How it works... The RegEx.IsMatch() method now accepts a parameter, which is matchTimeout of type TimeSpan, indicating the maximum time that we allow for the matching operation. If the execution time exceeds this amount, RegexMatchTimeoutException is launched. In our code, we have captured it with a try-catch statement to provide a custom message and of course to react upon a badly formed regex pattern taking too much time. We have tested it with an expression that will take some more time to validate and we got the timeout. When we changed the expression to a good one with a better execution time, the timeout was not reached. Additionally, we also watched out for the ArgumentOutOfRangeException, which is thrown when TimeSpan is zero, or negative, or greater than 24 days. There'smore... We could also set a global matchTimeout for the application through the "REGEX_DEFAULT_MATCH_TIMEOUT" property with the AppDomain.SetData method: AppDomain.CurrentDomain.SetData("REGEX_DEFAULT_MATCH_ TIMEOUT",TimeSpan.FromMilliseconds(200)); Anyway, if we specify the matchTimeout parameter, we will override the global value. Defining the culture for an application domain With .NET 4.5, we have in our hands a way of specifying the default culture for all of our application threads in a quick and efficient way. How to do it... We will now define the default culture for our application domain as follows: Create a new Visual Studio project of type Console Application named caCultureAppDomain. Open the Program.cs file and add the using clause for globalization: using System.Globalization; Next, add the following methods: static void DefineAppDomainCulture() {String CultureString = "en-US";DisplayCulture();CultureInfo.DefaultThreadCurrentCulture = CultureInfo.CreateSpecificCulture(CultureString);CultureInfo.DefaultThreadCurrentUICulture = CultureInfo.CreateSpecificCulture(CultureString);DisplayCulture();Console.ReadLine();}static void DisplayCulture() {Console.WriteLine("App Domain........: {0}", AppDomain.CurrentDomain.Id);Console.WriteLine("Default Culture...: {0}", CultureInfo.DefaultThreadCurrentCulture);Console.WriteLine("Default UI Culture: {0}", CultureInfo.DefaultThreadCurrentUICulture);} Then add a call to the DefineAppDomainCulture() method. If we execute it, we will observe that the initial default cultures are null and we specify them to become the default for the App Domain. How it works... We used the CultureInfo class to specify the culture and the UI of the application domain and all its threads. This is easily done through the DefaultThreadCurrentCulture and DefaultThreadCurrentUICulture properties. There's more... We must be aware that these properties affect only the current application domain, and if it changes we should control them.

(For more resources related to this topic, see here.) Citrix EdgeSight alert is a powerful rules- and actions-based system that instructs the EdgeSight agents to send an alert in real time when a predefined situation has occurred on a monitored object. Alerts are defined by rules. The action can be configured to send either an e-mail alert or SNMP trap. The generated alerts are also listed and organized within the EdgeSight web console. After an alert rule has been created, it should be mapped to a department. How to do it... To create an alert, navigate to Configure | Company Configuration | Alerts | Rules | New Alert Rule. We will create an alert rule based on an application, so select the Application Alerts radio button and click on Next. Select Application Performance as the alert type and click on Next. Give the alert rule a name, the name of the process we want to monitor, and the CPU time in percent. Click on Next. Select departments to assign this alert rule to and click on Next. Select the department you wish to edit alert actions in and click on Next. We now need to assign an action to this alert rule; we will create a new action. So select the Create New Alert Action radio button and click on Next. We will send an e-mail notification as the alert action so select Send an email notification radio button and click on Next. Enter a name, subject, and one or more recipient e-mail addresses for this e-mail action. You can click the Test Action button to test whether EdgeSight was able to successfully queue the message or not. Click on Finish. How it works... Creating too many real-time alerts can affect the XenApp server performance as each rule that is created requires more work to be performed by the agent. We should only create alerts for those critical situations that require immediate action. If the situation is not critical, the delivery of alerts based on the normal upload cycle will probably be sufficient. By default, the alert data and other statistics are uploaded to the server daily. There's more... When a new alert rule is created or any existing rule is modified, this change is applied to all the devices in the department when those devices next upload data to the EdgeSight Server; alternatively, you can manually upload the alert rule data by clicking on Run Remotely. Administrators can also force certain agent devices to perform a configuration check within the EdgeSight web console by navigating to Configure | Company Configuration | Agents and then selecting the device from the device picker. To suppress an alert, navigate to Monitor | Alert List, click on the down arrow , and then select Suppress Alert. To clear an alert navigate to Configure | Company Configuration | Alerts | Suppressions. This is per user basis and other EdgeSight administrators will still see those alerts suppressed by you. Summary In this article will learned EdgeSight alerts and saw how to create alerts and define action when the defined alert condition is met. Resources for Article : Further resources on this subject: Publishing applications [Article] Designing a XenApp 6 Farm [Article] The XenDesktop architecture [Article]

(For more resources related to this topic, see here.) Before the birth of Microsoft Kinect, few people were familiar with the technology of motion sensing. Similar devices have been invented and developed originally for monitoring aerial and undersea aggressors in wars. Then in the non-military cases, motion sensors are widely used in alarm systems, lighting systems and so on, which could detect if someone or something disrupts the waves throughout a room and trigger predefined events. Although radar sensors and modern infrared motion sensors are used more popularly in our life, we seldom notice their existence, and can hardly make use of these devices in our own applications. But Kinect changed everything from the time it was launched in North America at the end of 2010. Different from most other user input controllers, Kinect enables users to interact with programs without really touching a mouse or a pad, but only through gestures. In a top-level view, a Kinect sensor is made up of an RGB camera, a depth sensor, an IR emitter, and a microphone array, which consists of several microphones for sound and voice recognition. A standard Kinect (for Windows) equipment is shown as follows: The Kinect device The Kinect drivers and software, which are either from Microsoft or from third-party companies, can even track and analyze advanced gestures and skeletons of multiple players. All these features make it possible to design brilliant and exciting applications with handsfree user inputs. And until now, Kinect had already brought a lot of games and software to an entirely new level. It is believed to be the bridge between the physical world we exist in and the virtual reality we create, and a completely new way of interacting with arts and a pro fitable business opportunity for individuals and companies. In this article, we will try to make an interesting game with the popular Kinect technology for user inputs, As Kinect captures the camera and depth images as video streams, we can also merge this view of our real-world environment with virtual elements, which is called Augmented Reality (AR) . This enables users to feel as if they appear and live in a nonexistent world, or something unbelievable exists in the physical earth. In this article, we will first introduce the installation of Kinect hardware and software on personal computers, and then consider a good enough idea compounded of Kinect and augmented reality elements. Before installing the Kinect device on your PCs, obviously you should buy Kinect equipment first. In this article, we will depend on Kinect for Windows or Kinect for Xbox 360, which can be learned about and bought at: http://www.microsoft.com/en-us/kinectforwindows/ http://www.xbox.com/en-US/kinect Please note that you don't need to buy an Xbox 360 at all. Kinect will be connected to PCs so that we can make custom programs for it. An alternative choice is Kinect for Windows, which is located at: http://www.microsoft.com/en-us/kinectforwindows/purchase/ The uses and developments of both will be of no difference for our cases. Installation of Kinect It is strongly suggested that you have a Windows 7 operating system or higher. It can be either 32-bit or 64-bit and with dual-core or faster processors. Linux developers can also benefit from third-party drivers and SDKs to manipulate Kinect components. Before we start to discuss the software installation, you can download both the Microsoft Kinect SDK and the Developer Toolkit from: http://www.microsoft.com/en-us/kinectforwindows/develop/developerdownloads.aspx In this article, we prefer to develop Kinect-based applications using Kinect SDK Version 1.5 (or higher versions) and the C++ language. Later versions should be backward compatible so that the source code provided in this article doesn't need to be changed. Setting up your Kinect software on PCs After we have downloaded the SDK and the Developer Toolkit, it's time for us to install them on the PC and ensure that they can work with the Kinect hardware. Let's perform the following steps: Run the setup executable with administrator permissions. Select I agree to the license terms and conditions after reading the License Agreement. The Kinect SDK setup dialog Follow the steps until the SDK installation has finished. And then, install the toolkit following similar instructions. The hardware installation is easy: plug the ends of the cable into the USB port and a power point, and plug the USB into your PC. Wait for the drivers to be found automatically. Now, start the Developer Toolkit Browser, choose Samples: C++ from the tabs, and find and run the sample with the name Skeletal Viewer. You should be able to see a new window demonstrating the depth/ skeleton/color images of the current physical scene, which is similar to the following image: The depth (left), skeleton (middle), and color (right) images read from Kinect Why did I do that? We chose to set up the SDK software at first so that it will install the motor and camera drivers, the APIs, and the documentations, as well as the toolkit including resources and samples onto the PC. If the operation steps are inversed, that is, the hardware is connected before installing the SDK, your Windows OS may not be able to recognize the device. Just start the SDK setup at this time and the device should be identified again during the installation process. But before actually using Kinect, you still have to ensure there is nothing between the device and you (the player). And it's best to keep the play space at least 1.8 m wide and about 1.8 m to 3.6 m long from the sensor. If you have more than one Kinect device, don't keep them face-to-face as there may be infrared interference between them. If you have multiple Kinects to install on the same PC, please note that one USB root hub can have one and only one Kinect connected. The problem happens because Kinect takes over 50 percent of the USB bandwidth, and it needs an individual USB controller to run. So plugging more than one device on the same USB hub means only one of them will work. The depth image at the left in the preceding image shows a human (in fact, the author) standing in front of the camera. Some parts may be totally black if they are too near (often less than 80 cm), or too far (often more than 4 m). If you are using Kinect for Windows, you can turn on Near Mode to show objects that are near the camera; however, Kinect for Xbox 360 doesn't have such features. You can read more about the software and hardware setup at: http://www.microsoft.com/en-us/kinectforwindows/purchase/sensor_setup.aspx The idea of the AR-based Fruit Ninja game Now it's time for us to define the goal we are going to achieve in this article. As a quick but practical guide for Kinect and augmented reality, we should be able to make use of the depth detection, video streaming, and motion tracking functionalities in our project. 3D graphics APIs are also important here because virtual elements should also be included and interacted with irregular user inputs not common mouse or keyboard inputs). A fine example is the Fruit Ninja game, which is already a very popular game all over the world. Especially on mobile devices like smartphones and pads, you can see people destroy different kinds of fruits by touching and swiping their fingers on the screen. With the help of Kinect, our arms can act as blades to cut off flying fruits, and our images can also be shown along with the virtual environment so that we can determine the posture of our bodies and position of our arms through the screen display. Unfortunately, this idea is not fresh enough for now. Already, there are commercial products with similar purposes available in the market; for example: http://marketplace.xbox.com/en-US/Product/Fruit-Ninja-Kinect/66acd000-77fe-1000-9115-d80258410b79 But please note that we are not going to design a completely different product here, or even bring it to the market after finishing this article. We will only learn how to develop Kinect-based applications, work in our own way from the very beginning, and benefit from the experience in our professional work or as an amateur. So it is okay to reinvent the wheel this time, and have fun in the process and the results. Summary Kinect, which is a portmanteau of the words "kinetic" and "connect", is a motion sensor developed and released by Microsoft. It provides a natural user interface (NUI) for tracking and manipulating handsfree user inputs such as gestures and skeleton motions. It can be considered as one of the most successful consumer electronics device in recent years, and we will be using this novel device to build the Fruit Ninja game in this article. We will focus on developing Kinect and AR-based applications on Windows 7 or higher using the Microsoft Kinect SDK 1.5 (or higher) and the C++ programming language. Mainly, we have introduced how to install Kinect for Windows SDK in this article. Resources for Article : Further resources on this subject: So, what is KineticJS? [Article] Mission Running in EVE Online [Article] Making Money with Your Game [Article]

(For more resources related to this topic, see here.) Getting ready We will start by opening SciTE from the AutoIt Program Group and start writing source code. Open the Skype app (Version 3.0 and above must be installed). Download it from http://www.autoitscript.com/forum/topic/121767-skype-udf-v0-7- april-13-2013/, and copy and paste the Skype.au3 library file to the same folder as Skypetemplate.au3. How to do it... Copy and paste the source code into a new script called the Skypetemplate.au3 file, and run the script by pressing F5. #include "Skype.au3";Credits Basicos ; And to Firefox Skype.au3,SendChat UDFSendChat1("I will be at home in 10 minutes","echo123")SendChat2("I will be at home in 15 minutes","echo123")Func SendChat1($message,$destination)Local $iChatId, $oChat ;using chat id:$iChatId = _Skype_ChatCreate("echo123") ;Skype test Service_Skype_ChatMessage($iChatId, "test")EndFuncFunc SendChat2($message,$destination)Local $iChatId, $oChat ;using chat object:$oChat = _Skype_ChatCreateWith("maribelnv") ; your friend_Skype_ChatSendMessage($oChat, "test3")EndFunc How it works... In Skype.au3 functions, Skype4COM provides an ActiveX interface to the Skype API such as Visual Studio or Delphi. It is an external user defined function (UDF), created and provided free for a Firefox user. COM stands for Component Object Model. It is the Microsoft way to interconnect software using a common interface defined in a COM object. You can do most of your programming with AutoIt's built-in functions, use this only for special interfacing to some applications. Objects depend heavily on the operating system and the installed software. There's more... Skype.zip downloaded files include an example for auto call answer to answer calls automatically (and it joins calls if you're already in a call): Using an event call: _Skype_OnEventCallStatus("_CallIncomming", $cClsRinging) ;ifsomeone is calling you _Skype_OnEventCallStatus("_CallFinished",$cClsFinished) ;if a call has finishedWhile 1Sleep(60000)WEnd Using call answer and call join functions: If IsObj($oMainCall) Then ;if a call is running then join theincomming call to the main _Skype_CallJoin($oMainCall, $oCall)Else ;else answer_Skype_CallAnswer($oCall)EndIf Summary This article discussed about how you can automate calls, answer them, and use Skype to create automation in your communications. Resources for Article : Further resources on this subject: ASP.Net Site Performance: Reducing Page Load Time [Article] Why CoffeeScript? [Article] ASP.Net Site Performance: Improving JavaScript Loading [Article]

(For more resources related to this topic, see here.) We want to focus on a particular case here, because we have already seen how to add a new property, and for some components, we can easily add the plugins or features property, and then add the plugin configuration. But the components that have native plugins supported by the API do not allow us to do so, like for instance, the grid panel from Ext JS: We can only use the plugins and features that are available within Sencha Architect. What if we want to use a third-party plugin or feature such as the Filter Plugin? It is possible, but we need to use an advanced feature from Sencha Architect, which is "creating overrides". A disclaimer about overrides: this has to be avoided. Whenever you can use a set method to change a property, use it. Overrides should be your last resource and they should be used very carefully, because if you do not use them carefully, you can change the behavior of a component and something may stop working. But we will demonstrate how to do it in a safe way! We will use the BooksGrid as an example in this topic. Let's say we need to use the Filter Plugin on it, so we need to create an override first. To do it, select the BooksGrid from the project inspector, open the code editor, and click on the Create Override button (Step 1). Sencha Architect will display a warning (Step 2). We can click on Yes to continue: The code editor will open (Step 3) the override class so we can enter our code. In this case, we will have complete freedom to do whatever we need to on this file. So let's add the features() function with the declaration of the plugin and also the initComponent()function as shown in the following screenshot (Step 4): One thing that is very important is that we must call the callParent()function (callOverriden()is deprecated already in Ext JS 4.1 and later versions) to make sure we will continue to have all the original behavior of the component (in this case the BooksGridclass). The only thing we want to do is to add a new feature to it. And we are done with the override! To go back to the original class we can use the navigator as shown in the following screenshot: Notice that requires was added to the class Packt.view.override. BooksGrid, which is the class we just wrote. The next step is to add the plugin on the class requires. To do so, we need to select the BooksGrid, go to the config panel, and add the requires with the name of the plugin (Ext.ux.grid.FiltersFeature): Some developers like to add the plugin file directly as a JavaScript file on app.html/index.html. Sencha provides the dynamic loading feature so let's take advantage of it and use it! First, we cannot forget to add the uxfolder with the plugin on the project root folder as shown in the following screenshot: Next, we need to set the application loader. Select the Application from the project inspector (Step 5), then go to the config panel, locate the Loader Config property, click on the +icon (Step 6), then click on the arrow icon (Step 7). The details of the loader will be available on the config panel. Locate the paths property and click on it (Step 8). The code editor will be opened with the loader path's default value, which is {"Ext": "."}(Step 9). Do not remove it; simply add the path of the Ext.uxnamespace which is the uxfolder (Step 10): And we are almost done! We need to add the filterable option in each column we want to allow the user to filter its values (Step 11): we can use the config panel to add a new property (we need to select the desired column from the project inspector first—always remember to do this). And then, we can choose what type of property we want to add (Step 12 and Step 14). For example, we can add filterable: true(Step 13) for the id column and filterable: {type: 'string'}(Step 15 and Step 16) to the Name column as shown in the following screenshot: And the plugin is ready to be used! Summary In this article we learned some useful tricks that can help in our everyday tasks while working with Sencha projects using Sencha Architect. Also we covered advanced topics such as creating overrides to use third party plugins and features and implement multilingual apps. Resources for Article: Further resources on this subject: Sencha Touch: Layouts Revisited [Article] Sencha Touch: Catering Form Related Needs [Article] Creating a Simple Application in Sencha Touch [Article]

(For more resources related to this topic, see here.) Getting ready The first thing we need to do is to download the latest version of Sqoop from following location http://www.apache.org/dist/sqoop/ and extract it on your machine. Now I am calling the Sqoop installation dir as $SQOOP_HOME. Given here are the prerequisites for Sqoop import process. Installed and running Relational Database Management System (MySQL). Installed and running Hadoop Cluster. Set $HADOOP_HOME environment variable. Following are the common arguments of import process. Parameters Description --connect <jdbc-uri> This command specifies the server or database to connect. It also specifies the port. Example: --connect jdbc:mysql://host:port/databaseName --connection-manager <class-name>   Specify connection manager class name.   --driver <class-name> Specify the fully qualified name of JDBC driver class. --password <password> Set authentication password required to connect to input source. --username <username> Set authentication username. How to do it Let’s see how to work with import process First, we will start with import single RDBMS table into Hadoop. Query1: $ bin/sqoop import --connect jdbc:mysql://localhost:3306/db1 --username root --password password --table tableName --target-dir /user/abc/tableName The content of output file in HDFS will look like: Next, we will put some light on approach of import only selected rows and selected columns of RDBMS table into Hadoop. Query 2: Import selected columns $ bin/sqoop import --connect jdbc:mysql://localhost:3306/db1 --username root --password password --table student --target-dir /user/abc/student --columns “student_id,address,name” Query 3: Import selected rows. $ bin/sqoop import --connect jdbc:mysql://localhost:3306/db1 --username root --password password --table student --target-dir /user/abc/student --where ‘student_id<100’ Query 4: Import selected columns of selected rows. $ bin/sqoop import --connect jdbc:mysql://localhost:3306/db1 --username root --password password --table student --target-dir /user/abc/student --columns “student_id,address,name” -- where ‘student_id<100’ How it works… Now let’s see how the above steps work: Import single table: Apart from the common arguments of import process, as explained previously, this part covers some other arguments which are required to import a table into Hadoop Distributed File System. Parameters Description --table <table-name> Name of input table to fetch. --target-dir<dir> Location of output/target dir in HDFS. --direct If user want to use non-JDBC based access mechanism for faster database access --options-file <file-path> All the command line options that are common in most of commands can put in options file for convenience.  The Query1 will run a MapReduce job and import all the rows of given table to HDFS (where, /user/abc/tableName is the location of output files). The records imported in HDFS preserve their original columns order, which means, if input table contains four columns A, B, C and D, then content in HDFS file will look like: A1, B1, C1, D1 A2, B2, C2, D2 Import selected columns: By default, the import query will select all columns of input table for import, but we can select the subset of columns by specifying the comma separated list of columns in --columns argument. The Query2 will only fetch three columns (student_id, address and name) of student table. If import query contains the --columns argument, then the order of column in output files are same as order specified in --columns argument. The output in HDFS will look like: student_id, address, name 1, Delhi, XYZ 2, Mumbai, PQR .......... If the input query contains the column in following order -- “address, name, student_id”, then the output in HDFS will look like. address, name, student_id Delhi, XYZ, 1 Mumbai, PQR, 2 ............. Import selected rows: By default, all the rows of input table will be imported to HDFS, but we can control which rows need to be import by using a --where argument in the import statement. The Query3 will import only those rows into HDFS which has value of “student_id” column greater than 100. The Query4 use both --columns and --where arguments in one statement. For Query4, Sqoop will internally generates the query of the form “select student_id, address, name from student where student_id<100”. There’s more... This section covers some more examples of import process. Import all tables: So far we have imported a single table into HDFS, this section introduces an import-all-tables tool, by which we can import a set of tables from an RDBMS to HDFS. The import-all-tables tool creates a separate directory in HDFS for each RDBMS table. The following are the mandatory conditions for import-all-tables tool: All tables must have a single primary key column. User must intend to import all the columns of each table. No --where, --columns and --query arguments are permitted. Example: Query 5: $ bin/sqoop import-all-tables --connect jdbc:mysql://localhost:3306/db1 --username root --password password This query will import all tables (tableName and tableName1) of database db1 into HDFS. Output directories in HDFS look like: Summary We learned a lot in this article, about import single RDBMS table into HDFS, import selected columns and selected rows, and import set of RDBMS tables. Resources for Article : Further resources on this subject: Introduction to Logging in Tomcat 7 [Article] Configuring Apache and Nginx [Article] Geronimo Architecture: Part 2 [Article]

(For more resources related to this topic, see here.) A pivot table is the core business intelligence tool that helps to turn meaningless data from various sources to a meaningful result. By using different ways of presenting data, we are able to identify relations between seemingly separate data and reach conclusions to help us identify our strengths and areas of improvement. Getting ready Prepare the two files entitled DatabaseData_v2.xlsx and GDPData_v2.xlsx.We will be using these results along with other data sources to create a meaningful PowerPivot table that will be used for intelligent business analysis. How to do it... For each of the two files, we will build upon the file and add a pivot table to it, gaining exposure using the data we are already familiar with. The following are the steps to create a pivot table with the DatabaseData_v2.xlsx file, which results in the creation of a DatabaseData_v3.xlsx file: Open the PowerPivot window of the DatabaseData_v2.xlsx file with its 13 tables. Click on the PivotTable button near the middle of the top row and save as New Worksheet. Select the checkboxes as shown in the following screenshot: Select CountryRegion | Name and move it under Row Labels‰ Select Address | City and move it under Row Labels Select Address | AddressLine1 as Count of AddressLine1 and move it under Values Now, this shows the number of clients per city and per country. However, it is very difficult to navigate, as each country name has to be collapsed in order to see the next country. Let us move the CountryRegion | Name column to Slicers Vertical. Now, the PowerPivot Field List dashboard should appear as shown in the following screenshot: Now, the pivot table should display simple results: the number of clients in a region, filterable by the country using slicers. Let us apply some formatting to allow for a better understanding of the data. Right-click on Name under the Slicers Vertical area of the PowerPivot Field List dashboard. Select Field Settings, then change the name to Country Name. We now see that the title of the slicer has changed from Name to Country Name, allowing anyone who views this data to understand better what the data represents. Similarly, right-click on Count of AddressLine1 under Values, select Edit Measure, and then change its name to Number of Clients. Also change the data title City under the Row Labels area to City Name. The result should appear as shown in the following screenshot: Let's see our results change as we click on different country names. We can filter for multiple countries by holding the Ctrl key while clicking, and can remove all filters by clicking the small button on the top-right of slicers. This is definitely easier to navigate through and to understand compared to what we did at first without using slicers, which is how it would appear in Excel 2010 without PowerPivot. However, this table is still too big. Clicking on Canada gives too many cities whose names many of us have not heard about before. Let us break the data further down by including states/provinces. Select StateProvince | Name and move it under Slicers Horizontal and change its title to State Name. It is a good thing that we are renaming these as we go along. Otherwise, there would have been two datasets called Name, and anyone would be confused as we moved along. Now, we should see the state names filter on the top, the country name filter on the left, and a list of cities with the number of clients in the middle part. This, however, is kind of awkward. Let us rearrange the filters by having the largest filter (country) at the top and the sub-filter. (state) on the left-hand side This can be done simply by dragging the Country Name dataset to Slicers Horizontal and State Name to Slicers Vertical. After moving the slicers around a bit, the result should appear as shown in the following screenshot: Again, play with the results and try to understand the features: try filtering by a country—and by a state/province—now there are limited numbers of cities shown for each country and each state/province, making it easier to see the list of cities. However, for countries such as the United States, there are just too many states. Let us change the formatting of the vertical filter to display three states per line, so it is easier to find the state we are looking for. This can be done by right-clicking on the vertical filter, selecting Size and Properties| Position and Layout, and then by changing the Number of Columns value. Repeat the same step for Country Name to display six columns and then change the sizes of the filters to look more selectable. Change the name of the sheet as PivotTable and then save the file as DatabaseData_v3.xlsx. The following are the steps to create a pivot table with the GDPData_v2.xlsx file, which results in the creation of a GDPData_v3.xlsx file: Open the PowerPivot window of the GDPData_v2.xlsx file with its two tables. Click on the PivotTable button near the middle of the top row and save as New Worksheet. Move the dataset from the year 2000 to the year 2010 to the Value field, and move Country Name in the Row Labels field, and Country Name again into the Slicers Horizontal field. In the slicer, select five countries: Canada, China, Korea, Japan, and United States as shown in the following screenshot: Select all fields and reduce the number of its decimal places. We can now clearly see that GDP in China has tripled over the decade, and that only China and Korea saw an increase in GDP from 2008 to 2009 while the GDP of other nations dropped due to the 2008 financial crisis. Knowing the relevant background information of world finance events, we can make intelligent analysis such as which markets to invest in if we are worried about another financial crisis taking place. As the data get larger in size, looking at the GDP number becomes increasingly difficult. In such cases, we can switch the type of data displayed by using available buttons in the PivotTable Tools | Options menu, the Show Value As button. Play around with it and see how it works: % of Column Total shows each GDP as a percentage of the year, while % Different From allows the user to set one value as the standard and compare the rest to it, and the Rank Largest to Smallest option simply shows the ranking based on which country earns the most GDP. Change the name of the sheet as PivotTable and then save the file as GDPData_v3.xlsx. How it works... We looked at two different files and focused on two different fields. The first file was more qualitative and showed the relationship between regions and number of clients, using various features of pivot tables such as slicers. We also looked at how to format various aspects of a pivot table for easier processing and for a better understanding of the represented data. Slicers embedded in the pivot table are a unique and very powerful feature of PowerPivot that allow us to sort through data simply by clicking the different criteria. The increasing numbers of slicers help to customize the data further, enabling the user to create all sorts of data imaginable. There are no differences in horizontal and vertical slicers aside from the fact that they are at different locations. From the second file, we focused more on the quantitative data and different ways of representing the data. By using slicers to limit the number of countries, we were able to focus more on the data presented, and manage to represent the GDP in various formats such as percentages and ranks, and were able to compare the difference between the numbers by selecting one as a standard. A similar method of representing data in a different format could be applied to the first file to show the percentage of clients per nation, and so on. There's more... We covered the very basic setup of creating a pivot table. We can also analyze creating relationships between data and creating custom fields, so that better results are created. So don't worry about why the pivot table looks so small! For those who do not like working with a pivot table, there is also a feature that will convert all cells into Excel formula. Under the PowerPivot Tools | OLAP Tools option, the Convert to Formula button does exactly that. However, be warned that it cannot be undone as the changes are permanent. Summary In this article, we learned how to use the raw data to make some pivot tables that can help us make smart business decisions! Resources for Article: Further resources on this subject: SAP HANA integration with Microsoft Excel [Article] Managing Core Microsoft SQL Server 2008 R2 Technologies [Article] Eloquent relationships [Article]