How-To Tutorials

(For more resources on this topic, see here.) We will look at the mechanisms for storing data in Salesforce and at the concepts of objects and fields. The features that allow these data to be grouped and arranged within the application are then considered by looking at Apps, Tabs, Page Layouts, and Record Types. Finally, we take a look at some of the features that allow views of data to be presented and customized by looking in detail at related lists and list views. Relationship between profile and the features that it controls The following diagram describes the relationship that exists between the profile and the features that it controls: The profile is used to: Control access to the type of license specified for the user and any login hours or IP address restrictions that are set. Control access to objects and records using the role and sharing model. If the appropriate object-level permission is not set on the user's profile, then the user will be unable to gain access to the records of that object type in the application. In this article, we will look at the configurable elements that are set in conjunction with the profile. These are used to control the structure and the user interface for the Salesforce CRM application. Objects Objects are a key element in Salesforce CRM as they provide a structure for storing data and are incorporated in the interface, allowing users to interact with the data. Similar in nature to a database table, objects have properties such as: Fields which are similar in concept to a database column Records which are similar in concept to a database row Relationships to other objects Optional tabs which are user interface components to display the object data Standard objects Salesforce provides standard objects in the application when you sign up and these include Account, Contact, Opportunity, and so on. These are the tables that contain the data records in any standard tab such as Accounts, Contacts, or Opportunities. In addition to the standard objects, you can create custom objects and custom tabs. Custom objects Custom objects are the tables you create to store your data. You can create a custom object to store data specific to your organization. Once you have the custom objects and have created records for these objects, you can also create reports and dashboards based on the record data in your custom object. Fields Fields in Salesforce are similar in concept to a database column and store the data for the object records. An object record is analogous to a row in a database table. Standard fields Standard fields are predefined fields that are included as standard within the Salesforce CRM application. Standard fields cannot be deleted but non-required standard fields can be removed from page layouts whenever necessary. With standard fields, you can customize visual elements that are associated to the field such as field labels and field-level help as well certain data definitions such as picklist values, the formatting of auto-number fields (which are used as unique identifiers for the records), and setting of field history tracking. Some aspects, however, such as the field name cannot be customized and some standard fields (such as Opportunity Probability) do not allow the changing of the field label. Custom fields Custom fields are unique to your business needs and can not only be added and amended, but also deleted. Creating custom fields allow you to store the information that is necessary for your organization. Both standard and custom fields can be customized to include custom help text to help users understand how to use the field: Object relationships Object relationships can be set on both standard and custom objects and are used to define how records in one object relates to records in another object. Accounts, for example, can have a one-to-many relationship with opportunities and these relationships are presented in the application as related lists. Apps An app in Salesforce is a container for all the objects, tabs, processes, and services associated with a business function. There are standard and custom apps that are accessed using the App menu located at the top-right of the Salesforce page as shown in the following screenshot: When users select an app from the App menu, their screen changes to present the objects associated with that app. For example, when switching from an app that contains the Campaign tab to one that does not, the Campaign tab no longer appears. This feature is applied to both standard and custom apps. Standard apps Salesforce provides standard apps such as Sales, Call Center, and Marketing. Custom apps A custom app can optionally include a custom logo. Both standard and custom apps consist of a name, a description, and an ordered list of tabs. Tabs A tab is a user-interface element which, when clicked, displays the record data on a page specific to that object. Hiding and showing tabs To customize your personal tab settings follow the path Your Name Setup | My Personal Settings | Change My Display | Customize My Tabs|. Now, choose the tabs that will display in each of your apps by moving the tab name between the Available Tabs and the Selected Tabs sections and click Save. The following shows the section of tabs for the Sales app: To customize the tab settings of your users, follow the path Your Name Setup | Administration Setup | Manage Users | Profiles|. Now select a profile and click Edit. Scroll down to the tab settings section of the page as shown in the following screenshot: Standard tabs Salesforce provides tabs for each of the standard objects that are provided in the application when you sign up. For example, there are standard tabs for Accounts, Contacts, Opportunities, and so on: Visibility of the tab depends on the setting on the tab display setting for the app. Custom tabs You can create three different types of custom tabs: Custom Object Tabs, Web Tabs, and Visualforce Tabs. Custom Object Tabs allow you to create, read, update, and delete the data records in your custom objects. Web Tabs display any web URL in a tab within your Salesforce application. Visualforce Tabs display custom user-interface pages created using Visualforce. Creating custom tabs: The text displayed on the custom tab is set from the Plural label of the custom object which is entered when creating the custom object. If the tab text needs to be changed this can be done by changing the Plural label stored on the custom object. Salesforce.com recommends selecting the Append tab to users' existing personal customizations checkbox. This benefits your users as they will automatically be presented with the new tab and can immediately access the corresponding functionality without having to first customize their personal settings themselves. It is recommended that you do not show tabs by setting appropriate permissions so that the users in your organization cannot see any of your changes until you are ready to make them available. You can create up to 25 custom tabs in Enterprise Edition and as many as you require in Unlimited Edition. To create custom tabs for a custom object, follow the path Your Name Setup | App Setup | Create | Tabs|. Now select the appropriate tab type and/or object from the available selections as shown in the following screenshot: (Move the mouse over the image to enlarge.) Creating custom objects Custom objects are database tables that allow you to store data specific to your organization in Salesforce.com. You can use custom objects to extend Salesforce functionality or to build new application functionality. You can create up to 200 custom objects in Enterprise Edition and 2000 in Unlimited Edition. Once you have created a custom object, you can create a custom tab, custom-related lists, reports, and dashboards for users to interact with the custom object data. To create a custom object, follow the path Your Name Setup | App Setup | Create | Objects|. Now click New Custom Object, or click Edit to modify an existing custom object. The following screenshot shows the resulting screen: On the Custom Object Definition Edit page, you can enter the following: Label: This is the visible name that is displayed for the object within the Salesforce CRM user interface and shown on pages, views, and reports, for example. Plural Label: This is the plural name specified for the object which is used within the application in places such as reports and on tabs if you create a tab for the object. Gender (language dependent): This field appears if your organization-wide default language expects gender. This is used for organizations where the default language settings is for example, Spanish, French, Italian, German among many others. Your personal language preference setting does not affect whether the field appears or not. For example, if your organization's default language is English but your personal language is French, you will not be prompted for gender when creating a custom object. Starts with a vowel sound: Use of this setting depends on your organization's default language and is a linguistic check to allow you to specify whether your label is to be preceded by "an" instead of "a". For example, resulting in reference to the object as "an Order" instead of "a Order" as an example. Object Name: A unique name used to refer to the object. Here, the Object Name field must be unique and can only contain underscores and alphanumeric characters. It must also begin with a letter, not contain spaces, not contain two consecutive underscores, and not end with an underscore. Description: An optional description of the object. A meaningful description will help to explain the purpose for your custom objects when you are viewing them in a list. Context-Sensitive Help Setting: Defines what information is displayed when your users click the Help for this Page context-sensitive help link from the custom object record home (overview), edit, and detail pages, as well as list views and related lists. The Help & Training link at the top of any page is not affected by this setting. It always opens the Salesforce Help & Training window. Record Name: This is the name that is used in areas such page layouts, search results, key lists, and related lists as shown next. Data Type: The type of field for the record name. Here the data type can be either text or auto-number. If the data type is set to be text, then when a record is created, users must enter a text value which does not need to be unique. If the data type is set to be Auto Number, it becomes a read-only field whereby new records are automatically assigned a unique number: Display Format: As in the preceding example, this option only appears when the Data Type is set to Auto Number. It allows you to specify the structure and appearance of the Auto Number field. For example: {YYYY}{MM}-{000} is a display format that produces a 4-digit year, 2-digit month prefix to a number with leading zeros padded to 3 digits. Example data output would include: 201203-001; 201203-066; 201203-999; 201203-1234. It is worth noting that although you can specify the number to be 3 digits if the number of records created becomes over 999 the record will still be saved but the automatically incremented number becomes 1000, 1001, and so on. Starting Number: As described, Auto Number fields in Salesforce CRM are automatically incremented for each new record. Here you must enter the starting number for the incremental count (which does not have to be set to start from 1). Allow Reports: This setting is required if you want to include the record data from the custom object in any report or dashboard analytics. Such relationships can be either a lookup or a master-detail. Lookup relationships create a relationship between two records so you can associate them with each other. Master-detail relationship creates a relationship between records where the master record controls certain behaviors of the detail record such as record deletion and security. When the custom object has a master-detail relationship with a standard object or is a lookup object on a standard object, a new report type will appear in the standard report category. The new report type allows the user to create reports that relate the standard object to the custom object which is done by selecting the standard object for the report type category instead of the custom object. Allow Activities: Allows users to include tasks and events related to the custom object records which appear as a related list on the custom object page. Track Field History: Enables the tracking of data field changes on the custom object records, such as who changed the value of a field and when it was changed. Fields history tracking also stores the value of the field before and after the fields edit. This feature is useful for auditing and data quality measurement and is also available within the reporting tools. Deployment Status: Indicates whether the custom object is now visible and available for use by other users. This is useful as you can easily set the status to In Development until you are happy for users to start working with the new object. Add Notes & Attachments: This setting allows your users to record notes and attach files to the custom object records. When this is specified, a related list with New Note and Attach File buttons automatically appears on the custom object record page where your users can enter notes and attach documents. The Add Notes & Attachments option is only available when you create a new object. Launch the New Custom Tab Wizard: Starts the custom tab wizard after you save the custom object. The New Custom Tab Wizard option is only available when you create a new object. Creating custom object relationships Considerations to be observed when creating object relationships: Create the object relationships as a first step before starting to build the custom fields, page layouts, and any related list The Related To entry cannot be modified after you have saved the object relationship Each custom object can have up to two master-detail relationship and up to 25 total relationships. When planning to create a master-detail relationship on an object be aware that it can only be created before the object contains record data Clicking Edit List Layout allows you to choose columns for the key views and lookups The Standard Name field is required on all custom object-related lists and also on any page layouts

In this article by Robert Craig Layton, author of Learning Data Mining with Python, we will look at predicting the winner of games of the National Basketball Association (NBA) using a different type of classification algorithm—decision trees. Collecting the data The data we will be using is the match history data for the NBA, for the 2013-2014 season. The Basketball-Reference.com website contains a significant number of resources and statistics collected from the NBA and other leagues. Perform the following steps to download the dataset: Navigate to http://www.basketball-reference.com/leagues/NBA_2014_games.html in your web browser. Click on the Export button next to the Regular Season heading. Download the file to your data folder (and make a note of the path). This will download a CSV file containing the results of 1,230 games in the regular season of the NBA. We will load the file with the pandas library, which is an incredibly useful library for manipulating data. Python also contains a built-in library called csv that supports reading and writing CSV files. We will use pandas instead as it provides more powerful functions to work with datasets. For this article, you will need to install pandas. The easiest way to do that is to use pip3, which you may previously have used to install scikit-learn: $pip3 install pandas Using pandas to load the dataset We can load the dataset using the read_csv function in pandas as follows: import pandas as pddataset = pd.read_csv(data_filename) The result of this is a data frame, a data structure used by pandas. The pandas.read_csv function has parameters to fix some of the problems in the data, such as missing headings, which we can specify when loading the file: dataset = pd.read_csv(data_filename, parse_dates=["Date"],skiprows=[0,])dataset.columns = ["Date", "Score Type", "Visitor Team","VisitorPts", "Home Team", "HomePts", "OT?", "Notes"] We can now view a sample of the data frame: dataset.ix[:5] Extracting new features We extract our classes, 1 for a home win, and 0 for a visitor win. We can specify this using the following code to extract those wins into a NumPy array: dataset["HomeWin"] = dataset["VisitorPts"] < dataset["HomePts"] y_true = dataset["HomeWin"].values The first two new features we want to create are to indicate whether each of the two teams won their previous game. This would roughly approximate which team is currently playing well. We will compute this feature by iterating through the rows in order, and recording which team won. When we get to a new row, we look up whether the team won the last time: from collections import defaultdictwon_last = defaultdict(int) We can then iterate over all the rows and update the current row with the team's last result (win or loss): for index, row in dataset.iterrows():home_team = row["Home Team"]visitor_team = row["Visitor Team"]row["HomeLastWin"] = won_last[home_team]row["VisitorLastWin"] = won_last[visitor_team]dataset.ix[index] = row We then set our dictionary with each team's result (from this row) for the next time we see these teams: won_last[home_team] = row["HomeWin"]won_last[visitor_team] = not row["HomeWin"] Decision trees Decision trees are a class of classification algorithm such as a flow chart that consist of a sequence of nodes, where the values for a sample are used to make a decision on the next node to go to. We can use the DecisionTreeClassifier class to create a decision tree: from sklearn.tree import DecisionTreeClassifierclf = DecisionTreeClassifier(random_state=14) We now need to extract the dataset from our pandas data frame in order to use it with our scikit-learn classifier. We do this by specifying the columns we wish to use and using the values parameter of a view of the data frame: X_previouswins = dataset[["HomeLastWin", "VisitorLastWin"]].values Decision trees are estimators, and therefore, they have fit and predict methods. We can also use the cross_val_score method as before to get the average score: scores = cross_val_score(clf, X_previouswins, y_true,scoring='accuracy')print("Accuracy: {0:.1f}%".format(np.mean(scores) * 100)) This scores up to 56.1%; we are better off choosing randomly! Predicting sports outcomes We have a method for testing how accurate our models are using the cross_val_score method that allows us to try new features. For the first feature, we will create a feature that tells us whether the home team is generally better than the visitors by seeing whether they ranked higher in the previous season. To obtain the data, perform the following steps: Head to http://www.basketball-reference.com/leagues/NBA_2013_standings.html Scroll down to Expanded Standings. This gives us a single list for the entire league. Click on the Export link to the right of this heading. Save the download in your data folder. In your IPython Notebook, enter the following into a new cell. You'll need to ensure that the file was saved into the location pointed to by the data_folder variable: standings_filename = os.path.join(data_folder,"leagues_NBA_2013_standings_expanded-standings.csv")standings = pd.read_csv(standings_filename, skiprows=[0,1]) We then iterate over the rows and compare the team's standings: dataset["HomeTeamRanksHigher"] = 0for index, row in dataset.iterrows():home_team = row["Home Team"]visitor_team = row["Visitor Team"] Between 2013 and 2014, a team was renamed as follows: if home_team == "New Orleans Pelicans":home_team = "New Orleans Hornets"elif visitor_team == "New Orleans Pelicans":visitor_team = "New Orleans Hornets" Now, we can get the rankings for each team. We then compare them and update the feature in the row: home_rank = standings[standings["Team"] ==home_team]["Rk"].values[0]visitor_rank = standings[standings["Team"] ==visitor_team]["Rk"].values[0]row["HomeTeamRanksHigher"] = int(home_rank > visitor_rank)dataset.ix[index] = row Next, we use the cross_val_score function to test the result. First, we extract the dataset as before: X_homehigher = dataset[["HomeLastWin", "VisitorLastWin", "HomeTeamRanksHigher"]].values Then, we create a new DecisionTreeClassifier class and run the evaluation: clf = DecisionTreeClassifier(random_state=14)scores = cross_val_score(clf, X_homehigher, y_true,scoring='accuracy')print("Accuracy: {0:.1f}%".format(np.mean(scores) * 100)) This now scores up to 60.3%—even better than our previous result. Unleash the full power of Python machine learning with our 'Learning Data Mining with Python' book.

In this article by Ravi Kant Soni, author of the book Learning Spring Application Development, you will be closely acquainted with the Spring Framework. Spring is an open source framework created by Rod Johnson to address the complexity of enterprise application development. Spring is now a long time de facto standard for Java enterprise software development. The framework was designed with developer productivity in mind and this makes it easier to work with the existing Java and JEE APIs. Using Spring, we can develop standalone applications, desktop applications, two tier applications, web applications, distributed applications, enterprise applications, and so on. (For more resources related to this topic, see here.) Features of the Spring Framework Lightweight: Spring is described as a lightweight framework when it comes to size and transparency. Lightweight frameworks reduce complexity in application code and also avoid unnecessary complexity in their own functioning. Non intrusive: Non intrusive means that your domain logic code has no dependencies on the framework itself. Spring is designed to be non intrusive. Container: Spring's container is a lightweight container, which contains and manages the life cycle and configuration of application objects. Inversion of control (IoC): Inversion of Control is an architectural pattern. This describes the Dependency Injection that needs to be performed by external entities instead of creating dependencies by the component itself. Aspect-oriented programming (AOP): Aspect-oriented programming refers to the programming paradigm that isolates supporting functions from the main program's business logic. It allows developers to build the core functionality of a system without making it aware of the secondary requirements of this system. JDBC exception handling: The JDBC abstraction layer of the Spring Framework offers a exceptional hierarchy that simplifies the error handling strategy. Spring MVC Framework: Spring comes with an MVC web application framework to build robust and maintainable web applications. Spring Security: Spring Security offers a declarative security mechanism for Spring-based applications, which is a critical aspect of many applications. ApplicationContext ApplicationContext is defined by the org.springframework.context.ApplicationContext interface. BeanFactory provides a basic functionality, while ApplicationContext provides advance features to our spring applications, which make them enterprise-level applications. Create ApplicationContext by using the ClassPathXmlApplicationContext framework API. This API loads the beans configuration file and it takes care of creating and initializing all the beans mentioned in the configuration file: import org.springframework.context.ApplicationContext; import org.springframework.context.support.ClassPathXmlApplicationContext;   public class MainApp {   public static void main(String[] args) {      ApplicationContext context =    new ClassPathXmlApplicationContext("beans.xml");      HelloWorld helloWorld =    (HelloWorld) context.getBean("helloworld");      helloWorld.getMessage(); } } Autowiring modes There are five modes of autowiring that can be used to instruct Spring Container to use autowiring for Dependency Injection. You use the autowire attribute of the <bean/> element to specify the autowire mode for a bean definition. The following table explains the different modes of autowire: Mode Description no By default, the Spring bean autowiring is turned off, meaning no autowiring is to be performed. You should use the explicit bean reference called ref for wiring purposes. byName This autowires by the property name. If the bean property is the same as the other bean name, autowire it. The setter method is used for this type of autowiring to inject dependency. byType Data type is used for this type of autowiring. If the data type bean property is compatible with the data type of the other bean, autowire it. Only one bean should be configured for this type in the configuration file; otherwise, a fatal exception will be thrown. constructor This is similar to the byType autowire, but here a constructor is used to inject dependencies. autodetect Spring first tries to autowire by constructor; if this does not work, then it tries to autowire by byType. This option is deprecated. Stereotype annotation Generally, @Component, a parent stereotype annotation, can define all beans. The following table explains the different stereotype annotations: Annotation Use Description @Component Type This is a generic stereotype annotation for any Spring-managed component. @Service Type This stereotypes a component as a service and is used when defining a class that handles the business logic. @Controller Type This stereotypes a component as a Spring MVC controller. It is used when defining a controller class, which composes of a presentation layer and is available only on Spring MVC. @Repository Type This stereotypes a component as a repository and is used when defining a class that handles the data access logic and provide translations on the exception occurred at the persistence layer. Annotation-based container configuration For a Spring IoC container to recognize annotation, the following definition must be added to the configuration file: <?xml version="1.0" encoding="UTF-8"?> <beans xsi_schemaLocation="http://www.springframework.org/schema/beans    http://www.springframework.org/schema/beans/spring-beans.xsd    http://www.springframework.org/schema/context    http://www.springframework.org/schema/context/spring-context-    3.2.xsd">   <context:annotation-config />                             </beans> Aspect-oriented programming (AOP) supports in Spring AOP is used in Spring to provide declarative enterprise services, especially as a replacement for EJB declarative services. Application objects do what they're supposed to do—perform business logic—and nothing more. They are not responsible for (or even aware of) other system concerns, such as logging, security, auditing, locking, and event handling. AOP is a methodology of applying middleware services, such as security services, transaction management services, and so on on the Spring application. Declaring an aspect An aspect can be declared by annotating the POJO class with the @Aspect annotation. This aspect is required to import the org.aspectj.lang.annotation.aspect package. The following code snippet represents the aspect declaration in the @AspectJ form: import org.aspectj.lang.annotation.Aspect; import org.springframework.stereotype.Component;   @Aspect @Component ("myAspect") public class AspectModule { // ... } JDBC with the Spring Framework The DriverManagerDataSource class is used to configure the DataSource for application, which is defined in the Spring.xml configuration file. The central class of Spring JDBC's abstraction framework is the JdbcTemplate class that includes the most common logic in using the JDBC API to access data (such as handling the creation of connection, creation of statement, execution of statement, and release of resources). The JdbcTemplate class resides in the org.springframework.jdbc.core package. JdbcTemplate can be used to execute different types of SQL statements. DML is an abbreviation of data manipulation language and is used to retrieve, modify, insert, update, and delete data in a database. Examples of DML are SELECT, INSERT, or UPDATE statements. DDL is an abbreviation of data definition language and is used to create or modify the structure of database objects in a database. Examples of DDL are CREATE, ALTER, and DROP statements. The JDBC batch operation in Spring The JDBC batch operation allows you to submit multiple SQL DataSource to process at once. Submitting multiple SQL DataSource together instead of separately improves the performance: JDBC with batch processing Hibernate with the Spring Framework Data persistence is an ability of an object to save its state so that it can regain the same state. Hibernate is one of the ORM libraries that is available to the open source community. Hibernate is the main component available for a Java developer with features such as POJO-based approach and supports relationship definitions. The object query language used by Hibernate is called as Hibernate Query Language (HQL). HQL is an SQL-like textual query language working at a class level or a field level. Let's start learning the architecture of Hibernate. Hibernate annotations is the powerful way to provide the metadata for the object and relational table mapping. Hibernate provides an implementation of the Java Persistence API so that we can use JPA annotations with model beans. Hibernate will take care of configuring it to be used in CRUD operations. The following table explains JPA annotations: JPA annotation Description @Entity The javax.persistence.Entity annotation is used to mark a class as an entity bean that can be persisted by Hibernate, as Hibernate provides the JPA implementation. @Table The javax.persistence.Table annotation is used to define table mapping and unique constraints for various columns. The @Table annotation provides four attributes, which allows you to override the name of the table, its catalogue, and its schema. This annotation also allows you to enforce unique constraints on columns in the table. For now, we will just use the table name as Employee. @Id Each entity bean will have a primary key, which you annotate on the class with the @Id annotation. The javax.persistence.Id annotation is used to define the primary key for the table. By default, the @Id annotation will automatically determine the most appropriate primary key generation strategy to be used. @GeneratedValue javax.persistence.GeneratedValue is used to define the field that will be autogenerated. It takes two parameters, that is, strategy and generator. The GenerationType.IDENTITY strategy is used so that the generated id value is mapped to the bean and can be retrieved in the Java program. @Column javax.persistence.Column is used to map the field with the table column. We can also specify the length, nullable, and uniqueness for the bean properties. Object-relational mapping (ORM, O/RM, and O/R mapping) ORM stands for Object-relational Mapping. ORM is the process of persisting objects in a relational database such as RDBMS. ORM bridges the gap between object and relational schemas, allowing object-oriented application to persist objects directly without having the need to convert object to and from a relational format: Hibernate Query Language (HQL) Hibernate Query Language (HQL) is an object-oriented query language that works on persistence object and their properties instead of operating on tables and columns. To use HQL, we need to use a query object. Query interface is an object-oriented representation of HQL. The query interface provides many methods; let's take a look at a few of them: Method Description public int executeUpdate() This is used to execute the update or delete query public List list() This returns the result of the relation as a list public Query setFirstResult(int rowno) This specifies the row number from where a record will be retrieved public Query setMaxResult(int rowno) This specifies the number of records to be retrieved from the relation (table) public Query setParameter(int position, Object value) This sets the value to the JDBC style query parameter public Query setParameter(String name, Object value) This sets the value to a named query parameter The Spring Web MVC Framework Spring Framework supports web application development by providing comprehensive and intensive support. The Spring MVC framework is a robust, flexible, and well-designed framework used to develop web applications. It's designed in such a way that development of a web application is highly configurable to Model, View, and Controller. In an MVC design pattern, Model represents the data of a web application, View represents the UI, that is, user interface components, such as checkbox, textbox, and so on, that are used to display web pages, and Controller processes the user request. Spring MVC framework supports the integration of other frameworks, such as Struts and WebWork, in a Spring application. This framework also helps in integrating other view technologies, such as Java Server Pages (JSP), velocity, tiles, and FreeMarker in a Spring application. The Spring MVC Framework is designed around a DispatcherServlet. The DispatcherServlet dispatches the http request to handler, which is a very simple controller interface. The Spring MVC Framework provides a set of the following web support features: Powerful configuration of framework and application classes: The Spring MVC Framework provides a powerful and straightforward configuration of framework and application classes (such as JavaBeans). Easier testing: Most of the Spring classes are designed as JavaBeans, which enable you to inject the test data using the setter method of these JavaBeans classes. The Spring MVC framework also provides classes to handle the Hyper Text Transfer Protocol (HTTP) requests (HttpServletRequest), which makes the unit testing of the web application much simpler. Separation of roles: Each component of a Spring MVC Framework performs a different role during request handling. A request is handled by components (such as controller, validator, model object, view resolver, and the HandlerMapping interface). The whole task is dependent on these components and provides a clear separation of roles. No need of the duplication of code: In the Spring MVC Framework, we can use the existing business code in any component of the Spring MVC application. Therefore, no duplicity of code arises in a Spring MVC application. Specific validation and binding: Validation errors are displayed when any mismatched data is entered in a form. DispatcherServlet in Spring MVC The DispatcherServlet of the Spring MVC Framework is an implementation of front controller and is a Java Servlet component for Spring MVC applications. DispatcherServlet is a front controller class that receives all incoming HTTP client request for the Spring MVC application. DispatcherServlet is also responsible for initializing the framework components that will be used to process the request at various stages. The following code snippet declares the DispatcherServlet in the web.xml deployment descriptor: <servlet> <servlet-name>SpringDispatcher</servlet-name> <servlet-class>    org.springframework.web.DispatcherServlet </servlet-class> <load-on-startup>1</load-on-startup> </servlet>   <servlet-mapping> <servlet-name>SpringDispatcher</servlet-name> <url-pattern>/</url-pattern> </servlet-mapping> In the preceding code snippet, the user-defined name of the DispatcherServlet class is SpringDispatcher, which is enclosed with the <servlet-name> element. When our newly created SpringDispatcher class is loaded in a web application, it loads an application context from an XML file. DispatcherServlet will try to load the application context from a file named SpringDispatcher-servlet.xml, which will be located in the application's WEB-INF directory: <beans xsi_schemaLocation=" http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-3.0.xsd http://www.springframework.org/schema/context http://www.springframework.org/schema/context/spring-context- 3.0.xsd http://www.springframework.org/schema/mvc http://www.springframework.org/schema/mvc/spring-mvc-3.0.xsd">   <mvc:annotation-driven />   <context:component-scan base- package="org.packt.Spring.chapter7.springmvc" />   <beanclass="org.springframework.web.servlet.view. InternalResourceViewResolver">    <property name="prefix" value="/WEB-INF/views/" />    <property name="suffix" value=".jsp" /> </bean>   </beans> Spring Security The Spring Security framework is the de facto standard to secure Spring-based applications. The Spring Security framework provides security services for enterprise Java software applications by handling authentication and authorization. The Spring Security framework handles authentication and authorization at the web request and the method invocation level. The two major operations provided by Spring Security are as follows: Authentication: Authentication is the process of assuring that a user is the one who he/she claims to be. It's a combination of identification and verification. The identification process can be performed in a number of different ways, that is, username and password that can be stored in a database, LDAP, or CAS (single sign-out protocol), and so on. Spring Security provides a password encoder interface to make sure that the user's password is hashed. Authorization: Authorization provides access control to an authenticated user. It's the process of assurance that the authenticated user is allowed to access only those resources that he/she is authorized for use. Let's take a look at an example of the HR payroll application, where some parts of the application have access to HR and to some other parts, all the employees have access. The access rights given to user of the system will determine the access rules. In a web-based application, this is often done by URL-based security and is implemented using filters that play an primary role in securing the Spring web application. Sometimes, URL-based security is not enough in web application because URLs can be manipulated and can have relative pass. So, Spring Security also provides method level security. An authorized user will only able to invoke those methods that he is granted access for. Securing web application's URL access HttpServletRequest is the starting point of Java's web application. To configure web security, it's required to set up a filter that provides various security features. In order to enable Spring Security, add filter and their mapping in the web.xml file: <!—Spring Security --> <filter> <filter-name>springSecurityFilterChain</filter-name> <filter-class>org.springframework.web.filter. DelegatingFilterProxy</filter-class> </filter>   <filter-mapping> <filter-name>springSecurityFilterChain</filter-name> <url-pattern>/*</url-pattern> </filter-mapping> Logging in to a web application There are multiple ways supported by Spring security for users to log in to a web application: HTTP basic authentication: This is supported by Spring Security by processing the basic credentials presented in the header of the HTTP request. It's generally used with stateless clients, who on each request pass their credential. Form-based login service: Spring Security supports the form-based login service by providing a default login form page for users to log in to the web application. Logout service: Spring Security supports logout services that allow users to log out of this application. Anonymous login: This service is provided by Spring Security that grants authority to an anonymous user, such as a normal user. Remember-me support: This is also supported by Spring Security and remembers the identity of a user across multiple browser sessions. Encrypting passwords Spring Security supports some hashing algorithms such as MD5 (Md5PasswordEncoder), SHA (ShaPasswordEncoder), and BCrypt (BCryptPasswordEncoder) for password encryption. To enable the password encoder, use the <password-encoder/> element and set the hash attribute, as shown in the following code snippet: <authentication-manager> <authentication-provider>    <password-encoder hash="md5" />    <jdbc-user-service data-source-    ref="dataSource"    . . .   </authentication-provider> </authentication-manager> Mail support in the Spring Framework The Spring Framework provides a simplified API and plug-in for full e-mail support, which minimizes the effect of the underlying e-mailing system specifications. The Sprig e-mail supports provide an abstract, easy, and implementation independent API to send e-mails. The Spring Framework provides an API to simplify the use of the JavaMail API. The classes handle the initialization, cleanup operations, and exceptions. The packages for the JavaMail API provided by the Spring Framework are listed as follows: Package Description org.springframework.mail This defines the basic set of classes and interfaces to send e-mails. org.springframework.mail.java This defines JavaMail API-specific classes and interfaces to send e-mails. Spring's Java Messaging Service (JMS) Java Message Service is a Java Message-oriented middleware (MOM) API responsible for sending messages between two or more clients. JMS is a part of the Java enterprise edition. JMS is a broker similar to a postman who acts like a middleware between the message sender and the receiver. Message is nothing, but just bytes of data or information exchanged between two parties. By taking different specifications, a message can be described in various ways. However, it's nothing, but an entity of communication. A message can be used to transfer a piece of information from one application to another, which may or may not run on the same platform. The JMS application Let's look at the sample JMS application pictorial, as shown in the following diagram: We have a Sender and a Receiver. The Sender is responsible for sending a message and the Receiver is responsible for receiving a message. We need a broker or MOM between the Sender and Receiver, who takes the sender's message and passes it from the network to the receiver. Message oriented middleware (MOM) is basically an MQ application such as ActiveMQ or IBM-MQ, which are two different message providers. The sender promises loose coupling and it can be .NET or mainframe-based application. The receiver can be Java or Spring-based application and it sends back the message to the sender as well. This is a two-way communication, which is loosely coupled. Summary This article covered the architecture of Spring Framework and how to set up the key components of the Spring application development environment. Resources for Article: Further resources on this subject: Creating an Extension in Yii 2 [article] Serving and processing forms [article] Time Travelling with Spring [article]

Congratulations, you’ve written a web application! Now what? Part one of this post deals with steps to take after development, more specifically the creation of a Docker image that contains the application. In part two, I’ll lay out deploying that image to the Google Cloud Platform as well as some further reading that'll help you descend into the rabbit hole that is DevOps. For demonstration purposes, let’s say that you’re me and you want to share your adventures in TrapRap and Death Metal (not simultaneously, thankfully!) with the world. I’ve written a simple Ember frontend for this purpose, and through the course of this post, I will explain to you how I go about containerizing it. Of course, the beauty of this procedure is that it will work with any frontend application, and you are certainly welcome to Bring Your Own Code. Everything I use is publically available on GitHub, however, and you’re certainly welcome to work through this post with the material presented as well. So, I’ve got this web app. You can get it here, or you can run: $ git clone https://github.com/ndarwincorn/docker-demo.git Do this for wherever it is you keep your source code. You’ll need ember-cli and some familiarity with Ember to customize it yourself, or you can just cut to the chase and build the Docker image, which is what I’m going to do in this post. I’m using Docker 1.10, but there’s no reason this wouldn’t work on a Mac running Docker Toolbox (or even Boot2Docker, but don’t quote me on that) or a less bleeding edge Linux distro. Since installing Docker is well documented, I won’t get into that here and will continue with the assumption that you have a working, up-to-date Docker installed on your machine, and that the Docker daemon is running. If you’re working with your own app, feel free to skip below to my explanation of the process and then come back here when you’ve got a Dockerfile in the root of your application. In the root of the application, run the following (make sure you don’t have any locally-installed web servers listening on port 80 already): # docker build -t docker-demo . # docker run -d -p 80:80 --name demo docker-demo Once the command finishes by printing a container ID, launch a web browser and navigate to http://localhost. Hey! Now you can listen to my music served from a LXC container running on your very own computer. How did we accomplish this? Let’s take it piece-by-piece (here’s where to start reading again if you’ve approached this article with your own app): I created a simple Dockerfile using the official Nginx image because I have a deep-seated mistrust of Canonical and don’t want to use the Dockerfile here. Here’s what it looks like in my project: docker-demo/Dockerfile FROM nginx COPY dist/usr/share/nginx/html Running the docker build command reads the Dockerfile and uses it to configure a docker image based on the nginx image. During image configuration, it copies the contents of the dist folder in my project to /srv/http/docker-demo in the container, which the nginx configuration that was mentioned is pointed to. The -t flag tells Docker to ‘tag’ (name) the image we’ve just created as ‘docker-demo’. The docker run command takes that image and builds a container from it. The -d flag is short for ‘detach’, or run the /usr/bin/nginx command built into the image from our Dockerfile and leave the container running. The -p flag maps a port on the host to a port in the container, and --name names the container for later reference. The command should return a container ID that can be used to manipulate it later. In part two, I’ll show you how to push the image we created to the Google Cloud Platform and then launch it as a container in a specially-purposed VM on their Compute Engine. About the Author Darwin Corn is a Systems Analyst for the Consumer Direct Care Network. He is a mid-level professional with diverse experience in the Information Technology world.

In this article, by Richard Grimmett, author of Mastering BeagleBone Robotics, you'll build a quadruped, that is, a robot with four legs. You'll be using 12 servos so that each leg has three points that can move, or three Degrees of Freedom (DOF). In this project, you'll control 12 servos at the same time, so it will make more sense to use an external servo controller that can supply the control signals and supply voltages for all the 12 servos. (For more resources related to this topic, see here.) Since servos are the main component of this project, it is perhaps useful to go through a tutorial on servos and how to control them. Working of servomotors Servomotors are somewhat similar to DC motors; however, there is an important difference. While DC motors are generally designed to move in a continuous way—rotating 360 degrees at a given speed—servos are generally designed to move within a limited set of angles. In other words, in the case of a DC motor, you would generally want your motors to spin with continuous rotation speed that you control. But in the case of a servomotor, you would want your motor to move to a specific position that you control. This is done by sending a Pulse-Width-Modulated (PWM) signal to the control connector of the servo. PWM simply means that you are going to change the length of each pulse of electrical energy in order to control something. In this case, the length of this pulse will control the angle of the servo, as shown in the following diagram: These pulses are sent out with a repetition rate of 60 Hz. You can position the servo to any angle by setting the correct control pulse. Building the quadruped platform You'll first need some parts so that you can build your quadruped robot. There are several kit possibilities out there, including the one available at www.trossenrobotics.com/p/PhantomX-AX-12-Quadruped.aspx. However, such kits can be expensive, so for this example, you'll create your own kit using a set of Lynxmotion parts. These are available from several online retailers such as robotshop.com. To build this quadruped, you'll need four legs, each leg requires two Lynxmotion parts. Here are the parts with their Robotshop part numbers: Quantity Description 1 Lynxmotion symmetric quadruped body kit: Mini QBK-02 2 Lynxmotion 3'' aluminum femur pair 2 Lynxmotion Robot Leg "A" pair (No servo) RL-01 4 Lynxmotion aluminum multi-purpose servo bracket Two Pack ASB-04 2 Ball bearing with flange: 3mm ID (pair) Product code: RB-Lyn-317 This last part a bearing you'll need to connect the leg to the body. You'll also need 12 servos of a standard size. There are several possible choices, but I personally like the Hitec servos. They are very inexpensive and you can get them from most hobby shops and online electronic retailers. Now, for a moment on the model of servo: Servos come in different model numbers, primarily based on the amount of torque they can generate. Torque is the force that the servo can exert to move the part connected to it. In this case, your servos will need to lift and move the weight associated with your quadruped, so you'll need a servo with enough torque to do this. I suggest that you use eight Hitec model HS-485HB servos. You'll use these for the servos attached to the end of the leg and for the body. Then you'll use four Hitec model HS-645MG servos for the middle of the leg; this is the servo that will require the highest amount of torque. You can use the 12 Hitec model HS-645MG servos instead, but they are more expensive than the HS-485 servos, so using two different servos will be less expensive. Here are the steps required to assemble the quadruped: Put the two parts of the lower right leg together and insert the servo with the servo mounting screws. It should look like this: Now connect this assembly to the interconnect part, like this: Complete the leg by connecting two of the servo brackets together at right angles, mounting the HS-645MG on one of the brackets, and then connecting this servo to the interconnect piece, like this: Put another right leg together. Now put two left legs together following the same preceding steps, but in left leg configuration. They look like this: The next step is to build the body kit. There are some instructions given at www.lynxmotion.com/images/html/sq3u-assembly.htm, but it should be like the following image: Then connect each leg to the body kit. First connect the empty servo bracket to the body using the bearing, as shown in the following image: Now connect the other servo to the empty servo bracket and the body, like this: After performing all the preceding steps, your quadruped should now look like this: Now that you have the basic hardware assembled, you can turn your attention to the electronics. Using a servo controller to control the servos To make your quadruped walk, you will first need to connect the servomotor controller to the servos. The servo controller you are going to use for this project is a simple servomotor controller utilizing USB from Pololu (Pololu item number 1354, available at pololu.com) that can control 18 servomotors. Here is an image of the unit: Make sure that you order the assembled version. This piece of hardware will turn USB commands from the BeagleBone Black into signals that control your servomotors. Pololu makes a number of different versions of this controller, each able to control a certain number of servos. In this case, you may want to choose the 18-servo version, so you can control all 12 servos with one controller and also add an additional servo to control the direction of a camera or sensor. You could also choose the 12-servo version. One advantage of the 18-servo controller is the ease of connecting power to the unit via screw-type connectors. There are two connections you'll need to make to the servo controller to get started: the first is to the servomotors and the second is to a battery. First, connect the servos to the controller. In order to be consistent, let's connect your 12 servos to the connections marked 0 through 11 on the controller using the configuration shown in the following table: Servo Connector Servo 0 Right front lower leg 1 Right front middle leg 2 Right front upper leg 3 Right rear lower leg 4 Right rear middle leg 5 Right rear upper leg 6 Left front lower leg 7 Left front middle leg 8 Left front upper leg 9 Left rear lower leg 10 Left rear middle leg 11 Left rear upper leg Here is an image of the back of the controller. This will tell us where to connect our servos: Now you need to connect the servomotor controller to your battery. For this project, you can use a 2S RC LiPo battery. The 2S means that the battery will have two cells, with an output voltage of 7.2 volts. It will supply the voltage and current needed by your servos, which can be of the order of 2 amperes. Here is an image of the battery: This battery will come with two connectors: one with large gauge wires for normal usage and a smaller connector used to connect to the battery recharger. You'll want to build connectors that can connect to the screw-type connectors of the servo controller. I purchased some XT60 connector pairs, soldered some wires to the mating connector of the battery, and screwed these into the servo controller. Your system is now functional. Now you'll connect the motor controller to your personal computer to check if you can communicate with it. To do this, connect a mini USB cable between the servo controller and your personal computer. Communicating with the servo controller via a PC Now that the hardware is connected, you can use some software provided by Pololu to control the servos. Let's do this using your personal computer. First download the Pololu software from www.pololu.com/docs/0J40/3.a and install it according to the instructions on the website. Once it is installed, run the software, and you should see something like the following screenshot: You will first need to change the configuration of the serial settings, so select the Serial Settings tab and you should see this: Make sure that USB Chained is selected; this will allow you to communicate with and control the motor controller over USB. Now go back to the main screen by selecting the Status tab, and now you can actually turn on the 12 servos. The screen should look like this screenshot: Now you can use the sliders to actually control the servos. Check that servo 0 moves the right front lower servo, servo 1 moves the right front middle servo, servo 2 moves the right front upper servo, and so on. You can also use this to center the servos. Set all the servos such that the slider is in the middle. Now unscrew the servo horn on each servo until the servos are centered at this location. At the zero degree location of all servos, your quadruped should look like this: Your quadruped is now ready to actually do something. Now you'll need to send the servos the electronic signals they need to move your quadruped. Connecting the servo controller to the BeagleBone Black You've checked the servomotor controller and the servos. You'll now connect the motor controller to the BeagleBone Black and make sure you can control the servos from it. Remove the USB cable from the PC and connect it to the BeagleBone Black. The entire system will look like this: Let's now talk to the motor controller by downloading the Linux code from Pololu at www.pololu.com/docs/0J40/3.b. Perhaps the best way is to log in to your BeagleBone Black using PuTTY, then type wget http://www.pololu.com/file/download/maestro-linux-100507.tar.gz?file_id=0J315. Then move the file by typing mv maestro-linux-100507.tar.gz?file_id=0J315 maestro-linux-100507.tar.gz. Unpack the file by typing tar –xzfv maestro_linux_011507.tar.gz. This will create a directory called maestro_linux. Go to that directory by typing cd maestro_linux and then type ls. You should see something like this: The README.txt document will give you explicit instructions on how to install the software. Unfortunately, you can't run MaestroControlCenter on your BeagleBone Black. Your version of Windows doesn't support the graphics, but you can control your servos using the UscCmd command-line application to ensure that they are connected and working correctly. First, type ./UscCmd --list and you should see something like the following screenshot: The unit sees our servo controller. By just typing ./UscCmd, you can see all the commands that you can send to your controller, as shown in the following screenshot: Note that although you can send a servo a specific target angle, the target is not in angle values, so it makes it a bit difficult to know where you are sending your servo. Try typing ./UscCmd --servo 0, 10. The servo will move to its maximum angle position. Type ./UscCmd – servo 0, 0 and it will prevent the servo from trying to move. In the next section, you'll write some Python code that will translate your angles to the commands that the servo controller will want to receive to move it to specific angle locations. If you didn't run the Windows version of Maestro Controller and set the Serial Settings to USB Chained, your motor controller might not respond. Rerun the Maestro Controller code and set the Serial Settings to USB Chained. Creating a program on Linux to control your quadruped You now know that you can talk to your servomotor controller, and move your servos. In this section, you'll create a Python program that will let you talk to your servos to move them to specific angles. Let's start with a simple program that will make your legged mobile robot's servos go to 90 degrees (the middle of the 0 to 180 degrees you can set). This particular controller uses bytes of information, so the code will translate the input of the channel and angle to numbers that the controller can understand. For more details, see http://www.pololu.com/docs/0J40. Here is the code to move all the connected servos to the 90 degree point: Here is an explanation of the code: #! /usr/bin/python: This first line allows you to make this Python file executable from the command line. import serial: This line imports the serial library. You need the serial library to talk to your unit via USB. def setAngle(ser, channel, angle): This function converts your desired setting of servo and angle into the serial command that the servomotor controller needs. ser = serial.Serial("/dev/ttyACM0", 9600): This opens the serial port connection to your servo controller. for i in range(0, 15): This is for all 16 servo possibilities. setAngle(ser, i, 90): This allows you to set each servo to the middle (home) position. The default would be to set each servo to 90 degrees. If the legs of your robot aren't in their middle position, you can adjust them by adjusting the position of the servo horns on each servo. To access the serial port, you'll need to make sure that you have the Python serial library. If you don't, then type sudo apt-get install python-serial. After you have installed the serial library, you can run your program by typing sudo python quad.py. Once you have the basic home position set, you can now ask your robot to do some things. Let's start by making your quadruped wave an arm. Here is the Python code that waves the arm: In this case, you are using the setAngle command to set your servos to manipulate your robot's front-right arm. The middle servo raises the arm, and the lower servo then goes back and forth between angle 100 and 130. One of the most basic actions you'll want your robot to do is to walk forward. Here is an example of how to manipulate the legs to make this happen: This program lifts and moves each leg forward one at a time, and then moves all the legs to home position, which moves the robot forward. Not the most elegant motion, but it does work. There are more sophisticated algorithms to make your quadruped walk as shown at http://letsmakerobots.com/node/35354 and https://www.youtube.com/watch?v=jWP3RnYa_tw. Once you have the program working, you'll want to package all of your hardware onto the mobile robot. You can make your robot do many amazing things: walk forward, walk backward, dance, turn around—any kinds of movements are possible. The best way to learn is to try new and different positions with the servos. Issuing voice commands to your quadruped You should now have a mobile platform that you can program to move in any number of ways. Unfortunately, you still have your LAN cable connected, so the platform isn't completely mobile. And once you have begun the program, you can't alter the behavior of your program. You'll need to modify your voice recognition program so that it can run your Python program when it gets a voice command. You have to make a simple modification to the continuous.c program in /home/ubuntu/pocketsphinx-0.8/src/programs. To do this, type cd /home/ubuntu/ pocketsphinx-0.8/src/programs, and then type emacs continuous.c. The changes will occur in the same section as your other voice commands and will look like this: The additions are pretty straightforward. Let's walk through them: else if (strcmp(word, "FORWARD") == 0): This checks the word as recognized by your voice command program. If it corresponds with the word FORWARD, it will execute everything inside the if statement. We use { } to tell the system which commands go with this else if clause. system("/home/ubuntu/maestro_linux/robot.py"): This is the program we will execute. In this case, our mobile platform will do whatever the robot.py program tells it to do. After doing this, you will need to recompile the program, so type make and the pocketSphinx_continuous executable will be created. Run the program by typing ./pocketSphinx_continuous. Don't forget the ./ at the start of this command or you'll run a different version of the program. When the program is running, you can disconnect the LAN cable, and the mobile platform will now take the forward voice command and execute your program. Summary You now have a robot than can walk! You can also add other sensors, like the ones you discovered for your tracked robot, sensors that can watch for barriers, or even a webcam. Resources for Article: Further resources on this subject: Protecting GPG Keys in BeagleBone [Article] Home Security by BeagleBone [Article] Introducing BeagleBoard [Article]

(For more resources related to this topic, see here.) Regarding the first task, the multiple selection can be activated using an HTML5 input file attribute (multiple) and the JSF 2.2 pass-through attribute feature. When this attribute is present and its value is set to multiple, the file chooser can select multiple files. So, this task requires some minimal adjustments: <html > ... <h:form id="uploadFormId" enctype="multipart/form-data"> <h:inputFile id="fileToUpload" required="true" f5:multiple="multiple" requiredMessage="No file selected ..." value="#{uploadBean.file}"/> <h:commandButton value="Upload" action="#{uploadBean.upload()}"/> </h:form> The second task is a little bit tricky, because when multiple files are selected, JSF will overwrite the previous Part instance with each file in the uploaded set. This is normal, since you use an object of type Part, but you need a collection of Part instances. Fixing this issue requires us to focus on the renderer of the file component. This renderer is named FileRenderer (an extension of TextRenderer), and the decode method implementation is the key for our issue (the bold code is very important for us), as shown in the following code: @Override public void decode(FacesContext context, UIComponent component) { rendererParamsNotNull(context, component); if (!shouldDecode(component)) { return; } String clientId = decodeBehaviors(context, component); if (clientId == null) { clientId = component.getClientId(context); } assert(clientId != null); ExternalContext externalContext = context.getExternalContext(); Map<String, String> requestMap = externalContext.getRequestParameterMap(); if (requestMap.containsKey(clientId)) { setSubmittedValue(component, requestMap.get(clientId)); } HttpServletRequest request = (HttpServletRequest) externalContext.getRequest(); try { Collection<Part> parts = request.getParts(); for (Part cur : parts) { if (clientId.equals(cur.getName())) { component.setTransient(true); setSubmittedValue(component, cur); } } } catch (IOException ioe) { throw new FacesException(ioe); } catch (ServletException se) { throw new FacesException(se); } } The highlighted code causes the override Part issue, but you can easily modify it to submit a list of Part instances instead of one Part, as follows: try { Collection<Part> parts = request.getParts(); List<Part> multiple = new ArrayList<>(); for (Part cur : parts) { if (clientId.equals(cur.getName())) { component.setTransient(true); multiple.add(cur); } } this.setSubmittedValue(component, multiple); } catch (IOException | ServletException ioe) { throw new FacesException(ioe); } Of course, in order to modify this code, you need to create a custom file renderer and configure it properly in faces-config.xml. Afterwards, you can define a list of Part instances in your bean using the following code: ... private List<Part> files; public List<Part> getFile() { return files; } public void setFile(List<Part> files) { this.files = files; } ... Each entry in the list is a file; therefore, you can write them on the disk by iterating the list using the following code: ... for (Part file : files) { try (InputStream inputStream = file.getInputStream(); FileOutputStream outputStream = new FileOutputStream("D:" + File.separator + "files"+ File.separator + getSubmittedFileName())) { int bytesRead = 0; final byte[] chunck = new byte[1024]; while ((bytesRead = inputStream.read(chunck)) != -1) { outputStream.write(chunck, 0, bytesRead); } FacesContext.getCurrentInstance().addMessage(null, new FacesMessage("Upload successfully ended: " + file.getSubmittedFileName())); } catch (IOException e) { FacesContext.getCurrentInstance().addMessage(null, new FacesMessage("Upload failed !")); } } ... Upload and the indeterminate progress bar When users upload small files, the process happens pretty fast; however, when large files are involved, it may take several seconds, or even minutes, to end. In this case, it is a good practice to implement a progress bar that indicates the upload status. The simplest progress bar is known as an indeterminate progress bar, because it shows that the process is running, but it doesn't provide information for estimating the time left or the amount of processed bytes. In order to implement a progress bar, you need to develop an AJAX-based upload. The JSF AJAX mechanism allows us to determine when the AJAX request begins and when it completes. This can be achieved on the client side; therefore, an indeterminate progress bar can be easily implemented using the following code: <script type="text/javascript"> function progressBar(data) { if (data.status === "begin") { document.getElementById("uploadMsgId").innerHTML=""; document.getElementById("progressBarId"). setAttribute("src", "./resources/progress_bar.gif"); } if (data.status === "complete") { document.getElementById("progressBarId").removeAttribute("src"); } } </script> ... <h:body> <h:messages id="uploadMsgId" globalOnly="true" showDetail="false" showSummary="true" style="color:red"/> <h:form id="uploadFormId" enctype="multipart/form-data"> <h:inputFile id="fileToUpload" required="true" requiredMessage="No file selected ..." value="#{uploadBean.file}"/> <h:message showDetail="false" showSummary="true" for="fileToUpload" style="color:red"/> <h:commandButton value="Upload" action="#{uploadBean.upload()}"> <f:ajax execute="fileToUpload" onevent="progressBar" render=":uploadMsgId @form"/> </h:commandButton> </h:form> <div> <img id="progressBarId" width="250px;" height="23"/> </div> </h:body> A possible output is as follows: Upload and the determinate progress bar A determinate progress bar is much more complicated. Usually, such a progress bar is based on a listener capable to monitor the transferred bytes (if you have worked with Apache Commons' FileUpload, you must have had the chance to implement such a listener). In JSF 2.2, FacesServlet was annotated with @MultipartConfig for dealing multipart data (upload files), but there is no progress listener interface for it. Moreover, FacesServlet is declared final; therefore, we cannot extend it. Well, the possible approaches are pretty limited by these aspects. In order to implement a server-side progress bar, we need to implement the upload component in a separate class (servlet) and provide a listener. Alternatively, on the client side, we need a custom POST request that tricks FacesServlet that the request is formatted by jsf.js. In this article, you will see a workaround based on HTML5 XMLHttpRequest Level 2 (can upload/download streams as Blob, File, and FormData), HTML5 progress events (for upload it returns total transferred bytes and uploaded bytes), HTML5 progress bar, and a custom Servlet 3.0. If you are not familiar with these HTML5 features, then you have to check out some dedicated documentation. After you get familiar with these HTML5 features, it will be very easy to understand the following client-side code. First we have the following JavaScript code: <script type="text/javascript"> function fileSelected() { hideProgressBar(); updateProgress(0); document.getElementById("uploadStatus").innerHTML = ""; var file = document.getElementById('fileToUploadForm: fileToUpload').files[0]; if (file) { var fileSize = 0; if (file.size > 1048576) fileSize = (Math.round(file.size * 100 / (1048576)) / 100).toString() + 'MB'; else fileSize = (Math.round(file.size * 100 / 1024) / 100).toString() + 'KB'; document.getElementById('fileName').innerHTML = 'Name: ' + file.name; document.getElementById('fileSize').innerHTML = 'Size: ' + fileSize; document.getElementById('fileType').innerHTML = 'Type: ' + file.type; } } function uploadFile() { showProgressBar(); var fd = new FormData(); fd.append("fileToUpload", document.getElementById('fileToUploadForm: fileToUpload').files[0]); var xhr = new XMLHttpRequest(); xhr.upload.addEventListener("progress", uploadProgress, false); xhr.addEventListener("load", uploadComplete, false); xhr.addEventListener("error", uploadFailed, false); xhr.addEventListener("abort", uploadCanceled, false); xhr.open("POST", "UploadServlet"); xhr.send(fd); } function uploadProgress(evt) { if (evt.lengthComputable) { var percentComplete = Math.round(evt.loaded * 100 / evt.total); updateProgress(percentComplete); } } function uploadComplete(evt) { document.getElementById("uploadStatus").innerHTML = "Upload successfully completed!"; } function uploadFailed(evt) { hideProgressBar(); document.getElementById("uploadStatus").innerHTML = "The upload cannot be complete!"; } function uploadCanceled(evt) { hideProgressBar(); document.getElementById("uploadStatus").innerHTML = "The upload was canceled!"; } var updateProgress = function(value) { var pBar = document.getElementById("progressBar"); document.getElementById("progressNumber").innerHTML=value+"%"; pBar.value = value; } function hideProgressBar() { document.getElementById("progressBar").style.visibility = "hidden"; document.getElementById("progressNumber").style.visibility = "hidden"; } function showProgressBar() { document.getElementById("progressBar").style.visibility = "visible"; document.getElementById("progressNumber").style.visibility = "visible"; } </script> Further, we have the upload component that uses the preceding JavaScript code: <h:body> <hr/> <div id="fileName"></div> <div id="fileSize"></div> <div id="fileType"></div> <hr/> <h:form id="fileToUploadForm" enctype="multipart/form-data"> <h:inputFile id="fileToUpload" onchange="fileSelected();"/> <h:commandButton type="button" onclick="uploadFile()" value="Upload" /> </h:form> <hr/> <div id="uploadStatus"></div> <table> <tr> <td> <progress id="progressBar" style="visibility: hidden;" value="0" max="100"></progress> </td> <td> <div id="progressNumber" style="visibility: hidden;">0 %</div> </td> </tr> </table> <hr/> </h:body> A possible output can be seen in the following screenshot: The servlet behind this solution is UploadServlet that was presented earlier. For multiple file uploads and progress bars, you can extend this example, or choose a built-in solution, such as PrimeFaces Upload, RichFaces Upload, or jQuery Upload Plugin. Summary In this article, we saw how to upload multiple files using JSF 2.2 and the concepts of indeterminate and determinate progress bars. Resources for Article: Further resources on this subject: The Business Layer (Java EE 7 First Look) [article] The architecture of JavaScriptMVC [article] Differences in style between Java and Scala code [article]

We have already talked about a simple learning roadmap for you to develop your data science skills in the first resolution. We also talked about the importance of staying relevant in an increasingly automated job market, in our second resolution. Now it’s time to think about the kind of person you want to be and the legacy you will leave behind. 3rd Resolution: Choose projects wisely and be mindful of their impact. Your work has real consequences. And your projects will often be larger than what you know or can do. As such, the first step toward creating impact with intention is to define the project scope, purpose, outcomes and assets clearly. The next most important factor is choosing the project team. 1. Seek out, learn from and work with a diverse group of people To become a successful data scientist you must learn how to collaborate. Not only does it make projects fun and efficient, but it also brings in diverse points of view and expertise from other disciplines. This is a great advantage for machine learning projects that attempt to solve complex real-world problems. You could benefit from working with other technical professionals like web developers, software programmers, data analysts, data administrators, game developers etc. Collaborating with such people will enhance your own domain knowledge and skills and also let you see your work from a broader technical perspective. Apart from the people involved in the core data and software domain, there are others who also have a primary stake in your project’s success. These include UX designers, people with humanities background if you are building a product intended to participate in society (which most products often are), business development folks, who actually sell your product and bring revenue, marketing people, who are responsible for bringing your product to a much wider audience to name a few. Working with people of diverse skill sets will help market your product right and make it useful and interpretable to the target audience. In addition to working with a melange of people with diverse skill sets and educational background it is also important to work with people who think differently from you, and who have experiences that are different from yours to get a more holistic idea of the problems your project is trying to tackle and to arrive at a richer and unique set of solutions to solve those problems. 2. Educate yourself on ethics for data science As an aspiring data scientist, you should always keep in mind the ethical aspects surrounding privacy, data sharing, and algorithmic decision-making.  Here are some ways to develop a mind inclined to designing ethically-sound data science projects and models. Listen to seminars and talks by experts and researchers in fairness, accountability, and transparency in machine learning systems. Our favorites include Kate Crawford’s talk on The trouble with bias, Tricia Wang on The human insights missing from big data and Ethics & Data Science by Jeff Hammerbacher. Follow top influencers on social media and catch up with their blogs and about their work regularly. Some of these researchers include Kate Crawford, Margaret Mitchell, Rich Caruana, Jake Metcalf, Michael Veale, and Kristian Lum among others. Take up courses which will guide you on how to eliminate unintended bias while designing data-driven algorithms. We recommend Data Science Ethics by the University of Michigan, available on edX. You can also take up a course on basic Philosophy from your choice of University.   Start at the beginning. Read books on ethics and philosophy when you get long weekends this year. You can begin with Aristotle's Nicomachean Ethics to understand the real meaning of ethics, a term Aristotle helped develop. We recommend browsing through The Stanford Encyclopedia of Philosophy, which is an online archive of peer-reviewed publication of original papers in philosophy, freely accessible to Internet users. You can also try Practical Ethics, a book by Peter Singer and The Elements of Moral Philosophy by James Rachels. Attend or follow upcoming conferences in the field of bringing transparency in socio-technical systems. For starters, FAT* (Conference on Fairness, Accountability, and Transparency) is scheduled on February 23 and 24th, 2018 at New York University, NYC. We also have the 5th annual conference of FAT/ML, later in the year.  3. Question/Reassess your hypotheses before, during and after actual implementation Finally, for any data science project, always reassess your hypotheses before, during, and after the actual implementation. Always ask yourself these questions after each of the above steps and compare them with the previous answers. What question are you asking? What is your project about? Whose needs is it addressing? Who could it adversely impact? What data are you using? Is the data-type suitable for your type of model? Is the data relevant and fresh? What are its inherent biases and limitations? How robust are your workarounds for them? What techniques are you going to try? What algorithms are you going to implement? What would be its complexity? Is it interpretable and transparent? How will you evaluate your methods and results? What do you expect the results to be? Are the results biased? Are they reproducible? These pointers will help you evaluate your project goals from a customer and business point of view. Additionally, it will also help you in building efficient models which can benefit the society and your organization at large. With this, we come to the end of our new year resolutions for an aspiring data scientist. However, the beauty of the ideas behind these resolutions is that they are easily transferable to anyone in any job. All you gotta do is get your foundations right, stay relevant, and be mindful of your impact. We hope this gives a great kick start to your career in 2018. “Motivation is what gets you started. Habit is what keeps you going.” ― Jim Ryun Happy New Year! May the odds and the God(s) be in your favor this year to help you build your resolutions into your daily routines and habits!

(For more resources related to this topic, see here.) Microsoft created Windows Management Instrumentation ( WMI ) as a management layer to the Windows operating system. This management layer allows you to retrieve information pertaining to the operating system or physical hardware on a system. It also allows you to manipulate components within the operating system. A good example for this section is a hard drive. WMI provides the ability to view the physical hard drive as well as the logical components of the hard drive. Using a call to the Win32_diskdrive class, you have the ability to view the physical aspects of the disk drive, such as the tracks, sectors, manufacturer, and even the serial number. The win32_logicaldisk class provides you with the ability to see the logical aspects of a drive, such as partitions, free space, and volume names. Not limited to just Windows operating system management, WMI also has the extensibility to allow third-party developers to create WMI providers for use within their projects. This means that you can create the management hooks within your code that will allow remote management through a common standardized framework. Many companies have adopted the use of WMI providers for items such as storage subsystems, application virtualization, hardware virtualization, and enhanced management of the hardware connected to a workstation or server system. Microsoft chose to follow the Common Information Model ( CIM ) industry standard for WMI. The preceding diagram takes a simplified look at Microsoft's implementation of WMI through the use of the CIM standard. There are three layers to their model, which are as follows: WMI consumers: The WMI consumers are exactly what their name states. They consume the available APIs to access the managed component. The WMI consumers are the real users of the C/C++ and .NET clients, and they use scripting languages, such as PowerShell 3.0, to access management data and interact with the managed components. In the hard drive example, the WMI consumer is the PowerShell code that calls information about the hard drive. This would look as follows: get-wmiobject –class win32_logicaldisk WMI infrastructure: The WMI infrastructure includes the CIM Object Manager ( CIMOM ), which stores a repository of the available WMI providers. If a third-party WMI provider doesn't register with the CIM Object Manager, the Windows operating system will not be able to manage the component through WMI. In the hard drive example, the CIMOM will import the Managed Object Format ( MOF) file of the hard drive into the WMI repository. This will register the hard drive's available WMI properties and methods into use by a WMI consumer. WMI providers: The last components, WMI providers, are made up of a driver (DLL) and a MOF file. These two components are responsible for returning the management data to the WMI consumer, through the WMI infrastructure. This allows the WMI consumer to interact with the managed components. In the hard drive example, the WMI consumer will access the hard drive through the WMI infrastructure utilizing the hard drive driver (DLL). The WMI consumer will have the ability to retrieve the information pertaining to the physical components on that hard drive. WMI integration with PowerShell 3.0 PowerShell has the ability to interact with WMI through the use of the built-in cmdlets. These cmdlets act as the WMI consumers and interact with the WMI. As WMI evolved with the release of new operating systems, PowerShell also needed to evolve in parallel to manage those systems. With the release of Windows 8 and Windows Server 2012, Microsoft created a new iteration of the Microsoft Windows Management Framework ( WMF ), Version 3.0. The new release of Windows Management Framework updates WMI to Version 3.0, PowerShell to Version 3.0, and installs the new Windows Remote Management ( WinRM ), OData IIS Extensions, and Server Manager CIM Provider. PowerShell 3.0 includes new WMI management cmdlets, displayed in the following table, which leverage the use of the new functionality within Windows Management Framework 3.0. The new CIM cmdlets provide a richer WMI experience leveraging stateful communications to the remote systems. They also provide the ability to create CIM calls through PowerShell 3.0 to non-Windows-based WMI systems that support Web Services-Management ( WSman ). This provides engineers the ability to tap into a variety of systems for management purposes. Get-CimAssociatedInstance New-CimSession Get-CimClass New-CimSessionOption Get-CimInstance Register-CimIndicationEvent Get-CimSession Remove-CimInstance Invoke-CimMethod Remove-CimSession New-CimInstance Set-CimInstance Using PowerShell in your environment PowerShell is quickly becoming the de-facto standard for managing the Windows-based systems in small and large organizations. It is being used for tasks, such as automated software deployments, dynamic system provisioning, and system maintenance. It is also being heavily used in Microsoft products, such as System Center 2012 - Service Manager, that allow for business process automation. PowerShell 3.0 has introduced a variety of new cmdlets that further simplify the administration of systems through the use of WMI. Whatever the administrative task, the PowerShell community has several examples of the ways to manage systems through the use of the WMI consumers. The examples provided in this article are just the tip of the iceberg compared to what you can accomplish utilizing PowerShell 3.0 and Windows Management Instrumentation. You may be surprised by the number of manual administration steps that can be automated by creating PowerShell scripts. The sky is the limit when it comes to scripting with PowerShell! Summary In this article, we learned what Windows Management Instrumentation (WMI) is, how PowerShell 3.0 utilizes it, and why it's applicable to systems engineers. Resources for Article : Further resources on this subject: Using the Windows Azure Platform PowerShell Cmdlets [Article] Accessing Oracle [Article] Inventorying Servers with PowerShell [Article]

In this article written by Benjamin Cane, author of the book Red Hat Enterprise Linux Troubleshooting Guide the author goes on to explain how before starting to explore troubleshooting commands, we should first cover locations of useful information. Useful information is a bit of an ubiquitous term, pretty much every file, directory, or command can provide useful information. What he really plans to cover are places where it is possible to find information for almost any issue. (For more resources related to this topic, see here.) Log files Log files are often the first place to start looking for troubleshooting information. Whenever a service or server is experiencing an issue, checking the log files for errors can often answer many questions quickly. The default location By default, RHEL and most Linux distributions keep their log files in /var/log/, which is actually part of the Filesystem Hierarchy Standard (FHS) maintained by the Linux Foundation. However, while /var/log/ might be the default location not all log files are located there(http://en.wikipedia.org/wiki/Filesystem_Hierarchy_Standard). While /var/log/httpd/ is the default location for Apache logs, this location can be changed with Apache's configuration files. This is especially common when Apache was installed outside of the standard RHEL package. Like Apache, most services allow for custom log locations. It is not uncommon to find custom directories or file systems outside of /var/log created specifically for log files. Common log files The following table is a short list of common log files and a description of what you can find within them. Do keep in mind that this list is specific to Red Hat Enterprise Linux 7, and while other Linux distributions might follow similar conventions, they are not guaranteed. Log file Description /var/log/messages By default, this log file contains all syslog messages (except e-mail) of INFO or higher priority. /var/log/secure This log file contains authentication related message items such as: SSH logins User creations Sudo violations and privilege escalation /var/log/cron This log file contains a history of crond executions as well as start and end times of cron.daily, cron.weekly, and other executions. /var/log/maillog This log file is the default log location of mail events. If using postfix, this is the default location for all postfix-related messages. /var/log/httpd/ This log directory is the default location for Apache logs. While this is the default location, it is not a guaranteed location for all Apache logs. /var/log/mysql.log This log file is the default log file for mysqld. Much like the httpd logs, this is default and can be changed easily. /var/log/sa/ This directory contains the results of the sa commands that run every 10 minutes by default. For many issues, one of the first log files to review is the /var/log/messages log. On RHEL systems, this log file receives all system logs of INFO priority or higher. In general, this means that any significant event sent to syslog would be captured in this log file. The following is a sample of some of the log messages that can be found in /var/log/messages: Dec 24 18:03:51 localhost systemd: Starting Network Manager Script Dispatcher Service... Dec 24 18:03:51 localhost dbus-daemon: dbus[620]: [system] Successfully activated service 'org.freedesktop.nm_dispatcher' Dec 24 18:03:51 localhost dbus[620]: [system] Successfully activated service 'org.freedesktop.nm_dispatcher' Dec 24 18:03:51 localhost systemd: Started Network Manager Script Dispatcher Service. Dec 24 18:06:06 localhost kernel: e1000: enp0s3 NIC Link is Down Dec 24 18:06:06 localhost kernel: e1000: enp0s8 NIC Link is Down Dec 24 18:06:06 localhost NetworkManager[750]: <info> (enp0s3): link disconnected (deferring action for 4 seconds) Dec 24 18:06:06 localhost NetworkManager[750]: <info> (enp0s8): link disconnected (deferring action for 4 seconds) Dec 24 18:06:10 localhost NetworkManager[750]: <info> (enp0s3): link disconnected (calling deferred action) Dec 24 18:06:10 localhost NetworkManager[750]: <info> (enp0s8): link disconnected (calling deferred action) Dec 24 18:06:12 localhost kernel: e1000: enp0s3 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: RX Dec 24 18:06:12 localhost kernel: e1000: enp0s8 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: RX Dec 24 18:06:12 localhost NetworkManager[750]: <info> (enp0s3): link connected Dec 24 18:06:12 localhost NetworkManager[750]: <info> (enp0s8): link connected Dec 24 18:06:39 localhost kernel: atkbd serio0: Spurious NAK on isa0060/serio0. Some program might be trying to access hardware directly. Dec 24 18:07:10 localhost systemd: Starting Session 53 of user root. Dec 24 18:07:10 localhost systemd: Started Session 53 of user root. Dec 24 18:07:10 localhost systemd-logind: New session 53 of user root. As we can see, there are more than a few log messages within this sample that could be useful while troubleshooting issues. Finding logs that are not in the default location Many times log files are not in /var/log/, which can be either because someone modified the log location to some place apart from the default, or simply because the service in question defaults to another location. In general, there are three ways to find log files not in /var/log/. Checking syslog configuration If you know a service is using syslog for its logging, the best place to check to find which log file its messages are being written to is the rsyslog configuration files. The rsyslog service has two locations for configuration. The first is the /etc/rsyslog.d directory. The /etc/rsyslog.d directory is an include directory for custom rsyslog configurations. The second is the /etc/rsyslog.conf configuration file. This is the main configuration file for rsyslog and contains many of the default syslog configurations. The following is a sample of the default contents of /etc/rsyslog.conf: #### RULES #### # Log all kernel messages to the console. # Logging much else clutters up the screen. #kern.* /dev/console # Log anything (except mail) of level info or higher. # Don't log private authentication messages! *.info;mail.none;authpriv.none;cron.none /var/log/messages # The authpriv file has restricted access. authpriv.* /var/log/secure # Log all the mail messages in one place. mail.* -/var/log/maillog # Log cron stuff cron.* /var/log/cron By reviewing the contents of this file, it is fairly easy to identify which log files contain the information required, if not, at least, the possible location of syslog managed log files. Checking the application's configuration Not every application utilizes syslog; for those that don't, one of the easiest ways to find the application's log file is to read the application's configuration files. A quick and useful method for finding log file locations from configuration files is to use the grep command to search the file for the word log: $ grep log /etc/samba/smb.conf # files are rotated when they reach the size specified with "max log size". # log files split per-machine: log file = /var/log/samba/log.%m # maximum size of 50KB per log file, then rotate: max log size = 50 The grep command is a very useful command that can be used to search files or directories for specific strings or patterns. The simplest command can be seen in the preceding snippet where the grep command is used to search the /etc/samba/smb.conf file for any instance of the pattern "log". After reviewing the output of the preceding grep command, we can see that the configured log location for samba is /var/log/samba/log.%m. It is important to note that %m, in this example, is actually replaced with a "machine name" when creating the file. This is actually a variable within the samba configuration file. These variables are unique to each application but this method for making dynamic configuration values is a common practice. Other examples The following are examples of using the grep command to search for the word "log" in the Apache and MySQL configuration files: $ grep log /etc/httpd/conf/httpd.conf # ErrorLog: The location of the error log file. # logged here. If you *do* define an error logfile for a <VirtualHost> # container, that host's errors will be logged there and not here. ErrorLog "logs/error_log" $ grep log /etc/my.cnf # log_bin log-error=/var/log/mysqld.log In both instances, this method was able to identify the configuration parameter for the service's log file. With the previous three examples, it is easy to see how effective searching through configuration files can be. Using the find command The find command, is another useful method for finding log files. The find command is used to search a directory structure for specified files. A quick way of finding log files is to simply use the find command to search for any files that end in ".log": # find /opt/appxyz/ -type f -name "*.log" /opt/appxyz/logs/daily/7-1-15/alert.log /opt/appxyz/logs/daily/7-2-15/alert.log /opt/appxyz/logs/daily/7-3-15/alert.log /opt/appxyz/logs/daily/7-4-15/alert.log /opt/appxyz/logs/daily/7-5-15/alert.log The preceding is generally considered a last resort solution, and is mostly used when the previous methods do not produce results. When executing the find command, it is considered a best practice to be very specific about which directory to search. When being executed against very large directories, the performance of the server can be degraded. Configuration files As discussed previously, configuration files for an application or service can be excellent sources of information. While configuration files won't provide you with specific errors such as log files, they can provide you with critical information (for example, enabled/disabled features, output directories, and log file locations). Default system configuration directory In general, system, and service configuration files are located within the /etc/ directory on most Linux distributions. However, this does not mean that every configuration file is located within the /etc/ directory. In fact, it is not uncommon for applications to include a configuration directory within the application's home directory. So how do you know when to look in the /etc/ versus an application directory for configuration files? A general rule of thumb is, if the package is part of the RHEL distribution, it is safe to assume that the configuration is within the /etc/ directory. Anything else may or may not be present in the /etc/ directory. For these situations, you simply have to look for them. Finding configuration files In most scenarios, it is possible to find system configuration files within the /etc/ directory with a simple directory listing using the ls command: $ ls -la /etc/ | grep my -rw-r--r--. 1 root root 570 Nov 17 2014 my.cnf drwxr-xr-x. 2 root root 64 Jan 9 2015 my.cnf.d The preceding code snippet uses ls to perform a directory listing and redirects that output to grep in order to search the output for the string "my". We can see from the output that there is a my.cnf configuration file and a my.cnf.d configuration directory. The MySQL processes use these for its configuration. We were able to find these by assuming that anything related to MySQL would have the string "my" in it. Using the rpm command If the configuration files were deployed as part of a RPM package, it is possible to use the rpm command to identify configuration files. To do this, simply execute the rpm command with the –q (query) flag, and the –c (configfiles) flag, followed by the name of the package: $ rpm -q -c httpd /etc/httpd/conf.d/autoindex.conf /etc/httpd/conf.d/userdir.conf /etc/httpd/conf.d/welcome.conf /etc/httpd/conf.modules.d/00-base.conf /etc/httpd/conf.modules.d/00-dav.conf /etc/httpd/conf.modules.d/00-lua.conf /etc/httpd/conf.modules.d/00-mpm.conf /etc/httpd/conf.modules.d/00-proxy.conf /etc/httpd/conf.modules.d/00-systemd.conf /etc/httpd/conf.modules.d/01-cgi.conf /etc/httpd/conf/httpd.conf /etc/httpd/conf/magic /etc/logrotate.d/httpd /etc/sysconfig/htcacheclean /etc/sysconfig/httpd The rpm command is used to manage RPM packages and is a very useful command when troubleshooting. We will cover this command further as we explore commands for troubleshooting. Using the find command Much like finding log files, to find configuration files on a system, it is possible to utilize the find command. When searching for log files, the find command was used to search for all files where the name ends in ".log". In the following example, the find command is being used to search for all files where the name begins with "http". This find command should return at least a few results, which will provide configuration files related to the HTTPD (Apache) service: # find /etc -type f -name "http*" /etc/httpd/conf/httpd.conf /etc/sysconfig/httpd /etc/logrotate.d/httpd The preceding example searches the /etc directory; however, this could also be used to search any application home directory for user configuration files. Similar to searching for log files, using the find command to search for configuration files is generally considered a last resort step and should not be the first method used. The proc filesystem An extremely useful source of information is the proc filesystem. This is a special filesystem that is maintained by the Linux kernel. The proc filesystem can be used to find useful information about running processes, as well as other system information. For example, if we wanted to identify the filesystems supported by a system, we could simply read the /proc/filesystems file: $ cat /proc/filesystems nodev sysfs nodev rootfs nodev bdev nodev proc nodev cgroup nodev cpuset nodev tmpfs nodev devtmpfs nodev debugfs nodev securityfs nodev sockfs nodev pipefs nodev anon_inodefs nodev configfs nodev devpts nodev ramfs nodev hugetlbfs nodev autofs nodev pstore nodev mqueue nodev selinuxfs xfs nodev rpc_pipefs nodev nfsd This filesystem is extremely useful and contains quite a bit of information about a running system. The proc filesystem will be used throughout the troubleshooting steps. It is used in various ways while troubleshooting everything from specific processes, to read-only filesystems. Troubleshooting commands This section will cover frequently used troubleshooting commands that can be used to gather information from the system or a running service. While it is not feasible to cover every possible command, the commands used do cover fundamental troubleshooting steps for Linux systems. Command-line basics The troubleshooting steps used are primarily command-line based. While it is possible to perform many of these things from a graphical desktop environment, the more advanced items are command-line specific. As such, the reader has at least a basic understanding of Linux. To be more specific, we assumes that the reader has logged into a server via SSH and is familiar with basic commands such as cd, cp, mv, rm, and ls. For those who might not have much familiarity, I wanted to quickly cover some basic command-line usage that will be required. Command flags Many readers are probably familiar with the following command: $ ls -la total 588 drwx------. 5 vagrant vagrant 4096 Jul 4 21:26 . drwxr-xr-x. 3 root root 20 Jul 22 2014 .. -rw-rw-r--. 1 vagrant vagrant 153104 Jun 10 17:03 app.c Most should recognize that this is the ls command and it is used to perform a directory listing. What might not be familiar is what exactly the –la part of the command is or does. To understand this better, let's look at the ls command by itself: $ ls app.c application app.py bomber.py index.html lookbusy-1.4 lookbusy-1.4.tar.gz lotsofiles The previous execution of the ls command looks very different from the previous. The reason for this is because the latter is the default output for ls. The –la portion of the command is what is commonly referred to as command flags or options. The command flags allow a user to change the default behavior of the command providing it with specific options. In fact, the –la flags are two separate options, –l and –a; they can even be specified separately: $ ls -l -a total 588 drwx------. 5 vagrant vagrant 4096 Jul 4 21:26 . drwxr-xr-x. 3 root root 20 Jul 22 2014 .. -rw-rw-r--. 1 vagrant vagrant 153104 Jun 10 17:03 app.c We can see from the preceding snippet that the output of ls –la is exactly the same as ls –l –a. For common commands, such as the ls command, it does not matter if the flags are grouped or separated, they will be parsed in the same way. Will show both grouped and ungrouped. If grouping or ungrouping is performed for any specific reason it will be called out; otherwise, the grouping or ungrouping used for visual appeal and memorization. In addition to grouping and ungrouping, we will also show flags in their long format. In the previous examples, we showed the flag -a, this is known as a short flag. This same option can also be provided in the long format --all: $ ls -l --all total 588 drwx------. 5 vagrant vagrant 4096 Jul 4 21:26 . drwxr-xr-x. 3 root root 20 Jul 22 2014 .. -rw-rw-r--. 1 vagrant vagrant 153104 Jun 10 17:03 app.c The –a and the --all flags are essentially the same option; it can simply be represented in both short and long form. One important thing to remember is that not every short flag has a long form and vice versa. Each command has its own syntax, some commands only support the short form, others only support the long form, but many support both. In most cases, the long and short flags will both be documented within the commands man page. Piping command output Another common command-line practice that will be used several times is piping output. Specifically, examples such as the following: $ ls -l --all | grep app -rw-rw-r--. 1 vagrant vagrant 153104 Jun 10 17:03 app.c -rwxrwxr-x. 1 vagrant vagrant 29390 May 18 00:47 application -rw-rw-r--. 1 vagrant vagrant 1198 Jun 10 17:03 app.py In the preceding example, the output of the ls -l --all command is piped to the grep command. By placing | or the pipe character between the two commands, the output of the first command is "piped" to the input for the second command. The example preceding the ls command will be executed; with that, the grep command will then search that output for any instance of the pattern "app". Piping output to grep will actually be used quite often, as it is a simple way to trim the output into a maintainable size. Many times the examples will also contain multiple levels of piping: $ ls -la | grep app | awk '{print $4,$9}' vagrant app.c vagrant application vagrant app.py In the preceding code the output of ls -la is piped to the input of grep; however, this time, the output of grep is also piped to the input of awk. While many commands can be piped to, not every command supports this. In general, commands that accept user input from files or command-line also accept piped input. As with the flags, a command's man page can be used to identify whether the command accepts piped input or not. Gathering general information When managing the same servers for a long time, you start to remember key information about those servers. Such as the amount of physical memory, the size and layout of their filesystems, and what processes should be running. However, when you are not familiar with the server in question it is always a good idea to gather this type of information. The commands in this section are commands that can be used to gather this type of general information. w – show who is logged on and what they are doing Early in my systems administration career, I had a mentor who used to tell me I always run w when I log into a server. This simple tip has actually been very useful over and over again in my career. The w command is simple; when executed it will output information such as system uptime, load average, and who is logged in: # w 04:07:37 up 14:26, 2 users, load average: 0.00, 0.01, 0.05 USER TTY LOGIN@ IDLE JCPU PCPU WHAT root tty1 Wed13 11:24m 0.13s 0.13s -bash root pts/0 20:47 1.00s 0.21s 0.19s -bash This information can be extremely useful when working with unfamiliar systems. The output can be useful even when you are familiar with the system. With this command, you can see: When this system was last rebooted:04:07:37 up 14:26:This information can be extremely useful; whether it is an alert for a service like Apache being down, or a user calling in because they were locked out of the system. When these issues are caused by an unexpected reboot, the reported issue does not often include this information. By running the w command, it is easy to see the time elapsed since the last reboot. The load average of the system:load average: 0.00, 0.01, 0.05:The load average is a very important measurement of system health. To summarize it, the load average is the average number of processes in a wait state over a period of time. The three numbers in the output of w represent different times.The numbers are ordered from left to right as 1 minute, 5 minutes, and 15 minutes. Who is logged in and what they are running: USER TTY LOGIN@ IDLE JCPU PCPU WHAT root tty1 Wed13 11:24m 0.13s 0.13s -bash The final piece of information that the w command provides is users that are currently logged in and what command they are executing. This is essentially the same output as the who command, which includes the user logged in, when they logged in, how long they have been idle, and what command their shell is running. The last item in that list is extremely important. Oftentimes, when working with big teams, it is common for more than one person to respond to an issue or ticket. By running the w command immediately after login, you will see what other users are doing, preventing you from overriding any troubleshooting or corrective steps the other person has taken. rpm – RPM package manager The rpm command is used to manage Red Hat package manager (RPM). With this command, you can install and remove RPM packages, as well as search for packages that are already installed. We saw earlier how the rpm command can be used to look for configuration files. The following are several additional ways we can use the rpm command to find critical information. Listing all packages installed Often when troubleshooting services, a critical step is identifying the version of the service and how it was installed. To list all RPM packages installed on a system, simply execute the rpm command with -q (query) and -a (all): # rpm -q -a kpatch-0.0-1.el7.noarch virt-what-1.13-5.el7.x86_64 filesystem-3.2-18.el7.x86_64 gssproxy-0.3.0-9.el7.x86_64 hicolor-icon-theme-0.12-7.el7.noarch The rpm command is a very diverse command with many flags. In the preceding example the -q and -a flags are used. The -q flag tells the rpm command that the action being taken is a query; you can think of this as being put into a "search mode". The -a or --all flag tells the rpm command to list all packages. A useful feature is to add the --last flag to the preceding command, as this causes the rpm command to list the packages by install time with the latest being first. Listing all files deployed by a package Another useful rpm function is to show all of the files deployed by a specific package: # rpm -q --filesbypkg kpatch-0.0-1.el7.noarch kpatch /usr/bin/kpatch kpatch /usr/lib/systemd/system/kpatch.service In the preceding example, we again use the -q flag to specify that we are running a query, along with the --filesbypkg flag. The --filesbypkg flag will cause the rpm command to list all of the files deployed by the specified package. This example can be very useful when trying to identify a service's configuration file location. Using package verification In this third example, we are going to use an extremely useful feature of rpm, verify. The rpm command has the ability to verify whether or not the files deployed by a specified package have been altered from their original contents. To do this, we will use the -V (verify) flag: # rpm -V httpd S.5....T. c /etc/httpd/conf/httpd.conf In the preceding example, we simply run the rpm command with the -V flag followed by a package name. As the -q flag is used for querying, the -V flag is for verifying. With this command, we can see that only the /etc/httpd/conf/httpd.conf file was listed; this is because rpm will only output files that have been altered. In the first column of this output, we can see which verification checks the file failed. While this column is a bit cryptic at first, the rpm man page has a useful table (as shown in the following list) explaining what each character means: S: This means that the file size differs M: This means that the mode differs (includes permissions and file type) 5: This means that the digest (formerly MD5 sum) differs D: This means indicates the device major/minor number mismatch L: This means indicates the readLink(2) path mismatch U: This means that the user ownership differs G: This means that the group ownership differs T: This means that mTime differs P: This means that caPabilities differs Using this list we can see that the httpd.conf's file size, MD5 sum, and mtime (Modify Time) are not what was deployed by httpd.rpm. This means that it is highly likely that the httpd.conf file has been modified after installation. While the rpm command might not seem like a troubleshooting command at first, the preceding examples show just how powerful of a troubleshooting tool it can be. With these examples, it is simple to identify important files and whether or not those files have been modified from the deployed version. Summary Overall we learned that log files, configuration files, and the /proc filesystem are key sources of information during troubleshooting. We also covered the basic use of many fundamental troubleshooting commands. You also might have noticed that quite a few commands are also used in day-to-day life for nontroubleshooting purposes. While these commands might not explain the issue themselves, they can help gather information about the issue, which leads to a more accurate and quick resolution. Familiarity with these fundamental commands is critical to your success during troubleshooting. Resources for Article: Further resources on this subject: Linux Shell Scripting[article] Embedded Linux and Its Elements[article] Installing Red Hat CloudForms on Red Hat OpenStack [article]

(For more resources related to this topic, see here.) When Visio was first conceived of over 20 years ago, its first stated marketing aim was to outsell ABC Flowcharter, the best-selling process diagramming tool at the time. Therefore, Visio had to have all of the features from the start that are core in the creation of flowcharts, namely the ability to connect one shape to another and to have the lines route themselves around shapes. Visio soon achieved its aim, and looked for other targets to reach. So, process flow diagrams have long been a cornerstone of Visio's popularity and appeal and, although there have been some usability improvements over the years, there have been few enhancements to turn the diagrams into models that can be managed efficiently. Microsoft Visio 2010 saw the introduction of two features, structured diagrams and validation rules, that make process management achievable and customizable, and Microsoft Visio 2013 sees these features enhanced. In this article, you will be introduced to the new features that have been added to Microsoft Visio to support structured diagrams and validation. You will see where Visio fits in the Process Management stack, and explore the relevant out of the box content. Exploring the new process management features in Visio 2013 Firstly, Microsoft Visio 2010 introduced a new Validation API for structured diagrams and provided several examples of this in use, for example with the BPMN (Business Process Modeling Notation) Diagram and Microsoft SharePoint Workflow templates and the improvements to the Basic Flowchart and Cross-Functional Flowchart templates, all of which are found in the Flowchart category. Microsoft Visio 2013 has updated the version of BPMN from 1.1 to 2.0, and has introduced a new SharePoint 2013 Workflow template, in addition to the 2010 one. Templates in Visio consist of a predefined Visio document that has one or more pages, and may have a series of docked stencils (usually positioned on the left-hand side of workspace area). The template document may have an associated list of add-ons that are active while it is in use, and, with Visio 2013 Professional edition, an associated list of structured diagram validation rulesets as well. Most of the templates that contain validation rules in Visio 2013 are in the Flowchart category, as seen in the following screenshot, with the exception being the Six Sigma template in the Business category. Secondly, the concept of a Subprocess was introduced in Visio 2010. This enables processes to hyperlink to other pages describing the subprocesses in the same document, or even across documents. This latter point is necessary if subprocesses are stored in a document library, such as Microsoft SharePoint. The following screenshot illustrates how an existing subprocess can be associated with a shape in a larger process, selecting an existing shape in the diagram, before selecting the existing page that it links to from the drop-down menu on the Link to Existing button. In addition, a subprocess page can be created from an existing shape, or a selection of shapes, in which case they will be moved to the newly-created page. There were also a number of ease-of-use features introduced in Microsoft Visio 2010 to assist in the creation and revision of process flow diagrams. These include: Easy auto-connection of shapes Aligning and spacing of shapes Insertion and deletion of connected shapes Improved cross-functional flowcharts Subprocesses An infinite page option, so you need not go over the edge of the paper ever again Microsoft Visio 2013 has added two more notable features: Commenting (a replacement for the old reviewer's comments) Co-authoring However, this book is not about teaching the user how to use these features, since there will be many other authors willing to show you how to perform tasks that only need to be explained once. This book is about understanding the Validation API in particular, so that you can create, or amend, the rules to match the business logic that your business requires. Reviewing Visio Process Management capabilities Microsoft Visio now sits at the top of the Microsoft Process Management Product Stack, providing a Business Process Analysis (BPA) or Business Process Modeling (BPM) tool for business analysts, process owners/participants, and line of business software architects/developers. Understanding the Visio BMP Maturity Model If we look at the Visio BPM Maturity Model that Microsoft has previously presented to its partners, then we can see that Visio 2013 has filled some of the gaps that were still there after Visio 2010. However, we can also see that there are plenty of opportunities for partners to provide solutions on top of the Visio platform. The maturity model shows how Visio initially provided the means to capture paper-drawn business processes into electronic format, and included the ability to encapsulate data into each shape and infer the relationship and order between elements through connectors. Visio 2007 Professional added the ability to easily link shapes, which represent processes, tasks, decisions, gateways, and so on with a data source. Along with that, data graphics were provided to enable shape data to be displayed simply as icons, data bars, text, or to be colored by value. This enriched the user experience and provided quicker visual representation of data, thus increasing the comprehension of the data in the diagrams. Generic templates for specific types of business modeling were provided. Visio had a built-in report writer for many versions, which provided the ability to export to Excel or XML, but Visio 2010 Premium introduced the concept of validation and structured diagrams, which meant that the information could be verified before exporting. Some templates for specific types of business modeling were provided. Visio 2010 Premium also saw the introduction of Visio Services on SharePoint that provided the automatic (without involving the Visio client) refreshing of data graphics that were linked to specific types of data sources. Throughout this book we will be going into detail about Level 5 (Validation) in Visio 2013, because it is important to understand the core capabilities provided in Visio 2013. We will then be able to take the opportunity to provide custom Business Rule Modeling and Visualization. Reviewing the foundations of structured diagramming A structured diagram is a set of logical relationships between items, where these relationships provide visual organization or describe special interaction behaviors between them. The Microsoft Visio team analyzed the requirements for adding structure to diagrams and came up with a number of features that needed to be added to the Visio product to achieve this: Container Management: The ability to add labeled boxes around shapes to visually organize them Callout Management: The ability to associate callouts with shapes to display notes List Management: To provide order to shapes within a container Validation API: The ability to test the business logic of a diagram Connectivity API: The ability to create, remove, or traverse connections easily The following diagram demonstrates the use of Containers and Callouts in the construction of a basic flowchart, that has been validated using the Validation API, which in turn uses the Connectivity API.

  GIMP 2.6 cookbook Over 50 recipes to produce amazing graphics with the GIMP         Read more about this book       (For more resources on GIMP, see here.) Get exclusive offers on Open Source Graphic Application and Library books through out this month. For more information click here. This article by Juan Manuel Ferreyra, author of GIMP 2.6 Cookbook, is about working with text. GIMP uses the fonts installed in the system. Many times, they are not enough for even the average user. There are many websites that let you download cheap fonts for a low price and also, fortunately, there are hundreds of websites with free, open-source fonts for download, a huge community of font-loving enthusiasts that have no problem with sharing their creations. GIMP comes with a wide variety of filters and effects. Now, we are going to use some of them with just text layers, to give you a clear idea of how to create great looking text starting from a solid-color font. In this article, we will cover: Creating 3D text Creating glowing text effects Creating shining plastic text Creating gold text Creating icy cold and fiery text Creating paper cutout and rubber stamp text Installing a font to use it with GIMP is a really quick and easy task. Just copy the font files to the .fonts folder or to your .gimp-[version]/fonts folder in your home directory. Then, restart GIMP. Once you have your fonts set up, they will be available for you with which to work. GIMP comes with a wide variety of filters and effects. We saw a few of them in action in the previous article on Creating a Logo. Now, we are going to use some of them with just text layers, to give you a clear idea of how to create great looking text starting from a solid-color font. Creating 3D text This is a fairly simple recipe where we are going to create 3D text. How to do it... Follow these steps to create text with a 3D effect: In a new file, create your text. Pick a font, color, and size. Select the Perspective Tool, and drag the layer's corners to simulate perspective until you are satisfied. Duplicate the text layer, and make the copy invisible. With the original text layer selected, go to Filters | Map | Bump Map and experiment with the settings. You can see mine in the following image: Once you click OK, duplicate the layer five or six times. These layers are the basis of the 3D text effect. Now select one of the layers, click the Move Tool, and press the down key once: Select another layer, click the Move Tool again and press the down key twice. Repeat this operation on each of the duplicated layers, for each layer press the down key one more time than the previous layer. When you are satisfied with the thickness of the text, merge all the duplicated layers and your image will be ready: Creating glowing text effects This is a simple recipe. We will create glowing text with a subtle outline. I'll be using a gradient to fill it and make it a little more psychedelic, but you can pick a solid color if you want, or create your own gradient for it. How to do it... You can create glowing text by following these steps: Create the text, right-click on its name inside the Layers dialog, and select the Alpha to Selection option: Delete the text layer, and create a new one. Name it text. Don't erase the selection. Now, you can choose to use a solid color or a gradient for the glow effect. I'll pick a colorful gradient and apply it from left to right with the Blend Tool: You should still have the selection around the text, create a new layer (name it text2), and fill it with a solid white using the Bucket Fill Tool. Now, create another new layer (name it text3), and fill it with a solid black. Delete the selection by going to Select | None from the menu. Make these two layers invisible by clicking on the eye icon to the left of the layer's name in the Layers dialog: Select the text layer, and go to Filters | Blur | Gaussian Blur. Set the horizontal and vertical value to around 10 or 15 pixels: Enable and select the text2 layer, and apply the same Gaussian Blur, but with a Blur Radius of around 5: Just to give the glow effect another twist, select the text3 layer and set it to Overlay mode. Be aware that this works best with a dark, colored background. If you have a bright or transparent background, the effect won't be so noticeable: Here's what the final piece looks like:

(For more resources related to this topic, see here.) Quick text editing Geany provides quick text-editing facilities. You can make use of these by going to Edit | Commands, Edit | Format and the Documents menu. The following are some examples; please note that there are many facilities and I cannot list all of them here: Use the Tab key to quickly indent a block of code: Use a shortcut key or a command to quickly comment out a block of code: For other tools, you can see the keyboard shortcut on the menu and you can set it by going to Edit | Preferences | Keybindings: The <Primary> key in the preceding screenshot is the Ctrl key. Navigating through the source code I feel that navigating through source code in Geany is better and faster than any other IDE I've ever tried. To navigate in the same file, you can use the following features as shown in the following screenshot and explained thereafter: Symbol list: Click on one item in the list and you will be brought to where the symbol is defined. Very few free IDEs or editors provide this feature; if they do, it is not as capable or fast as Geany Go to Tag Definition: Clicking on this will take you to where a function is defined and from where it is used. This feature has limitations when used alone in Geany. It can only look for definitions in the files being opened. So, to enable it to look for definitions in other closed files in the same folder, you must use this feature inside a certain project created by Geany's built-in Project feature, with the File patterns defined, and combine it with other project plugins that support Generate file list on load. Find Usage and Find Document Usage: These features help you look for the lines in which a function/variable is used. You can also jump to these lines from the result list. These are more convenient than the usual find/search dialog. To navigate between files, you may find the tree browser (1) and addons (2) plugins useful, as shown in the following screenshot: Project Not only can Geany work with a single file on demand, but it also provides some basic management for working with a group of files in a project. This feature helps you reopen files from the last session when reopening Geany. It also helps you change some default parameters (indentation type, command to build and run, and so on) and apply those on all files. And don't forget its role in navigating through the source code, as described previously. Using the Project feature is easy. All things related to it can be found in the Project menu. Clearing files With the increasing use of source version control (SVN and Git), cleaning the space automatically gets important in order to avoid polluting the Commit history with meaningless changes, caused accidentally by some whitespace additions/removals. You should make Geany do this automatically, by con figuring the settings after going to Edit | Preferences | Files as shown in the following screenshot: Plugins If you want a feature that you cannot find built in to Geany, or you want to discover something new and funny, you need to pick and turn on some Geany plugins. To see the list of installed plugins, please go to Tools | Plugin Manager. Some noteworthy plugins are as follows: Addons: It shows a button with a dropdown menu displaying a list of opened files, to allow you to switch quickly between them Code navigation: It adds more support to the built-in code-navigation feature Debugger: It helps to integrate with GDB ( GNU Debugger) and other debuggers Summary In this article, we discussed the most commonly used tasks in Geany and how to perform them, along with the most widely used features and plugins of Geany. Resources for Article: Further resources on this subject: 10 Minute Guide to the Enterprise Service Bus and the NetBeans SOA Pack [Article] User Extensions and Add-ons in Selenium 1.0 Testing Tools [Article] Installing and Setting up JavaFX for NetBeans and Eclipse IDE [Article]

In this article by, Govardhan Gunnala and Daniele Tosatto, authors of the book Mastering Citrix® XenDesktop®, we will discuss the process of designing XenDesktop® deployments. The uniqueness of the XenDesktop architecture is its modular five layer model. It covers all the key decisions in designing the XenDesktop deployment. User layer: Defines the users and their requirements Access layer: Defines how the users will access the resources Desktop/resource layer: Defines what resources will be delivered Control layer: Defines managing and maintaining the solution Hardware layer: Defines what resources it needs for implementing the chosen solution (For more resources related to this topic, see here.) While FMA is simple at a high level, its implementation can become complex depending on the technologies/options that are chosen for each component across the layers of FMA. Along with great flexibility, comes the responsibility of diligently choosing the technologies/options for fulfilling your business requirements. Importantly, the decisions made in the first three layers impact the last two layers of the deployment. It means that fixing a wrong decision anywhere in the first three layers during/after implementation stage would have less or no scope, and may even lead to implement the solution from the scratch again. Your design decisions speak for your solution's effectiveness in helping with the given business requirements. The layered architecture of the XenDesktop FMA, featuring the components at each layer is given in the following diagram. Each component of XenDesktop will fall under one of the layers shown in the succeeding diagram. We'll see what decisions are to be made for each of these components at each layer in the next sub section. Decisions to be made at each layer I will have to write a separate book for discussing all the possible technologies/options that are available at each layer. Following is a highly summarized list of the decisions to be made at each layer. This will help you in realizing the breadth of designing XenDesktop. This high level coverage of the various options will help you in locating and considering all the possible options that are available for making the right decisions and avoiding any slippages and missing any considerations. User layer The user layer refers to the specification of the users who will utilize the XenDesktop deployment. A business requirement statement may mention that the service users can either be the internal business users or the external customers accessing the service from Internet. Furthermore, both of these users may also need mobile access to the XenDesktop services. The Citrix receiver is the only component that belongs to the user layer, and XenDesktop is dependent on it for successfully delivering a XenDesktop session. By correlating this technical aspect with the preceding business requirement statement, one needs to consider all the possible aspects of receiver software on the client devices. This involves making the following decisions: Endpoint/user devices related: What are the devices that the users are supposed to access the services from? Who owns and administrates those devices throughout their lifecycle? Endpoints supported: Corporate computers, laptops, or mobiles running on Windows or thin clients. User smart devices, such as Android tablets, Apple iPads, and so on. In case of service providers, the endpoints can usually be any device and they need to be supported. Endpoint ownership: Device management includes security, availability, and compliance. Maintaining the responsibility of the devices on network. Endpoint lifecycle: Devices become either outdated or limited very quickly. Define minimum device hardware requirements to run your business workloads. Endpoint form factor: Choose the devices that may either be fully featured or have limited thin clients, or be a mix of both to support features, such as HDX graphics, multi-monitors, and so on. Thin client selection: Choose if the thin clients, such as Dell Wyse Zero clients, running on the limited functionality operating systems would suffice your user requirements. Understand its licensing cost. Receiver selection: Once you determine your endpoint device and its capabilities, you need to decide on the receiver selection that can be run on the devices. The greatest thing is that receiver is available for almost any device. Receiver type: Choose the receiver that is required for your device. Since the Receiver software for each platform (OS) differs, it is important to use the appropriate Receiver software for your devices while considering the platform that it runs on. You can download the appropriate Receiver software for your device from http://www.Citrix.com/go/receiver.html page. Initial deployment: Receiver is like any other software that will fit into your overall application portfolio. Determine how you would deploy this application on your devices. For corporate desktops and mobiles, you may use the enterprise application deployment and the mobile device management software. Otherwise, the users will be prompted to install it when they access the StoreFront URL, or they can even download it from Citrix for facilitating the installation process. For user-managed mobile devices, you can get it from the respective Windows or Google Apple stores/marketplaces. Initial configuration: Similar to other applications, Receiver requires certain initial configuration. It can be configured either manually or by using a provisioning file, group policy, and e-mail based discovery. Keeping the Receiver software up-to-date: Once you have installed Receiver on user devices, you will also require a mechanism for deploying the updates to Receiver. This can also be the way of initial deployments. Access layer An access layer refers to the specification of how the users gain access to the resources. A business requirement statement may usually state that the users should be validated for gaining access, and the access should be secured when the user is connected over the Internet. The technical components that fall under this layer include firewall(s), NetScaler, and StoreFront. These components play a broader role in the overall networking infrastructure of the company, which also includes the XenDesktop, as well as complete Citrix solutions in the environment. Their key activities include firewalling, external to internal IP address NATing, NetScaler Gateway to secure the connection between the virtual desktop and the user device, global load balancing, user validation/authentication, and GUI presentation of the enumerated resources to the end users. It involves making the following decisions: Authentication: Authentication point: A user can be authenticated at the NetScaler Gateway or StoreFront. Authentication policy: Various business use cases and compliance makes certain modes of authentication mandatory. You can choose from the different authentication methods supported at: StoreFront: Basic authentication by using a username and a password; Domain Pass-through, the NS Gateway pass-through, smart card, and even unauthenticated access. NetScaler Gateway: LDAP, RADIUS (token), client certificates. StoreFront: The decisions that are to be made around the scope of StoreFront are as follows: Unauthenticated access: Provides access to the users who don't require a username and a password, but they are still able to access the administrator allowed resources. Usually, this fits well with public or Kiosk systems. High availability: Making the StoreFront servers available at all times. Hardware load balancing, DNS Round Robin, Windows network load balancing, and so on. Delivery controller high availability and StoreFront: Building high availability for the delivery controller is recommended since they are needed for forming successful connections. Defining more than one delivery controller for the stores makes StoreFront auto failover to the next server in the list. Security - Inbound traffic: Consider securing the user connection to virtual desktops from the internal StoreFront and the external NetScaler Gateway. Security – Backend traffic: Consider securing the communication between the StoreFront and the XML services running on the controller servers. As these will be within the internal network, they can be secured by using the internal private certificate. Routing Receiver with Beacons: Receiver supports websites called Beacons to identify whether the user connection is internal or external. StoreFront provides Receiver with the http(s) addresses of the Beacon points during the initial connection. Resource Presentation: StoreFront presents a webpage, which provides self-service of the resources by the user. Scalability: The StoreFront server load and capacity for the user workload. Multi-site App synchronization: StoreFront can connect to the controllers at multiple site deployments. StoreFront can replicate the user subscribed applications across the servers. NetScaler Gateway: In the NetScaler Gateway, the decision regarding the secured external user access from public Internet involves the following: Topology: NetScaler supports two topologies: 1-Arm (normal security) and 2-Arm (high security). High availability: The NetScaler Gateways can be configured in pairs to provide high availability. Platform: NetScaler is available in different platforms, such as VPX, MDX, and SDX. They have different SSL throughput and SSL Transaction Per Second (TPS) metrics. Pre-authentication policy: Specifies about the Endpoint Analysis (EPA) scans for evaluating whether the endpoints meet the pre-set security criteria. This is available when NetScaler is chosen as the authentication point. Session management: The session policies define the overall user experience by classifying the endpoints into mobile and non-mobile devices. Session profile defines the details needed for gaining access to the environment. These are in two forms: SSLVPN and HDX proxy. Preferred data center: In multi-active data center deployments, StoreFront can determine the user resources primary data center and NetScaler can direct the user connections to that. Static and dynamic methods are used for specifying the preferred data center. Desktop/resource layer The desktop or resource layer refers to the specification of which resources (applications and desktops) users will receive. This layer comes with various options, which are tailored for business user roles and their requirements. This layer makes XenDesktop a better fit for achieving the varying user needs across each of their departments. It includes specification of the FlexCast model (type of desktop), user personalization, and delivering the application to the users in the desktop session. An example business requirement statement may specify that all the permanent employees would require a desktop with all the basic applications pre-installed based on their team and role, with their user settings and data to be retained. For all the contract employees, provide a basic desktop with controlled access to the applications on-demand and do not retain their user data. It includes various components, such as profile management solutions (including Windows profiles, the Citrix profile management, AppSense), Citrix print server, Windows operating systems, application delivery, and so on. It involves making decisions, such as: Images: It involves choosing the FlexCast model that is tailored to the user requirements, thereby delivering the expected desktop behavior to the end users, as follows: Operating system related: It requires choosing the desktop or the server operating systems for your master image, which depends on the FlexCast model that you are choosing from. Hosted Shared Hosted VDI: Pooled-static, pooled-random, pooled with PvD, dedicated, existing, physical/remote PC, streamed and streamed with PvD Streamed VHD Local VM On-demand apps Local apps In case of the desktop OS, it's also important to choose the right OS architecture according to the 32-bit or 64-bit processor architecture of the desktop. Computer policies: Define the controls over the user connection, security and bandwidth settings, devices or connection types, and so on. Specify all the policy features similar to that of the user policies. Machine catalogs: Define your catalog settings, including the FlexCast model, AD computer accounts, provisioning method, OS of the base image, and so on. Delivery groups: Assign desktops or applications to the user groups. Application folders: This is a tidy interface feature in Studio for organizing the applications into folders for easy management. StoreFront integration: This is an option for specifying the StoreFront URL for the Receiver in the master image so that the users will be auto connected to the storefront in the session. Resource allocation: This defines the hardware resources for the desktop VMs. It primarily involves hosts and storage. Depending on your estimated workloads, you can define the resources, such as number of virtual processors (vCPU), amount of virtual memory (vRAM), storage requirements for the needed disk space, and also the following resources Graphics (GPU): For the advanced use cases, you may choose to allocate the pass-through GPU, hardware vGPU, or the software vGPUs. IOPS: Depending on the operating system, the FlexCast model, and estimated workloads, you can analyze the overall IOPS load from the system and plan the corresponding hardware to support that load. Optimizations: Depending on the operating system, you can apply various optimizations to Windows that run on the master image. This greatly reduces the overall load later. Bandwidth requirements: Bandwidth can be a limiting factor in case of WAN and remote user connections of slow networks. Bandwidth consumption and user experience depend on various factors, such as the operating system being used, the application design, and screen resolution. To retain high user experience, it's important to consider the bandwidth requirements and optimization technologies, as follows: Bandwidth minimizing technologies: These include Quality of Service (QoS), HDX RealTime, and WAN Optimization, with Citrix's own CloudBridge solution. HDX Encoding Method: HDX encoding method also affects the bandwidth usage. For XenDesktop 7.x, there are three encoding methods that are available. These will appropriately be employed by the HDX protocol. These are Desktop Composition Redirection, H.264 Enhanced SuperCodec, and Legacy Mode (XenDesktop 5.X Adaptive Display). Session Bandwidth: Bandwidth needed in a session depends on the user interaction with desktop and applications. Latency: HDX can typically perform well up to 300 ms latency and the experience begins to degrade as latency increases. Personalization: This is an essential element of the desktop environment. It involves the decisions that are critical for the end user experience/acceptance and for the overall success of the solution during implementation. Following are the decisions that are involved in personalization. User profiles: This involves the decisions that are related to the user login, roaming of their settings, and seamless profile experience across overall Windows network: Profile type: Choose which profile type works for your user requirements. Possible options include local, roaming, mandatory, and hybrid profile with Citrix Profile Management. Citrix Profile Management provides various additional features, such as profile streaming, active write back, and configuring profiles using an .ini file, and so on. Folder redirection: This option saves the user's application settings in the profile. Represents special folders, such as AppData, Desktop, and so on. Folder exclusion: This option is for setting the exclusion of folders that are to be saved in the user profile. Usually, it refers to the local and IE Temp folders of a user profile. Profile caching: Caching profiles on a local system improves the user login experience and it occurs by default. You need to consider this depending on the type of virtual desktop FlexCast mode. Profile permissions: Specify whether the administrator needs access to the user profiles based on information sensitivity. Profile path: The decision to place the user profiles on a network location for high availability. It affects the logon performance depending on how close the profile is to the virtual desktop from which the user is logging on. It can be managed either from Active Directory or through Citrix Profile Management. User profile replication between data centers: This involves making the user profiles highly available and supporting the profile roaming among multiple data centers. User policies: Involves the decision regarding deploying the user settings and controlling those using management policies providing consistent settings for users, such as: Preferred policy engine: This requires choosing the policy processing for the Windows systems. The Citrix policies can be defined and managed from either Citrix Studio or the Active Directory group policy. Policy filtering: The Citrix policies can be applied to the users and their desktop with the various filter options that are available in the Citrix policy engine. If the group policies have been used, then you'll use the group policy filtering options. Policy precedence: The Citrix policies are processed in the order of LCSDOU (Local, Citrix, Site, Domain, OU policies). Baseline policy: This defines the policy with default and common settings for all the desktop images. Citrix provides the policy templates that suit specific business use cases. A baseline should cover security requirements, common network conditions, and managing the user device or the user profile requirements. Such a baseline can be configured using the security policies, connection-based policies, device-based policies, and profile-based policies. Printing: This is one of the most common desktop user requirements. XenDesktop supports printing, which can work for various scenarios. The printing technology involves deploying and using appropriate drivers. Provisioning printers: These can either be a static or dynamic set of printers. The options for dynamic printers do and do not auto-create all the client printers and auto-create the non-network client printers only. You can also set the options for session printers through the Citrix policy, which can include either static or dynamic printers. Furthermore, you can also set proximity printers. Managing print drivers: This option can be configured so that printer drivers are auto-installed during the session creation. It can be installed by using either the generic Citrix universal printer driver, or the manual option. You can also have all the known drivers preinstalled on the master image. Citrix even provides the Citrix universal print server, which extends XenDesktop universal printing support to network printing. Print job routing: It can be routed among client device or through the network server. The ICA protocol is used for compressing and sending data. Personal vDisk: Desktops with personal vDisks retain the user changes. Choosing the personal vDisk depends on the user requirements and the FlexCast Model that was opted for. Personal vDisk can be set to thin provisioned for estimated growth, but it can't be shrunk later. Applications: The application separation into another layer improves the scalability of the overall desktop solution. Applications are critical elements, which the users require from a desktop environment: Application delivery method: Applications can be installed on the base image, on the Personal vDisks, streamed into the session, or through the on-demand XenApp hosted mode. It also depends on application compatibility, and it requires technical expertise and tools, such as AppDNA, for effectively resolving them. Application streaming: XenDesktop supports App-V to build isolated application packages, which can be streamed to desktops. 16-bit legacy application delivery: If there are any legacy 16-bit applications to be supported, then you can choose from the 32 bit OS, VM hosted App, or a parallel XenApp5 deployment. Control layer Control layer speaks about all the backend systems that are required for managing and maintaining the overall solution through its life cycle. The control layer includes most of the XenDesktop components that are further classified into categories, such as resource/access controllers, image/desktop controllers, and infrastructure controllers. These respectively correspond to the first three layers of FMA, as shown here: Resource/access controllers: Supports the access layer Image/desktop controllers: Supports the desktop/resource layer Infrastructure controllers: Provides the underlying hardware for the overall FMA components/environment This layer involves the specification of capacity, configuration, and the topology of the environment. Building required/planned redundancy for each of these components enables achieving the enterprise business capabilities, such as HA, scalability, disaster recovery, load balancing, and so on. Components and technologies that operate under this layer include Active Directory, group policies, site database, Citrix licensing, XenDesktop delivery controllers, XenClient hypervisor, the Windows server and the Desktop operating systems, provisioning services, which can be either MCS or PVS and their controllers, and so on. An example business requirement statement may be as follows: Build a highly available desktop environment for a fast growing business users group. We currently have a head count of 30 users, which is expected to double in a year. It involves making the following decisions: Infrastructure controllers: It includes common infrastructure, which is required for XenDesktop to function in the Windows domain network. Active Directory: This is used for the authentication and authorization of users in a Citrix environment. It's also responsible for providing and synchronizing time on the systems, which is critical for Kerberos. For the most part, your AD structure will be in-place, and it may require certain changes for accommodating your XenDesktop requirements, such as: Forest design: It involves choosing the AD forest and domain decisions, such as multi-domain, multi-forest, domain and forest trusts, and so on, which will define the users of the XenDesktop resources. Site design: It involves choosing the number of sites that represent your geographical locations, the number of domain controllers, the subnets that accommodate the IP addresses, site links for replication, and so on. Organizational unit structure: Planning the OU structure for easier management of XenDesktop Workers and VDAs. In the case of multi-forest deployment scenarios (as supported in App Orchestration), having the same OU structure is critical. Naming standards: Planning proper conventions for XenDesktop AD objects, which includes users, security groups, XenDesktop servers, OUs, and so on. User groups: This helps in choosing the individual user names or groups. The user security groups are recommended as they reduce validation to just one object despite the number of users in it. Policy control: This helps in planning GPOs ordering and sizing, inheritance, filtering, enforcement, blocking, and loopback processing for reducing the overall processing time on the VDAs and servers. Database: Citrix uses the Microsoft SQL server database for most of its products, as follows: Edition: Microsoft ships the SQL server database in different editions, which provide varying features and capabilities. Using the standard edition for typical XenDesktop production deployments is recommended. For larger/enterprise deployments, depending on the requirement, a higher edition may be required. Database and Transaction Log Sizing: This involves estimating the storage requirements for the Site Configuration database, Monitoring database, and configuration logging databases. Database Location: By default, the Configuration Logging and the Monitoring databases are located within the Site Configuration database. Separating these into separate databases and relocating the Monitoring database to a different SQL server is recommended. High availability: Choose from VM-level HA, Mirroring, AlwaysOn Failover Cluster, and AlwaysOn Availability Groups. Database Creation: Usually, the database is automatically recreated during the XenDesktop installation. Alternatively, they can be created by using the scripts. Citrix licensing: Citrix licensing for XenDesktop requires the existence of a Citrix license server on the network. You can install and manage the multiple Citrix licenses. License type: Choose from user, device, and concurrent licenses. Version: Citrix's new license servers are backward compatible. Sizing: A license server can be scaled out to support a higher number of license requests per second. High availability: License server comes with a 30 day grace period to usually help in recovering from failures. High Availability for license server can be implemented through Window clustering technology or duplication of the virtual server. Optimization: Optimize the number of the receiving and processing threads depending on your hardware. This is generally required in large and heavily-loaded enterprise environments. Resource controllers: The resource controllers include the XenDesktop, the XenApp controllers, and the XenClient synchronizer, as shown here: XenDesktop and XenApp delivery controller. Number of sites: It is considered to have been based on network, risk tolerance, security requirements. Delivery controller sizing: Delivery controller scalability is based on CPU utilization. The more processor cores are available, the more virtual desktops a controller can support. High availability: Always plan for the N+1 deployment of the controllers for achieving the HA. Then, update the controllers' details on VDA through policy. Host connection configuration: Host connections define the hosts, storage repositories, and guest network to be used by the virtual machines on hypervisors. XML service encryption: The XML service protocol running on delivery controllers uses clear text for exchanging all data except passwords. Consider using an SSL encryption for sending the StoreFront data over a secure HTTP connection. Server OS load management: The default maximum number of sessions per server has been set to 250. Using real time usage monitoring and loading analysis, you can define appropriate load management policies. Session PreLaunch and Session Linger: Designed for helping the users in quickly accessing the applications by starting the sessions before they are requested (session prelaunch) and by keeping the user sessions active after a user closes all the applications in a session (session linger). XenClient synchronizer: It includes considerations for its architecture, processor specification, memory specification, network specification, high availability, the SQL database, remote synchronizer servers, storage repository size and location, and external access, and Active Directory integration. Image controllers: This includes all the image provisioning controllers. MCS is built-into the delivery controller. We'll have PVS considerations, such as the following: Farms: A farm represents the top level of the PVS infrastructure. Depending on your networking and administration boundaries, you can define the number of farms to be deployed in your environment. Sites: Each Farm consists of one or more sites, which contain all the PVS objects. While multiple sites share the same database, the target devices can only failover to the other Provisioning Servers that are within the same site. Your networking and organization structure determines the number of sites in your deployment. High availability: If implemented, PVS will be a critical component of the virtual desktop infrastructure. HA should be considered for its database, PVS servers, vDisks and storage, networking and TFTP, and so on. Bootstrap delivery: There are three methods in which the target device can receive the bootstrap program. This can be done by using the DHCP options, the PXE broadcasts, and the boot device manager. Write cache placement: Write cache uniquely identifies the target device by including the target device's MAC address and disk identifier. Write cache can be placed on the following: Cache on the Device Hard Drive, Cache on the Device Hard Drive Persisted, Cache in the Device RAM, Cache in the Device RAM with overflow on the hard disk, and Cache on the Provisioning Server Disk, and Cache on the Provisioning Server Disk Persisted. vDisk format and replication: PVS supports the use of fixed-size or dynamic vDisks. vDisks hosted on a SAN, local, or Direct Attached Storage must be replicated between the vDisk stores whenever a vDisk is created or changed. It can be replicated either manually or automatically. Virtual or physical servers, processor and memory: The virtual Provisioning Servers are preferred when sufficient processor, memory, disk and networking resources are guaranteed. Scale up or scale out: Determining whether to scale up or scale out the servers requires considering factors like redundancy, failover times, datacenter capacity, hardware costs, hosting costs, and so on. Bandwidth requirements and network configuration: PVS can boot 500 devices simultaneously. A 10Gbps network is recommended for provisioning services. Network configuration should consider the PVS Uplink, the Hypervisor Uplink, and the VM Uplink. Recommended switch settings include either Disable Spanning Tree or Enable Portfast, Storm Control, and Broadcast Helper. Network interfaces: Teaming the multiple network interfaces with link aggregation can provide a greater throughput. Consider the NIC features TCP Offloading and Receive Side Scaling (RSS) while selecting NICs. Subnet affinity: It is a load balancing algorithm, which helps in ensuring that the target devices are connected to the most appropriate Provisioning Server. It can be configured to Best Effort and Fixed. Auditing: By default, the auditing feature is disabled. When enabled, the audit trail information is written in the provisioning services database along with the general configuration data. Antivirus: The antivirus software can cause file-locking issues on the PVS server by contending with the files being accessed by PVS Services. The vDisk store and the write cache should be excluded from any antivirus scans in order to prevent file contention issues. Hardware layer The hardware layer involves choosing the right capacity, make, and hardware features of the backend systems that are required for the overall solution as defined in the control layer. In-line with the control layer, the hardware layer decisions will change if any of the first three layer decisions are changed. Components and technologies that operate under this layer include server hardware, storage technologies, hard disks and the RAID configurations, hypervisors and their management software, backup solutions, monitoring, network devices and connectivity, and so on. It involves making the decisions shown here: Hardware Sizing: The hardware sizing is usually done in two ways. The first, and the preferred, way is to plan ahead and purchase the hardware based on the workload requirements. The second way to size the hosts to use the existing hardware in the best configuration to support the different workload requirements, as follows: Workload separation: Workloads can either be separated into dedicated resource clusters or be mixed in the same physical hosts. Control host sizing: The VM resource allocation for each control component should be determined in the control layer and it should be allocated accordingly. Desktop host sizing: This involves choosing the physical resources required for the virtual desktops as well as the hosted server deployments. It includes estimating the pCPU, pRAM, GPU, and the number of hosts. Hypervisors: This involves choosing from the supported hypervisors that include major players, such as Hyper-V, XenServer, and ESX. Choosing from these requires considering a vast range of parameters, such as host hardware - processor and memory, storage requirements, network requirements, scale up/out, and host scalability. Further considerations to be made also include the following: Networking: Networks, physical NIC, NIC teaming, virtual NICs—hosts, virtual NICs—guests, and IP addressing VM provisioning: Templates High availability: Microsoft Hyper-V: Failover clustering, cluster shared volumes, CSV cache VMware ESXi: VMware vSphere high availability cluster Citrix XenServer: XenServer high availability by using the server pool Monitoring: Use the hypervisor specific vendor provided management and monitoring tools for hypervisor monitoring; use hardware specific vendor provided monitoring tools for hardware level monitoring. Backup and recovery: Backup method and components to be backed up. Storage: Storage architecture, RAID level, numbers of disks, disk type, storage bandwidth, tiered storage, thin provisioning, and data de-duplication Disaster recovery Data center utilization: The XenDesktop deployments can leverage multiple data centers for improving the user performance and the availability of resources. Multiple data centers can be deployed in an active/active or an active/passive configuration. An active/active configuration allows for both data centers to be utilized, although the individual users are tied to a specific location. Data center connectivity: An active/active data center configuration utilizing GSLB (Global Server Load Balancing) ensures that the users will be able to establish a connection even if one datacenter is unavailable. In the active/active configuration, the considerations that should be made are as follows: data center failover time, application servers, and StoreFront optimal routing. Capacity in the secondary data center: Planning of the secondary data center capacity is determined by the cost and by the management to support full capacity in each data center. A percent of the overall users, or a percent of the users per application, may be considered for the secondary data center facility. Then, it also needs the consideration of the type and amount of resources that will be made available in a failover scenario. Tools for designing XenDesktop® In the previous section, we saw a broad list of components, technologies, and configuration options, and so on, which we learned are involved in the process of designing the XenDesktop deployment. Obviously, designing the XenDesktop deployment for large, advanced, and complex business scenarios is a mammoth task, which requires operational knowledge of a broad range of technologies. Understanding the maze of this complexity, Citrix constantly helps the customers with great learning resources through handbooks, reviewer guides, blueprints, online eDocs, and training sessions. To ease the life of technical architects and XenDesktop designing and deployment consultants, Citrix has developed an online designing portal called Project Accelerator, which automates, streamlines, and covers all the broad aspects that are involved in the XenDesktop deployment. Project Accelerator Citrix designed the Project Accelerator web based designing tool, and it is available to the customers after they login. Its design is based on the Citrix consulting best practices for the XenDesktop deployment and implementation. It follows the layered FMA and allows you to create a close to deployment architecture. It covers all the key decisions and facilitates modifying them and evaluating their impact on the overall architecture. Upon completion of the design, it generates an architectural diagram and a deployment sizing plan. One can define more than one project and customize them in parallel to achieve multiple deployment plans. I highly recommended starting your Production XenDesktop deployment with the Project Accelerator architecture and the sizing design. Virtual Desktop Handbook Citrix provides the handbook along with new XenDesktop releases. The handbook covers the latest features of that XenDesktop version and provides detailed information on the design decisions. It provides all the possible options for each of the decisions involved, and these options are evaluated and validated in an in-depth manner by the Citrix Solutions lab. They include the Citrix Consulting leading best practices as well. This helps architects and engineers to consider the recommended technologies, and then evaluate them further for fulfilling the business requirements. The Virtual Desktop Handbook for latest the version of XenDesktop, that is, 7.x, can be found at: http://support.Citrix.com/article/CTX139331. XenDesktop® Reviewer's Guide The Reviewer's Guide is also released along with the new versions of XenDesktop. They are designed for helping businesses in quickly installing and configuring the XenDesktop for evaluation. They provide a step-by-step screencast of the installation and configuration wizards of XenDesktop. This provides practical guidance to the IT administrators for successfully installing and delivering the XenDesktop sessions. The XenDesktop Reviewers Guide for the latest version of XenDesktop, that is, 7.6, can be found at https://www.citrix.com/content/dam/citrix/en_us/documents/products-solutions/xendesktop-reviewers-guide.pdf. Summary We learnt the decision making that is involved in designing the XenDesktop in general, and we also saw the deployment designs of the complex environments involving the cloud capabilities. We also saw different tools for designing XenDesktop. Resources for Article: Further resources on this subject: Understanding Citrix®Provisioning Services 7.0 [article] Installation and Deployment of Citrix Systems®' CPSM [article] Designing, Sizing, Building, and Configuring Citrix VDI-in-a-Box [article]

Python Geospatial Development If you want to follow through the examples in this article, make sure you have the following Python libraries installed on your computer: GDAL/OGR version 1.7 or later (http://gdal.org) pyproj version 1.8.6 or later (http://code.google.com/p/pyproj) Shapely version 1.2 or later (http://trac.gispython.org/lab/wiki/Shapely) Reading and writing geo-spatial data In this section, we will look at some examples of tasks you might want to perform that involve reading and writing geo-spatial data in both vector and raster format. Task: Calculate the bounding box for each country in the world In this slightly contrived example, we will make use of a Shapefile to calculate the minimum and maximum latitude/longitude values for each country in the world. This "bounding box" can be used, among other things, to generate a map of a particular country. For example, the bounding box for Turkey would look like this: Start by downloading the World Borders Dataset from:   http://thematicmapping.org/downloads/world_borders.php Decompress the .zip archive and place the various files that make up the Shapefile (the .dbf, .prj, .shp, and .shx files) together in a suitable directory. We next need to create a Python program that can read the borders of each country. Fortunately, using OGR to read through the contents of a Shapefile is trivial: import osgeo.ogr shapefile = osgeo.ogr.Open("TM_WORLD_BORDERS-0.3.shp") layer = shapefile.GetLayer(0) for i in range(layer.GetFeatureCount()): feature = layer.GetFeature(i) The feature consists of a geometry and a set of fields. For this data, the geometry is a polygon that defines the outline of the country, while the fields contain various pieces of information about the country. According to the Readme.txt file, the fields in this Shapefile include the ISO-3166 three-letter code for the country (in a field named ISO3) as well as the name for the country (in a field named NAME). This allows us to obtain the country code and name like this: countryCode = feature.GetField("ISO3") countryName = feature.GetField("NAME") We can also obtain the country's border polygon using: geometry = feature.GetGeometryRef() There are all sorts of things we can do with this geometry, but in this case we want to obtain the bounding box or envelope for the polygon: minLong,maxLong,minLat,maxLat = geometry.GetEnvelope() Let's put all this together into a complete working program: # calcBoundingBoxes.py import osgeo.ogr shapefile = osgeo.ogr.Open("TM_WORLD_BORDERS-0.3.shp") layer = shapefile.GetLayer(0) countries = [] # List of (code,name,minLat,maxLat, # minLong,maxLong) tuples. for i in range(layer.GetFeatureCount()): feature = layer.GetFeature(i) countryCode = feature.GetField("ISO3") countryName = feature.GetField("NAME") geometry = feature.GetGeometryRef() minLong,maxLong,minLat,maxLat = geometry.GetEnvelope() countries.append((countryName, countryCode, minLat, maxLat, minLong, maxLong)) countries.sort() for name,code,minLat,maxLat,minLong,maxLong in countries: print "%s (%s) lat=%0.4f..%0.4f, long=%0.4f..%0.4f" % (name, code,minLat, maxLat,minLong, maxLong) Running this program produces the following output: % python calcBoundingBoxes.py Afghanistan (AFG) lat=29.4061..38.4721, long=60.5042..74.9157 Albania (ALB) lat=39.6447..42.6619, long=19.2825..21.0542 Algeria (DZA) lat=18.9764..37.0914, long=-8.6672..11.9865 ... Task: Save the country bounding boxes into a Shapefile While the previous example simply printed out the latitude and longitude values, it might be more useful to draw the bounding boxes onto a map. To do this, we have to convert the bounding boxes into polygons, and save these polygons into a Shapefile. Creating a Shapefile involves the following steps: Define the spatial reference used by the Shapefile's data. In this case, we'll use the WGS84 datum and unprojected geographic coordinates (that is, latitude and longitude values). This is how you would define this spatial reference using OGR: import osgeo.osr spatialReference = osgeo.osr.SpatialReference() spatialReference.SetWellKnownGeogCS('WGS84') We can now create the Shapefile itself using this spatial reference: import osgeo.ogr driver = osgeo.ogr.GetDriverByName("ESRI Shapefile") dstFile = driver.CreateDataSource("boundingBoxes.shp")) dstLayer = dstFile.CreateLayer("layer", spatialReference) After creating the Shapefile, you next define the various fields that will hold the metadata for each feature. In this case, let's add two fields to store the country name and its ISO-3166 code: fieldDef = osgeo.ogr.FieldDefn("COUNTRY", osgeo.ogr.OFTString) fieldDef.SetWidth(50) dstLayer.CreateField(fieldDef) fieldDef = osgeo.ogr.FieldDefn("CODE", osgeo.ogr.OFTString) fieldDef.SetWidth(3) dstLayer.CreateField(fieldDef) We now need to create the geometry for each feature—in this case, a polygon defining the country's bounding box. A polygon consists of one or more linear rings; the first linear ring defines the exterior of the polygon, while additional rings define "holes" inside the polygon. In this case, we want a simple polygon with a square exterior and no holes: linearRing = osgeo.ogr.Geometry(osgeo.ogr.wkbLinearRing) linearRing.AddPoint(minLong, minLat) linearRing.AddPoint(maxLong, minLat) linearRing.AddPoint(maxLong, maxLat) linearRing.AddPoint(minLong, maxLat) linearRing.AddPoint(minLong, minLat) polygon = osgeo.ogr.Geometry(osgeo.ogr.wkbPolygon) polygon.AddGeometry(linearRing) You may have noticed that the coordinate (minLong, minLat)was added to the linear ring twice. This is because we are defining line segments rather than just points—the first call to AddPoint()defines the starting point, and each subsequent call to AddPoint()adds a new line segment to the linear ring. In this case, we start in the lower-left corner and move counter-clockwise around the bounding box until we reach the lower-left corner again:   Once we have the polygon, we can use it to create a feature: feature = osgeo.ogr.Feature(dstLayer.GetLayerDefn()) feature.SetGeometry(polygon) feature.SetField("COUNTRY", countryName) feature.SetField("CODE", countryCode) dstLayer.CreateFeature(feature) feature.Destroy() Notice how we use the setField() method to store the feature's metadata. We also have to call the Destroy() method to close the feature once we have finished with it; this ensures that the feature is saved into the Shapefile. Finally, we call the Destroy() method to close the output Shapefile: dstFile.Destroy() Putting all this together, and combining it with the code from the previous recipe to calculate the bounding boxes for each country in the World Borders Dataset Shapefile, we end up with the following complete program: # boundingBoxesToShapefile.py import os, os.path, shutil import osgeo.ogr import osgeo.osr # Open the source shapefile. srcFile = osgeo.ogr.Open("TM_WORLD_BORDERS-0.3.shp") srcLayer = srcFile.GetLayer(0) # Open the output shapefile. if os.path.exists("bounding-boxes"): shutil.rmtree("bounding-boxes") os.mkdir("bounding-boxes") spatialReference = osgeo.osr.SpatialReference() spatialReference.SetWellKnownGeogCS('WGS84') driver = osgeo.ogr.GetDriverByName("ESRI Shapefile") dstPath = os.path.join("bounding-boxes", "boundingBoxes.shp") dstFile = driver.CreateDataSource(dstPath) dstLayer = dstFile.CreateLayer("layer", spatialReference) fieldDef = osgeo.ogr.FieldDefn("COUNTRY", osgeo.ogr.OFTString) fieldDef.SetWidth(50) dstLayer.CreateField(fieldDef) fieldDef = osgeo.ogr.FieldDefn("CODE", osgeo.ogr.OFTString) fieldDef.SetWidth(3) dstLayer.CreateField(fieldDef) # Read the country features from the source shapefile. for i in range(srcLayer.GetFeatureCount()): feature = srcLayer.GetFeature(i) countryCode = feature.GetField("ISO3") countryName = feature.GetField("NAME") geometry = feature.GetGeometryRef() minLong,maxLong,minLat,maxLat = geometry.GetEnvelope() # Save the bounding box as a feature in the output # shapefile. linearRing = osgeo.ogr.Geometry(osgeo.ogr.wkbLinearRing) linearRing.AddPoint(minLong, minLat) linearRing.AddPoint(maxLong, minLat) linearRing.AddPoint(maxLong, maxLat) linearRing.AddPoint(minLong, maxLat) linearRing.AddPoint(minLong, minLat) polygon = osgeo.ogr.Geometry(osgeo.ogr.wkbPolygon) polygon.AddGeometry(linearRing) feature = osgeo.ogr.Feature(dstLayer.GetLayerDefn()) feature.SetGeometry(polygon) feature.SetField("COUNTRY", countryName) feature.SetField("CODE", countryCode) dstLayer.CreateFeature(feature) feature.Destroy() # All done. srcFile.Destroy() dstFile.Destroy() The only unexpected twist in this program is the use of a sub-directory called bounding-boxes to store the output Shapefile. Because a Shapefile is actually made up of multiple files on disk (a .dbf file, a .prj file, a .shp file, and a .shx file), it is easier to place these together in a sub-directory. We use the Python Standard Library module shutil to delete the previous contents of this directory, and then os.mkdir() to create it again. If you aren't storing the TM_WORLD_BORDERS-0.3.shp Shapefile in the same directory as the script itself, you will need to add the directory where the Shapefile is stored to your osgeo.ogr.Open() call. You can also store the boundingBoxes.shp Shapefile in a different directory if you prefer, by changing the path where this Shapefile is created. Running this program creates the bounding box Shapefile, which we can then draw onto a map. For example, here is the outline of Thailand along with a bounding box taken from the boundingBoxes.shp Shapefile:  

(For more resources related to this topic, see here.) Understanding the SAP HANA architecture Architecture is the key for SAP HANA to be a game changing innovative technology. SAP HANA has been designed so well architecture-wise such that it makes a lot of difference when compared to other traditional databases available today. This section explains us the various components of SAP HANA and its functionalities. Getting ready Enterprise application requirements have become more demanding—complex reports with high computation on huge volumes of transaction data and also business data of other formats (both structured and semi-structured). Data is being written or updated, and also read from the database in parallel. Thus, integration of both transactional and analytical data into single database is required, where SAP HANA has evolved. Columnar storage exploits modern hardware and technology (multiple CPU cores, large main memory, and caches) in achieving the requirements of enterprise applications. Apart from this, it should also support procedural logic where certain tasks cannot be completed with simple SQL. How it works… The SAP HANA database consists of several services (servers). Index server is the most important component of all the servers. Other servers are name server, preprocessor server, statistics server, and XS Engine: Index server: This server holds the actual data and the engines for processing the data. When SQL or MDX is fired against the SAP HANA system in the case of authenticated sessions and transactions, an index server takes care of these commands and processes them. Name server: This server holds complete information about the system landscape. Name server is responsible for the topology of the SAP HANA system. In a distributed system, SAP HANA instances will be running on multiple hosts. In this kind of setup, the name server knows where the components are running and how data is spread on different servers. Preprocessor server: This server comes into the picture during text data analysis. Index server utilizes the capabilities of preprocessor server in text data analysis and searching. This helps to extract the information on which text search capabilities are based. Statistics server: This server helps in collecting the data for the system monitor and helps you know the health of the SAP HANA system. The statistics server is responsible for collecting the data related to status, resource allocation/consumption and performance of the SAP HANA system. Monitoring the clients and getting the status of various alert monitors use the data collected by Statistics server. This server also provides a history of measurement data for further analysis. XS Engine: The XS Engine allows external applications and application developers to access the SAP HANA system via the XS Engine clients, for example, a web browser accesses SAP HANA apps built by application developers via HTTP. Application developers build applications by using the XS Engine, and the users access the app via HTTP by using a web browser. The persistent model in the SAP HANA database is converted into a consumption model for clients to access it via HTTP. This allows an organization to host system services that are a part of the SAP HANA database (for example, Search service, a built-in web server that provides access to static content in the repository). The following diagram shows the architecture of SAP HANA: There's is more... Let us continue learning about the different components: SAP Host Agent: According to the new approach from SAP, the SAP Host Agent should be installed on all machines that are related to the SAP landscape. It is used by Adaptive Computing Controller (ACC) to manage the system and Software Update Manager (SUM) for automatic updates. LM-structure: LM-structure for SAP HANA contains the information about current installation details. This information will be used by SUM during automatic updates. SAP Solution Manager diagnostic agent: This agent provides all the data to SAP Solution Manager (SAP SOLMAN) to monitor the SAP HANA system. After the SAP SOLMAN is integrated with the SAP HANA system, this agent provides information about the database at a glance, which includes the database state and general information about the system, such as alerts, CPU, or memory and disk usage. SAP HANA Studio repository: This helps the end users to update the SAP HANA studio to higher versions. The SAP HANA Studio repository is the code that does this process. Software Update Manager for SAP HANA: This helps in automatic updates of SAP HANA from the SAP Marketplace and patching the SAP host agent. It also allows distribution of the Studio repository to the end users. See also http://help.sap.com/hana/SAP_HANA_Installation_Guide_en.pdf SAP Notes:1793303 and 1514967 Explaining IMCE and its components We have seen the architecture of SAP HANA and its components. In this section, we will learn about IMCE (in-memory computing engine) and how its components and its functionalities. Getting Ready The SAP in-memory computing engine (formerly Business Analytic Engine (BAE)) is the core engine for SAP's next generation high-performance, in-memory solutions as it leverages technologies such as in-memory computing, columnar databases, massively parallel processing (MPP), and data compression, to allow organizations to instantly explore and analyze large volumes of transactional and analytical data from across the enterprise in real time. How it works... In-memory computing allows the processing of massive quantities of real-time data in the main memory of the server, providing immediate results from analyses and transactions. The SAP in-memory computing database delivers the following capabilities: In-memory computing functionality with native support for row and columnar datastores providing full ACID (atomicity, consistency, isolation, and durability) transactional capabilities Integrated lifecycle management capabilities and data integration capabilities to access SAP and non-SAP data sources SAP IMCE Studio, which includes tools for data modeling, data and life cycle management, and data security The SAP IMCE that resides at the heart of SAP HANA is an integrated database and calculation layer that allows the processing of massive quantities of real-time data in the main memory to provide immediate results from analysis and transactions. Like any standard database, the SAP IMCE not only supports industry standards such as SQL and MDX, but also incorporates a high-performance calculation engine that embeds procedural language support directly into the database kernel. This approach is designed to remove the need to read data from the database, process it, and then write data back to the database, that is, process the data near the database and return the results. The IMCE is an in-memory, column-oriented database technology. It is a powerful calculation engine at the heart of SAP HANA. As data resides in the Random Access Memory (RAM), highly accelerated performance can be achieved compared to systems that read data from disks. The heart lies within the IMCE, which allows us to create and perform calculations on data. SAP IMCE Studio includes tools for data modeling activities, data and life cycle management, and also tools that are related to data security. The following diagram shows the components of IMCE alone: There's more… SAP HANA database has the following two engines: Column-based store: This engine stores the huge amounts of relational data in column-optimized tables, which are aggregated and used in analytical operations. Row-based store: This engine stores the relational data in rows, similar to the storage mechanism of traditional database systems. The row store is more optimized for write operations and has a lower compression rate. Also, the query performance is lower when compared to the column-based store. The engine that is used to store data can be selected on a per-table basis at the time of creating a table. Tables in the row-based store are loaded at start up time. In the case of column-based stores, tables can be either loaded at start up or on demand, that is, during normal operation of the SAP HANA database. Both engines share a common persistence layer, which provides data persistency that is consistent across both engines. Like a traditional database, we have page management and logging in SAP HANA. The changes made to the in-memory database pages are persisted through savepoints. These savepoints are written to those data volumes on the persistent storage for which the storage medium is hard drives. All transactions committed in the SAP HANA database are stored/saved/referenced by the logger of the persistency layer in a log entry written to the log volumes on the persistent storage. To get high I/O performance and low latency, log volumes use the flash technology storage. The relational engines can be accessed through a variety of interfaces. The SAP HANA database supports SQL (JDBC/ODBC), MDX (ODBO), and BICS (SQLDBC). The calculation engine performs all the calculations in the database. No data moves into the application layer until calculations are completed. It also contains the business functions library that is called by applications to perform calculations based on the business rules and logic. The SAP HANA-specific SQL script language is an extension of SQL that can be used to push down data-intensive application logic into the SAP HANA database for specific requirements. Session management This component creates and manages sessions and connections for the database clients. When a session is created, a set of parameters are maintained. These parameters are like auto-commit settings or the current transaction isolation level. After establishing a session, database clients communicate with the SAP HANA database using SQL statements. SAP HANA database treats all the statements as transactions while processing them. Each new session created will be assigned to a new transaction. Transaction manager The transaction manager is the component that coordinates database transactions, takes care of controlling transaction isolation, and keeps track of running and closed transactions. The transaction manager informs the involved storage engines about the running or closed transactions, so that they can execute necessary actions, when a transaction is committed or rolled back. The transaction manager cooperates with the persistence layer to achieve atomic and durable transactions. The client requests are analyzed and executed by a set of components summarized as request processing and execution control. The client requests are analyzed by a request parser, and then it is dispatched to the responsible component. The transaction control statements are forwarded to the transaction manager. The data definition statements are sent to the metadata manager. The object invocations are dispatched to the object store. The data manipulation statements are sent to the optimizer, which creates an optimized execution plan that is given to the execution layer. The SAP HANA database also has built-in support for domain-specific models (such as for financial planning domain) and it offers scripting capabilities that allow application-specific calculations to run inside the database. It has its own scripting language named SQLScript that is designed to enable optimizations and parallelization. This SQLScript is based on free functions that operate on tables by using SQL queries for set processing. The SAP HANA database also contains a component called the planning engine that allows financial planning applications to execute basic planning operations in the database layer. For example, while applying filters/transformations, a new version of a dataset will be created as a copy of an existing one. An example of planning operation is disaggregation operation in which based on a distribution function; target values from higher to lower aggregation levels are distributed. Metadata manager Metadata manager helps to access metadata. SAP HANA database's metadata consists of a variety of objects, such as definitions of tables, views and indexes, SQLScript function definitions, and object store metadata. All these types of metadata are stored in one common catalog for all the SAP HANA database stores. Metadata is stored in tables in the row store. The SAP HANA features such as transaction support and multi-version concurrency control (MVCC) are also used for metadata management. Central metadata is shared across the servers in the case of a distributed database systems. The background mechanism of metadata storage and sharing is hidden from the components that use the metadata manager. As row-based tables and columnar tables can be combined in one SQL statement, both the row and column engines must be able to consume the intermediate results. The main difference between the two engines is the way they process data: the row store operators process data in a row-at-a-time fashion, whereas column store operations (such as scan and aggregate) require the entire column to be available in contiguous memory locations. To exchange intermediate results created by each other, the row store provides results to the column store. The result materializes as complete rows in the memory, while the column store can expose results using the iterators interface needed by the row store. Persistence layer The persistence layer is responsible for durability and atomicity of transactions. The persistent layer ensures that the database is restored to the most recent committed state after a restart, and makes sure that transactions are either completely executed or completely rolled back. To achieve this in an efficient way, the persistence layer uses a combination of write-ahead logs, shadow paging, and savepoints. Moreover, the persistence layer also offers interfaces for writing and reading data. It also contains SAP HANA's logger that manages the transaction log. Authorization manager The authorization manager is invoked by other SAP HANA database components to check the required privileges for users to execute the requested operations. Privileges to other users or roles can be granted. A privilege grants the right to perform a specified operation (such as create, update, select, and execute data manipulation languages) on a specified object such as a table, view, and SQLScript function. Analytic privileges represent filters or hierarchy, and they drill down limitations for analytic queries. Analytic privileges such as granting access to values with a certain combination of dimension attributes are supported in SAP HANA. Users are authenticated either by the SAP HANA database itself (log in with username and password), or authentication can be delegated to external authentication providers third-party such as an LDAP directory. See also SAP HANA in-memory analytics and in-memory computing available at http://scn.sap.com/people/vitaliy.rudnytskiy/blog/2011/03/22/time-to-update-your-sap-hana-vocabulary Summary This article explains the SAP architecture and the IMCE feature in brief. Resources for Article: Further resources on this subject: SAP HANA integration with Microsoft Excel [Article] Data Migration Scenarios in SAP Business ONE Application- part 2 [Article] Data Migration Scenarios in SAP Business ONE Application- part 1 [Article]