Search icon CANCEL
Subscription
0
Cart icon
Your Cart (0 item)
Close icon
You have no products in your basket yet
Arrow left icon
Explore Products
Best Sellers
New Releases
Books
Events
Videos
Audiobooks
Packt Hub
Free Learning
Arrow right icon
timer SALE ENDS IN
0 Days
:
00 Hours
:
00 Minutes
:
00 Seconds

How-To Tutorials

7019 Articles
article-image-creating-installing-and-tweaking-your-theme-using-plone-3
Packt
26 May 2010
12 min read
Save for later

Creating, Installing and Tweaking your theme using Plone 3

Packt
26 May 2010
12 min read
(For more resources on Plone here.) We will first inspect a few structural changes and install them, and then finally examine the various components and skin layer items that have been changed, one at a time. Where restarting Zope or rerunning your buildout would be required, this will be noted. About the theme This theme and its design are available for personal and professional use to anyone, and can be freely modified. You can (and should) download the files from https://svn.plone.org/svn/collective/plonetheme.guria/trunk using the following command: svn co https://svn.plone.org/svn/collective/plonetheme.guria/trunk plonetheme.guria Note the space between the words trunk and plonetheme.guria. This theme is intended for installation on Plone 3 web sites. The finished theme should look like the following, but we have work to do to make this happen: This theme was created by me, for use by a charity group in India, called Guria (http://www.guriaindia.org), dedicated to ending human trafficking and prostitution. The finished site is currently in development, and is generously hosted free of charge by the talented folks at Six Feet Up (sixfeetup.com). Additionally, most of the code and lessons learned come courtesy of similar themes created by the staff at ONE/Northwest in Seattle, Washington. The design for this theme was created with the assumption that most of the tasks would need to be present in this theme. In fact, the only task not covered here is the creation of a new viewlet manager. Creation of viewlet managers is discussed at http://plone.org/documentation/how-to/adding-portlet-managers and http://plone.org/documentation/manual/theme-reference/elements/viewletmanager/override. Creating a theme product I created a theme product named plonetheme.guria, using the command line syntax paster create –t plone3_theme, while we were located in the src/ directory of our buildout, as seen next: [bash: /opt/mybuildout/src] paster create -t plone3_theme plonetheme.guriaSelected and implied templates: ZopeSkel#basic_namespace A project with a namespace package ZopeSkel#plone A Plone project ZopeSkel#plone3_theme A Theme for Plone 3.0Variables: egg: plonetheme.guria package: plonethemeguria project: plonetheme.guriaEnter namespace_package (Namespace package (like plonetheme)) ['plonetheme']:Enter package (The package contained namespace package (like example)) ['example']: guriaEnter skinname (The skin selection to be added to 'portal_skins' (like 'My Theme')) ['']: Guria Theme for the Plone Theming BookEnter skinbase (Name of the skin selection from which the new one will be copied) ['Plone Default']:Enter empty_styles (Override default public stylesheets with empty ones?) [True]: FalseEnter include_doc (Include in-line documentation in generated code?) [False]:Enter zope2product (Are you creating a Zope 2 Product?) [True]:Enter version (Version) ['0.1']:Enter description (One-line description of the package) ['An installable theme for Plone 3.0']:Enter long_description (Multi-line description (in reST)) ['']:Enter author (Author name) ['Plone Collective']: Veda WilliamsEnter author_email (Author email) ['product-developers@lists. plone.org']: email@email.comEnter keywords (Space-separated keywords/tags) ['web zope plone theme']:Enter url (URL of homepage) ['http://svn.plone.org/svn/collective/']:Enter license_name (License name) ['GPL']:Enter zip_safe (True/False: if the package can be distributed as a .zip file) [False]:Creating template basic_namespaceCreating directory ./plonetheme.guria[snip] You may wish to generate a new Plone theme product yourself, so that you can compare and contrast the differences between the Guria theme and a vanilla Plone theme. Notice that the full name of the theme is plonetheme.guria, and where an item shows as blank, it defaults to the example value in that step. In other words, the namespace package defaults to plonetheme, because there was no reason to change it. The skinname is set to a single lowercase word out of stylistic preference. It's important to also note that you should not use hyphens or spaces in your theme names, as they will not be recognized by your buildout. We've chosen not to override Plone's default stylesheets, and instead, we want to build on top of Plone's default (and excellent!) stylesheets. I prefer this method mostly because the layout needed for Plone's Contents view and other complex structural pieces are already taken care of by Plone's base stylesheets. It's easier than trying to rebuild those from scratch every time, but this is merely a personal preference. Following the creation of the theme, we register the theme product in our buildout.cfg, using the following syntax [buildout] ...develop = src/plonetheme.guria...[instance]eggs = plonetheme.guria...zcml = plonetheme.guria... If we were using the eggtractor egg, there would be no need to add these lines of code to our buildout.cfg; all we would need to do is rebuild our buildout and it would automatically recognize the new egg. eggtractor can be found at http://pypi.python.org/pypi/buildout.eggtractor, and is documented thoroughly. Assuming we are not using eggtractor, we must rebuild our buildout, as we have altered ZCML code and added a new egg: [bash: /opt/mybuildout/src/] ./bin/buildout This would be a good time to check your vanilla theme product into Subversion, so that you can track back to the original version, if needed. However, since this is an existing theme, there is no need to do so. For the purposes of following along, it might be best if you do not yet install the theme. We want to make some changes first. However, we will point out some caveats along the way, in case you installed the theme prematurely. Altering the theme product's structure Several modifications have been made to the theme product's structure to shorten folder names and change the default behavior. Again, this is mostly a personal preference. Let's take a look at these changes and how they were achieved. Renaming the theme In our theme product, you will see a file named profiles.zcml, located at mybuildout/src/plonetheme.guria/plonetheme/guria/profiles.zcml. The code looks like this: <configure i18n_domain="plonetheme.guria"> <genericsetup:registerProfile name="default" title="Guria Theme for the Plone Theming Book" directory="profiles/default" description='Extension profile for the "Guria Theme for the Plone Theming Book" Plone theme.' provides="Products.GenericSetup.interfaces.EXTENSION" /></configure> If you named your theme in a way that was less descriptive, you could alter the title. Naming your theme product properly is important, because you may have different types of products used for a given web site—for example, a policy product for content that might be used in tandem with your theme product. This text is what you see in the portal_quickinstaller at http://localhost:8080/mysite/portal_quickinstaller/manage_installProductsForm, where mysite is the name of your Plone site. You can also see this name if you install your theme product via Site Setup Add-on Products|, found at http://localhost:8080/mysite/prefs_install_products_form. If you change your XML here, and your theme product is already installed, you'll need to start (or restart) your Zope instance, using: [bash: /opt/mybuildout] ./bin/instance fg Shortening folder names Next, we look at the folder structure of our theme product. The standard Plone 3 theme produces folders with names like plonetheme_guria_custom_images, plonetheme_guria_custom_templates, and plonetheme_guria_styles. While there is nothing wrong with keeping this structure, it can be cumbersome to type or tab through (especially when checking items into Subversion). However, you might want to keep the existing folder names to help you distinguish which items of base Plone you modified. This can make migrations easier. If you choose this route, you probably want to create additional folders for non-base-Plone items. I personally prefer the shorter folder names and don't worry too much about the migration issues. In the case of this theme product, I opted to make the folder names shorter. First, I altered the names of the folders in the skins/ folder to guria_images, guria_styles, and guria_templates. Then, in the theme, go to mybuildout/plonetheme.guria/plonetheme/guria/skins.zcml. The code in this file is altered to appear as follows: <configure i18n_domain="plonetheme.guria"> <!-- File System Directory Views registration --> <cmf:registerDirectory name="guria_images"/> <cmf:registerDirectory name="guria_templates"/> <cmf:registerDirectory name="guria_styles"/></configure> One more step is required here. In plonetheme.guria/plonetheme/guria/profiles/default/skins.xml, the code is changed to read as follows: <?xml version="1.0"?><object name="portal_skins" allow_any="False" cookie_persistence="False" default_skin=" Guria Theme for the Plone Theming Book "> <object name="guria_images" meta_type="Filesystem Directory View" directory="plonetheme.guria:skins/guria_images"/> <object name="guria_templates" meta_type="Filesystem Directory View" directory="plonetheme.guria:skins/guria_templates"/> <object name="guria_styles" meta_type="Filesystem Directory View" directory="plonetheme.guria:skins/guria_styles"/> <skin-path name=" Guria Theme for the Plone Theming Book " based- on="Plone Default"> <layer name="guria_images" insert-after="custom"/> <layer name="guria_templates" insert-after="guria_images"/> <layer name="guria_styles" insert-after="guria_templates"/> </skin-path></object> Basically, the steps are the following:   Rename the folders on the filesystem. Modify the skins.zcml file to change the name of the filesystem directory view (what you see in the portal_skins/properties area of the ZMI). Modify the skins.xml file in the profiles/default folder to match. This alters the basic profile of your theme product. If you wanted to add additional folders and filesystem directory views here (a scripts/ folder, for example), you'd just add code by following the conventions given to you in these files and then create additional folders. Making changes to the ZCML file means that you would need to do a restart of your Zope instance. If you installed your theme product before making the changes to the skin layer names, you might want to inspect the skin layers at http://localhost:8080/mysite/ portal_skins/manage_propertiesForm, to make sure that the correct skin layers are listed. You might even need to reimport the "skins tool" step via portal_setup at http://localhost:8080/mysite/portal_setup/manage_importSteps. Make sure you choose the correct profile first by choosing your theme product's name from the drop-down list at the top of the import page. The theme product's name is the same name as you find in your profiles.zcml file. Adjusting how stylesheets and images are used Next, we remove some of the default behavior given to us by the plone3_theme recipe. In a vanilla theme product, folders named images/ and stylesheets/ are inserted into the plonetheme.guria/plonetheme/guria/browser/ directory. Additionally, a file named main.css is included in the stylesheets/ directory. I chose not to place the theme's images or stylesheets in the browser/ directory, as this is generally unnecessary for most themes. Advanced programmers may wish to expose these items to the browser layer, but this is generally a personal choice and carries with it additional consequences. I deleted the folders mentioned above, as well as the i file. Then, I opened the file named configure.zcml, located at plonetheme.guria/plonetheme/guria/browser/, and removed all of the following boilerplate text: <!-- Viewlets registration --> <!-- Zope 3 browser resources --> <!-- Resource directory for images --> <browser:resourceDirectory name="plonetheme.guria.images" directory="images" layer=".interfaces.IThemeSpecific" /> <!-- Resource directory for stylesheets --> <browser:resourceDirectory name="plonetheme.guria.stylesheets" directory="stylesheets" layer=".interfaces.IThemeSpecific" /> I then removed the highlighted code below fromI then removed the highlighted code below from plonetheme.guria/plonetheme/guria/profiles/default/cssregistry.xml:: <stylesheet title="" id="++resource++plonetheme.guria.stylesheets/main.css" media="screen" rel="stylesheet" rendering="import" cacheable="True" compression="safe" cookable="True" enabled="1" expression=""/> And replaced it with the following: <stylesheet title="" id="guria.css" media="screen" rel="stylesheet" rendering="import" cacheable="True" compression="safe" cookable="True" enabled="1" expression=""/> This, in effect, tells our theme product that we will be using a stylesheet named guria.css (or more correctly, guria.css.dtml, as we'll see in a moment). This stylesheet does not yet exist, so we have to create it. I wanted the option of making use of the DTML behavior provided by Plone, so that I could use certain base properties provided to us via the base_properties.props file (also located in our skins/guria_styles/ folder). DTML essentially allows us to use property-sheet variables and apply changes on a more global scale. The easiest way to create this new stylesheet is to go to your mybuildout/buildout-cache/eggs/Plone[some version number]/Products/CMFPlone/skins/plone_styles/ploneCustom.css and copy the contents of that file into a new stylesheet (named guria.css.dtml) in your theme's guria_styles/ folder (located in the skins/ directory at mybuildout/plonetheme.guria/plonetheme/guria/skins/guria_styles). The important bits of code you want are as follows: /* <dtml-with base_properties> (do not remove this :) *//* <dtml-call "REQUEST.set('portal_url', portal_url())"> (not this either :) *//* DELETE THIS LINE AND PUT YOUR CUSTOM STUFF HERE */ /* </dtml-with> */ Again, we would need to restart our Zope at this point, as we have modified our ZCML. If we had already installed our theme product, we'd also have to import our cssregistry.xml file via portal_setup in the ZMI, to capture the new GenericSetup profile settings. However, we have not yet installed the product, so we do not need to worry about this.
Read more
  • 0
  • 0
  • 4702

article-image-planning-your-site-adobe-muse
Packt
27 Sep 2012
11 min read
Save for later

Planning Your Site in Adobe Muse

Packt
27 Sep 2012
11 min read
Page layouts The layout of your website can be a deciding factor on whether your visitors will stay on your website or leave with an impatient click. You could think of the layout as a map. If it's easy for the visitors to "read" the map, they are more likely to stick around and find the content you're providing. Let's take a look at some of the typical layouts we see on the Web today. Bread and butter layouts When we're designing a layout for a web page, there are a number of sections that we need to include. These sections can be broken into the following: Consistent content: This does not change throughout the site. Examples of this type of content are logos, navigation bars, and footers. Changing content: This is the part of the page that changes throughout the site, usually the main content. In some situations, the content of the sidebar may also change. A web designer's job is to create a layout that keeps the visitor focused on the content while keeping it nice and easy to navigate around the site. Some examples of conventional or bread and butter site layouts are shown in the following figure: You have a very short amount of time to capture a visitor's attention. So by choosing one of these basic layouts, you're using a tried and tested setup, which many web users will feel at home with. Don't worry that these layouts look "boxy". You can use images, colors, and typefaces, which complement the purpose of your site to completely disguise the fact that every web page is essentially made up of boxes. The bread and butter layouts featured previously are well-tested guides; however, there is absolutely no obligation for you to use one of these. What appears on a typical web page? So we've seen some basic layouts. Now we'll look at some of the elements that appear on (nearly) every web page. Logo The logo is the part of a company's overall branding and identity, and appears at the top of each page on the site along with the company name and tagline, just as it would on printed forms of marketing, such as business cards, brochures, and letterheads. This identity block increases brand recognition and ensures users know that the pages they're viewing are part of a single site. Frequently, the logo is also a link back to the home page of the site. Navigation bar The navigation for your site should be easy to use and easy to find. Just like the logo, it should appear near the top of the page. You may decide to use a horizontal menu across the top of the page, a vertical menu in a sidebar, or a combination of the two. Either way, your main navigation should be visible "above the fold", that is, any area of a web page that can be viewed without visitors having to scroll. Content Content is the King. This is what your visitors have come for. If the visitors can't find what they're looking for, they will move on very quickly. The main content is an important focal point in your design; don't waste time fling it with unnecessary "stuff". Footer Sitting at the bottom of the page, the footer usually holds copyright information, contact links information, and legalities of the site. Some designers have become very imaginative with footers and use this area to hold additional links, tweets, and "about me" information. The footer clearly separates the main content from the end of the page and indicates to users that they're at the bottom of the page. In the following screenshot, you can see a page from the Apple website, which is highly regarded for its aesthetic design. Each section is clearly delineated. If you keep in mind the idea of your site's layout as a map, then you can determine where you want to lead your visitors on the site. Wireframes Wireframing is an important part of the design process for both simple and complex projects. If you're creating a website for a client, a wireframe is a great tool for communicating your ideas visually at an early stage rather than just having a verbal description. If you're creating a website for yourself, the wireframe helps to clarify what is required on each page of your website. It can be considered an overlap between the planning process and the design process. Creating a simple wireframe ensures that your page designs take into account all of the elements you'll add to your pages and where they will be positioned. Wireframes are cost-effective because the time spent in the initial stages potentially saves you from losing much more time revising the design at a later stage. Wireframes can be created in several ways, including pen and paper and computer-based tools. When it comes to computer-based applications for wireframing, there are many options available. Some designers use Photoshop, Illustrator, or even InDesign to put together their wireframes. Specific wireframing software packages that are popular with web designers include Balsamiq and OmniGraffe. Wireframes and Mockups and Prototypes. Oh my! You may hear web designers refer to wireframes, mockups, and prototypes. Although these terms are sometimes used interchangeably, it's important to understand that they are three different things. A wireframe is a basic illustration showing the structure and components of a web page. A mockup is an image file focusing on the design elements in the site. It contains the graphics and other page elements that make up the web page but may contain dummy text and images. A prototype is an almost-complete or semi-functional web page, constructed with HTML and CSS. Prototypes give the client (or yourself) the ability to click around and check out how the final site will work. What to include in a wireframe? Think about which design elements will appear on each page of your website. As mentioned already, most websites will have elements such as logos, navigation, search bars, and footers in consistent positions throughout the site. Next, think about any extra elements that may be specific to individual pages. These include graphics and dynamic widgets. Once you know what's required, you can start to create your wireframe based on these elements. Some designers like to create their wireframes with the "big picture" in mind. Once the basic layout is in place, they get feedback from the client and revise the wireframe if necessary. Others like to create a very detailed wireframe for each page on the site, including every single design element on the list before showing it to the client. Wireframes let us try out several different ideas before settling on our favorite design, which can then be brought forward to the mockup stage. Obviously our focus in this book is on using Muse, but I would urge you not to rule out using paper sketches. It's a great way to quickly get ideas out of your head and into a tangible, visible layout. Web.without.words (www.webwithoutwords.com) is an interesting website dedicated to showing popular and well-known sites as wireframes. The text and images on each site are blocked out and it's a nice way to look at web pages and see how they can be broken down into simple boxes without getting caught up in the content. Wireframes with Muse So what are the advantages of using Muse to create our wireframes? Well, Muse not only lets you create wireframes, but it also allows you to quickly create prototypes using those wireframes. This means you can show clients the functionality of the website with the basic layout. The prototype produced by Muse can be reviewed on any web browser giving the client a good idea of how the site will appear. This kind of early testing can help alleviate time-consuming problems further down the line of the design process. We're going to prepare a site and wireframe now for a fictitious website about windsurfing. First, we'll create a new site, and then add pages in the Plan view. Site structure with Plan view. Let's start by creating a new site. Open Muse. Choose File | New Site. In the New Site dialog box, set Page Width to 960 and Min Height to 800 pixels. Set Margins to 0 all around and Padding Top and Bottom to 10 pixels each. Set the number of Columns to 16. The columns appear as guidelines on the page and we use them to help us align the design elements on our layout. Note that Gutter is set to 20 by default, leave this as it is. The Column Width is calculated by Muse and you should see a value of 41 appear automatically in that field. Remember that all of these values can be changed later if necessary. Click on OK. The Plan view opens and you'll see a thumbnail representing the Home page at the top left, and a thumbnail representing the A-Master page on the bottom pane. Save your site right away by selecting File | Save Site. Give it a descriptive name you'll recognize, such as Windsurf.muse. To create new pages, click on the plus (+) sign to the right of or below the existing pages, and then click on the page's name field to type its name. Click on the plus sign to the right of the Home page and name the new page Gear. Click on the plus sign below the Gear page to add a subpage and name that page Sails. Click on the plus sign to the right of the Sails page and name the new page Boards. Sails and Boards are now on the same level and are subpages of the Gear page. Click on the plus sign to the right of the Gear page and name the new page Learning. Click on the plus sign to the right of Learning and add one more page called Contact. Your Plan view should now look like the following screenshot: Working with thumbnails in the Plan view It's easy to add, delete, reposition, or duplicate pages when working in the Plan view. Right-click (Win) or Ctrl + click (Mac) on a thumbnail to see a contextual menu. This menu provides every option for managing your pages. In the previous screenshot, you can see the menu that appears when you right-click/Ctrl + click.   New Child Page: This option creates a new blank page at a lower level as the current thumbnail. New Sibling Page: This option creates a new blank page at the same level as the current thumbnail. Duplicate Page: This option makes an exact copy of the current page. This is most useful when you have added content and applied some formatting. Delete Page: This option gets rid of the page. Rename Page: This option allows us to change the name of the page. Go to Page: This option opens up the current page in the Design view. Page Properties: This option opens the Page Properties dialog box allowing you to set properties for the current page only. Reset Page Properties: This option reverts to the original settings for the page. Export Page: This option allows you to export your page as HTML and CSS. Menu Options: This option allows you to choose how the page will be included (or not included) in the automatically-created menu. Masters: This option lets you choose which Master design will be applied to the page. The context menu is not the only way to get to these options, for example the most common tasks in the Plan view can be completed as follows: You can rename a page by double-clicking on the page name You can delete a page by hovering your mouse over the thumbnail and then clicking on the x icon that appears in the top-right corner To reposition a page in your site map hierarchy, you can drag-and-drop a thumbnail on the same level or on a sublevel. Spend a couple of minutes adding, deleting, and repositioning pages so that you get a feel of creating the site structure. You'll find the Plan view to be intuitive to use and extremely fast for creating site maps. You can choose Edit | Undo to undo any of the steps you've taken. Muse tracks all the page names, and later in the design process it allows us to create menus quickly using menu widgets. All links created in the Plan view are maintained and are updated automatically if we make a change to the site structure. You can come back to the Plan view at any point during your web design process.
Read more
  • 0
  • 0
  • 4697

article-image-users-profiles-and-connections-elgg
Packt
23 Oct 2009
8 min read
Save for later

Users, Profiles, and Connections in Elgg

Packt
23 Oct 2009
8 min read
Connecting to Friends and Users I hope you're convinced how important friends are to a social network. Initially, you'll have to manually invite your friends over to join. I say initially, because membership on a social network is viral. Once your friends are registered members of your network, they can also bring in their own friends. This means that soon your friends would have invited their own friends as well. Chances are that you might not know these friends of your friends. So, Elgg not only allows you to invite friends from outside, but also connect with users already on the network. Let's understand these situations in real-life terms. You invite your friends over to a party with you at your new Star Trek themed club. That's what you'll do with Elgg, initially. So your friends like the place and next time around they bring in more friends from work. These friends of friends from work talk about your place with their friends and so on, until you're hosting a bunch of people in the club that you haven't ever met in your life. You overhear some people discussing Geordi La Forge, your favorite character from the show. You invite them over for drinks. That's connecting with users already on the network. So let's head on over to Elgg and invite some friends! Inviting Friends to Join There are two ways of inviting users to join your network. Either send them an email with a link to join the website, or let Elgg handle sending them emails. If you send them emails, you can include a direct link to the registration page. This link is also on the front page of your network, which every visitor will see. It asks visitors to register an account if they like what's on the network. Let Elgg Handle Registration This is the most popular method of inviting users to join the network. It's accessible not only to you, but also to your friends once they've registered with the network. To allow Elgg to send emails on your behalf, you'll have to be logged into Elgg. Once you login, click on the Your Network button on the top navigation bar. This will take you to a page, which links to tools that'll help you connect with others. The last link in this bar (Invite a Friend) does exactly what it says. When you click on this link, it'll explain to you some benefits of inviting friends over. The page has three fields; Their name: Enter the name of the friend you're sending the invitation to. Their email address: Very important. This is the address to where the invitation is sent. An optional message: Elgg sends an email composed using a template. If you want to add a personal message to Elgg's email, you can do so here. In the email, which Elgg sends on behalf of the network's administrator, that means you, it displays the optional message (if you've sent one), along with a link to the registration page. The invitation is valid for seven days, after which the registration link in the email isn't valid. When your friends click on the registration form, it asks them to enter their: Name: This is your friend's real name. When he arrives here by clicking the link in the email, this field already has the same name as the one in the email. Of course, your friend can choose to change it if he pleases. Username: The name your friend wants to use to log in to the network. Elgg automatically suggests one based on your friend's real name. Password: The last two fields ask your friend to enter (and then re-enter to confirm) a password. This is used along with the username to authenticate him on the system. Once your friends enter all the details and click on join, Elgg creates an account for them, logs them in, and dispatches a message to them containing the log in details for reference. Build a Profile The first thing a new user has to do on the network is to create his profile. If you haven't yet built up a profile yourself, now is a good time. To recap, your profile is your digital self. By filling in a form, Elgg helps you define yourself in terms that'll help other members find and connect to you. This is again where socializing using Elgg outscores socializing in real life. You can find people with similar tastes, likes, and dislikes, as soon as you enter the network. So let's steam ahead and create a digital you. The Various Profile Options Once you are logged into your Elgg network, select the Your Profile option from the top navigation-bar. In the page that opens, click the first link, Edit this profile. This opens up a form, divided into five tabs—Basic details, Location, Contact, Employment, and Education. Each tab helps you fill in details regarding that particular area. You don't necessarily have to fill in each and every detail. And you definitely don't have to fill them all in one go. Each tab has a Save your profile button at the end. When you press this button, Elgg updates your profile instantaneously. You can fill in as much detail as you want, and keep coming back to edit your profile, and append new information. Let's look at the various tabs: Basic details: Although filling information in any tab is optional, I'd advise you to fill in all details in this tab. This will make it easy, for you to find others, and for others to find you. The tab basically asks you to introduce yourself, list your interests, your likes, your dislikes, your goals in life, and your main skills. Location: This tab requests information that'll help members reach you physically. Fill in your street address, town, state, postal code, and country. Contact: Do you want members to contact you outside your Elgg network? This tab requests both physical as well as electronic means which members can use to get in touch with you. Physical details include your work, home, and mobile telephone number. Electronic details include your email address, your personal, and official websites. Elgg can also list information to help users connect to you on instant messenger. It supports ICQ, MSN, AIM, Skype, and Jabber. Employment: List your occupation, the industry, and company you work in, your job title, and description. Elgg also lets you list your career goals and suggests you do so to "let colleagues and potential employers know what you'd like to get out of your career. Education: Here you can specify your level of education, and which high school, university or college you attended, and the degree you hold. As you can clearly see, Elgg's profiling options are very diverse and detailed. Rather than serve the sole purpose of describing you to the visitors, the profile also helps you find new friends as well, as we'll see later in this article. What is FOAF? While filling the profile, you must have noticed an Upload a FOAF file area down at the bottom of all tabs. FOAF or Friend of a Friend is a project (http://www.foaf-project.org/) to help create "machine-readable pages that describe people, the links between them, and the things they create, and do". The FOAF file includes lots of details about you, and if you have already created a FOAF profile, Elgg can use that to pick out information describing you from in there. You can modify the information once it's imported into Elgg, if you feel the need to do so. The FOAF-a-Matic tool (http://www.ldodds.com/foaf/foaf-a-matic.en.html) is a simple Web-based program you can use to create a FOAF profile. A Face for Your Profile Once you have created your digital self, why not give it a face as well. The default Elgg picture with a question mark doesn't look like you! To upload your picture, head over to Your Profile and select the Change site picture link. From this page, click Browse to find and select the picture on your computer. Put in an optional description, and then choose to make it your default icon. When you click the Upload new icon button, Elgg will upload the picture. Once the upload completes, Elgg will display the picture. Click the Save button to replace Elgg's default icon with this picture.   Elgg will automatically resize your picture to fit into its small area. You should use a close-up of yourself, otherwise the picture will lose clarity when resizing. If you don't like the picture when it appears on the website, or you want to replace it with a new one, simply tick the Delete check-box associated with the picture you don't like. When you click Save, Elgg will revert to the default question-mark guy.
Read more
  • 0
  • 0
  • 4695

article-image-creating-your-first-collection-simple
Packt
26 Jun 2013
7 min read
Save for later

Creating your first collection (Simple)

Packt
26 Jun 2013
7 min read
(For more resources related to this topic, see here.) Getting ready Assuming that you have walked through the tutorial, you should be nearly ready with the setup. Still, it does not hurt to go through the checklist: Be familiar that you know how to start your operating system's shell (cmd.exe on Windows, Terminal/iTerm on Mac, and sh/bash/tch/zsh on Unix). Ensure that running the java –version command on the shell's prompt returns at least Version 1.6. You may need to upgrade if you have an older version. Ensure that you know where you unpacked the Solr distribution and the full path to the example directory within that. You needed that directory for the tutorial, but that's also where we are going to start our own Solr instance. That allows us to easily run an embedded Jetty web server and to also find all the additional JAR files that Solr needs to operate properly. Now, create a directory where we will store our indexes and experiments. It can be anywhere on your drive. As Solr can run on any operating system where Java can run, we will use SOLRINDEXING as a name whenever we refer to that directory. Make sure to use absolute path names when substituting with your real path for the directory. How to do it... As our first example, we will create an index that stores and allows for the searching of simplified e-mail information. For now, we will just look at the addr_from and addr_to e-mail addresses and the subject line. You will see that it takes only two simple configuration files to get the basic Solr index working. Under the SOLR-INDEXING directory, create a collection1 directory and inside that create a conf directory. In the conf directory, create two files: schema.xml and solrconfig.xml. The schema.xml file should have the following content: <?xml version="1.0" encoding="UTF-8" ?><schema version="1.5"><fields><field name="id" type="string" indexed="true" stored="true"required="true"/><field name="addr_from" type="string" indexed="true"stored="true" required="true"/><field name="addr_to" type="string" indexed="true"stored="true" required="true"/><field name="subject" type="string" indexed="true"stored="true" required="true"/></fields><uniqueKey>id</uniqueKey><types><fieldType name="string" class="solr.StrField" /></types></schema> The solrconfig.xml file should have the following content: <?xml version="1.0" encoding="UTF-8" ?><config><luceneMatchVersion>LUCENE_43</luceneMatchVersion><requestDispatcher handleSelect="false"><httpCaching never304="true" /></requestDispatcher><requestHandler name="/select" class="solr.SearchHandler" /><requestHandler name="/update" class="solr.UpdateRequestHandler" /><requestHandler name="/admin" class="solr.admin.AdminHandlers" /><requestHandler name="/analysis/field" class="solr.FieldAnalysisRequestHandler" startup="lazy" /></config> That is it. Now, let's start our just-created Solr instance. Open a new shell (we'll need the current one later). On that shell's command prompt, change the directory to the example directory of the Solr distribution and run the following command: java -Dsolr.solr.home=SOLR-INDEXING -jar start.jar Notice that solr.solr.home is not a typo; you do need the solr part twice. And, as always, if you have spaces in your paths (now or later), you may need to escape them in platform-specific ways, such as with backslashes on Unix/Linux or by quoting the whole value. In the window of your shell, you should see a long list of messages that you can safely ignore (at least for now). You can verify that everything is working fine by checking for the following three elements: The long list of messages should finish with a message like Started SocketConnector@0.0.0.0:8983. This means that Solr is now running on port 8983 successfully. You should now have a directory called data, right next to the directory called conf that we created earlier. If you open the web browser and go to the http:// localhost:8983/ solr/, you should see a web-based admin interface that makes testing and troubleshooting your Solr instance much easier. We will be using this interface later, so do spend a couple of minutes clicking around now. Now, let's load some actual content into our collection: Copy post.jar from the Solr distribution's example/exampledocs directory to our root SOLR-INDEXING directory. Create a file called input1.csv in the collection1 directory, next to the conf and data directories with the following three-line content: id,addr_from,addr_to,subjectemail1,fulan@acme.example.com,kari@acme.example.com,"Kari,we need more Junior Java engineers"email2,kari@acme.example.com,maija@acme.example.com,"Updating vacancy description" Run the import command from the command line in the SOLR-INDEXING directory (one long command; do not split it across lines): java -Dauto -Durl=http://localhost:8983/solr/collection1/update -jar post.jar collection1/input1.csv You should see the following in one of the message lines: "1 files indexed". If you now open a web browser and go to http:// localhost:8983/solr/ collection1/select?q=*%3A*&wt=ruby&indent=true, you should see Solr output with all the three documents displayed on the screen in a somewhat readable format. How it works... We have created two files to get our example working. Let's review what they mean and how they fit together: The schema.xml file in the collection's conf directory defines the actual shape of data that you want to store and index. The fields define a structure of a record. Each field has a type, which is also defined in the same file. The field defines whether it is stored, indexed, required, multivalued, or a small number of other, more advanced properties. On the other hand, the field type defines what is actually done to the field when it is indexed and when it is searched. We will explore all of these later. The solrconfig.xml file also in the collection's conf directory defines and tunes the components that make up Solr's runtime environment. At the very least, it needs to define which URLs can be called to add records to a collection (here, /update), which to query a collection (here, /select), and which to do various administrative tasks (here, /admin and /analysis/field). Once Solr started, it created a single collection with the default name of collection1, assigned an update handler to it at the /solr/collection1/update URL and search handler at the /solr/collection1/select URL (as per solrconfig.xml). At that point, Solr was ready for the data to be imported into the four required fields (as per schema.xml). We then proceeded to populate the index from a CSV file (one of many update formats available) and then verified that the records are all present in an indented Ruby format (again, one of many result formats available). Summary This article helped you create a basic Solr collection and populate it with a simple dataset in CSV format. Resources for Article : Further resources on this subject: Integrating Solr: Ruby on Rails Integration [Article] Indexing Data in Solr 1.4 Enterprise Search Server: Part2 [Article] Text Search, your Database or Solr [Article]
Read more
  • 0
  • 0
  • 4693

article-image-setting-synchronous-replication
Packt
10 Aug 2015
17 min read
Save for later

Setting Up Synchronous Replication

Packt
10 Aug 2015
17 min read
In this article by the author, Hans-Jürgen Schönig, of the book, PostgreSQL Replication, Second Edition, we learn how to set up synchronous replication. In asynchronous replication, data is submitted and received by the slave (or slaves) after the transaction has been committed on the master. During the time between the master's commit and the point when the slave actually has fully received the data, it can still be lost. Here, you will learn about the following topics: Making sure that no single transaction can be lost Configuring PostgreSQL for synchronous replication Understanding and using application_name The performance impact of synchronous replication Optimizing replication for speed Synchronous replication can be the cornerstone of your replication setup, providing a system that ensures zero data loss. (For more resources related to this topic, see here.) Synchronous replication setup Synchronous replication has been made to protect your data at all costs. The core idea of synchronous replication is that a transaction must be on at least two servers before the master returns success to the client. Making sure that data is on at least two nodes is a key requirement to ensure no data loss in the event of a crash. Setting up synchronous replication works just like setting up asynchronous replication. Just a handful of parameters discussed here have to be changed to enjoy the blessings of synchronous replication. However, if you are about to create a setup based on synchronous replication, we recommend getting started with an asynchronous setup and gradually extending your configuration and turning it into synchronous replication. This will allow you to debug things more easily and avoid problems down the road. Understanding the downside to synchronous replication The most important thing you have to know about synchronous replication is that it is simply expensive. Synchronous replication and its downsides are two of the core reasons for which we have decided to include all this background information in this book. It is essential to understand the physical limitations of synchronous replication, otherwise you could end up in deep trouble. When setting up synchronous replication, try to keep the following things in mind: Minimize the latency Make sure you have redundant connections Synchronous replication is more expensive than asynchronous replication Always cross-check twice whether there is a real need for synchronous replication In many cases, it is perfectly fine to lose a couple of rows in the event of a crash. Synchronous replication can safely be skipped in this case. However, if there is zero tolerance, synchronous replication is a tool that should be used. Understanding the application_name parameter In order to understand a synchronous setup, a config variable called application_name is essential, and it plays an important role in a synchronous setup. In a typical application, people use the application_name parameter for debugging purposes, as it allows users to assign a name to their database connection. It can help track bugs, identify what an application is doing, and so on: test=# SHOW application_name; application_name ------------------ psql (1 row)   test=# SET application_name TO 'whatever'; SET test=# SHOW application_name; application_name ------------------ whatever (1 row) As you can see, it is possible to set the application_name parameter freely. The setting is valid for the session we are in, and will be gone as soon as we disconnect. The question now is: What does application_name have to do with synchronous replication? Well, the story goes like this: if this application_name value happens to be part of synchronous_standby_names, the slave will be a synchronous one. In addition to that, to be a synchronous standby, it has to be: connected streaming data in real-time (that is, not fetching old WAL records) Once a standby becomes synced, it remains in that position until disconnection. In the case of cascaded replication (which means that a slave is again connected to a slave), the cascaded slave is not treated synchronously anymore. Only the first server is considered to be synchronous. With all of this information in mind, we can move forward and configure our first synchronous replication. Making synchronous replication work To show you how synchronous replication works, this article will include a full, working example outlining all the relevant configuration parameters. A couple of changes have to be made to the master. The following settings will be needed in postgresql.conf on the master: wal_level = hot_standby max_wal_senders = 5   # or any number synchronous_standby_names = 'book_sample' hot_standby = on # on the slave to make it readable Then we have to adapt pg_hba.conf. After that, the server can be restarted and the master is ready for action. We recommend that you set wal_keep_segments as well to keep more transaction logs. We also recommend setting wal_keep_segments to keep more transaction logs on the master database. This makes the entire setup way more robust. It is also possible to utilize replication slots. In the next step, we can perform a base backup just as we have done before. We have to call pg_basebackup on the slave. Ideally, we already include the transaction log when doing the base backup. The --xlog-method=stream parameter allows us to fire things up quickly and without any greater risks. The --xlog-method=stream and wal_keep_segments parameters are a good combo, and in our opinion, should be used in most cases to ensure that a setup works flawlessly and safely. We have already recommended setting hot_standby on the master. The config file will be replicated anyway, so you save yourself one trip to postgresql.conf to change this setting. Of course, this is not fine art but an easy and pragmatic approach. Once the base backup has been performed, we can move ahead and write a simple recovery.conf file suitable for synchronous replication, as follows: iMac:slavehs$ cat recovery.conf primary_conninfo = 'host=localhost                    application_name=book_sample                    port=5432'   standby_mode = on The config file looks just like before. The only difference is that we have added application_name to the scenery. Note that the application_name parameter must be identical to the synchronous_standby_names setting on the master. Once we have finished writing recovery.conf, we can fire up the slave. In our example, the slave is on the same server as the master. In this case, you have to ensure that those two instances will use different TCP ports, otherwise the instance that starts second will not be able to fire up. The port can easily be changed in postgresql.conf. After these steps, the database instance can be started. The slave will check out its connection information and connect to the master. Once it has replayed all the relevant transaction logs, it will be in synchronous state. The master and the slave will hold exactly the same data from then on. Checking the replication Now that we have started the database instance, we can connect to the system and see whether things are working properly. To check for replication, we can connect to the master and take a look at pg_stat_replication. For this check, we can connect to any database inside our (master) instance, as follows: postgres=# x Expanded display is on. postgres=# SELECT * FROM pg_stat_replication; -[ RECORD 1 ]----+------------------------------ pid            | 62871 usesysid         | 10 usename         | hs application_name | book_sample client_addr     | ::1 client_hostname | client_port     | 59235 backend_start   | 2013-03-29 14:53:52.352741+01 state           | streaming sent_location   | 0/30001E8 write_location   | 0/30001E8 flush_location   | 0/30001E8 replay_location | 0/30001E8 sync_priority   | 1 sync_state       | sync This system view will show exactly one line per slave attached to your master system. The x command will make the output more readable for you. If you don't use x to transpose the output, the lines will be so long that it will be pretty hard for you to comprehend the content of this table. In expanded display mode, each column will be in one line instead. You can see that the application_name parameter has been taken from the connect string passed to the master by the slave (which is book_sample in our example). As the application_name parameter matches the master's synchronous_standby_names setting, we have convinced the system to replicate synchronously. No transaction can be lost anymore because every transaction will end up on two servers instantly. The sync_state setting will tell you precisely how data is moving from the master to the slave. You can also use a list of application names, or simply a * sign in synchronous_standby_names to indicate that the first slave has to be synchronous. Understanding performance issues At various points in this book, we have already pointed out that synchronous replication is an expensive thing to do. Remember that we have to wait for a remote server and not just the local system. The network between those two nodes is definitely not something that is going to speed things up. Writing to more than one node is always more expensive than writing to only one node. Therefore, we definitely have to keep an eye on speed, otherwise we might face some pretty nasty surprises. Consider what you have learned about the CAP theory earlier in this book. Synchronous replication is exactly where it should be, with the serious impact that the physical limitations will have on performance. The main question you really have to ask yourself is: do I really want to replicate all transactions synchronously? In many cases, you don't. To prove our point, let's imagine a typical scenario: a bank wants to store accounting-related data as well as some logging data. We definitely don't want to lose a couple of million dollars just because a database node goes down. This kind of data might be worth the effort of replicating synchronously. The logging data is quite different, however. It might be far too expensive to cope with the overhead of synchronous replication. So, we want to replicate this data in an asynchronous way to ensure maximum throughput. How can we configure a system to handle important as well as not-so-important transactions nicely? The answer lies in a variable you have already seen earlier in the book—the synchronous_commit variable. Setting synchronous_commit to on In the default PostgreSQL configuration, synchronous_commit has been set to on. In this case, commits will wait until a reply from the current synchronous standby indicates that it has received the commit record of the transaction and has flushed it to the disk. In other words, both servers must report that the data has been written safely. Unless both servers crash at the same time, your data will survive potential problems (crashing of both servers should be pretty unlikely). Setting synchronous_commit to remote_write Flushing to both disks can be highly expensive. In many cases, it is enough to know that the remote server has accepted the XLOG and passed it on to the operating system without flushing things to the disk on the slave. As we can be pretty certain that we don't lose two servers at the very same time, this is a reasonable compromise between performance and consistency with respect to data protection. Setting synchronous_commit to off The idea is to delay WAL writing to reduce disk flushes. This can be used if performance is more important than durability. In the case of replication, it means that we are not replicating in a fully synchronous way. Keep in mind that this can have a serious impact on your application. Imagine a transaction committing on the master and you wanting to query that data instantly on one of the slaves. There would still be a tiny window during which you can actually get outdated data. Setting synchronous_commit to local The local value will flush locally but not wait for the replica to respond. In other words, it will turn your transaction into an asynchronous one. Setting synchronous_commit to local can also cause a small time delay window, during which the slave can actually return slightly outdated data. This phenomenon has to be kept in mind when you decide to offload reads to the slave. In short, if you want to replicate synchronously, you have to ensure that synchronous_commit is set to either on or remote_write. Changing durability settings on the fly Changing the way data is replicated on the fly is easy and highly important to many applications, as it allows the user to control durability on the fly. Not all data has been created equal, and therefore, more important data should be written in a safer way than data that is not as important (such as log files). We have already set up a full synchronous replication infrastructure by adjusting synchronous_standby_names (master) along with the application_name (slave) parameter. The good thing about PostgreSQL is that you can change your durability requirements on the fly: test=# BEGIN; BEGIN test=# CREATE TABLE t_test (id int4); CREATE TABLE test=# SET synchronous_commit TO local; SET test=# x Expanded display is on. test=# SELECT * FROM pg_stat_replication; -[ RECORD 1 ]----+------------------------------ pid             | 62871 usesysid         | 10 usename         | hs application_name | book_sample client_addr     | ::1 client_hostname | client_port     | 59235 backend_start   | 2013-03-29 14:53:52.352741+01 state           | streaming sent_location   | 0/3026258 write_location   | 0/3026258 flush_location   | 0/3026258 replay_location | 0/3026258 sync_priority   | 1 sync_state       | sync   test=# COMMIT; COMMIT In this example, we changed the durability requirements on the fly. This will make sure that this very specific transaction will not wait for the slave to flush to the disk. Note, as you can see, sync_state has not changed. Don't be fooled by what you see here; you can completely rely on the behavior outlined in this section. PostgreSQL is perfectly able to handle each transaction separately. This is a unique feature of this wonderful open source database; it puts you in control and lets you decide which kind of durability requirements you want. Understanding the practical implications and performance We have already talked about practical implications as well as performance implications. But what good is a theoretical example? Let's do a simple benchmark and see how replication behaves. We are performing this kind of testing to show you that various levels of durability are not just a minor topic; they are the key to performance. Let's assume a simple test: in the following scenario, we have connected two equally powerful machines (3 GHz, 8 GB RAM) over a 1 Gbit network. The two machines are next to each other. To demonstrate the impact of synchronous replication, we have left shared_buffers and all other memory parameters as default, and only changed fsync to off to make sure that the effect of disk wait is reduced to practically zero. The test is simple: we use a one-column table with only one integer field and 10,000 single transactions consisting of just one INSERT statement: INSERT INTO t_test VALUES (1); We can try this with full, synchronous replication (synchronous_commit = on): real 0m6.043s user 0m0.131s sys 0m0.169s As you can see, the test has taken around 6 seconds to complete. This test can be repeated with synchronous_commit = local now (which effectively means asynchronous replication): real 0m0.909s user 0m0.101s sys 0m0.142s In this simple test, you can see that the speed has gone up by us much as six times. Of course, this is a brute-force example, which does not fully reflect reality (this was not the goal anyway). What is important to understand, however, is that synchronous versus asynchronous replication is not a matter of a couple of percentage points or so. This should stress our point even more: replicate synchronously only if it is really needed, and if you really have to use synchronous replication, make sure that you limit the number of synchronous transactions to an absolute minimum. Also, please make sure that your network is up to the job. Replicating data synchronously over network connections with high latency will kill your system performance like nothing else. Keep in mind that throwing expensive hardware at the problem will not solve the problem. Doubling the clock speed of your servers will do practically nothing for you because the real limitation will always come from network latency. The performance penalty with just one connection is definitely a lot larger than that with many connections. Remember that things can be done in parallel, and network latency does not make us more I/O or CPU bound, so we can reduce the impact of slow transactions by firing up more concurrent work. When synchronous replication is used, how can you still make sure that performance does not suffer too much? Basically, there are a couple of important suggestions that have proven to be helpful: Use longer transactions: Remember that the system must ensure on commit that the data is available on two servers. We don't care what happens in the middle of a transaction, because anybody outside our transaction cannot see the data anyway. A longer transaction will dramatically reduce network communication. Run stuff concurrently: If you have more than one transaction going on at the same time, it will be beneficial to performance. The reason for this is that the remote server will return the position inside the XLOG that is considered to be processed safely (flushed or accepted). This method ensures that many transactions can be confirmed at the same time. Redundancy and stopping replication When talking about synchronous replication, there is one phenomenon that must not be left out. Imagine we have a two-node cluster replicating synchronously. What happens if the slave dies? The answer is that the master cannot distinguish between a slow and a dead slave easily, so it will start waiting for the slave to come back. At first glance, this looks like nonsense, but if you think about it more deeply, you will figure out that synchronous replication is actually the only correct thing to do. If somebody decides to go for synchronous replication, the data in the system must be worth something, so it must not be at risk. It is better to refuse data and cry out to the end user than to risk data and silently ignore the requirements of high durability. If you decide to use synchronous replication, you must consider using at least three nodes in your cluster. Otherwise, it will be very risky, and you cannot afford to lose a single node without facing significant downtime or risking data loss. Summary Here, we outlined the basic concept of synchronous replication, and showed how data can be replicated synchronously. We also showed how durability requirements can be changed on the fly by modifying PostgreSQL runtime parameters. PostgreSQL gives users the choice of how a transaction should be replicated, and which level of durability is necessary for a certain transaction. Resources for Article: Further resources on this subject: Introducing PostgreSQL 9 [article] PostgreSQL – New Features [article] Installing PostgreSQL [article]
Read more
  • 0
  • 0
  • 4683

article-image-creating-sample-application-simple
Packt
04 Sep 2013
8 min read
Save for later

Creating a sample application (Simple)

Packt
04 Sep 2013
8 min read
(For more resources related to this topic, see here.) How to do it... To create an application, include the JavaScript and CSS files in your page. Perform the following steps: Create an HTML document, index.html, under your project directory. Please note that this directory should be placed in the web root of your web server. Create the directories styles and scripts under your project directory. Copy the CSS file kendo.mobile.all.min.css, from <downloaded directory>/styles to the styles directory created in step 2. Then add a reference to the CSS file in the head section of the document. Download the jQuery library from jQuery.com. Place this file in the scripts directory and add a reference to this file in the document before closing the body tag. You can also specify the CDN location of the file in the document. Copy the JavaScript file kendo.mobile.min.js, from the <downloaded directory>/js tag to the scripts directory created in step 2. Then add a reference to this JavaScript file in the document (after jQuery). Add the text "Hello Kendo!!" in the body tag of the index.html file as follows: <!DOCTYPE HTML><html><head><title>My first Kendo Mobile Application</title><link rel="stylesheet"type="text/css"href="styles/kendo.mobile.all.min.css"></head><body>Hello Kendo!!<script type="text/javascript"src = "scripts/jquery.min.js"></script><script type="text/javascript"src = "scripts/kendo.mobile.min.js"></script></body></html> The preceding code snippet is a simple HTML page with references to Kendo Mobile CSS and JavaScript files. These files are minified and contain all the features, themes, and widgets. In production, you would like to include only those that are required. The downloaded ZIP file includes CSS and JavaScript files for specific features. However, in development you can use the minified files that contain all features. Another thing to note is that apart from the reference to the script kendo.mobile.min.js, the page also includes a reference to jQuery. It is the only external dependency for Kendo UI. When you view this page on a mobile device, you will see the text Hello Kendo!! shown. This page does not include any of the widgets that come as a part of the library. Now let's build on top of our Hello World application and add some visual elements; that is, UI widgets to the page. This can be done with the following steps: Add a layout first. A mobile application generally has a header, a footer, and multiple views. It is also observed that while navigating through different views in an application, the header and footer remain constant. The framework allows you to define a global layout that may contain a header and a footer for all the views in the application. Also, the framework allows you to define multiple views that can share the same layout. The following is the same page that now includes a header and footer defined in the layout: <body><div data-role="layout" data-id="defaultLayout"> <header data-role="header"> <div data-role="navbar"> My first application </div> </header> <footer data-role="footer"> <div data-role="tabstrip"> <a data-icon="about">About</a> <a data-icon="settings">Settings</a> </div> </footer> </div></body> The body contains a few div tags with data attributes. Let's look into one of these tags in detail. <div data-role="layout" data-id="defaultLayout"> Here, the div tag contains two data attributes, role and id. The role data attribute is used to initialize and configure a widget. The data-role attribute has a value, layout, identifying the target element as a layout widget. All the widgets are expected to have a role data attribute that helps in marking the target element for a specific purpose. It instructs the library which widget needs to be added to the page. The id data attribute is used to identify the widget (the layout widget) in the page. A page may define several layout widgets and each one of these must be identified by a unique ID. Here, the data-id attribute has defaultLayout as its value. Now there can be many views referring to this layout by its id. Similarly, there are other elements in the page with the data-role attribute, defining them as one of widgets in the page. Let's take a look at the header and footer widgets defined inside the layout. <header data-role="header">... </header><footer data-role="footer">...</footer> The header and footer tags have the role data attribute set to header and footer respectively. This aligns them to the top and bottom of the page, giving the rest of the available space for different views to render. Also, note that there is a navbar widget in the header and a tabstrip widget defined in the footer. As mentioned earlier, the framework comes with several widgets that can help you build the application rapidly. Now add views to the page. The index.html page now has a layout defined and when you run the page in the browser, you will see an error message in the console which says: Uncaught Error: Your kendo mobile application element does not contain any direct child elements with data-role="view" attribute set. Make sure that you instantiate the mobile application using the correct container. Views represent the actual content that has to be displayed between the header and the footer that we defined while creating a layout. A layout cannot exist without a view and hence you see that error message in the console. To fix this error, you need to define a view for your mobile application. Add the following to your index.html page: <div data-role="view" data-layout="defaultLayout"> Hello Kendo!!</div> As mentioned earlier, every widget needs to have a role data attribute to identify itself as a particular widget in the page. Here, the target element is defined as a view widget and tied to the layout by defining the data-layout attribute. The data-layout attribute has a value defaultLayout that is the same as the value for the data-id attribute of the layout that we defined earlier. This attaches the view to the layout and you will not see the error message anymore. Similarly, you can have multiple Views defined in the page that can make use of the same layout. Now, there's only one pending task for the application to start working: initializing the application. A Kendo Mobile application can be initialized using the Application object. To do that, add the following code to the page: <script> var app = new kendo.mobile.Application();</script> Include the previous script block right after references to jQuery and Kendo Mobile and before closing the body tag. This single line of JavaScript code will initialize your Kendo Mobile application and all the widgets with the data-role attribute. The Application object is used for many other purposes . How it works... When you run the index.html page in a browser, you will see a navbar and a tabstrip in the header and footer of the page. Also, the message Hello Kendo!! being shown in the body of the page. The following screenshot shows how it will look like when you view the page on an iPhone: If you have noticed, this looks like a native iOS application. The framework has the capability to render the application that looks like a native application on a device. When you view the same page on an Android device, it will look like an native Android application, as shown in the following screenshot: The framework identifies the platform on which the mobile application is being run and then provides native look and feel to the application. There are ways in which you can customize this behavior. Summary Creating a sample application (Simple)got us started with the Kendo UI Mobile framework and showed us how to build a sample application using the same. We also saw some of the Mobile UI widgets, such as layouts, views, navbar, and tabstrip in brief. Resources for Article : Further resources on this subject: Working with remote data [Article] The Decider: External APIs [Article] Constructing and Evaluating Your Design Solution [Article]
Read more
  • 0
  • 0
  • 4682
Unlock access to the largest independent learning library in Tech for FREE!
Get unlimited access to 7500+ expert-authored eBooks and video courses covering every tech area you can think of.
Renews at $19.99/month. Cancel anytime
article-image-neural-network-azure-ml
Packt
15 Jun 2015
3 min read
Save for later

Neural Network in Azure ML

Packt
15 Jun 2015
3 min read
In this article written by Sumit Mund, author of the book Microsoft Azure Machine Learning, we will learn about neural network, which is a kind of machine learning algorithm inspired by the computational models of a human brain. It builds a network of computation units, neurons, or nodes. In a typical network, there are three layers of nodes. First, the input layer, followed by the middle layer or hidden layer, and in the end, the output layer. Neural network algorithms can be used for both classification and regression problems. (For more resources related to this topic, see here.) The number of nodes in a layer depends on the problem and how you construct the network to get the best result. Usually, the number of nodes in an input layer is equal to the number of features in the dataset. For a regression problem, the number of nodes in the output layer is one while for a classification problem, it is equal to the number of class or label. Each node in a layer gets connected to all the nodes in the next layer. Each edge that connects between nodes is assigned a weight. So, a neural network can well be imagined as a weighted directed acyclic graph. In a typical neural network, as shown in the preceding figure, the middle layer or hidden layer contains the number nodes, which are chosen to make the computation right. While there is no formula or agreed convention for this, it is often optimized after trying out different options. Azure Machine Learning supports neural network for regression, two-class classification, and multiclass classification. It provides a separate module for each kind of problem and lets the users tune it with different parameters, such as the number of hidden nodes, number of iterations to train the model, and so on. A special kind of neural network algorithms where there are more than one hidden layers is known as deep networks or deep learning algorithms. Azure Machine Learning allows us to choose the number of hidden nodes as a property value of the neural network module. These kind of neural networks are getting increasingly popular these days because of their remarkable results and because they allow us to model complex and nonlinear scenarios. There are many kinds of deep networks, but recently, a special kind of deep network known as the convolutional neural network got very popular because of its significant performance in image recognition or classification problems. Azure Machine Learning supports the convolutional neural network. For simple networks with three layers, this can be done through a UI just by choosing parameters. However, to build a deep network like a convolutional deep network, it’s not easy to do so through a UI. So, Azure Machine Learning supports a new kind of language called Net#, which allows you to script different kinds of neural network inside ML Studio by defining different node, the connections (edges), and kind of connections. While deep networks are complex to build and train, Net# makes things relatively easy and simple. Though complex, neural networks are very powerful and Azure Machine Learning makes it fun to work with these be it three-layered shallow networks or multilayer deep networks. Resources for Article: Further resources on this subject: Security in Microsoft Azure [article] High Availability, Protection, and Recovery using Microsoft Azure [article] Managing Microsoft Cloud [article]
Read more
  • 0
  • 0
  • 4680

article-image-customizing-xtext-components
Packt
08 Sep 2016
30 min read
Save for later

Customizing Xtext Components

Packt
08 Sep 2016
30 min read
In this article written by Lorenzo Bettini, author of the book Implementing Domain Specific Languages Using Xtend and Xtext, Second Edition, the author describes the main mechanism for customizing Xtext components—Google Guice, a Dependency Injection framework. With Google Guice, we can easily and consistently inject custom implementations of specific components into Xtext. In the first section, we will briefly show some Java examples that use Google Guice. Then, we will show how Xtext uses this dependency injection framework. In particular, you will learn how to customize both the runtime and the UI aspects. This article will cover the following topics: An introduction to Google Guice dependency injection framework How Xtext uses Google Guice How to customize several aspects of an Xtext DSL (For more resources related to this topic, see here.) Dependency injection The Dependency Injection pattern (see the article Fowler, 2004) allows you to inject implementation objects into a class hierarchy in a consistent way. This is useful when classes delegate specific tasks to objects referenced in fields. These fields have abstract types (that is, interfaces or abstract classes) so that the dependency on actual implementation classes is removed. In this first section, we will briefly show some Java examples that use Google Guice. Of course, all the injection principles naturally apply to Xtend as well. If you want to try the following examples yourself, you need to create a new Plug-in Project, for example, org.example.guice and add com.google.inject and javax.inject as dependencies in the MANIFEST.MF. Let's consider a possible scenario: a Service class that abstracts from the actual implementation of a Processor class and a Logger class. The following is a possible implementation: public class Service {   private Logger logger;   private Processor processor;     public void execute(String command) { logger.log("executing " + command); processor.process(command); logger.log("executed " + command);   } }   public class Logger {   public void log(String message) { out.println("LOG: " + message);   } }   public interface Processor {   public void process(Object o); }   public classProcessorImplimplements Processor {   private Logger logger;     public void process(Object o) { logger.log("processing"); out.println("processing " + o + "...");   } } These classes correctly abstract from the implementation details, but the problem of initializing the fields correctly still persists. If we initialize the fields in the constructor, then the user still needs to hardcode the actual implementation classnames. Also, note that Logger is used in two independent classes; thus, if we have a custom logger, we must make sure that all the instances use the correct one. These issues can be dealt with using dependency injection. With dependency injection, hardcoded dependencies will be removed. Moreover, we will be able to easily and consistently switch the implementation classes throughout the code. Although the same goal can be achieved manually by implementing factory method or abstract factory patterns (see the book Gamma et al, 1995), with dependency injection framework it is easier to keep the desired consistency and the programmer needs to write less code. Xtext uses the dependency injection framework Google Guice, https://github.com/google/guice. We refer to the Google Guice documentation for all the features provided by this framework. In this section, we just briefly describe its main features. You annotate the fields you want Guice to inject with the @Inject annotation (com.google.inject.Inject): public class Service {   @Inject private Logger logger;   @Inject private Processor processor;     public void execute(String command) { logger.log("executing " + command); processor.process(command); logger.log("executed " + command);   } }   public class ProcessorImpl implements Processor {   @Inject private Logger logger;     public void process(Object o) { logger.log("processing"); out.println("processing " + o + "...");   } } The mapping from injection requests to instances is specified in a Guice Module, a class that is derived from com.google.inject.AbstractModule. The method configure is implemented to specify the bindings using a simple and intuitive API. You only need to specify the bindings for interfaces, abstract classes, and for custom classes. This means that you do not need to specify a binding for Logger since it is a concrete class. On the contrary, you need to specify a binding for the interface Processor. The following is an example of a Guice module for our scenario: public class StandardModule extends AbstractModule {   @Override   protected void configure() {     bind(Processor.class).to(ProcessorImpl.class);   } } You create an Injector using the static method Guice.createInjector by passing a module. You then use the injector to create instances: Injector injector = Guice.createInjector(newStandardModule()); Service service = injector.getInstance(Service.class); service.execute("First command"); The initialization of injected fields will be done automatically by Google Guice. It is worth noting that the framework is also able to initialize (inject) private fields, like in our example. Instances of classes that use dependency injection must be created only through an injector. Creating instances with new will not trigger injection, thus all the fields annotated with @Inject will be null. When implementing a DSL with Xtext you will never have to create a new injector manually. In fact, Xtext generates utility classes to easily obtain an injector, for example, when testing your DSL with JUnit. We also refer to the article Köhnlein, 2012 for more details. The example shown in this section only aims at presenting the main features of Google Guice. If we need a different configuration of the bindings, all we need to do is define another module. For example, let's assume that we defined additional derived implementations for logging and processing. Here is an example where Logger and Processor are bound to custom implementations: public class CustomModule extends AbstractModule {   @Override   protected void configure() {     bind(Logger.class).to(CustomLogger.class);     bind(Processor.class).to(AdvancedProcessor.class);   } } Creating instances with an injector obtained using this module will ensure that the right classes are used consistently. For example, the CustomLogger class will be used both by Service and Processor. You can create instances from different injectors in the same application, for example: executeService(Guice.createInjector(newStandardModule())); executeService(Guice.createInjector(newCustomModule()));   voidexecuteService(Injector injector) {   Service service = injector.getInstance(Service.class); service.execute("First command"); service.execute("Second command"); } It is possible to request injection in many different ways, such as injection of parameters to constructors, using named instances, specification of default implementation of an interface, setter methods, and much more. In this book, we will mainly use injected fields. Injected fields are instantiated only once when the class is instantiated. Each injection will create a new instance, unless the type to inject is marked as @Singleton(com.google.inject.Singleton). The annotation @Singleton indicates that only one instance per injector will be used. We will see an example of Singleton injection. If you want to decide when you need an element to be instantiated from within method bodies, you can use a provider. Instead of injecting an instance of the wanted type C, you inject a com.google.inject.Provider<C> instance, which has a get method that produces an instance of C. For example: public class Logger {   @Inject   private Provider<Utility>utilityProvider;     public void log(String message) { out.println("LOG: " + message + " - " + utilityProvider.get().m());    } } Each time we create a new instance of Utility using the injected Provider class. Even in this case, if the type of the created instance is annotated with @Singleton, then the same instance will always be returned for the same injector. The nice thing is that to inject a custom implementation of Utility, you do not need to provide a custom Provider: you just bind the Utility class in the Guice module and everything will work as expected: public classCustomModule extends AbstractModule {   @Override   protected void configure() {     bind(Logger.class).to(CustomLogger.class);     bind(Processor.class).to(AdvancedProcessor.class);     bind(Utility.class).to(CustomUtility.class);   } }   It is crucial to keep in mind that once classes rely on injection, their instances must be created only through an injector; otherwise, all the injected elements will be null. In general, once dependency injection is used in a framework, all classes of the framework must rely on injection. Google Guice in Xtext All Xtext components rely on Google Guice dependency injection, even the classes that Xtext generates for your DSL. This means that in your classes, if you need to use a class from Xtext, you just have to declare a field of such type with the @Inject annotation. The injection mechanism allows a DSL developer to customize basically every component of the Xtext framework. This boils down to another property of dependency injection, which, in fact, inverts dependencies. The Xtext runtime can use your classes without having a dependency to its implementer. Instead, the implementer has a dependency on the interface defined by the Xtext runtime. For this reason, dependency injection is said to implement inversion of control and the dependency inversion principle. When running the MWE2 workflow, Xtext generates both a fully configured module and an empty module that inherits from the generated one. This allows you to override generated or default bindings. Customizations are added to the empty stub module. The generated module should not be touched. Xtext generates one runtime module that defines the non-user interface-related parts of the configuration and one specific for usage in the Eclipse IDE. Guice provides a mechanism for composing modules that is used by Xtext—the module in the UI project uses the module in the runtime project and overrides some bindings. Let's consider the Entities DSL example. You can find in the src directory of the runtime project the Xtend class EntitiesRuntimeModule, which inherits from AbstractEntitiesRuntimeModule in the src-gen directory. Similarly, in the UI project, you can find in the src directory the Xtend class EntitiesUiModule, which inherits from AbstractEntitiesUiModule in the src-gen directory. The Guice modules in src-gen are already configured with the bindings for the stub classes generated during the MWE2 workflow. Thus, if you want to customize an aspect using a stub class, then you do not have to specify any specific binding. The generated stub classes concern typical aspects that the programmer usually wants to customize, for example, validation and generation in the runtime project, and labels, and outline in the UI project (as we will see in the next sections). If you need to customize an aspect which is not covered by any of the generated stub classes, then you will need to write a class yourself and then specify the binding for your class in the Guice module in the src folder. We will see an example of this scenario in the Other customizations section. Bindings in these Guice module classes can be specified as we saw in the previous section, by implementing the configure method. However, Xtext provides an enhanced API for defining bindings; Xtext reflectively searches for methods with a specific signature in order to find Guice bindings. Thus, assuming you want to bind a BaseClass class to your derived CustomClass, you can simply define a method in your module with a specific signature, as follows: def Class<? extendsBaseClass>bindBaseClass() {   returnCustomClass } Remember that in Xtend, you must explicitly specify that you are overriding a method of the base class; thus, in case the bind method is already defined in the  base class, you need to use override instead of def. These methods are invoked reflectively, thus their signature must follow the expected convention. We refer to the official Xtext documentation for the complete description of the module API. Typically, the binding methods that you will see in this book will have the preceding shape, in particular, the name of the method must start with bind followed by the name of the class or interface we want to provide a binding for. It is important to understand that these bind methods do not necessarily have to override a method in the module base class. You can also make your own classes, which are not related to Xtext framework classes at all, participants of this injection mechanism, as long as you follow the preceding convention on method signatures. In the rest of this article, we will show examples of customizations of both IDE and runtime concepts. For most of these customizations, we will modify the corresponding Xtend stub class that Xtext generated when running the MWE2 workflow. As hinted before, in these cases, we will not need to write a custom Guice binding. We will also show an example of a customization, which does not have an automatically generated stub class. Xtext uses injection to inject services and not to inject state (apart from EMF Singleton registries). Thus, the things that are injected are interfaces consisting of functions that take state as arguments (for example, the document, the resource, and so on.). This leads to a service-oriented architecture, which is different from an object-oriented architecture where state is encapsulated with operations. An advantage of this approach is that there are far less problems with synchronization of multiple threads. Customizations of IDE concepts In this section, we show typical concepts of the IDE for your DSL that you may want to customize. Xtext shows its usability in this context as well, since, as you will see, it reduces the customization effort. Labels Xtext UI classes make use of an ILabelProvider interface to obtain textual labels and icons through its methods getText and getImage, respectively. ILabelProvider is a standard component of Eclipse JFace-based viewers. You can see the label provider in action in the Outline view and in content assist proposal popups (as well as in various other places). Xtext provides a default implementation of a label provider for all DSLs, which does its best to produce a sensible representation of the EMF model objects using the name feature, if it is found in the corresponding object class, and a default image. You can see that in the Outline view when editing an entities file, refer to the following screenshot: However, you surely want to customize the representation of some elements of your DSL. The label provider Xtend stub class for your DSL can be found in the UI plug-in project in the subpackageui.labeling. This stub class extends the base class DefaultEObjectLabelProvider. In the Entities DSL, the class is called EntitiesLabelProvider. This class employs a Polymorphic Dispatcher mechanism, which is also used in many other places in Xtext. Thus, instead of implementing the getText and getImage methods, you can simply define several versions of methods text and image taking as parameter an EObject object of the type you want to provide a representation for. Xtext will then search for such methods according to the runtime type of the elements to represent. For example, for our Entities DSL, we can change the textual representation of attributes in order to show their names and a better representation of types (for example, name : type). We then define a method text taking Attribute as a parameter and returning a string: classEntitiesLabelProviderextends ... {     @Inject extensionTypeRepresentation   def text(Attribute a) { a.name +       if (a.type != null)          " : " + a.type.representation       else ""   } } To get a representation of the AttributeType element, we use an injected extension, TypeRepresentation, in particular its method representation: classTypeRepresentation { def representation(AttributeType t) { valelementType = t.elementType valelementTypeRepr =       switch (elementType) { BasicType : elementType.typeName EntityType : elementType?.entity.name       } elementTypeRepr + if (t.array) "[]"else""   } } Remember that the label provider is used, for example, for the Outline view, which is refreshed when the editor contents change, and its contents might contain errors. Thus, you must be ready to deal with an incomplete model, and some features might still be null. That is why you should always check that the features are not null before accessing them. Note that we inject an extension field of type TypeRepresentation instead of creating an instance with new in the field declaration. Although it is not necessary to use injection for this class, we decided to rely on that because in the future we might want to be able to provide a different implementation for that class. Another point for using injection instead of new is that the other class may rely on injection in the future. Using injection leaves the door open for future and unanticipated customizations. The Outline view now shows as in the following screenshot: We can further enrich the labels for entities and attributes using images for them. To do this, we create a directory in the org.example.entities.ui project where we place the image files of the icons we want to use. In order to benefit from Xtext's default handling of images, we call the directory icons, and we place two gif images there, Entity.gif and Attribute.gif (for entities and attributes, respectively). You fill find the icon files in the accompanying source code in the org.example.entities.ui/icons folder. We then define two image methods in EntitiesLabelProvider where we only need to return the name of the image files and Xtext will do the rest for us: class EntitiesLabelProvider extends DefaultEObjectLabelProvider {   ... as before def image(Entity e) { "Entity.gif" }   def image(Attribute a) { "Attribute.gif" } } You can see the result by relaunching Eclipse, as seen in the following screenshot: Now, the entities and attributes labels look nicer. If you plan to export the plugins for your DSL so that others can install them in their Eclipse, you must make sure that the icons directory is added to the build.properties file, otherwise that directory will not be exported. The bin.includes section of the build.properties file of your UI plugin should look like the following: bin.includes = META-INF/,                ., plugin.xml,                icons/ The Outline view The default Outline view comes with nice features. In particular, it provides toolbar buttons to keep the Outline view selection synchronized with the element currently selected in the editor. Moreover, it provides a button to sort the elements of the tree alphabetically. By default, the tree structure is built using the containment relations of the metamodel of the DSL. This strategy is not optimal in some cases. For example, an Attribute definition also contains the AttributeType element, which is a structured definition with children (for example, elementType, array, and length). This is reflected in the Outline view (refer to the previous screenshot) if you expand the Attribute elements. This shows unnecessary elements, such as BasicType names, which are now redundant since they are shown in the label of the attribute, and additional elements which are not representable with a name, such as the array feature. We can influence the structure of the Outline tree using the generated stub class EntitiesOutlineTreeProvider in the src folder org.example.entities.ui.outline. Also in this class, customizations are specified in a declarative way using the polymorphic dispatch mechanism. The official documentation, https://www.eclipse.org/Xtext/documentation/, details all the features that can be customized. In our example, we just want to make sure that the nodes for attributes are leaf nodes, that is, they cannot be further expanded and they have no children. In order to achieve this, we just need to define a method named _isLeaf (note the underscore) with a parameter of the type of the element, returning true. Thus, in our case we write the following code: classEntitiesOutlineTreeProviderextends DefaultOutlineTreeProvider { def _isLeaf(Attribute a) { true } } Let's relaunch Eclipse, and now see that the attribute nodes do not expose children anymore. Besides defining leaf nodes, you can also specify the children in the tree for a specific node by defining a _createChildren method taking as parameters the type of outline node and the type of the model element. This can be useful to define the actual root elements of the Outline tree. By default, the tree is rooted with a single node for the source file. In this example, it might be better to have a tree with many root nodes, each one representing an entity. The root of the Outline tree is always represented by a node of type DefaultRootNode. The root node is actually not visible, it is just the container of all nodes that will be displayed as roots in the tree. Thus, we define the following method (our Entities model is rooted by a Model element): public classEntitiesOutlineTreeProvider ... {   ... as before def void _createChildren(DocumentRootNodeoutlineNode,                            Model model) { model.entities.forEach[       entity | createNode(outlineNode, entity);    ]   } } This way, when the Outline tree is built, we create a root node for each entity instead of having a single root for the source file. The createNode method is part of the Xtext base class. The result can be seen in the following screenshot: Customizing other aspects We will show how to customize the content assistant. There is no need to do this for the simple Entities DSL since the default implementation already does a fine job. Custom formatting An editor for a DSL should provide a mechanism for rearranging the text of the program in order to improve its readability, without changing its semantics. For example, nested regions inside blocks should be indented, and the user should be able to achieve that with a menu. Besides that, implementing a custom formatter has also other benefits, since the formatter is automatically used by Xtext when you change the EMF model of the AST. If you tried to apply the quickfixes, you might have noticed that after the EMF model has changed, the editor immediately reflects this change. However, the resulting textual representation is not well formatted, especially for the quickfix that adds the missing referred entity. In fact, the EMF model representing the AST does not contain any information about the textual representation, that is, all white space characters are not part of the EMF model (after all, the AST is an abstraction of the actual program). Xtext keeps track of such information in another in-memory model called the nodemodel. The node model carries the syntactical information, that is, offset and length in the textual document. However, when we manually change the EMF model, we do not provide any formatting directives, and Xtext uses the default formatter to get a textual representation of the modified or added model parts. Xtext already generates the menu for formatting your DSL source programs in the Eclipse editor. As it is standard in Eclipse editors (for example, the JDT editor), you can access the Format menu from the context menu of the editor or using the Ctrl + Shift + F key combination. The default formatter is OneWhitespaceFormatter and you can test this in the Entities DSL editor; this formatter simply separates all tokens of your program with a space. Typically, you will want to change this default behavior. If you provide a custom formatter, this will be used not only when the Format menu is invoked, but also when Xtext needs to update the editor contents after a manual modification of the AST model, for example, a quickfix performing a semantic modification. The easiest way to customize the formatting is to have the Xtext generator create a stub class. To achieve this, you need to add the following formatter specification in the StandardLanguage block in the MWE2 workflow file, requesting to generate an Xtend stub class: language = StandardLanguage {     name = "org.example.entities.Entities" fileExtensions = "entities"     ...     formatter = { generateStub = true generateXtendStub = true     } } If you now run the workflow, you will find the formatter Xtend stub class in the main plugin project in the formatting2 package. For our Entities DSL, the class is org.example.entities.formatting2.EntitiesFormatter. This stub class extends the Xtext class AbstractFormatter2. Note that the name of the package ends with 2. That is because Xtext recently completely changed the customization of the formatter to enhance its mechanisms. The old formatter is still available, though deprecated, so the new formatter classes have the 2 in the package in order not to be mixed with the old formatter classes. In the generated stub class, you will get lots of warnings of the shape Discouraged access: the type AbstractFormatter2 is not accessible due to restriction on required project org.example.entities. That is because the new formatting API is still provisional, and it may change in future releases in a non-backward compatible way. Once you are aware of that, you can decide to ignore the warnings. In order to make the warnings disappear from the Eclipse project, you configure the specific project settings to ignore such warnings, as shown in the following screenshot: The Xtend stub class already implements a few dispatch methods, taking as parameters the AST element to format and an IFormattableDocument object. The latter is used to specify the formatting requests. A formatting request will result in a textual replacement in the program text. Since it is an extension parameter, you can use its methods as extension methods (for more details on extension methods. The IFormattableDocument interface provides a Java API for specifying formatting requests. Xtend features such as extension methods and lambdas will allow you to specify formatting request in an easy and readable way. The typical formatting requests are line wraps, indentations, space addition and removal, and so on. These will be applied on the textual regions of AST elements. As we will show in this section, the textual regions can be specified by the EObject of AST or by its keywords and features. For our Entities DSL, we decide to perform formatting as follows: Insert two newlines after each entity so that entities will be separated by an empty line; after the last entity, we want a single empty line. Indent attributes between entities curly brackets. Insert one line-wrap after each attribute declaration. Make sure that entity name, super entity, and the extends keyword are surrounded by a single space. Remove possible white spaces around the ; of an attribute declaration. To achieve the empty lines among entities, we modify the stub method for the Entities Model element: def dispatch void format(Model model,                                 extensionIFormattableDocument document) { vallastEntity = model.entities.last   for (entity : model.entities) { entity.format     if (entity === lastEntity) entity.append[setNewLines(1)]     else entity.append[setNewLines(2)]   } } We append two newlines after each entity. This way, each entity will be separated by an empty line, since each entity, except for the first one, will start on the second added newline. We append only one newline after the last entity. Now start a new Eclipse instance and manually test the formatter with some entities, by pressing Ctrl + Shift + F. We modify the format stub method for the Entity elements. In order to separate each attribute, we follow a logic similar to the previous format method. For the sake of the example, we use a different version of setNewLines, that is setNewLines(intminNewLines, intdefaultNewLines, intmaxNewLines), whose signature is self-explanatory: for (attribute : entity.attributes) { attribute.append[setNewLines(1, 1, 2)] } Up to now, we referred to a textual region of the AST by specifying the EObject. Now, we need to specify the textual regions of keywords and features of a given AST element. In order to specify that the "extends" keyword is surrounded by one single space we write the following: entity.regionFor.keyword("extends").surround[oneSpace] We also want to have no space around the terminating semicolon of attributes, so we write the following: attribute.regionFor.keyword(";").surround[noSpace] In order to specify that the the entity's name and the super entity are surrounded by one single space we write the following: entity.regionFor.feature(ENTITY__NAME).surround[oneSpace] entity.regionFor.feature(ENTITY__SUPER_TYPE).surround[oneSpace] After having imported statically all the EntitiesPackage.Literals members, as follows: import staticorg.example.entities.entities.EntitiesPackage.Literals.* Finally, we want to handle the indentation inside the curly brackets of an entity and to have a newline after the opening curly bracket. This is achieved with the following lines: val open = entity.regionFor.keyword("{") val close = entity.regionFor.keyword("}") open.append[newLine] interior(open, close)[indent] Summarizing, the format method for an Entity is the following one: def dispatch void format(Entity entity,                           extensionIFormattableDocument document) { entity.regionFor.keyword("extends").surround[oneSpace] entity.regionFor.feature(ENTITY__NAME).surround[oneSpace] entity.regionFor.feature(ENTITY__SUPER_TYPE).surround[oneSpace]   val open = entity.regionFor.keyword("{") val close = entity.regionFor.keyword("}") open.append[newLine]   interior(open, close)[indent]     for (attribute : entity.attributes) { attribute.regionFor.keyword(";").surround[noSpace] attribute.append[setNewLines(1, 1, 2)]   } } Now, start a new Eclipse instance and manually test the formatter with some attributes and entities, by pressing Ctrl + Shift + F. In the generated Xtend stub class, you also find an injected extension for accessing programmatically the elements of your grammar. In this DSL it is the following: @Inject extensionEntitiesGrammarAccess For example, to specify the left curly bracket of an entity, we could have written this alternative line: val open = entity.regionFor.keyword(entityAccess.leftCurlyBracketKeyword_3) Similarly, to specify the terminating semicolon of an attribute, we could have written this alternative line: attribute.regionFor.keyword(attributeAccess.semicolonKeyword_2)   .surround[noSpace] Eclipse content assist will help you in selecting the right method to use. Note that the method names are suffixed with numbers that relate to the position of the keyword in the grammar's rule. Changing a rule in the DSL's grammar with additional elements or by removing some parts will make such method invocations invalid since the method names will change. On the other hand, if you change a keyword in your grammar, for example, you use square brackets instead of curly brackets, then referring to keywords with string literals as we did in the original implementation of the format methods will issue no compilation errors, but the formatting will not work anymore as expected. Thus, you need to choose your preferred strategy according to the likeliness of your DSL's grammar evolution. You can also try and apply our quickfixes for missing entities and you will see that the added entity is nicely formatted, according to the logic we implemented. What is left to be done is to format the attribute type nicely, including the array specification. This is left as an exercise. The EntitiesFormatter you find in the accompanying sources of this example DSL contains also this formatting logic for attribute types. You should specify formatting requests avoiding conflicting requests on the same textual region. In case of conflicts, the formatter will throw an exception with the details of the conflict. Other customizations All the customizations you have seen so far were based on modification of a generated stub class with accompanying generated Guice bindings in the module under the src-gen directory. However, since Xtext relies on injection everywhere, it is possible to inject a custom implementation for any mechanism, even if no stub class has been generated. If you installed Xtext SDK in your Eclipse, the sources of Xtext are available for you to inspect. You should learn to inspect these sources by navigating to them and see what gets injected and how it is used. Then, you are ready to provide a custom implementation and inject it. You can use the Eclipse Navigate menu. In particular, to quickly open a Java file (even from a library if it comes with sources), use Ctrl + Shift + T (Open Type…). This works both for Java classes and Xtend classes. If you want to quickly open another source file (for example, an Xtext grammar file) use Ctrl + Shift + R (Open Resource…). Both dialogs have a text field where, if you start typing, the available elements soon show up. Eclipse supports CamelCase everywhere, so you can just type the capital letters of a compound name to quickly get to the desired element. For example, to open the EntitiesRuntimeModule Java class, use the Open Type… menu and just digit ERM to see the filtered results. As an example, we show how to customize the output directory where the generated files will be stored (the default is src-gen). Of course, this output directory can be modified by the user using the Properties dialog that Xtext generated for your DSL, but we want to customize the default output directory for Entities DSL so that it becomes entities-gen. The default output directory is retrieved internally by Xtext using an injected IOutputConfigurationProvider instance. If you take a look at this class (see the preceding tip), you will see the following: importcom.google.inject.ImplementedBy; @ImplementedBy(OutputConfigurationProvider.class) public interfaceIOutputConfigurationProvider {   Set<OutputConfiguration>getOutputConfigurations();   ...  The @ImplementedByGuice annotation tells the injection mechanism the default implementation of the interface. Thus, what we need to do is create a subclass of the default implementation (that is, OutputConfigurationProvider) and provide a custom binding for the IOutputConfigurationProvider interface. The method we need to override is getOutputConfigurations; if we take a look at its default implementation, we see the following: public Set<OutputConfiguration>getOutputConfigurations() { OutputConfigurationdefaultOutput = new OutputConfiguration(IFileSystemAccess.DEFAULT_OUTPUT); defaultOutput.setDescription("Output Folder"); defaultOutput.setOutputDirectory("./src-gen"); defaultOutput.setOverrideExistingResources(true); defaultOutput.setCreateOutputDirectory(true); defaultOutput.setCleanUpDerivedResources(true); defaultOutput.setSetDerivedProperty(true); defaultOutput.setKeepLocalHistory(true);   returnnewHashSet(defaultOutput); } Of course, the interesting part is the call to setOutputDirectory. We define an Xtend subclass as follows: classEntitiesOutputConfigurationProviderextends OutputConfigurationProvider {     public static valENTITIES_GEN = "./entities-gen"     overridegetOutputConfigurations() { super.getOutputConfigurations() => [ head.outputDirectory = ENTITIES_GEN     ]   } } Note that we use a public constant for the output directory since we might need it later in other classes. We use several Xtend features: the with operator, the implicit static extension method head, which returns the first element of a collection, and the syntactic sugar for setter method. We create this class in the main plug-in project, since this concept is not just an UI concept and it is used also in other parts of the framework. Since it deals with generation, we create it in the generatorsubpackage. Now, we must bind our implementation in the EntitiesRuntimeModule class: classEntitiesRuntimeModuleextends AbstractEntitiesRuntimeModule {   def Class<? extendsIOutputConfigurationProvider> bindIOutputConfigurationProvider() {     returnEntitiesOutputConfigurationProvider   } } If we now relaunch Eclipse, we can verify that the Java code is generated into entities-gen instead of src-gen. If you previously used the same project, the src-gen directory might still be there from previous generations; you need to manually remove it and set the new entities-gen as a source folder. Summary In this article, we introduced the Google Guice dependency injection framework on which Xtext relies. You should now be aware of how easy it is to inject custom implementations consistently throughout the framework. You also learned how to customize some basic runtime and IDE concepts for a DSL. Resources for Article: Further resources on this subject: Testing with Xtext and Xtend [article] Clojure for Domain-specific Languages - Design Concepts with Clojure [article] Java Development [article]
Read more
  • 0
  • 0
  • 4676

article-image-wireshark
Packt
24 Oct 2014
16 min read
Save for later

Wireshark

Packt
24 Oct 2014
16 min read
In this article by James H. Baxter, author of Wireshark Essentials, we will learn how to install Wireshark, perform a packet capture, use display filters to isolate traffic of interest, and save a filtered packet trace file. (For more resources related to this topic, see here.) Installing Wireshark Wireshark can be installed on machines running 32- and 64-bit Windows (XP, Win7, Win8.1, and so on), Mac OS X (10.5 and higher), and most flavors of Linux/Unix. Installation on Windows and Mac machines is quick and easy because installers are available from the Wireshark website download page. Wireshark is a standard package available on many Linux distributions, and there is a list of links to third-party installers provided on the Wireshark download page for a variety of popular *nix platforms. Alternatively, you can download the source code and compile Wireshark for your environment if a precompiled installation package isn't available. Wireshark relies on the WinPcap (Windows) or libpcap (Linux/Unix/Mac) libraries to provide the packet capture and capture filtering function; the appropriate library is installed during the Wireshark installation. You might need the administrator (Windows) or root (Linux/Unix/Mac) privileges to install Wireshark and the WinPcap/libpcap utilities on your workstation. Assuming that you're installing Wireshark on a Windows or Mac machine, you need to go to the Wireshark website (https://www.wireshark.org/) and click on the Download button at the top of the page. This will take you to the download page, and at the same time attempt to perform an autodiscovery of your operating system type and version from your browser info. The majority of the time, the correct Wireshark installation package for your machine will be highlighted, and you only have to click on the highlighted link to download the correct installer. If you already have Wireshark installed, an autoupdate feature will notify you of available version updates when you launch Wireshark. Installing Wireshark on Windows In the following screenshot, the Wireshark download page has identified that a 64-bit Windows installer is appropriate for this Windows workstation: Clicking on the highlighted link downloads a Wireshark-win64-1.10.8.exe file or similar executable file that you can save on your hard drive. Double-clicking on the executable starts the installation process. You need to follow these steps: Agree to the License Agreement. Accept all of the defaults by clicking on Next for each prompt, including the prompt to install WinPcap, which is a library needed to capture packets from the Network Interface Card (NIC) on your workstation. Early in the Wireshark installation, the process will pause and prompt you to click on Install and several Next buttons in separate windows to install WinPcap. After the WinPcap installation is complete, click through the remaining Next prompts to finish the Wireshark installation. Installing Wireshark on Mac OS X The process to install Wireshark on Mac is the same as the process for Windows, except that you will not be prompted to install WinPcap; libpcap, the packet capture library for Mac and *nix machines, gets installed instead (without prompting). There are, however, two additional requirements that may need to be addressed in a Mac installation: The first is to install X11, a windowing system library. If this is needed for your system, you will be informed and provided a link that ultimately takes you to the XQuartz project download page so you can install this package. The second requirement that might come up is if upon starting Wireshark, you are informed that there are no interfaces on which a capture can be done. This is a permissions issue on the Berkeley packet filter (BPF) that can be resolved by opening a terminal window and typing the following command: bash-3.2$ sudo chmod 644 /dev/bpf* If this process needs to be repeated each time you start Wireshark, you can perform a web search for more permanent permissions solution for your environment. Installing Wireshark on Linux/Unix The requirements and process to install Wireshark on a Linux or Unix platform can vary significantly depending on the particular environment. Performing your first packet capture When you first start Wireshark, you are presented with an initial Start Page as shown in the following screenshot: Don't get too fond of this screen. Although you'll see this every time you start Wireshark, once you do a capture, open a trace file, or perform any other function within Wireshark, this screen will be replaced with the standard Wireshark user interface and you won't see it again until the next time you start Wireshark. So, we won't spend much time here. Selecting a network interface If you have a number of network interfaces on your machine, you may not be sure which one to select to capture packets, but there's a fairly easy way to figure this out. On the Wireshark start page, click on Interface List (alternatively, click on Interfaces from the Capture menu or click on the first icon on the icon bar). The Wireshark Capture Interfaces window that opens provides a list and description of all the network interfaces on your machine, the IP address assigned to each one (if an address has been assigned), and a couple of counters, such as the total number of packets seen on the interface since this window starts and a packets/s (packets per second) counter. If an interface has an IPv6 address assigned (which may start with fe80:: and contain a number of colons) and this is being displayed, you can click on the IPv6 address and it will toggle to display the IPv4 address. This is shown in the following screenshot: On Linux/Unix/Mac platforms, you might also see a loopback interface that can be selected to capture packets being sent between applications on the same machine. However, in most cases, you'll only be interested in capturing packets from a network interface. The goal is to identify the active interface that will be used to communicate with the Internet when you open a browser and navigate to a website. If you have a wired local area network connection and the interface is enabled, that's probably the active interface, but you might also have a wireless interface that is enabled and you may or may not be the primary interface. The most reliable indicator of the active network interface is that it will have greater number of steadily increasing packets with a corresponding active number of packets/s (which will vary over time). Another possible indicator is if an interface has an IP address assigned and others do not. If you're still unsure, open a browser window and navigate to one of your favorite websites and watch the packets and packets/s counters to identify the interface that shows the greatest increase in activity. Performing the packet capture Once you've identified the correct interface, select the checkbox on the left-hand side of that interface and click on the Start button at the bottom of the Capture Interfaces window. Wireshark will start capturing all the packets that can be seen from that interface, including the packets sent to and from your workstation. If you don't see this, try a different interface. It's a bit amazing just how much background traffic there is on a typical network such as broadcast packets from devices advertising their names, addresses, and services and from other devices asking for addresses of stations they want to communicate with. Also, a fair amount of traffic is generated from your own workstation for applications and services that are running in the background, and you had no idea they were creating this much noise. Your Wireshark's Packet List pane may look similar to the following screenshot; however, we can ignore all this for now: We're ready to generate some traffic that we'll be interested in analyzing. Open a new Internet browser window, enter www.wireshark.org in the address box, and press Enter. When the https://www.wireshark.org/ home page finishes loading, stop the Wireshark capture by either selecting Stop from the Capture menu or by clicking on the red square stop icon that's between the View and Go menu headers. Wireshark user interface essentials Once you have completed your first capture, you will see the normal Wireshark user interface main screen. So before we go much further, a quick introduction to the primary parts of this user interface will be helpful so you'll know what's being referred to when as we continue the analysis process. There are eight significant sections or elements of the default Wireshark user interface as shown in the following screenshot: Let's look at the eight significant sections in detail: Title: This area reflects the interface from where a capture is being taken or the filename of an open packet trace file Menu: This is the standard row of main functions and subfunctions in Wireshark Main toolbar (icons): These provide a quick way to access the most useful Wireshark functions and are well worth getting familiar with and using Display filter toolbar: This allows you to quickly create, edit, clear, apply, and save filters to isolate packets of interest for analysis Packet list pane: This section contains a summary info line for each captured packet, as well as a packet number and relative timestamp Packet details pane: This section provides a hierarchical display of information about a single packet that has been selected in the packet list pane, which is divided into sections for the various protocols contained in a packet Packet bytes pane: This section displays the selected packets' contents in hex bytes or bits form, as well as an ASCII display of the data that can be helpful Status bar: This section provides an expert info indicator, edit capture comments icon, trace file path, name, and size information, data on the number of packets captured and displayed and other info, and a profile display and selection section. Filtering out the noise Somewhere in your packet capture, there are packets involved with loading the Wireshark home page—but how do you find and view just those packets out of all the background noise? The simplest and most reliable method is to determine the IP address of the Wireshark website and filter out all the packets except those flowing between that IP address and the IP address of your workstation by using a display filter. The best approach—and the one that you'll likely use as a first step for most of your post-capture analysis work in future—is to investigate a list of all the conversations by IP address and/or hostnames, sorted by the most active nodes, and identify your target hostname, website name, or IP address from this list. From the Wireshark menu, select Conversations from the Statistics menu and in the Conversations window that opens, select the IPv4 tab at the top. You'll see a list of network conversations identified by Address A and Address B, with columns for total Packets, Bytes, Packets A→B, Bytes A→B, Packets A←B, and Bytes A←B. Scrolling over to the right-hand side of this window, there are Relative Start values. These are the times when each particular conversation was first observed in the capture, relative to the start of the capture in seconds. The next column is Duration, which is how long this conversation persisted in the capture (first to last packet seen). Finally, there are average data rates in bits per second (bps) in each direction for each conversation, which is the network impact for this conversation. All these are shown in the following screenshot: We want to sort the list of conversations to get the busiest ones—called the Top Talkers in the network jargon—at the top of the list. Click on the Bytes column header and then click it again. Your list should look something like the preceding screenshot, and if you didn't get a great deal on other background traffic flowing to/from your workstation, the traffic from https://www.wireshark.org/ should have the greatest volume and therefore be at the top of the list. In this example, the conversation between IP addresses 162.159.241.165 and 192.168.1.116 has the greatest overall volume, and looking at the Bytes A->B column, it's apparent that the majority of the traffic was from the 162.159.241.165 address to the 192.168.1.116 address. However, at this point, how do we know if this is really the conversation that we're after? We will need to resolve the IP addresses from our list to hostnames or website addresses, and this can be done from within Wireshark by turning on Network Name Resolution and trying to get hostnames and/or website addresses resolved for those IP addresses using reverse DNS queries (using what is known as a pointer (PTR) DNS record type). If you just installed or started Wireshark, the Name Resolution option may not be turned on by default. This is usually a good thing, as Wireshark can create traffic of its own by transmitting the DNS queries trying to resolve all the IP addresses that it comes across during the capture, and you don't really want that going on during a capture. However, the Name Resolution option can be very helpful to resolve IP addresses to proper hostnames after a capture is complete. To enable Name Resolution, navigate to View | Name Resolution | Enable for Network Layer (click to turn on the checkmark) and make sure Use External Network Name Resolver is enabled as well. Wireshark will attempt to resolve all the IP addresses in the capture to their hostname or website address, and the resolved names will then appear (replacing the previous IP addresses) in the packet list as well as the Conversations window. Note that the Name Resolution option at the bottom of the Conversations window must be enabled as well (it usually is by default), and this setting affects whether resolved names or IP addresses appear in that Conversations window (if Name Resolution is enabled in the Wireshark main screen), as shown in the following screenshot: At this point, you should see the conversation pair between wireshark.org and your workstation at or near the top of the list, as shown in the following screenshot. Of course, your workstation will have a different name or may only appear as an IP address, but identifying the conversation to wireshark.org has been achieved. Applying a display filter You now want to see just the conversation between your workstation and wireshark.org, and get rid of all the extraneous conversations so you can focus on the traffic of interest. This is accomplished by creating a filter that only displays the desired traffic. Right-click on the line containing the wireshark.org entry and navigate to Apply as Filter | Selected | A<->B, as shown in the following screenshot: Wireshark will create and apply a display filter string that isolates the displayed traffic to just the conversation between the IP addresses of wireshark.org and your workstation, as shown in the following screenshot. Note that if you create or edit a display filter entry manually, you will need to click on Apply to apply the filter to the trace file (or Clear to clear it). This particular display filter syntax works with IP addresses, not with hostnames, and uses an ip.addr== (IP address equals) syntax for each node along with the && (and) logic operator to build a string that says display any packet that contains this IP address *and* that IP address. This is the type of display filter that you will be using a great deal for packet analysis. You'll notice as you scroll up and down in the Packet List pane that all the other packets, except those between your workstation and wireshark.org are gone. They're not gone in the strict sense, they're just hidden—as you can observe by inspecting the Packet No. column, there are gaps in the numbering sequence, those are for the hidden packets. Saving the packet trace Now that you've isolated the traffic of interest using a display filter, you can save a new packet trace file that contains just the filtered packets. This serves two purposes. Firstly, you can close Wireshark, come back to it later, open the filtered trace file, and pick up where you left off in your analysis, as well as have a record of the capture in case you need to reference it later such as in a troubleshooting scenario. Secondly, it's much easier and quicker to work in the various Wireshark screens and functions with a smaller, more focused trace file that contains just the packets that you want to analyze. To create new packet trace file containing just the filtered/displayed packets, select Export Specified Packets from the Wireshark File menu. You can navigate to and/or create a folder to hold your Wireshark trace files, and then enter a filename for the trace file that you want to save. In this example, the filename is wireshark_website.pcapng. By default, Wireshark will save the trace file in the pcapng format (which is the preferred and more recent format). If you don't specify a file extension with the filename, Wireshark will provide the appropriate extension based on the Save as type selection, as shown in the following screenshot: Also, by default, Wireshark will have the All packets option selected, and if a display filter is applied (as it is in this scenario), the Displayed option will be selected as opposed to the Captured option that saves all the packets regardless of whether a filter was applied. Having entered a filename and confirmed that all the save selections are correct, you can click on Save to save the new packet trace file. Note that when you have finished this trace file save activity, Wireshark still has all the original packets from the capture in memory, and they can still be viewed by clicking on Clear in the Display Filter Toolbar menu. If you want work further with the new trace file you just saved, you'll need to open it by clicking on Open in the File menu (or Open Recent in the File menu). Summary Congratulations! If you accomplished all the activities covered in this article, you have successfully installed Wireshark, performed a packet capture, created a filter to isolate and display just the packets you were interested in from all the extraneous noise, and created a new packet trace file containing just those packets so you can analyze them later. Moreover, in the process, you gained an initial familiarity with the Wireshark user interface and you learned how to use several of its most useful and powerful features. Resources for Article: Further resources on this subject: Wireshark: Working with Packet Streams [Article] The Kendo MVVM Framework [Article] Kali Linux – Wireless Attacks [Article]
Read more
  • 0
  • 0
  • 4670

article-image-oracle-goldengate-considerations-designing-solution
Packt
24 Feb 2011
8 min read
Save for later

Oracle GoldenGate: Considerations for Designing a Solution

Packt
24 Feb 2011
8 min read
  Oracle GoldenGate 11g Implementer's guide Design, install, and configure high-performance data replication solutions using Oracle GoldenGate The very first book on GoldenGate, focused on design and performance tuning in enterprise-wide environments Exhaustive coverage and analysis of all aspects of the GoldenGate software implementation, including design, installation, and advanced configuration Migrate your data replication solution from Oracle Streams to GoldenGate Design a GoldenGate solution that meets all the functional and non-functional requirements of your system Written in a simple illustrative manner, providing step-by-step guidance with discussion points Goes way beyond the manual, appealing to Solution Architects, System Administrators and Database Administrators          At a high level, the design must include the following generic requirements: Hardware Software Network Storage Performance All the above must be factored into the overall system architecture. So let's take a look at some of the options and the key design issues. Replication methods So you have a fast reliable network between your source and target sites. You also have a schema design that is scalable and logically split. You now need to choose the replication architecture; One to One, One to Many, active-active, active-passive, and so on. This consideration may already be answered for you by the sheer fact of what the system has to achieve. Let's take a look at some configuration options. Active-active Let's assume a multi-national computer hardware company has an office in London and New York. Data entry clerks are employed at both sites inputting orders into an Order Management System. There is also a procurement department that updates the system inventory with volumes of stock and new products related to a US or European market. European countries are managed by London, and the US States are managed by New York. A requirement exists where the underlying database systems must be kept in synchronisation. Should one of the systems fail, London users can connect to New York and vice-versa allowing business to continue and orders to be taken. Oracle GoldenGate's active-active architecture provides the best solution to this requirement, ensuring that the database systems on both sides of the pond are kept synchronised in case of failure. Another feature the active-active configuration has to offer is the ability to load balance operations. Rather than have effectively a DR site in both locations, the European users could be allowed access to New York and London systems and viceversa. Should a site fail, then the DR solution could be quickly implemented. Active-passive The active-passive bi-directional configuration replicates data from an active primary database to a full replica database. Sticking with the earlier example, the business would need to decide which site is the primary where all users connect. For example, in the event of a failure in London, the application could be configured to failover to New York. Depending on the failure scenario, another option is to start up the passive configuration, effectively turning the active-passive configuration into active-active. Cascading The Cascading GoldenGate topology offers a number of "drop-off" points that are intermediate targets being populated from a single source. The question here is "what data do I drop at which site?" Once this question has been answered by the business, it is then a case of configuring filters in Replicat parameter files allowing just the selected data to be replicated. All of the data is passed on to the next target where it is filtered and applied again. This type of configuration lends itself to a head office system updating its satellite office systems in a round robin fashion. In this case, only the relevant data is replicated at each target site. Another design, is the Hub and Spoke solution, where all target sites are updated simultaneously. This is a typical head office topology, but additional configuration and resources would be required at the source site to ship the data in a timely manner. The CPU, network, and file storage requirements must be sufficient to accommodate and send the data to multiple targets. Physical Standby A Physical Standby database is a robust Oracle DR solution managed by the Oracle Data Guard product. The Physical Standby database is essentially a mirror copy of its Primary, which lends itself perfectly for failover scenarios. However , it is not easy to replicate data from the Physical Standby database, because it does not generate any of its own redo. That said, it is possible to configure GoldenGate to read the archived standby logs in Archive Log Only (ALO) mode. Despite being potentially slower, it may be prudent to feed a downstream system on the DR site using this mechanism, rather than having two data streams configured from the Primary database. This reduces network bandwidth utilization, as shown in the following diagram: Reducing network traffic is particularly important when there is considerable distance between the primary and the DR site. Networking The network should not be taken for granted. It is a fundamental component in data replication and must be considered in the design process. Not only must it be fast, it must be reliable. In the following paragraphs, we look at ways to make our network resilient to faults and subsequent outages, in an effort to maintain zero downtime. Surviving network outages Probably one of your biggest fears in a replication environment is network failure. Should the network fail, the source trail will fill as the transactions continue on the source database, ultimately filling the filesystem to 100% utilization, causing the Extract process to abend. Depending on the length of the outage, data in the database's redologs may be overwritten causing you the additional task of configuring GoldenGate to extract data from the database's archived logs. This is not ideal as you already have the backlog of data in the trail files to ship to the target site once the network is restored. Therefore, ensure there is sufficient disk space available to accommodate data for the longest network outage during the busiest period. Disks are relatively cheap nowadays. Providing ample space for your trail files will help to reduce the recovery time from the network outage. Redundant networks One of the key components in your GoldenGate implementation is the network. Without the ability to transfer data from the source to the target, it is rendered useless. So, you not only need a fast network but one that will always be available. This is where redundant networks come into play, offering speed and reliability. NIC teaming One method of achieving redundancy is Network Interface Card (NIC) teaming or bonding. Here two or more Ethernet ports can be "coupled" to form a bonded network supporting one IP address. The main goal of NIC teaming is to use two or more Ethernet ports connected to two or more different access network switches thus avoiding a single point of failure. The following diagram illustrates the redundant features of NIC teaming: Linux (OEL/RHEL 4 and above) supports NIC teaming with no additional software requirements. It is purely a matter of network configuration stored in text files in the /etc/sysconfig/network-scripts directory. The following steps show how to configure a server for NIC teaming: First, you need to log on as root user and create a bond0 config file using the vi text editor. # vi /etc/sysconfig/network-scripts/ifcfg-bond0 Append the following lines to it, replacing the IP address with your actual IP address, then save file and exit to shell prompt: DEVICE=bond0 IPADDR=192.168.1.20 NETWORK=192.168.1.0 NETMASK=255.255.255.0 USERCTL=no BOOTPROTO=none ONBOOT=yes Choose the Ethernet ports you wish to bond, and then open both configurations in turn using the vi text editor, replacing ethn with the respective port number. # vi /etc/sysconfig/network-scripts/ifcfg-eth2 # vi /etc/sysconfig/network-scripts/ifcfg-eth4 Modify the configuration as follows: DEVICE=ethn USERCTL=no ONBOOT=yes MASTER=bond0 SLAVE=yes BOOTPROTO=none Save the files and exit to shell prompt. To make sure the bonding module is loaded when the bonding interface (bond0) is brought up, you need to modify the kernel modules configuration file: # vi /etc/modprobe.conf Append the following two lines to the file: alias bond0 bonding options bond0 mode=balance-alb miimon=100 Finally, load the bonding module and restart the network services: # modprobe bonding # service network restart You now have a bonded network that will load balance when both physical networks are available, providing additional bandwidth and enhanced performance. Should one network fail, the available bandwidth will be halved, but the network will still be available. Non-functional requirements (NFRs) Irrespective of the functional requirements, the design must also include the nonfunctional requirements (NFR) in order to achieve the overall goal of delivering a robust, high performance, and stable system. Latency One of the main NFRs is performance. How long does it take to replicate a transaction from the source database to the target? This is known as end-to-end latency that typically has a threshold that must not be breeched in order to satisfy the specified NFR. GoldenGate refers to latency as lag, which can be measured at different intervals in the replication process. These are: Source to Extract: The time taken for a record to be processed by the Extract compared to the commit timestamp on the database Replicat to Target: The time taken for the last record to be processed by the Replicat compared to the record creation time in the trail file A well designed system may encounter spikes in latency but it should never be continuous or growing. Trying to tune GoldenGate when the design is poor is a difficult situation to be in. For the system to perform well you may need to revisit the design.  
Read more
  • 0
  • 0
  • 4669
article-image-working-microsoft-windows-workflow-foundation-40-wf-program
Packt
29 Sep 2010
5 min read
Save for later

Working with a Microsoft Windows Workflow Foundation 4.0 (WF) Program

Packt
29 Sep 2010
5 min read
  Microsoft Windows Workflow Foundation 4.0 Cookbook Over 70 recipes with hands-on, ready to implement solutions for authoring workflows Customize Windows Workflow 4.0 applications to suit your needs A hands-on guide with real-world illustrations, screenshots, and step-by-step instructions Explore various functions that you can perform using WF 4.0 with running code examples A hands-on guide with real-world illustrations, screenshots, and step-by-step instructions Read more about this book (For more resources on this subject, see here.) Introduction Considering workflow programs as imperative program, we need to think of three fundamental things: How to define workflow programs How to build (compile) workflow programs How to execute workflow programs In WF4, we can define a workflow in either managed .NET code or in XAML. There are two kinds of code workflow authoring styles: Custom Activity class Creating workflow dynamically in the runtime There are also two ways to author workflow in XAML: By WF designer (recommended) Typing XML tags manually Essentially, workflow program is .NET program, no matter how we create it. After defining workflows, we can build workflow applications as we build normal .NET applications. When it comes to workflow execution, we need to consider three basic things: How to flow data into and out of a workflow How to store a temporary data when workflow is executing How to manipulate data in workflow This article is going to focus on answering these questions. Before moving ahead, make sure we have the following installed on our computer: Windows Vista/7 or Windows Server 2008 Visual Studio 2010 and .NET framework 4.0 We can also use Windows XP; however, its usage is not recommended. Creating the first WF program: HelloWorkflow In this task we will create our first workflow to print "Hello Workflow" to console application. How to do it... Create a Workflow Console Application project:After starting Visual Studio 2010, select File | New Project. A dialog is presented, as shown in the following screenshot. Under Visual C# section, select Workflow, and choose Workflow Console Application. Name the project HelloWorkflow. Name the solution Chapter01 and make sure to create a directory for the solution. Author workflow program:First, drag a Sequence activity to designer from Toolbox, next drag a WriteLine activity into Sequence activity. Finally, input "Hello Workflow" in the expression box of WriteLine activity. We can see the following screenshot: Run it:Press Ctrl+F5 to run project without debugging. The result is as shown in the following screenshot: How it works... When we press Ctrl+F5, Visual Studio saves the current project, and then it runs the project from Main method in Program.cs file. WorkflowInvoker.Invoke(new Workflow1()); The preceding statement starts the workflow. After the workflow starts running, WriteLine activity prints the "Hello Workflow" to Console Application. The workflow we created in WF designer is actually an XML file. We can open Workflow1.xaml with XML editor to check it. Right-click on Workflow1.xaml then click Open With..., and choose XML Editor to open the Workflow1.xaml as XML file. All XAML files will be compiled to .dll or .exe file. That is why when we press Ctrl+F5, the program just runs like a normal C# program. There's more... So far, there are no officially published WF4 Designer add-ins for Visual Studio 2008. We need a copy of Visual Studio 2010 installed on our computer to use WF4 designer, otherwise we can only create workflows by imperative code or by writing pure XAML file. Creating a WF program using C# Code In this task, we will create the same "HelloWorkflow" function workflow using pure C# code, beginning from a Console Application. How to do it... Create a Console Application project:Create a new Console Application project under Chapter01 solution. Name the project HelloCodeWorkflow. The following screenshot shows the Console Application new project dialog: Add reference to System.Activities assembly:By default, a new Console Application doesn't have reference to System.Activities assembly, due to which we need to perform this step. Create workflow definition code:Open Program.cs file and change the code present as follows: Open Program.cs file and change the code present as follows:using System.Activities;using System.Activities.Statements;namespace HelloCodeWorkflow { class Program { static void Main(string[] args) { WorkflowInvoker.Invoke(new HelloWorkflow()); } } public class HelloWorkflow:Activity { public HelloWorkflow() { this.Implementation = () => new Sequence { Activities = { new WriteLine(){Text="Hellow Workflow"} } }; } }} Run it:Set HelloCodeWorkflow as StartUp project and press Ctrl+F5 to run it. As expected, the result should be just like the previous result showed. How it works... We use the following namespace: using System.Activities;using System.Activities.Statements; Because WorflowInvoker class belongs to System.Activities namespace and Sequence activity, WriteLine activity belongs to System.Activities.Statements. public class HelloWorkflow:Activity { public HelloWorkflow() { this.Implementation = () => new Sequence { Activities = { new WriteLine(){Text="Hellow Workflow"} } }; }} By implementing a class inherited from Activity, we define a workflow using imperative code. WorkflowInvoker.Invoke(s); This code statement loads a workflow instance up and runs it automatically. WorkflowInvoker.Invoke method is synchronous and invokes the workflow on the same thread as the caller. There's more WF4 also provide us a class DynamicActivity by which we can create a workflow instance dynamically in the runtime. In other words, by using DynamicActivity, there is no need to define a workflow class before initializing a workflow instance. Here is a sample code: public static DynamicActivity GetWF() { return new DynamicActivity() { Implementation = () => new Sequence() { Activities ={ new WriteLine(){Text="Hello Workflow"} } } };}
Read more
  • 0
  • 0
  • 4667

article-image-getting-started-apache-cassandra
Packt
29 Jul 2011
8 min read
Save for later

Getting started with Apache Cassandra

Packt
29 Jul 2011
8 min read
The Apache Cassandra Project develops a highly scalable second-generation distributed database, bringing together a fully distributed design and a ColumnFamily-based data model. The article contains recipes that allow users to hit the ground running with Cassandra. We show several recipes to set up Cassandra. These include cursory explanations of the key configuration files. It also contains recipes for connecting to Cassandra and executing commands both from the application programmer interface and the command-line interface. Also described are the Java profiling tools such as JConsole. The recipes in this article should help the user understand the basics of running and working with Cassandra. A simple single node Cassandra installation Cassandra is a highly scalable distributed database. While it is designed to run on multiple production class servers, it can be installed on desktop computers for functional testing and experimentation. This recipe shows how to set up a single instance of Cassandra. Getting ready Visit http://cassandra.apache.org in your web browser and find a link to the latest binary release. New releases happen often. For reference, this recipe will assume apache-cassandra-0.7.2-bin.tar.gz was the name of the downloaded file. How to do it... Download a binary version of Cassandra: $ mkdir $home/downloads $ cd $home/downloads $ wget <url_from_getting_ready>/apache-cassandra-0.7.2-bin.tar.gz Choose a base directory that the user will run as he has read and write access to: Default Cassandra storage locations Cassandra defaults to wanting to save data in /var/lib/cassandra and logs in /var/log/cassandra. These locations will likely not exist and will require root-level privileges to create. To avoid permission issues, carry out the installation in user-writable directories. Create a cassandra directory in your home directory. Inside the cassandra directory, create commitlog, log, saved_caches, and data subdirectories: $ mkdir $HOME/cassandra/ $ mkdir $HOME/cassandra/{commitlog,log,data,saved_caches} $ cd $HOME/cassandra/ $ cp $HOME/downloads/apache-cassandra-0.7.2-bin.tar.gz . $ tar -xf apache-cassandra-0.7.2-bin.tar.gz Use the echo command to display the path to your home directory. You will need this when editing the configuration file: $ echo $HOME /home/edward This tar file extracts to apache-cassandra-0.7.2 directory. Open up the conf/cassandra.yaml file inside in your text editor and make changes to the following sections: data_file_directories: - /home/edward/cassandra/data commitlog_directory: /home/edward/cassandra/commit saved_caches_directory: /home/edward/cassandra/saved_caches Edit the $HOME/apache-cassandra-0.7.2/conf/log4j-server.properties file to change the directory where logs are written: log4j.appender.R.File=/home/edward/cassandra/log/system.log Start the Cassandra instance and confirm it is running by connecting with nodetool: $ $HOME/apache-cassandra-0.7.2/bin/cassandra INFO 17:59:26,699 Binding thrift service to /127.0.0.1:9160 INFO 17:59:26,702 Using TFramedTransport with a max frame size of 15728640 bytes. $ $HOME/apache-cassandra-0.7.2/bin/nodetool --host 127.0.0.1 ring Address Status State Load Token 127.0.0.1 Up Normal 385 bytes 398856952452... How it works... Cassandra comes as a compiled Java application in a tar file. By default, it is configured to store data inside /var. By changing options in the cassandra.yaml configuration file, Cassandra uses specific directories created. YAML: YAML Ain't Markup Language YAML™ (rhymes with "camel") is a human-friendly, cross-language, Unicode-based data serialization language designed around the common native data types of agile programming languages. It is broadly useful for programming needs ranging from configuration files and Internet messaging to object persistence and data auditing. See http://www.yaml.org for more information. After startup, Cassandra detaches from the console and runs as a daemon. It opens several ports, including the Thrift port 9160 and JMX port on 8080. For versions of Cassandra higher than 0.8.X, the default port is 7199. The nodetool program communicates with the JMX port to confirm that the server is alive. There's more... Due to the distributed design, many of the features require multiple instances of Cassandra running to utilize. For example, you cannot experiment with Replication Factor, the setting that controls how many nodes data is stored on, larger than one. Replication Factor dictates what Consistency Level settings can be used for. With one node the highest Consistency Level is ONE. Reading and writing test data using the command-line interface The command-line interface (CLI) presents users with an interactive tool to communicate with the Cassandra server and execute the same operations that can be done from client server code. This recipe takes you through all the steps required to insert and read data. How to do it... Start the Cassandra CLI and connect to an instance: $ <cassandra_home>/bin/cassandra-cli [default@unknown] connect 127.0.0.1/9160; Connected to: "Test Cluster" on 127.0.0.1/9160 New clusters do not have any preexisting keyspaces or column families. These need to be created so data can be stored in them: [default@unknown] create keyspace testkeyspace [default@testkeyspace] use testkeyspace; Authenticated to keyspace: testkeyspace [default@testkeyspace] create column family testcolumnfamily; Insert and read back data using the set and get commands: [default@testk..] set testcolumnfamily['thekey'] ['thecolumn']='avalue'; Value inserted. [default@testkeyspace] assume testcolumnfamily validator as ascii; [default@testkeyspace] assume testcolumnfamily comparator as ascii; [default@testkeyspace] get testcolumnfamily['thekey']; => (column=thecolumn, value=avalue, timestamp=1298580528208000) How it works... The CLI is a helpful interactive facade on top of the Cassandra API. After connecting, users can carry out administrative or troubleshooting tasks. Running multiple instances on a single machine Cassandra is typically deployed on clusters of multiple servers. While it can be run on a single node, simulating a production cluster of multiple nodes is best done by running multiple instances of Cassandra. This recipe is similar to A simple single node Cassandra installation earlier in this article. However in order to run multiple instances on a single machine, we create different sets of directories and modified configuration files for each node. How to do it... Ensure your system has proper loopback address support. Each system should have the entire range of 127.0.0.1-127.255.255.255 configured as localhost for loopback. Confirm this by pinging 127.0.0.1 and 127.0.0.2: $ ping -c 1 127.0.0.1 PING 127.0.0.1 (127.0.0.1) 56(84) bytes of data. 64 bytes from 127.0.0.1: icmp_req=1 ttl=64 time=0.051 ms $ ping -c 1 127.0.0.2 PING 127.0.0.2 (127.0.0.2) 56(84) bytes of data. 64 bytes from 127.0.0.2: icmp_req=1 ttl=64 time=0.083 ms Use the echo command to display the path to your home directory. You will need this when editing the configuration file: $ echo $HOME /home/edward Create a hpcas directory in your home directory. Inside the cassandra directory, create commitlog, log, saved_caches, and data subdirectories: $ mkdir $HOME/hpcas/ $ mkdir $HOME/hpcas/{commitlog,log,data,saved_caches} $ cd $HOME/hpcas/ $ cp $HOME/downloads/apache-cassandra-0.7.2-bin.tar.gz . $ tar -xf apache-cassandra-0.7.2-bin.tar.gz Download and extract a binary distribution of Cassandra. After extracting the binary, move/rename the directory by appending '1' to the end of the filename.$ mv apache-cassandra-0.7.2 apache-cassandra-0.7.2-1 Open the apachecassandra- 0.7.2-1/conf/cassandra.yaml in a text editor. Change the default storage locations and IP addresses to accommodate our multiple instances on the same machine without clashing with each other: data_file_directories: - /home/edward/hpcas/data/1 commitlog_directory: /home/edward/hpcas/commitlog/1 saved_caches_directory: /home/edward/hpcas/saved_caches/1 listen_address: 127.0.0.1 rpc_address: 127.0.0.1 Each instance will have a separate logfile. This will aid in troubleshooting. Edit conf/log4j-server.properties: log4j.appender.R.File=/home/edward/hpcas/log/system1.log Cassandra uses JMX (Java Management Extensions), which allows you to configure an explicit port but always binds to all interfaces on the system. As a result, each instance will require its own management port. Edit cassandra-env.sh: JMX_PORT=8001 Start this instance: $ ~/hpcas/apache-cassandra-0.7.2-1/bin/cassandra INFO 17:59:26,699 Binding thrift service to /127.0.0.101:9160 INFO 17:59:26,702 Using TFramedTransport with a max frame size of 15728640 bytes. $ bin/nodetool --host 127.0.0.1 --port 8001 ring Address Status State Load Token 127.0.0.1 Up Normal 385 bytes 398856952452... At this point your cluster is comprised of single node. To join other nodes to the cluster, carry out the preceding steps replacing '1' with '2', '3', '4', and so on: $ mv apache-cassandra-0.7.2 apache-cassandra-0.7.2-2 Open ~/hpcas/apache-cassandra-0.7.2-2/conf/cassandra.yaml in a text editor: data_file_directories: - /home/edward/hpcas/data/2 commitlog_directory: /home/edward/hpcas/commitlog/2 saved_caches_directory: /home/edward/hpcas/saved_caches/2 listen_address: 127.0.0.2 rpc_address: 127.0.0.2 Edit ~/hpcas/apache-cassandra-0.7.2-2/conf/log4j-server. properties: log4j.appender.R.File=/home/edward/hpcas/log/system2.log Edit ~/hpcas/apache-cassandra-0.7.2-2/conf/cassandra-env.sh: JMX_PORT=8002 Start this instance: $ ~/hpcas/apache-cassandra-0.7.2-2/bin/cassandra How it works... The Thrift port has to be the same for all instances in a cluster. Thus, it is impossible to run multiple nodes in the same cluster on one IP address. However, computers have multiple loopback addresses: 127.0.0.1, 127.0.0.2, and so on. These addresses do not usually need to be configured explicitly. Each instance also needs its own storage directories. Following this recipe you can run as many instances on your computer as you wish, or even multiple distinct clusters. You are only limited by resources such as memory, CPU time, and hard disk space.
Read more
  • 0
  • 0
  • 4660

article-image-article-packt-technology-books-kindle
Packt
10 Jun 2011
1 min read
Save for later

Packt Books are Now Available in Kindle Format on Amazon + Win a Kindle!

Packt
10 Jun 2011
1 min read
If you’re one of the millions of Kindle users out there you’ll be pleased to hear that Packt’s entire catalogue of technology books is now available to purchase in Kindle format from Amazon, and with over 540 titles available you'll be spoilt for choice.   Whether its Moodle or Microsoft you can now download your Packt book from Amazon and refer to it quickly and easily wherever you are, learning has never been easier.   Packt's current best-selling Kindle books include:   And if you don't yet own a Kindle, well now's your opportunity to take full advantage of this milestone with: Packt's Kindle Competition   Simply identify the key pictured at the top and send your answers A, B, C, or D to KindleComp@PacktPub.com, all correct answers will then be entered into a prize draw to win a Kindle                                                  
Read more
  • 0
  • 0
  • 4660
article-image-how-build-dropdown-menu-using-canjs
Liz Tom
17 Mar 2017
7 min read
Save for later

How to build a dropdown menu using Can.js

Liz Tom
17 Mar 2017
7 min read
This post describes how to build a dropdown menu using Can.js. In this example, we will build a dropdown menu of names. If you'd like to see the complete example of what you'll be building, you can check it out here. Setup The very first thing you will need to do is to import Can.js and jQuery. <script src="https://code.jquery.com/jquery-2.2.4.js"></script> <script src="https://rawgit.com/canjs/canjs/v3.0.0-pre.6/dist/global/can.all.js"></script> Our First Model To make a model, you use can DefineMap. If you're following along using Code Pen or JSBin in the js tab, type the following piece of code[ZB1] : var Person = can.DefineMap.extend({ id: "string", name: "string", }); Here, we have a model named Person that defined the properties of id and name as string types. You can read about the different types that Can.js has here: https://canjs.com/doc/can-map-define._type.html. Can.js 3.0 allows us to declare types in two different ways. We could have also written the following piece of code[ZB2] : var Person = can.DefineMap.extend({ id: { type: "string", }, name: { type: "string", }, }); I tend to use the second syntax only when I have other settings I need to define on a particular property. The short hand of the first way makes things a bit easier. Getting it on the Page Since we're building a dropdown, we will most likely want the user to be able to see it. We're going to use can.stache to help us with this. In our HTML tab, write the following lines of code: <script type='text/stache' id='person-template'> <h1>Person Template</h1> <input placeholder="{{person.test}}"/> </script> The {{person.test}} is there, so you can see if you have it working. We'll add a test property to our model. var Person = can.DefineMap.extend({ id: "string", name: "string", test: { value: 'It's working!' } }); Now, we need to create a View Model. We're going to use Define Map again. Add the following to your js file: var PersonVM = can.DefineMap.extend({ person: {Value: Person}, }); You might notice that I'm using Value with a capitol "V". You have the option of using both value and Value. The difference is that Value causes the new to be used. Now, to use this as our View Model, you'll need to add the following to your js tab. can.Component.extend({ tag: 'person', view: can.stache.from('person-template'), ViewModel: PersonVM }); var vm = new PersonVM(); var template = can.stache.from('person-template') var frag = template(vm); document.body.appendChild(frag); The can.stache.from ('person-template') uses the ID from our script tag. The tag value person is so that we can use this component elsewhere, like <person>. If you check out the preview tab, you should see a header followed by an input box with the placeholder text we set. If you change the value of our test property, you should see the live binding updating. Fixtures Can.js allows us to easily add fixtures so we can test our UI without needing the API set up. This is great for development as the UI and the API don't always sync up in terms of development. We start off by setting up our set Algebra. Put the following at the top of your js tab: var personAlgebra = new set.Algebra( set.props.id('id'), set.props.sort('sort') ); var peopleStore = can.fixture.store([ { name: "Mary", id: 5 }, { name: "John", id: 6 }, { name: "Peter", id: 7 } ], personAlgebra); The set.Algebra helps us with some things. The set.props.id allows us to change the ID property. A very common example is that Mongo uses _id. We can easily change the ID property to map responses from the server with _id to our can model's id. In our fixture, we are faking some data that might already be stored in our database. Here, we have three people that have already been added. We need to add in a fixture route to catch our requests so we can send back our fixture data instead of trying to make a call to our API: can.fixture("/api/people/{id}", peopleStore); Here, we're telling can to use the people store whenever we have any requests using /api/people/{id}. Next, we will need to tell can.js how to use everything we just set up. We're going to use can-connect for that. Add this to your js tab: Person.connection = can.connect.superMap({ Map: Person, List: Person.List, url: "/api/people", name: "person", algebra: personAlgebra }); Does it work? Let's see if it's working. We'll write a function in our viewModel that allows us to save. Can-connect comes with some helper functions that allow us to do basic CRUD functionality. Keeping this in mind, update your Person View Model as follows: var PersonCreateVM = can.DefineMap.extend({ person: {Value: Person}, createPerson: function(){ this.person.save().then(function(){ this.person = new Person(); }.bind(this)); } }); Now, we have a createPerson function that saves a new person to the database and updates the person to be our new person. In order to use this, we can update our input tag to the following: <input placeholder="Name" {($value)}="person.name" ($enter)="createPerson()"/> This two-way binds the value of the input to our viewModel. Now, when we update the input, person.name also gets updated, and when we update person.name, the input updates as well. ($enter)=createPerson() will call createPerson whenever we press Enter. Populating the Select Now that we can create people and save them, we should be able to easily create a list of names. Since we may want to use this list of names at many places in our app, we're making the list its own component. Add this to the HTML tab. First, we will create a view model for our People. We're going to end up passing our people into the component. This way, we can use different people, depending on where this dropdown is being used. var PeopleListVM = can.DefineMap.extend({ peoplePromise: Promise, }); can.Component.extend({ tag: "people-list", view: can.stache.from("people-list-template"), ViewModel: PeopleListVM }); Then update your HTML with a template. Since peoplePromise is a Promise, we want to make sure it is resolved before we populate the select menu. We also have the ability to check isRejected, and isPending. value gives us result of the promise. We also use {{#each}} to cycle through each item in a list. <script type='text/stache' id='people-list-template'> {{#if peoplePromise.isResolved}} <select> {{#each peoplePromise.value}} <option>{{name}}</option> {{/each}} </select> {{/if}} </script> Building Blocks We can use these components, such as building blocks, in various parts of our app. If we create an app view model, we can put people there. We are using a getter in this case to get back a list of people. .getList({}) comes with DefineMap. This will return a promise. var AppVM = can.DefineMap.extend({ people: { get: function(){ return Person.getList({}); } } }); We will update our HTML to use these components. Now, we're using the tags we set up earlier. We can use the following to pass people into our people-list component: <people-list {people-promise}="people"/>. We can't use camel case in our stache file, so we will use hypens. can.js knows how to convert this into camel case for us. <script type='text/stache' id='names-template'> <div id="nameapp"> <h1>Names</h1> <person-create/> <people-list {people-promise}="people"/> </div> </script> Update the vm to use the app view model instead of the people view model. var vm = new AppVM(); var template = can.stache.from("app-template") var frag = template(vm); document.body.appendChild(frag); And that's it! You should have a drop-down menu that updates as you add more people. About the author Liz Tom is a developer at Bitovi in Portland, OR, focused on JavaScript. When she’s not in the office, you can find Liz attempting parkour and going to check out interactive displays at museums.
Read more
  • 0
  • 0
  • 4659

article-image-soa-service-oriented-architecture
Packt
20 Oct 2009
17 min read
Save for later

SOA—Service Oriented Architecture

Packt
20 Oct 2009
17 min read
What is SOA? SOA is the acronym for Service Oriented Architecture. As it has come to be known, SOA is an architectural design pattern by which several guiding principles determine the nature of the design. Basically, SOA states that every component of a system should be a service, and the system should be composed of several loosely-coupled services. A service here means a unit of a program that serves a business process. "Loosely-coupled" here means that these services should be independent of each other, so that changing one of them should not affect any other services. SOA is not a specific technology, nor a specific language. It is just a blueprint, or a system design approach. It is an architecture model that aims to enhance the efficiency, agility, and productivity of an enterprise system. The key concepts of SOA are services, high interoperability and loose coupling. Several other architecture/technologies such as RPC, DCOM, and CORBA have existed for a long time, and attempted to address the client/server communication problems. The difference between SOA and these other approaches is that SOA is trying to address the problem from the client side, and not from the server side. It tries to decouple the client side from the server side, instead of bundling them, to make the client side application much easier to develop and maintain. This is exactly what happened when object-oriented programming (OOP) came into play 20 years ago. Prior to object-oriented programming, most designs were procedure-oriented, meaning the developer had to control the process of an application. Without OOP, in order to finish a block of work, the developer had to be aware of the sequence that the code would follow. This sequence was then hard-coded into the program, and any change to this sequence would result in a code change. With OOP, an object simply supplied certain operations; it was up to the caller of the object to decide the sequence of those operations. The caller could mash up all of the operations, and finish the job in whatever order needed. There was a paradigm shift from the object side to the caller side. This same paradigm shift is happening today. Without SOA, every application is a bundled, tightly coupled solution. The client-side application is often compiled and deployed along with the server-side applications, making it impossible to quickly change anything on the server side. DCOM and CORBA were on the right track to ease this problem by making the server-side components reside on remote machines. The client application could directly call a method on a remote object, without knowing that this object was actually far away, just like calling a method on a local object. However, the client-side applications continue to remain tightly coupled with these remote objects, and any change to the remote object will still result in a recompiling or redeploying of the client application. Now, with SOA, the remote objects are truly treated as remote objects. To the client applications, they are no longer objects; they are services. The client application is unaware of how the service is implemented, or of the signature that should be used when interacting with those services. The client application interacts with these services by exchanging messages. What a client application knows now is only the interfaces, or protocols of the services, such as the format of the messages to be passed in to the service, and the format of the expected returning messages from the service. Historically, there have been many other architectural design approaches, technologies, and methodologies to integrate existing applications. EAI (Enterprise Application Integration) is just one of them. Often, organizations have many different applications, such as order management systems, accounts receivable systems, and customer relationship management systems. Each application has been designed and developed by different people using different tools and technologies at different times, and to serve different purposes. However, between these applications, there are no standard common ways to communicate. EAI is the process of linking these applications and others in order to realize financial and operational competitive advantages. It may seem that SOA is just an extension of EAI. The similarity is that they are both designed to connect different pieces of applications in order to build an enterprise-level system for business. But fundamentally, they are quite different. EAI attempts to connect legacy applications without modifying any of the applications, while SOA is a fresh approach to solve the same problem. Why SOA? So why do we need SOA now? The answer is in one word—agility. Business requirements change frequently, as they always have. The IT department has to respond more quickly and cost-effectively to those changes. With a traditional architecture, all components are bundled together with each other. Thus, even a small change to one component will require a large number of other components to be recompiled and redeployed. Quality assurance (QA) effort is also huge for any code changes. The processes of gathering requirements, designing, development, QA, and deployment are too long for businesses to wait for, and become actual bottlenecks. To complicate matters further, some business processes are no longer static. Requirements change on an ad-hoc basis, and a business needs to be able to dynamically define its own processes whenever it wants. A business needs a system that is agile enough for its day-to-day work. This is very hard, if not impossible, with existing traditional infrastructure and systems. This is where SOA comes into play. SOA's basic unit is a service. These services are building blocks that business users can use to define their own processes. Services are designed and implemented so that they can serve different purposes or processes, and not just specific ones. No matter what new processes a business needs to build or what existing processes a business needs need to modify, the business users should always be able to use existing service blocks, in order to compete with others according to current marketing conditions. Also, if necessary, some new service blocks can be used. These services are also designed and implemented so that they are loosely coupled, and independent of one another. A change to one service does not affect any other service. Also, the deployment of a new service does not affect any existing service. This greatly eases release management and makes agility possible. For example, a GetBalance service can be designed to retrieve the balance for a loan. When a borrower calls in to query the status of a specific loan, this GetBalance service may be called by the application that is used by the customer service representatives. When a borrower makes a payment online, this service can also be called to get the balance of the loan, so that the borrower will know the balance of his or her loan after the payment. Yet in the payment posting process, this service can still be used to calculate the accrued interest for a loan, by multiplying the balance with the interest rate. Even further, a new process can be created by business users to utilize this service if a loan balance needs to be retrieved. The GetBalance service is developed and deployed independently from all of the above processes. Actually, the service exists without even knowing who the client will be or even how many clients there will be. All of the client applications communicate with this service through its interface, and its interface will remain stable once it is in production. If we have to change the implementation of this service, for example by fixing a bug, or changing an algorithm inside a method of the service, all of the client applications can still work without any change. When combined with the more mature Business Process Management (BPM) technology, SOA plays an even more important role in an organization's efforts to achieve agility. Business users can create and maintain processes within BPM, and through SOA they can plug a service into any of the processes. The front-end BPM application is loosely coupled to the back-end SOA system. This combination of BPM and SOA will give an organization much greater flexibility in order to achieve agility. How do we implement SOA? Now that we've established why SOA is needed by the business, the question becomes—how do we implement SOA? To implement SOA in an organization, three key elements have to be evaluated—people, process, and technology. Firstly, the people in the organization must be ready to adopt SOA. Secondly, the organization must know the processes that the SOA approach will include, including the definition, scope, and priority. Finally, the organization should choose the right technology to implement it. Note that people and processes take precedence over technology in an SOA implementation, but they are out of the scope of this article. In this article, we will assume people and processes are all ready for an organization to adopt SOA. Technically, there are many SOA approaches. At certain degrees, traditional technologies such as RPC, DCOM, CORBA, or some modern technologies such as IBM WebSphere MQ, Java RMI, and .NET Remoting could all be categorized as service-oriented, and can be used to implement SOA for one organization. However, all of these technologies have limitations, such as language or platform specifications, complexity of implementation, or the ability to support binary transports only. The most important shortcoming of these approaches is that the server-side applications are tightly coupled with the client-side applications, which is against the SOA principle. Today, with the emergence of web service technologies, SOA becomes a reality. Thanks to the dramatic increase in network bandwidth, and given the maturity of web service standards such as WS-Security, and WS-AtomicTransaction, an SOA back-end can now be implemented as a real system. SOA from different users' perspectives However, as we said earlier, SOA is not a technology, but only a style of architecture, or an approach to building software products. Different people view SOA in different ways. In fact, many companies now have their own definitions for SOA. Many companies claim they can offer an SOA solution, while they are really just trying to sell their products. The key point here is—SOA is not a solution. SOA alone can't solve any problem. It has to be implemented with a specific approach to become a real solution. You can't buy an SOA solution. You may be able to buy some kinds of products to help you realize your own SOA, but this SOA should be customized to your specific environment, for your specific needs. Even within the same organization, different players will think about SOA in quite different ways. What follows are just some examples of how different players in an organization judge the success of an SOA initiative using different criteria. [Gartner, Twelve Common SOA Mistakes and How to Avoid Them, Publication Date: 26 October 2007 ID Number: G00152446] To a programmer, SOA is a form of distributed computing in which the building blocks (services) may come from other applications or be offered to them. SOA increases the scope of a programmer's product and adds to his or her resources, while also closely resembling familiar modular software design principles. To a software architect, SOA translates to the disappearance of fences between applications. Architects turn to the design of business functions rather than to self-contained and isolated applications. The software architect becomes interested in collaboration with a business analyst to get a clear picture of the business functionality and scope of the application. SOA turns software architects into integration architects and business experts. For the Chief Investment Officers (CIOs), SOA is an investment in the future. Expensive in the short term, its long-term promises are lower costs, and greater flexibility in meeting new business requirements. Re-use is the primary benefit anticipated as a means to reduce the cost and time of new application development. For business analysts, SOA is the bridge between them and the IT organization. It carries the promise that IT designers will understand them better, because the services in SOA reflect the business functions in business process models. For CEOs, SOA is expected to help IT become more responsive to business needs and facilitate competitive business change. Complexities in SOA implementation Although SOA will make it possible for business parties to achieve agility, SOA itself is technically not simple to implement. In some cases, it even makes software development more complex than ever, because with SOA you are building for unknown problems. On one hand, you have to make sure that the SOA blocks you are building are useful blocks. On the other, you need a framework within which you can assemble those blocks to perform business activities. The technology issues associated with SOA are more challenging than vendors would like users to believe. Web services technology has turned SOA into an affordable proposition for most large organizations by providing a universally-accepted, standard foundation. However, web services play a technology role only for the SOA backplane, which is the software infrastructure that enables SOA-related interoperability and integration. The following figure shows the technical complexity of SOA. It has been taken from Gartner, Twelve Common SOA Mistakes and How to Avoid Them, Publication Date: 26 October 2007 ID Number: G00152446. As Gartner says, users must understand the complex world of middleware, and point-to-point web service connections only for small-scale, experimental SOA projects. If the number of services deployed grows to more than 20 or 30, then use a middleware-based intermediary—the SOA backplane. The SOA backplane could be an Enterprise Service Bus (ESB), a Message-Oriented Middleware (MOM), or an Object Request Broker (ORB). However, in this article, we will not cover it. We will build only point-to-point services using WCF. Web services There are many approaches to realizing SOA, but the most popular and practical one is—using web services. What is a web service? A web service is a software system designed to support interoperable machine-to-machine interaction over a network. A web service is typically hosted on a remote machine (provider), and called by a client application (consumer) over a network. After the provider of a web service publishes the service, the client can discover it and invoke it. The communications between a web service and a client application use XML messages. A web service is hosted within a web server and HTTP is used as the transport protocol between the server and the client applications. The following diagram shows the interaction of web services: Web services were invented to solve the interoperability problem between applications. In the early 90s, along with the LAN/WAN/Internet development, it became a big problem to integrate different applications. An application might have been developed using C++, or Java, and run on a Unix box, a Windows PC, or even a mainframe computer. There was no easy way for it to communicate with other applications. It was the development of XML that made it possible to share data between applications across hardware boundaries and networks, or even over the Internet. For example, a Windows application might need to display the price of a particular stock. With a web service, this application can make a request to a URL, and/or pass an XML string such as <QuoteRequest><GetPrice Symble='XYZ'/></QuoteRequest>. The requested URL is actually the Internet address of a web service, which, upon receiving the above quote request, gives a response, <QuoteResponse><QuotePrice Symble='XYZ'>51.22</QuotePrice></QuoteResponse/>. The Windows application then uses an XML parser to interpret the response package, and display the price on the screen. The reason it is called a web service is that it is designed to be hosted in a web server, such as Microsoft Internet Information Server, and called over the Internet, typically via the HTTP or HTTPS protocols. This is to ensure that a web service can be called by any application, using any programming language, and under any operating system, as long as there is an active Internet connection, and of course, an open HTTP/HTTPS port, which is true for almost every computer on the Internet. Each web service has a unique URL, and contains various methods. When calling a web service, you have to specify which method you want to call, and pass the required parameters to the web service method. Each web service method will also give a response package to tell the caller the execution results. Besides new applications being developed specifically as web services, legacy applications can also be wrapped up and exposed as web services. So, an IBM mainframe accounting system might be able to provide external customers with a link to check the balance of an account. Web service WSDL In order to be called by other applications, each web service has to supply a description of itself, so that other applications will know how to call it. This description is provided in a language called a WSDL. WSDL stands for Web Services Description Language. It is an XML format that defines and describes the functionalities of the web service, including the method names, parameter names, and types, and returning data types of the web service. For a Microsoft ASMX web service, you can get the WSDL by adding ?WSDL to the end of the web service URL, say http://localhost/MyService/MyService.asmx?WSDL. Web service proxy A client application calls a web service through a proxy. A web service proxy is a stub class between a web service and a client. It is normally auto-generated by a tool such as Visual Studio IDE, according to the WSDL of the web service. It can be re-used by any client application. The proxy contains stub methods mimicking all of methods of the web service so that a client application can call each method of the web service through these stub methods. It also contains other necessary information required by the client to call the web service such as custom exceptions, custom data and class types, and so on. The address of the web service can be embedded within the proxy class, or it can be placed inside a configuration file. A proxy class is always for a specific language. For each web service, there could be a proxy class for Java clients, a proxy class for C# clients, and yet another proxy class for COBOL clients. To call a web service from a client application, the proper proxy class first has to be added to the client project. Then, with an optional configuration file, the address of the web service can be defined. Within the client application, a web service object can be instantiated, and its methods can be called just as for any other normal method. SOAP There are many standards for web services. SOAP is one of them. SOAP was originally an acronym for Simple Object Access Protocol, and was designed by Microsoft. As this protocol became popular with the spread of web services, and its original meaning was misleading, the original acronym was dropped with version 1.2 of the standard. It is now merely a protocol, maintained by W3C. SOAP, now, is a protocol for exchanging XML-based messages over computer networks. It is widely-used by web services and has become its de-facto protocol. With SOAP, the client application can send a request in XML format to a server application, and the server application will send back a response in XML format. The transport for SOAP is normally HTTP / HTTPS, and the wide acceptance of HTTP is one of the reasons why SOAP is widely accepted today.
Read more
  • 0
  • 0
  • 4659
Modal Close icon
Modal Close icon