How-To Tutorials

  Foreword by Alexander Limi, co-founder of Plone It's always fascinating how life throws you a loop now and then that changes your future in a profound way—and you don't realize it at the time. As I sit here almost six years after the Plone project started, it seems like a good time to reflect on how the last years changed everything, and some of the background of why you are holding this book in your hands—because the story about the Plone community is at least as remarkable as the software itself. It all started out in a very classic way—I had just discovered Zope and Python, and wanted to build a simple web application to teach myself how they worked. This was back in 1999, when Zope was still a new, unproven technology, and had more than a few rough spots. I have never been a programmer, but Python made it all seem so simple that I couldn't resist trying to build a simple web application with it. After reading what I could find of documentation at the time, I couldn't quite figure it out—so I ended up in the online Zope chat rooms to see if I could get any help with building my web application. Little did I know that what happened that evening would change my life in a significant way. I met Alan Runyan online, and after trying to assist me, we ended up talking about music instead. We also reached the conclusion that I should focus on what I was passionate about—instead of coding, I wanted to build great user interfaces and make things easy to use. Alan wanted to provide the plumbing to make the system work. For some reason, it just clicked at that point, and we collaborated online and obsessed over the details of the system for months. External factors were probably decisive here too: I was without a job, and my girlfriend had left me a few months prior; Alan had just given up his job as a Java programmer at a failed dot-com company and decided to start his own company doing Python instead—so we both ended up pouring every living hour into the project, and moving at a break-neck pace towards getting the initial version out. We ended up getting a release ready just before the EuroPython Conference in 2002, and this was actually the first time I met Alan in person. We had been working on Plone for the past year just using email and IRC chat—two technologies that are still cornerstones of Plone project communication. I still remember the delight in discovering that we had excellent communication in person as well. What happened next was somewhat surreal for people new to this whole thing: we were sitting in the audience in the "State of Zope" talk held by Paul Everitt. He got to the part of his talk where he called attention to people and projects that he was especially impressed with. When he called out our names and talked about how much he liked Plone—which at this point was still mostly the effort of a handful of people—it made us feel like we were really onto something. This was our defining moment. For those of you who don't know Paul, he is one of the founders of Zope Corporation, and would go on to become our most tireless and hard-working supporter. He got involved in all the important steps that would follow—he put a solid legal and marketing story in place and helped create the Plone Foundation—and did some great storytelling along the way. There is no way to properly express how much Paul has meant to us personally—and to Plone—five years later. His role was crucial in the story of Plone's success, and the project would not be where it is now without him. Looking back, it sounds a bit like the classic romanticized start-up stories of Silicon Valley, except that we didn't start a company together. We chose to start two separate companies—in hindsight a very good decision. It never ceases to amaze me how much of an impact the project has had since. We are now an open-source community of hundreds of companies doing Plone development, training, and support. In just the past month, large companies like Novell and Akamai—as well as government agencies like the CIA, and NGOs like Oxfam—have revealed that they are using Plone for their web content management, and more will follow. The Plone Network site, plone.net, lists over 150 companies that offer Plone services, and the entire ecosystem is estimated to have revenues in the hundreds of millions of US dollars annually. This year's Plone Conference in Naples, Italy is expected to draw over 300 developers and users from around the world. Not bad for a system that was conceived and created by a handful of people standing on the shoulders of the giants of the Zope and Python communities. But the real story here is about an amazing community of people—individuals and organizations, large and small—all coming together to create the best content management system on the planet. We meet in the most unlikely locations—from ancient castles and mountain-tops in Austria, to the archipelagos and fjords of Norway, the sandy beaches of Brazil, and the busy corporate offices of Google in Silicon Valley. These events are at the core of the Plone experience, and developers nurture deep friendships within the community. I can say without a doubt that these are the smartest, kindest, most amazing people I have ever had the pleasure to work with. One of those people is Martin Aspeli, whose book you are reading right now. Even though we're originally from the same country, we didn't meet that way. Martin was at the time—and still is—living in London. He had contributed some code to one of our community projects a few months prior, and suggested that we should meet up when he was visiting his parents in Oslo, Norway. It was a cold and dark winter evening when we met at the train station—and ended up talking about how to improve Plone and the community process at a nearby café. I knew there and then that Martin would become an important part of the Plone project. Fast-forward a few years, and Martin has risen to become one of Plone's most important and respected—not to mention prolific—developers. He has architected and built several core components of the Plone 3 release; he has been one of the leaders on the documentation team, as well as an active guide in Plone's help forums. He also manages to fit in a day job at one of the "big four" consulting companies in the world. On top of all this, he was secretly working on a book to coincide with the Plone 3.0 release—which you are now the lucky owner of. This brings me to why this book is so unique, and why we are lucky to have Martin as part of our community. In the fast-paced world of open-source development—and Plone in particular—we have never had the chance to have a book that was entirely up-to-date on all subjects. There have been several great books in the past, but Martin has raised the bar further—by using the writing of a book to inform the development of Plone. If something didn't make sense, or was deemed too complex for the problem it was trying to solve—he would update that part of Plone so that it could be explained in simpler terms. It made the book better, and it has certainly made Plone better. Another thing that sets Martin's book apart is his unparalleled ability to explain advanced and powerful concepts in a very accessible way. He has years of experience developing with Plone and answering questions on the support forums, and is one of the most patient and eloquent writers around. He doesn't give up until you know exactly what's going on. But maybe more than anything, this book is unique in its scope. Martin takes you through every step from installing Plone, through professional development practices, unit tests, how to think about your application, and even through some common, non-trivial tasks like setting up external caching proxies like Varnish and authentication mechanisms like LDAP. In sum, this book teaches you how to be an independent and skillful Plone developer, capable of running your own company—if that is your goal—or provide scalable, maintainable services for your existing organization. Five years ago, I certainly wouldn't have imagined sitting here, jet-lagged and happy in Barcelona this Sunday morning after wrapping up a workshop to improve the multilingual components in Plone. Nor would I have expected to live halfway across the world in San Francisco and work for Google, and still have time to lead Plone into the future. Speaking of which, how does the future of Plone look like in 2007? Web development is now in a state we could only have dreamt about five years ago—and the rise of numerous great Python web frameworks, and even non-Python solutions like Ruby on Rails has made it possible for the Plone community to focus on what it excels at: content and document management, multilingual content, and solving real problems for real companies—and having fun in the process. Before these frameworks existed, people would often try to do things with Plone that it was not built or designed to do—and we are very happy that solutions now exist that cater to these audiences, so we can focus on our core expertise. Choice is good, and you should use the right tool for the job at hand. We are lucky to have Martin, and so are you. Enjoy the book, and I look forward to seeing you in our help forums, chat rooms, or at one of the many Plone conferences and workshops around the world. — Alexander Limi, Barcelona, July 2007 http://limi.net Alexander Limi co-founded the Plone project with Alan Runyan, and continues to play a key role in the Plone community. He is Plone's main user interface developer, and currently works as a user interaction designer at Google in California.

(For more resources related to this topic, see here.) Using an external API APIs are provided as a service from many different companies. This is not an entirely altruistic move on the part of the company. The expectation is that by providing the information and access to the company's data, the company gets more usage for their service and more customers. With this in mind, most (if not all) companies will require you to have an account on their system in order to access their API. This allows you to access their systems and information from within your application, but more importantly from the company's perspective, it allows them to maintain control over how their data can be used. If you violate the company's usage policies, they can shut off your application's access to the data, so play nice. The API key Most APIs require a key in order to use them. An API key is a long string of text that gets sent as an extra parameter on any request you send to the API. The key is often composed of two separate pieces and it uniquely identifies your application to the system much like a username and a password would for a regular user account. As such it's also a good idea to keep this key hidden in your application so that your users can't easily get it. While each company is different, an API key is typically a matter of filling out a web form and getting the key. Most companies do not charge for this service. However, some do limit the usage available to outside applications, so it's a good idea to look at any restrictions the company sets on their service. Once you have an API key you should take a look at the available functions for the API. API functions API functions typically come in two types – public and protected: The public functions can simply be requested with the API key The protected functions will also require that a user be logged into the system in order to make the request If the API function is protected, your application will also need to know how to log in correctly with the remote system. The login functions will usually be a part of the API or a web standard such as Facebook and Google's OAuth. It should be noted that while OAuth is a standard, its implementation will vary depending on the service. You will need to consult the documentation for the service you are using to make sure that the features and functions you need are supported. Be sure to read through the service's API documentation to understand which functions you will need and if they require a login. Another thing to understand about APIs is that they don't always do exactly what you need them to do. You may find that you need to do a little more work than you expect to get the data you need. In this case, it's always good to do a little bit of testing. Many APIs offer a console interface where you can type commands directly into the system and examine the results: Image This can be really helpful for digging into the data, but consoles are not always available for every API service. Another option is to send the commands in your application (along with your API credentials) and examine the data returned in the Safari console. The drawback of this method is that the data is often returned as a single-line string that is very difficult to read as shown in the screenshot: Image This is where a tool like JSONLint comes in handy. You can copy and paste the single-line string from your Safari console into the page at http://jsonlint.com and have the string formatted so that it is much easier to read and validate the string as JSON at the same time: Image Once you get a hold of what data is being sent and received, you will need to set it all up in Sencha Touch. External APIs and Sencha Touch As we have talked about earlier in the book, you cannot use a standard AJAX request to get data from another domain. You will need to use a JSONP proxy and store to request data from an external API. Using the API or the Safari console, you can get a good idea of the data that is coming back to you and use it to set up your model. For this example, let's use a simple model called Category. code We can then set up a store to load data from the API: This will set up a store with our Category model and call the url property for our external API. Remember that we have to send our credentials along with the request so we set these as extraParams on the proxy section. The apiKey and appSecret properties shown here are examples. You will need your own API key information to use an API. We also need to set a property called rootProperty in the reader section. Most API's send back a ton of detailed information along with the request and the store needs some idea of where to start loading in the category records. We can also add additional parameters later by calling the setExtraParam() function on our store proxy. This will let us add additional parameters to be sent to our external API URL. Please note that setExtraParam() will add an additional parameter but setExtraParams() will replace all of our extraParams with the new values. The basic application The Decider application is designed to use a combination of local storage, Google's Map API, and the Foursquare API. The application will take a list of people and their food preferences, and then use Foursquare and Google Maps to find nearby places to eat that will match everyone's food preferences. This screenshot provides a pictorial representation of the preceding explanation: Image Our contacts and categories will be stored using local storage. External APIs from Google and Foursquare will generate our maps and restaurant listings respectively. We will start with a quick overview of the basic application structure and forms, before diving into the store setup and API integration. Our main container is a simple card layout: code In this viewport we will add two cards: a navigation view and a form panel. Our navigationvew will serve as our main window for display. We will add additional containers to it via our controller: code This mainView contains our navigationBar and our homeScreen container with the big Get Started button. This button will add new containers to the navigation view (we will look at this later in the controller). A DataStage project stores jobs and define their environment, such as their security and execution resources. Your project, as well as your user account, is typically created by your DataStage administrator. The second item that is added to our viewport is our form panel. This will contain text fields for first and last name, as well as a selectable list for our different food categories: code We close out the form with a segmentedbutton property, which has options for Save and Cancel. We will add the handler functions for these buttons later on in our controller. We also include a title bar at the top of the form to give the user some idea of what they are doing. One of the key pieces of this form is the categories list, so let's take a closer look at how it works. Creating the categories list Since we will be getting our list of potential restaurants from the Foursquare API, we need to use their categories as well so that we can match things up with some degree of accuracy. The Foursquare API can be found at https://developer.foursquare.com/. As mentioned before, you will need a Foursquare account to access the API. You will also need an API key in order to integrate Foursquare with your application. We can use the Foursquare's API to get a list of categories, however the API returns a list of a few hundred categories including Airports, Trains, Taxis, Museums, and Restaurants. Additionally, each of these has its own subcategories. All we really want is the subcategories for Restaurants. To make things more complicated, Foursquare's API also returns the data like this: code This means we can only get at a specific category by its order in the array of categories. For example, if Restaurants is the twenty-third category in the array, we can get to it as: categories[23], but we cannot get to it by calling categories['Restaurants']. Unfortunately, if we use categories[23] and Foursquare adds a new category or changes the order, our application will break. This is a situation where it pays to be adaptable. Foursquare's API includes a console where we can try out our API requests. We can use this console to request the data for all of our categories and then pull the data we need into a flat file for our application. Check this URL to see the output: https://developer.foursquare.com/docs/explore#req=venues/categories We can copy just the Restaurant information that we need from categories and save this as a file called categories.json and call it from our store. A better solution to this conundrum would be to write some server code that would request the full category list from Foursquare and then pull out just the information we are interested in. But for the sake of brevity, we will just use a flat json file. Each of our categories are laid out like this: code The main pieces we care about are the id, name, shortname and icon values. This gives us a data model that looks like this: code Notice that we also add a function to create an image URL for the icons we need. We do this with the convert configuration, which lets us assemble the data for image URL based on the other data in the record: code The convert function is automatically passed both the data value (v), which we ignore in this case, and the record (rec), which lets us create a valid Foursquare URL by combining the icon.prefix value, a number, and the icon.suffix value in our record. If you take a look at our previous category data example, this would yield a URL of: https://foursquare.com/img/categories_v2/food/argentinian_32.png By changing the number we can control the size of the icon (this is part of the Foursquare API as well). We combine this with our XTemplate: code This gives us a very attractive list for choosing our categories: Images Next we need to take a look at the controller for the contact form. Creating the contact controller The contact controller handles saving the contact and canceling the action. We start out the controller by declaring our references and controls: code Remember that our refs (references) provide a handy shortcut we can use anywhere in the controller to get to the pieces we need. Our control section attaches tap listeners to our cancel and save buttons. Next we need to add our two functions after the controls section. The doCancel function is really simple: code We just use our references to clear the contact editor, deselect all the items in our category list, and switch back to our main view. The save function is a little more complex, but similar to the functions we have covered elsewhere in this book: code As with our previous save functions, we create a new MyApp.model.Contact and add the values from our form. However, since our list isn't really a standard form component we need to grab its selections separately and add them to the contact data as a comma-separated list. We do this by creating an empty array and using Ext.each() to loop through and run a function on all our categories. We then use join to implode the array into a comma-separated list. Finally, we save the contact and run our doCancel function to clean up and return to our main view. Now that we can add contacts we need to create a controller to handle our requests to the Foursquare and Google APIs, and get the data back to our users. Integrating with Google Maps and Foursquare Our application still has a couple of tasks to accomplish. It needs to: Handle the click of the Get Started button Add our maps panel and offer to adjust the current location via Google Maps API Display a list of friends to include in our search Display the search results in a list Display the details for a selected result We will start out with the basic skeleton of the controller, create the views and stores, and then finish up the controller to complete the application. Starting the mainView.js controller We will start the mainView.js controller file with some placeholders for the stores. We will add views later on and some references for those components. Keep in mind that when working with placeholders in this fashion the application will not be testable until all the files are actually in place. We create the mainView.js file in our controllers folder: code At the top of this configuration we require Ext.DateExtras. This file provides us with formatting options for date objects. If this file is not included, only the now() method for date objects will be available in your application. In our views section we have added placeholders for confirmLocation, restaurantList, friendChooser,and restaurantDetails. We will add these files later on, along with the RestaurantStore file listed in our stores section. We also have a number of references for these views, stores, and some of their sub-components. We will need to create these views before getting to the rest of our controller. We will take these views in the order the user will see them, starting with the confirmLocation view. Creating the confirmLocation view The confirmLocation view first appears when the user clicks on the Get Started button. This view will present the user with a map showing their current location and offer an option to switch to a different location if the user desires. The following screenshot gives a pictorial representation of the preceding code: Image In order to give ourselves a bit more flexibility, we will be using the Google Maps Tracker plugin as part of this view. You can find this plugin in your Sencha Touch 2 folder in examples/map/lib/plugin/google/Tracker.js. Copy the file into a lib/google folder in your main application folder and be sure to add it into the requires section of your app.js file: code This plugin will let us easily drop markers on the map. Once the Google Tracker plugin file is included in the application, we can set up our confirmLocation.js view like so: code The view itself is a simple container with some HTML at the top asking the user to confirm their location. Next we have a map container that uses our Google Tracker plugin to configure the map and animate the location marker to drop from the top of the screen to the current location of the user. The position configuration is a default location, which is used when the user denies the application access to their current location. This one is set to the Sencha Headquarters. Next we need a few options for the user to choose from: Cancel, New Location, and Next. We will add these as a segmented button under our map container. We add the code to the end of our items container (after the map container): code Each of our buttons has an associated action. This allows us to assign functions to each button within the mainView.js controller. By creating buttons in this fashion, we maintain separation between the display of the application and the functionality of the application. This is really helpful when you want to re-use a view component. The next view the user encounters is the Friends Chooser. Creating the Friends Chooser view The friendsChooser.js file uses a similar list to our previous category chooser. This lets our users select multiple people to include in the restaurant search: Image Our friendChooser extends the Ext.Container component and allows the user to select from a list of friends: code As with our previous panel, we have a container with HTML at the top to provide some instructions to the user. Below that is our list container, which, like our category list, allows for selection of multiple items via the mode: 'MULTI' configuration. We also set grouped to true. This allows our store to group the contacts together by last name. If you take a look at the ContactStore.js file, you can see where we do: code This configuration returns the first letter of the last name for grouping. The last thing we need to do with our friendChooser.js file is add the buttons at the bottom to Cancel or Finish the search. The buttons go out in the items section, just below the list: code As in our previous view, we use a segmentedbutton property with actions assigned to each of our individual buttons. Once the user clicks on Finish, we will need to return a list of restaurants they can select from. Creating the restaurant list, store, and details Our restaurant list will use a store and the Foursquare API to return a list of restaurants based on the shared preferences of everyone the user selected. The following screenshot exemplifies the preceding explanation: Image This component is pretty basic: code This component uses a simple list with a configuration option for onItemDisclosure:true. This places an arrow next to the restaurant name in the list. The user will be able to click on the arrow and see the details for that restaurant (which we will create after the store). We also set grouped to true, only this time our store will use a function to calculate and sort by distance. Creating the restaurant store and model The restaurant store is where we set up our request to the Foursquare API: code The RestaurantStore.js file sets a model and storeId field for our store and then defines our proxy. The proxy section is where we set up our request to Foursquare. As we mentioned at the start of the article, this needs to be a jsonp request since it is going to another domain. We make our request to https://api.foursquare. com/v2/venues/search and we are looking for the responses.venues section of the JSON array that gets returned. You will note that this store currently has no other parameters to send to Foursquare. We will add these later on in the controller before we load the store. For the model, we can consult the Foursquare API documentation to see the information that is returned for a restaurant (called a venue in Foursquare terms) at https://developer.foursquare.com/docs/responses/venue You can include any of the fields listed on the page. For this app, we have chosen to include the following code in our model: code You can add more fields if you want to display more information in the details view. Creating the details view The details view is a simple panel and XTemplate combination. Using our controller, the panel will receive the data record when a user clicks on a restaurant in the list: code Since the tpl tag is basically HTML, you can use any CSS styling you like here. Keep in mind that certain fields such as contact, location, and categories can have more than one entry. You will need to use <tpl for="fieldname"> to loop through these values. Now that the views are complete, we need to head back to our controller and add the functions to put everything together. Finishing the main view controller When we started out with our main controller, we added all of our views, stores, and references. Now it's time to add the functionality for the application. We start by adding a control section to the end of our config: code The controls are based on the references in the controller and they add functions to specific listeners on the component. These are each in the format of: code Once these controls are in place, we can add our functions after the config section of our controller. Our first function is doStart. This function loads our Contacts store and checks to see if we have any existing contacts. If not, we alert the user and offer to let them add some. If they have contacts we create a new instance of our confirmLocation container and push it onto the main navigation view: code Remember that since the mainView is a navigation view, a Back button will automatically be created in the top toolbar. This function will show the user our initial map panel with the users current location. This panel needs four functions: one to cancel the request, one to pop up a new location window, one to set the new location, and one to move on to the next step: code We actually want to be able to use the doCancel function from anywhere in the process. As we add new panels to our mainView navigation, these panels simply pile up in a stack. This means we need to get the number of panels currently on the mainView stack. We use length-1 to always leave the initial panel (the one with our big Get Started button) on the stack. We use pop to remove all but the first panel from the stack. This way the Cancel button will take us all the way back to the beginning of our stack, while the Back button will take us back just to the previous step. The next function is doNewLocation(), which uses Ext.Msg.prompt to ask the user to enter a new location: code If the user enters a new location, we call setNewLocation to process the text the user entered in the prompt textbox: code This code gets our map and encodes the text the user passed us as a geocode location. If Google returns a valid address, we center the map on the location and drop a marker to show the exact location. We also set the latitude and longitude so that we can reference them later. If we fail to get a valid address, we alert the user so they can fix it and try again. Once the user is happy with the location they can click on the Next button, which fires our doChooseFriends function: This function pushes our friendchooser view onto the stack for display. The friendchooser view allows the user to select multiple friends and click on Cancel or Finish. Since we have already taken care of our Cancel button with our doCancel function, we just need to write the doShowRestaurants function. This function starts by looping through the selected friends. For the first one in the list, we grab the restaurant categories we have stored for the friend and convert it from a comma-separated list (which is how we stored it) into an array. This lets us grab every subsequent selection and run Ext.Array.intersect() to find the common categories between all of the selected friends: code Next, we load the store based on the common categories by categoryID, the location data we have stored in our map, client_id, and client_secret that comprise our API key for Foursquare and a radius value (in meters). We also send a required field called v that is set to the current date. Finally, we push our restaurant list component onto the stack of containers. This will display our list of results and allow the user to click on for details. This brings us to our doShowRestaurantDetails function: code When the user taps one of the disclosure icons in our list of restaurants, we push a restaurantdetails view onto the stack of containers and set its data to the record that was tapped. This displays the details for the restaurant in our details XTemplate Homework There are a number of additional features that can be added to this type of application, including: Editing for contacts (or automatically pulling friends from Facebook) Setting up a live feed for the categories menu Adding additional venues other than restaurants Combining the application with additional APIs such as Yelp for reviews Just remember the key requirements of using additional APIs: the API key(s), studying the API documentation, and using the JSONP store for grabbing the data. Summary In this article we talked about using external APIs to enhance your Sencha Touch applications. This included: An overview of API basics Putting together the basic application Interaction with Google Maps and Foursquare Building the views, models, and stores Building the application controller Resources for Article : Further resources on this subject: How to Use jQuery Mobile Grid and Columns Layout [Article] iPhone JavaScript: Installing Frameworks [Article] An Introduction to Rhomobile [Article]

With SAX and DOM APIs, node lists have to be iterated over to access a particular node. Another advantage of navigating an XML document with XPath is that an attribute node may be selected directly. With DOM and SAX APIs, an element node has to be selected before an element attribute can be selected. Here we will discuss XPath support in JDeveloper. What is XPath? XPath is a language for addressing an XML document's elements and attributes. As an example, say you receive an XML document that contains the details of a shipment and you want to retrieve the element/attribute values from the XML document. You don't just want to list the values of all the nodes, but also want to output the values of specific elements or attributes. In such a case, you would use XPath to retrieve the values of those elements and attributes. XPath constructs a hierarchical structure of an XML document, a tree of nodes, which is the XPath data model. The XPath data model consists of seven node types. The different types of nodes in the XPath data model are discussed in the following table: Node Type Description Root Node The root node is the root of the DOM tree. The document element (the root element) is a child of the root node. The root node also has the processing instructions and comments as child nodes. Element Node It represents an element in an XML document. The character data, elements, processing instruction, and comments within an element are the child nodes of the element node. Attribute Node It represents an attribute other than the valign="top"> Text Node The character data within an element is a text node. A text node has at least one character of data. A whitespace is also considered as a character of data.  By default, the ignorable whitespace after the end of an element and before the start of the following element is also a text node. The ignorable whitespace can be excluded from the DOM tree built by parsing an XML document. This can be done by setting the whitespace-preserving mode to false with the setPreserveWhitespace(boolean flag) method. Comment Node It represents a comment in an XML document, except the comments within the DOCTYPE declaration. Processing Instruction Node It represents a processing instruction in an XML document except the processing instruction within the DOCTYPE declaration. The XML declaration is not considered as a processing instruction node. Namespace Node It represents a namespace mapping, which consists of a . A namespace node consists of a namespace prefix (xsd in the example) and a namespace URI (http://www.w3.org/2001/XMLSchema in the example). Specific nodes including element, attribute, and text nodes may be accessed with XPath. XPath supports nodes in a namespace. Nodes in XPath are selected with an XPath expression. An expression is evaluated to yield an object of one of the following four types: node set, Boolean, number, or string. For an introduction on XPath refer to the W3C Recommendation for XPath (http://www.w3.org/TR/xpath). As a brief review, expression evaluation in XPath is performed with respect to a context node. The most commonly used type of expression in XPath is a location path . XPath defines two types of location paths: relative location paths and absolute location paths. A relative location path is defined with respect to a context node and consists of a sequence of one or more location steps separated by "/". A location step consists of an axis, a node test, and predicates. An example of a location step is: child::journal[position()=2] In the example, the child axis contains the child nodes of the context node. Node test is the journal node set, and predicate is the second node in the journal node set. An absolute location path is defined with respect to the root node, and starts with "/". The difference between a relative location path and an absolute location path is that a relative location path starts with a location step, and an absolute location path starts with "/". XPath in Oracle XDK 11g Oracle XML Developer's Kit 11g, which is included in JDeveloper, provides the DOMParser class to parse an XML document and construct a DOM structure of the XML document. An XMLDocument object represents the DOM structure of an XML document. An XMLDocument object may be retrieved from a DOMParser object after an XML document has been parsed. The XMLDocument class provides select methods to select nodes in an XML document with an XPath expression. In this article we shall parse an example XML document with the DOMParser class, obtain an XMLDocument object for the XML document, and select nodes from the document with the XMLDocument class select methods. The different select methods in theXMLDocument class are discussed in the following table: Method Name Description selectSingleNode(String XPathExpression) Selects a single node that matches an XPath expression. If more than one node matches the specified expression, the first node is selected. Use this method if you want to select the first node that matches an XPath expression. selectNodes(String XPathExpression) Selects a node list of nodes that match a specified XPath expression. Use this method if you want to select a collection of similar nodes. selectSingleNode(String XPathExpression, NSResolver resolver) Selects a single namespace node that matches a specified XPath expression. Use this method if the XML document has nodes in namespaces and you want to select the first node, which is in a namespace and matches an XPath expression. selectNodes(String XPathExpression, NSResolver resolver) Selects a node list of nodes that match a specified XPath expression. Use this method if you want to select a collection of similar nodes that are in a namespace. The example XML document that is parsed in this article has a namespace declaration for elements in the namespace with the prefix journal. For an introduction on namespaces in XML refer to the W3C Recommendation on Namespaces in XML 1.0 (http://www.w3.org/TR/REC-xml-names/). catalog.xml, the example XML document, is shown in the following listing: <?xml version="1.0" encoding="UTF-8"?><catalog title="Oracle Magazine" publisher="Oracle Publishing"><journal:journal journal_date="November-December 2008"> <journal:article journal_section="ORACLE DEVELOPER"> <title>Instant ODP.NET Deployment</title> <author>Mark A. Williams</author></journal:article><journal:article journal_section="COMMENT"> <title>Application Server Convergence</title> <author>David Baum</author> </journal:article></journal:journal><journal date="March-April 2008"> <article section="TECHNOLOGY"> <title>Oracle Database 11g Redux</title> <author>Tom Kyte</author> </article><article section="ORACLE DEVELOPER"> <title>Declarative Data Filtering</title> <author>Steve Muench</author> </article> </journal></catalog Setting the environment Create an application (called XPath, for example) and a project (called XPath) in JDeveloper. The XPath API will be demonstrated in a Java application. Therefore, create a Java class in the XPath project with File | New. In the New Gallery window select < >Categories | General and Items | Java Class. In the Create Java Class window, specify the class name (XPathParser, for example), the package name (xpath in the example application), and click on the OK button. To develop an application with XPath, add the required libraries to the project classpath. Select the project node in Application Navigator and select Tools | Project Properties. In the Project Properties window, select the Libraries and Classpath node. To add a library, select the Add Library button. Select the Oracle XML Parser v2 library. Click on the OK button in the Project Properties window. We also need to add an XML document that is to be parsed and navigated with XPath. To add an XML document, select File | New. In the New Gallery window, select Categories | General | XML and Items | XML Document. Click on the OK button. In the Create XML File window specify the file name catalog.xml in the File Name field, and click on the OK button. Copy the catalog.xml listing to the catalog.xml file in the Application Navigator. The directory structure of the XPath project is shown in the following illustration: XPath Search In this section, we shall select nodes from the example XML document, catalog.xml, with the XPath Search tool of JDeveloper 11g. The XPath Search tool consists of an Expression field for specifying an XPath expression. Specify an XPath expression and click on OK to select nodes matching the XPath expression. The XPath Search tool has the provision to search for nodes in a specific namespace. An XML namespace is a collection of element and attribute names that are identified by a URI reference. Namespaces are specified in an XML document using namespace declarations. A namespace declaration is an > To navigate catalog.xml with XPath, select catalog.xml in the Application Navigator and select Search | XPath Search. In the following subsections, we shall select example nodes using absolute location paths and relative location paths. Use a relative location path if the XML document is large and a specifi c node is required. Also, use a relative path if the node from which subnodes are to be selected and the relative location path are known. Use an absolute location path if the XML document is small, or if the relative location path is not known. The objective is to use minimum XPath navigation. Use the minimum number nodes to navigate in order to select the required node. Selecting nodes with absolute location paths Next, we shall demonstrate with various examples of selecting nodes using XPath. As an example, select all the title elements in catalog.xml. Specify the XPath expression for selecting the title elements in the Expression field of the Apply an XPath Expression on catalog.xml window. The XPath expression to select all title elements is /catalog/journal/article/title. Click on the OK button to select the title elements. The title elements get selected. Title elements from the journal:article elements in the journal namespace do not get selected because a namespace has not been applied to the XPath expression. As an other example, select the title element in the first article element using the XPath expression /catalog/journal/article[1]/title. We are not using namespaces yet. The XPath expression is specified in the Expression field. The title of the first article element gets selected as shown in the JDeveloper output: Attribute nodes may also be selected with XPath. Attributes are selected by using the "@" prefix. As an example, select the section attribute in the first article element in the journal element. The XPath expression for selecting the section attribute is /catalog/journal/article[1]/@section and is specified in the Expression field. Click on the OK button to select the section attribute. The attribute section gets outputted in JDeveloper. Selecting nodes with relative location paths In the previous examples, an absolute location is used to select nodes. Next, we shall demonstrate selecting an element with a relative location path. As an example, select the title of the first article element in the journal element. The relative location path for selecting the title element is child::catalog/journal/article[position()=1]/title. Specifying the axis as child and node test as catalog selects all the child nodes of the catalog node and is equivalent to an absolute location path that starts with /catalog. If the child nodes of the journal node were required to be selected, specify the node test as journal. Specify the XPath expression in the Expression field and click on the OK button. The title of the first article element in the journal element gets selected as shown here: Selecting namespace nodes XPath Search also has the provision to select elements and attributes in a namespace. To illustrate, select all the title elements in the journal element (that is, in the journal namespace) using the XPath expression /catalog/journal:journal/journal:article/title. First, add the namespaces of the elements and attributes to be selected in the Namespaces text area. Prefix and URI of namespaces are added with the Add button. Specify the prefix in the Prefix column, and the URI in the URI column. Multiple namespace mappings may be added. XPath expressions that select namespace nodes are similar to no-namespace expressions, except that the namespace prefixes are included in the expressions. Elements in the default namespace, which does not have a namespace prefix, are also considered to be in a namespace. Click on the OK button to select the nodes with XPath. The title elements in the journal element (in the journal namespace) get selected and outputted in JDeveloper. Attributes in a namespace may also be selected with XPath Search. As an example, select the section attributes in the journal namespace. Specify the XPath expression to select the section attributes in the Expression field and click on the OK button. Section attributes in the journal namespace get selected.

(For more resources related to this topic, see here.) Getting ready To run this article, the phantomjs binary will need to be accessible to the continuous integration server, which may not necessarily share the same permissions or PATH as our user. We will also need a target URL. We will use the PhantomJS port of the YSlow library to execute the performance analysis on our target web page. The YSlow library must be installed somewhere on the filesystem that is accessible to the continuous integration server. For our example, we have placed the yslow.js script in the tmp directory of the jenkins user's home directory. To find the jenkins user's home directory on a POSIX-compatible system, first switch to that user using the following command: sudo su - jenkins Then print the home directory to the console using the following command: echo $HOME We will need to have a continuous integration server set up where we can configure the jobs that will execute our automated performance analyses. The example that follows will use the open source Jenkins CI server. Jenkins CI is too large a subject to introduce here, but this article does not assume any working knowledge of it. For information about Jenkins CI, including basic installation or usage instructions, or to obtain a copy for your platform, visit the project website at http://jenkins-ci.org/. Our article uses version 1.552. The combination of PhantomJS and YSlow is in no way unique to Jenkins CI. The example aims to provide a clear illustration of automated performance testing that can easily be adapted to any number of continuous integration server environments. The article also uses several plugins on Jenkins CI to help facilitate our automated testing. These plugins include: Environment Injector Plugin JUnit Attachments Plugin TAP Plugin xUnit Plugin To run that demo site, we must have Node.js installed. In a separate terminal, change to the phantomjs-sandbox directory (in the sample code's directory), and start the app with the following command: node app.js How to do it… To execute our automated performance analyses in Jenkins CI, the first thing that we need to do is set up the job as follows: Select the New Item link in Jenkins CI. Give the new job a name (for example, YSlow Performance Analysis), select Build a free-style software project, and then click on OK. To ensure that the performance analyses are automated, we enter a Build Trigger for the job. Check off the appropriate Build Trigger and enter details about it. For example, to run the tests every two hours, during business hours, Monday through Friday, check Build periodically and enter the Schedule as H 9-16/2 * * 1-5. In the Build block, click on Add build step and then click on Execute shell. In the Command text area of the Execute Shell block, enter the shell commands that we would normally type at the command line, for example: phantomjs ${HOME}/tmp/yslow.js -i grade -threshold "B" -f junit http ://localhost:3000/css-demo > yslow.xml In the Post-build Actions block, click on Add post-build action and then click on Publish JUnit test result report. In the Test report XMLs field of the Publish JUnit Test Result Report block, enter *.xml. Lastly, click on Save to persist the changes to this job. Our performance analysis job should now run automatically according to the specified schedule; however, we can always trigger it manually by navigating to the job in Jenkins CI and clicking on Build Now. After a few of the performance analyses have completed, we can navigate to those jobs in Jenkins CI and see the results shown in the following screenshots: The landing page for a performance analysis project in Jenkins CI Note the Test Result Trend graph with the successes and failures. The Test Result report page for a specific build Note that the failed tests in the overall analysis are called out and that we can expand specific items to view their details. The All Tests view of the Test Result report page for a specific build Note that all tests in the performance analysis are listed here, regardless of whether they passed or failed, and that we can click into a specific test to view its details. How it works… The driving principle behind this article is that we want our continuous integration server to periodically and automatically execute the YSlow analyses for us so that we can monitor our website's performance over time. This way, we can see whether our changes are having an effect on overall site performance, receive alerts when performance declines, or even fail builds if we fall below our performance threshold. The first thing that we do in this article is set up the build job. In our example, we set up a new job that was dedicated to the YSlow performance analysis task. However, these steps could be adapted such that the performance analysis task is added onto an existing multipurpose job. Next, we configured when our job will run, adding Build Trigger to run the analyses according to a schedule. For our schedule, we selected H 9-16/2 * * 1-5, which runs the analyses every two hours, during business hours, on weekdays. While the schedule that we used is fine for demonstration purposes, we should carefully consider the needs of our project—chances are that a different Build Trigger will be more appropriate. For example, it may make more sense to select Build after other projects are built, and to have the performance analyses run only after the new code has been committed, built, and deployed to the appropriate QA or staging environment. Another alternative would be to select Poll SCM and to have the performance analyses run only after Jenkins CI detects new changes in source control. With the schedule configured, we can apply the shell commands necessary for the performance analyses. As noted earlier, the Command text area accepts the text that we would normally type on the command line. Here we type the following: phantomjs: This is for the PhantomJS executable binary ${HOME}/tmp/yslow.js: This is to refer to the copy of the YSlow library accessible to the Jenkins CI user -i grade: This is to indicate that we want the "Grade" level of report detail -threshold "B": This is to indicate that we want to fail builds with an overall grade of "B" or below -f junit: This is to indicate that we want the results output in the JUnit format http://localhost:3000/css-demo: This is typed in as our target URL > yslow.xml: This is to redirect the JUnit-formatted output to that file on the disk What if PhantomJS isn't on the PATH for the Jenkins CI user? A relatively common problem that we may experience is that, although we have permission on Jenkins CI to set up new build jobs, we are not the server administrator. It is likely that PhantomJS is available on the same machine where Jenkins CI is running, but the jenkins user simply does not have the phantomjs binary on its PATH. In these cases, we should work with the person administering the Jenkins CI server to learn its path. Once we have the PhantomJS path, we can do the following: click on Add build step and then on Inject environment variables; drag-and-drop the Inject environment variables block to ensure that it is above our Execute shell block; in the Properties Content text area, apply the PhantomJS binary's path to the PATH variable, as we would in any other script as follows: PATH=/path/to/phantomjs/bin:${PATH} After setting the shell commands to execute, we jump into the Post-build Actions block and instruct Jenkins CI where it can find the JUnit XML reports. As our shell command is redirecting the output into a file that is directly in the workspace, it is sufficient to enter an unqualified *.xml here. Once we have saved our build job in Jenkins CI, the performance analyses can begin right away! If we are impatient for our first round of results, we can click on Build Now for our job and watch as it executes the initial performance analysis. As the performance analyses are run, Jenkins CI will accumulate the results on the filesystem, keeping them until they are either manually removed or until a discard policy removes old build information. We can browse these accumulated jobs in the web UI for Jenkins CI, clicking on the Test Result link to drill into them. There's more… The first thing that bears expanding upon is that we should be thoughtful about what we use as the target URL for our performance analysis job. The YSlow library expects a single target URL, and as such, it is not prepared to handle a performance analysis job that is otherwise configured to target two or more URLs. As such, we must select a strategy to compensate for this, for example: Pick a representative page: We could manually go through our site and select the single page that we feel best represents the site as a whole. For example, we could pick the page that is "most average" compared to the other pages ("most will perform at about this level"), or the page that is most likely to be the "worst performing" page ("most pages will perform better than this"). With our representative page selected, we can then extrapolate performance for other pages from this specimen. Pick a critical page: We could manually select the single page that is most sensitive to performance. For example, we could pick our site's landing page (for example, "it is critical to optimize performance for first-time visitors"), or a product demo page (for example, "this is where conversions happen, so this is where performance needs to be best"). Again, with our performance-sensitive page selected, we can optimize the general cases around the specific one. Set up multiple performance analysis jobs: If we are not content to extrapolate site performance from a single specimen page, then we could set up multiple performance analysis jobs—one for each page on the site that we want to test. In this way, we could (conceivably) set up an exhaustive performance analysis suite. Unfortunately, the results will not roll up into one; however, once our site is properly tuned, we need to only look for the telltale red ball of a failed build in Jenkins CI. The second point worth considering is—where do we point PhantomJS and YSlow for the performance analysis? And how does the target URL's environment affect our interpretation of the results? If we are comfortable running our performance analysis against our production deploys, then there is not much else to discuss—we are assessing exactly what needs to be assessed. But if we are analyzing performance in production, then it's already too late—the slow code has already been deployed! If we have a QA or staging environment available to us, then this is potentially better; we can deploy new code to one of these environments for integration and performance testing before putting it in front of the customers. However, these environments are likely to be different from production despite our best efforts. For example, though we may be "doing everything else right", perhaps our staging server causes all traffic to come back from a single hostname, and thus, we cannot properly mimic a CDN, nor can we use cookie-free domains. Do we lower our threshold grade? Do we deactivate or ignore these rules? How can we tell apart the false negatives from the real warnings? We should put some careful thought into this—but don't be disheartened—better to have results that are slightly off than to have no results at all! Using TAP format If JUnit formatted results turn out to be unacceptable, there is also a TAP plugin for Jenkins CI. Test Anything Protocol (TAP) is a plain text-based report format that is relatively easy for both humans and machines to read. With the TAP plugin installed in Jenkins CI, we can easily configure our performance analysis job to use it. We would just make the following changes to our build job: In the Command text area of our Execute shell block, we would enter the following command: phantomjs ${HOME}/tmp/yslow.js -i grade -threshold "B" -f tap http ://localhost:3000/css-demo > yslow.tap In the Post-build Actions block, we would select Publish TAP Results instead of Publish JUnit test result report and enter yslow.tap in the Test results text field. Everything else about using TAP instead of JUnit-formatted results here is basically the same. The job will still run on the schedule we specify, Jenkins CI will still accumulate test results for comparison, and we can still explore the details of an individual test's outcomes. The TAP plugin adds an additional link in the job for us, TAP Extended Test Results, as shown in the following screenshot: One thing worth pointing out about using TAP results is that it is much easier to set up a single job to test multiple target URLs within a single website. We can enter multiple tests in the Execute Shell block (separating them with the && operator) and then set our Test Results target to be *.tap. This will conveniently combine the results of all our performance analyses into one. Summary In this article, we saw setting up of an automated performance analysis task on a continuous integration server (for example, Jenkins CI) using PhantomJS and the YSlow library. Resources for Article: Further resources on this subject: Getting Started [article] Introducing a feature of IntroJs [article] So, what is Node.js? [article]

OmniGraffle 5 Diagramming Essentials Create better diagrams with less effort using OmniGraffle Produce high-quality professional-looking diagrams that communicate information much better than words Makes diagramming fun and simple for Macintosh users Master the art of illustrating your ideas with OmniGraffle Learn to draw engaging charts and graphs to grasp your viewers' attention to your presentations A hands-on guide filled with visual step-by-step examples that cover both the basics and the advanced features of OmniGraffle        Easy shape selection When diagrams become complex and you want to change the appearance of many of the same objects, you can either hold down the shift key on your keyboard and select the shapes you need to change, or you can use the built-in selection functions in OmniGraffle. There are three built in methods of selecting shapes: The Edit | Select | Similar Objects menu command. The context sensitive menu when right-clicking on selected shapes (Select | Similar Objects). Using the Canvas: Selection inspector. If shapes are connected to each other, it's also possible to select connected shapes from the application menu or the context sensitive menu. You will deal with connected shapes later. What OmniGraffle defines as similar shapes Similar shapes are shapes that have exactly the same styling, not the form, type or size of the shapes. The styling can be; the fi lling color or blend of the shape, the stroke thickness, the corner radius, the stroke color, the shadow, if the shape is filled with a picture and if the shape has an associated text. Neither the size of two or more shapes, nor the content of the text will make OmniGraffle define the shapes as dissimilar. If you change the font type, weight or color–then OmniGraffle will define the shapes as dissimilar. As the few examples in the table below show more often than not, shapes that are of the same type (rectangle, circle, and so on) are dissimilar. ShapesExplanationResultSame font, color and weight, but different text.Similar shapes.Difference in the fuzziness of the shadow.Dissimilar shapes.Different font size.Dissimilar shapes.Different color. Different text, but same font, color and weight.Dissimilar shapes. Let's explore how easy it is to select similar shapes. Create an OmniGraffle document with only 4 shapes: Rectangle, Circle, Diamond, and Triangle. Mix them all together so it's not easy to manually select various shapes. In the download bundle, Ch:7, there is a file named Experiments in shape selections.graffle—you can open this if you do not want to create your own version. In the Canvas: Selection inspector (seen in the previous screenshot), you will see the four different shapes. Under each shape, there are two numbers separated by a forward slash (/). The number on the left indicates how many of the given shape are selected, and the number to the right of the slash is the total count of shapes on the current canvas. As you can see in the Selection inspector, we have 11 rectangles, 9 triangles, 10 circles, and 10 diamonds. To select all the diamonds on the canvas, click on the diamond shape in the Selection inspector. Notice that the numbers below the shape are now reading 10/10. If you click on the Fill style inspector and use the color named Honeydew from the Crayon color palette—all the diamonds change into this color. Now, click on the rectangle in the upper-left corner and right-click to get the context sensitive menu. Then select then use the Select | Similar Objects menu command, as seen next: Now all the rectangles are selected. Fill the rectangles with the color named Tin from the Crayons color palette. The next thing you are going to do is to select four of the circles and fill these with a yellow color. Notice that the Selection inspector now has five different shapes: You can now continue on your own to experiment changing various style properties to the shape groups. Selecting connected shapes You are now going to perform some experiments with connected shapes. Start with a diagram like the one next. You can either create this by hand, or you can open the file named Experiments in shape selections.graffle found in the Ch:7 directory in the download bundle. The diagram is found on the second canvas. Notice that all shapes are connected to each other. If you look at the Selection inspector, you'll see four shapes, where the lines are one of those shapes. OmniGraffle defines descendants to a selected shape as all shapes that are either connected to the right of or below the selected shape. OmniGraffle defines ancestors to a selected shape as all shapes that are either connected to the left of or above the selected shape. The following table lists a few actions and their corresponding results while working with connected shapes. ActionResultChoose (select) circle number 1 and execute Descendants from the Edit | Selection menu. This action results in the selection of the shapes connected below and to the right of the chosen shape.Select circle number 2 and choose Ancestors from the Edit | Selection menu. This action results in the selection of shapes connected to the left of the chosen shape.Select circle number 3 and choose Ancestors from the Edit | Selection menu. This action results in the selection of shapes connected above and to the left of the chosen shape.Select square B and choose Descendants from the Edit | Selection menu. This action result in every shape except circle number 1 and square A being selected.Select square C and choose Ancestors from the Edit | Selection menu. The shapes connected to the left of the chosen shape are selected.Select square C and choose Descendants from the Edit | Selection menu. The shapes connected below and to the right of the chosen shape are selected.Select any shape and choose Connected Objects from the Edit | Selection menu. This action results in the whole diagram being selected since every shape is connect to every other shape, either directly or indirectly. Selecting all these shapes—either by form or by connections seems like a very powerful tool, and indeed it is. With this way of selecting shapes, you can easily change the look and feel in a consistent manner throughout your whole diagram. Easy re-styling of shapes Instead of selecting similar shapes, and then changing their appearance by using the style inspectors, you can also drag and drop styles from the Style Tray directly onto the Selection inspector. Start your experiments in the easy re-styling of shapes by creating various shapes, and fill some of these with various colors. To save you some time and get right into the experimentation, you can also open the file named Experiments in re-styling shapes.graffle, which is found in the Ch: 6 folder of the download bundle. We're not quite happy with only having four yellow circles. Your first task is to make the rest of the circles yellow. Unfortunately, you do not know which yellow color these circles are – you could work with the color picker to get the right yellow. This is too much work. Let's do this the simple way. In the Selection inspector, you will see all the different shapes, including the four yellow circles. If you use the file from the download bundle, you should have the yellow circles as the first shape in the inspector. To select all the yellow circles, you could click on one of the circles and then use the Edit | Select | Similar Objects menu command. This is too much work really. Just click on the shape inside the inspector. What happens now is that all the yellow circles have been selected. In the Style Tray, the styling for these shapes is now shown: You can now drag the color chit (//Image 43) on to the white circle in the Selection inspector. Suddenly all your circles have this shade of yellow. And it's not only the color we can change in a second like this. In fact, your next job is to eradicate all of those gray and boring squares and instead replace these with the green and happy diamonds. This is also just a two-step process. Start by clicking on one of the diamonds in your canvas. Now, drag the Complete Style Chit (the leftmost of all the chits) onto the gray squares inside the selection inspector. Not only does this change the color of the square shapes— but it also changes the shapes into diamonds. You can thus drag any chit from the Style Tray onto any shapes both on the canvas , and also onto the Selection inspector. This is a very powerful tool to aid you in getting a consistent look for your shapes. You can also use this method to search and replace a lot of shapes in one go. No need to manually hunt around on a shape-by-shape basis.

In this article by Chitij Chauhan, author of the book PostgreSQL Cookbook, we will talk about various high availability and replication solutions, including some popular third-party replication tools such as Slony-I and Londiste. In this article, we will cover the following recipes: Setting up hot streaming replication Replication using Slony-I Replication using Londiste The important components for any production database is to achieve fault tolerance, 24/7 availability, and redundancy. It is for this purpose that we have different high availability and replication solutions available for PostgreSQL. From a business perspective, it is important to ensure 24/7 data availability in the event of a disaster situation or a database crash due to disk or hardware failure. In such situations, it becomes critical to ensure that a duplicate copy of the data is available on a different server or a different database, so that seamless failover can be achieved even when the primary server/database is unavailable. Setting up hot streaming replication In this recipe, we are going to set up a master-slave streaming replication. Getting ready For this exercise, you will need two Linux machines, each with the latest version of PostgreSQL installed. We will be using the following IP addresses for the master and slave servers: Master IP address: 192.168.0.4 Slave IP address: 192.168.0.5 Before you start with the master-slave streaming setup, it is important that the SSH connectivity between the master and slave is setup. How to do it... Perform the following sequence of steps to set up a master-slave streaming replication: First, we are going to create a user on the master, which will be used by the slave server to connect to the PostgreSQL database on the master server: psql -c "CREATE USER repuser REPLICATION LOGIN ENCRYPTED PASSWORD 'charlie';" Next, we will allow the replication user that was created in the previous step to allow access to the master PostgreSQL server. This is done by making the necessary changes as mentioned in the pg_hba.conf file: Vi pg_hba.conf host   replication   repuser   192.168.0.5/32   md5 In the next step, we are going to configure parameters in the postgresql.conf file. These parameters need to be set in order to get the streaming replication working: Vi /var/lib/pgsql/9.3/data/postgresql.conf listen_addresses = '*' wal_level = hot_standby max_wal_senders = 3 wal_keep_segments = 8 archive_mode = on       archive_command = 'cp %p /var/lib/pgsql/archive/%f && scp %p postgres@192.168.0.5:/var/lib/pgsql/archive/%f' checkpoint_segments = 8 Once the parameter changes have been made in the postgresql.conf file in the previous step, the next step will be to restart the PostgreSQL server on the master server, in order to let the changes take effect: pg_ctl -D /var/lib/pgsql/9.3/data restart Before the slave can replicate the master, we will need to give it the initial database to build off. For this purpose, we will make a base backup by copying the primary server's data directory to the standby. The rsync command needs to be run as a root user: psql -U postgres -h 192.168.0.4 -c "SELECT pg_start_backup('label', true)" rsync -a /var/lib/pgsql/9.3/data/ 192.168.0.5:/var/lib/pgsql/9.3/data/ --exclude postmaster.pid psql -U postgres -h 192.168.0.4 -c "SELECT pg_stop_backup()" Once the data directory, mentioned in the previous step, is populated, the next step is to enable the following parameter in the postgresql.conf file on the slave server: hot_standby = on The next step will be to copy the recovery.conf.sample file in the $PGDATA location on the slave server and then configure the following parameters: cp /usr/pgsql-9.3/share/recovery.conf.sample /var/lib/pgsql/9.3/data/recovery.conf standby_mode = on primary_conninfo = 'host=192.168.0.4 port=5432 user=repuser password=charlie' trigger_file = '/tmp/trigger.replication′ restore_command = 'cp /var/lib/pgsql/archive/%f "%p"' The next step will be to start the slave server: service postgresql-9.3 start Now that the above mentioned replication steps are set up, we will test for replication. On the master server, log in and issue the following SQL commands: psql -h 192.168.0.4 -d postgres -U postgres -W postgres=# create database test;   postgres=# c test;   test=# create table testtable ( testint int, testchar varchar(40) );   CREATE TABLE test=# insert into testtable values ( 1, 'What A Sight.' ); INSERT 0 1 On the slave server, we will now check whether the newly created database and the corresponding table, created in the previous step, are replicated: psql -h 192.168.0.5 -d test -U postgres -W test=# select * from testtable; testint | testchar ---------+--------------------------- 1 | What A Sight. (1 row) How it works... The following is the explanation for the steps performed in the preceding section. In the initial step of the preceding section, we create a user called repuser, which will be used by the slave server to make a connection to the primary server. In the second step of the preceding section, we make the necessary changes in the pg_hba.conf file to allow the master server to be accessed by the slave server using the repuser user ID that was created in step 1. We then make the necessary parameter changes on the master in step 3 of the preceding section to configure a streaming replication. The following is a description of these parameters: listen_addresses: This parameter is used to provide the IP address associated with the interface that you want to have PostgreSQL listen to. A value of * indicates all available IP addresses. wal_level: This parameter determines the level of WAL logging done. Specify hot_standby for streaming replication. wal_keep_segments: This parameter specifies the number of 16 MB WAL files to be retained in the pg_xlog directory. The rule of thumb is that more such files might be required to handle a large checkpoint. archive_mode: Setting this parameter enables completed WAL segments to be sent to the archive storage. archive_command: This parameter is basically a shell command that is executed whenever a WAL segment is completed. In our case, we are basically copying the file to the local machine and then using the secure copy command to send it across to the slave. max_wal_senders: This parameter specifies the total number of concurrent connections allowed from the slave servers. checkpoint_segments: This parameter specifies the maximum number of logfile segments between automatic WAL checkpoints. Once the necessary configuration changes have been made on the master server, we then restart the PostgreSQL server on the master in order to let the new configuration changes take effect. This is done in step 4 of the preceding section. In step 5 of the preceding section, we are basically building the slave by copying the primary server's data directory to the slave. Now, with the data directory available on the slave, the next step is to configure it. We will now make the necessary parameter replication related parameter changes on the slave in the postgresql.conf directory on the slave server. We set the following parameters on the slave: hot_standby: This parameter determines whether you can connect and run queries when the server is in the archive recovery or standby mode. In the next step, we are configuring the recovery.conf file. This is required to be set up so that the slave can start receiving logs from the master. The parameters explained next are configured in the recovery.conf file on the slave. standby_mode: This parameter, when enabled, causes PostgreSQL to work as a standby in a replication configuration. primary_conninfo: This parameter specifies the connection information used by the slave to connect to the master. For our scenario, our master server is set as 192.168.0.4 on port 5432 and we are using the repuser userid with the password charlie to make a connection to the master. Remember that repuser was the userid which was created in the initial step of the preceding section for this purpose, that is, connecting to the master from the slave. trigger_file: When a slave is configured as a standby, it will continue to restore the XLOG records from the master. The trigger_file parameter specifies what is used to trigger a slave, in order to switch over its duties from standby and take over as master or primary server. At this stage, the slave is fully configured now and we can start the slave server; then, the replication process begins. This is shown in step 8 of the preceding section. In steps 9 and 10 of the preceding section, we are simply testing our replication. We first begin by creating a test database, then we log in to the test database and create a table by the name testtable, and then we begin inserting some records into the testtable table. Now, our purpose is to see whether these changes are replicated across the slave. To test this, we log in to the slave on the test database and then query the records from the testtable table, as seen in step 10 of the preceding section. The final result that we see is that all the records that are changed/inserted on the primary server are visible on the slave. This completes our streaming replication's setup and configuration. Replication using Slony-I Here, we are going to set up replication using Slony-I. We will be setting up the replication of table data between two databases on the same server. Getting ready The steps performed in this recipe are carried out on a CentOS Version 6 machine. It is also important to remove the directives related to hot streaming replication prior to setting up replication using Slony-I. We will first need to install Slony-I. The following steps need to be performed in order to install Slony-I: First, go to http://slony.info/downloads/2.2/source/ and download the given software. Once you have downloaded the Slony-I software, the next step is to unzip the .tar file and then go the newly created directory. Before doing this, please ensure that you have the postgresql-devel package for the corresponding PostgreSQL version installed before you install Slony-I: tar xvfj slony1-2.2.3.tar.bz2  cd slony1-2.2.3 In the next step, we are going to configure, compile, and build the software: ./configure --with-pgconfigdir=/usr/pgsql-9.3/bin/ make make install How to do it... You need to perform the following sequence of steps, in order to replicate data between two tables using Slony-I replication: First, start the PostgreSQL server if you have not already started it: pg_ctl -D $PGDATA start In the next step, we will be creating two databases, test1 and test2, which will be used as the source and target databases respectively: createdb test1 createdb test2 In the next step, we will create the t_test table on the source database, test1, and insert some records into it: psql -d test1 test1=# create table t_test (id numeric primary key, name varchar);   test1=# insert into t_test values(1,'A'),(2,'B'), (3,'C'); We will now set up the target database by copying the table definitions from the test1 source database: pg_dump -s -p 5432 -h localhost test1 | psql -h localhost -p 5432 test2 We will now connect to the target database, test2, and verify that there is no data in the tables of the test2 database: psql -d test2 test2=# select * from t_test; We will now set up a slonik script for the master-slave, that is source/target, setup. In this scenario, since we are replicating between two different databases on the same server, the only different connection string option will be the database name: cd /usr/pgsql-9.3/bin vi init_master.slonik   #!/bin/sh cluster name = mycluster; node 1 admin conninfo = 'dbname=test1 host=localhost port=5432 user=postgres password=postgres'; node 2 admin conninfo = 'dbname=test2 host=localhost port=5432 user=postgres password=postgres'; init cluster ( id=1); create set (id=1, origin=1); set add table(set id=1, origin=1, id=1, fully qualified name = 'public.t_test'); store node (id=2, event node = 1); store path (server=1, client=2, conninfo='dbname=test1 host=localhost port=5432 user=postgres password=postgres'); store path (server=2, client=1, conninfo='dbname=test2 host=localhost port=5432 user=postgres password=postgres'); store listen (origin=1, provider = 1, receiver = 2);  store listen (origin=2, provider = 2, receiver = 1); We will now create a slonik script for subscription to the slave, that is, target: cd /usr/pgsql-9.3/bin vi init_slave.slonik #!/bin/sh cluster name = mycluster; node 1 admin conninfo = 'dbname=test1 host=localhost port=5432 user=postgres password=postgres'; node 2 admin conninfo = 'dbname=test2 host=localhost port=5432 user=postgres password=postgres'; subscribe set ( id = 1, provider = 1, receiver = 2, forward = no); We will now run the init_master.slonik script created in step 6 and run this on the master, as follows: cd /usr/pgsql-9.3/bin   slonik init_master.slonik We will now run the init_slave.slonik script created in step 7 and run this on the slave, that is, target: cd /usr/pgsql-9.3/bin   slonik init_slave.slonik In the next step, we will start the master slon daemon: nohup slon mycluster "dbname=test1 host=localhost port=5432 user=postgres password=postgres" & In the next step, we will start the slave slon daemon: nohup slon mycluster "dbname=test2 host=localhost port=5432 user=postgres password=postgres" & Next, we will connect to the master, that is, the test1 source database, and insert some records in the t_test table: psql -d test1 test1=# insert into t_test values (5,'E'); We will now test for the replication by logging on to the slave, that is, the test2 target database, and see whether the inserted records in the t_test table are visible: psql -d test2   test2=# select * from t_test; id | name ----+------ 1 | A 2 | B 3 | C 5 | E (4 rows) How it works... We will now discuss the steps performed in the preceding section: In step 1, we first start the PostgreSQL server if it is not already started. In step 2, we create two databases, namely test1 and test2, that will serve as our source (master) and target (slave) databases. In step 3, we log in to the test1 source database, create a t_test table, and insert some records into the table. In step 4, we set up the target database, test2, by copying the table definitions present in the source database and loading them into test2 using the pg_dump utility. In step 5, we log in to the target database, test2, and verify that there are no records present in the t_test table because in step 4, we only extracted the table definitions into the test2 database from the test1 database. In step 6, we set up a slonik script for the master-slave replication setup. In the init_master.slonik file, we first define the cluster name as mycluster. We then define the nodes in the cluster. Each node will have a number associated with a connection string, which contains database connection information. The node entry is defined both for the source and target databases. The store_path commands are necessary, so that each node knows how to communicate with the other. In step 7, we set up a slonik script for the subscription of the slave, that is, the test2 target database. Once again, the script contains information such as the cluster name and the node entries that are designated a unique number related to connection string information. It also contains a subscriber set. In step 8, we run the init_master.slonik file on the master. Similarly, in step 9, we run the init_slave.slonik file on the slave. In step 10, we start the master slon daemon. In step 11, we start the slave slon daemon. The subsequent steps, 12 and 13, are used to test for replication. For this purpose, in step 12 of the preceding section, we first log in to the test1 source database and insert some records into the t_test table. To check whether the newly inserted records have been replicated in the target database, test2, we log in to the test2 database in step 13. The result set obtained from the output of the query confirms that the changed/inserted records on the t_test table in the test1 database are successfully replicated across the target database, test2. For more information on Slony-I replication, go to http://slony.info/documentation/tutorial.html. There's more... If you are using Slony-I for replication between two different servers, in addition to the steps mentioned in the How to do it… section, you will also have to enable authentication information in the pg_hba.conf file existing on both the source and target servers. For example, let's assume that the source server's IP is 192.168.16.44 and the target server's IP is 192.168.16.56 and we are using a user named super to replicate the data. If this is the situation, then in the source server's pg_hba.conf file, we will have to enter the information, as follows: host         postgres         super     192.168.16.44/32           md5 Similarly, in the target server's pg_hba.conf file, we will have to enter the authentication information, as follows: host         postgres         super     192.168.16.56/32           md5 Also, in the shell scripts that were used for Slony-I, wherever the connection information for the host is localhost that entry will need to be replaced by the source and target server's IP addresses. Replication using Londiste In this recipe, we are going to show you how to replicate data using Londiste. Getting ready For this setup, we are using the same host CentOS Linux machine to replicate data between two databases. This can also be set up using two separate Linux machines running on VMware, VirtualBox, or any other virtualization software. It is assumed that the latest version of PostgreSQL, version 9.3, is installed. We used CentOS Version 6 as the Linux operating system for this exercise. To set up Londiste replication on the Linux machine, perform the following steps: Go to http://pgfoundry.org/projects/skytools/ and download the latest version of Skytools 3.2, that is, tarball skytools-3.2.tar.gz. Extract the tarball file, as follows: tar -xvzf skytools-3.2.tar.gz Go to the new location and build and compile the software: cd skytools-3.2 ./configure --prefix=/var/lib/pgsql/9.3/Sky –with-pgconfig=/usr/pgsql-9.3/bin/pg_config   make   make install Also, set the PYTHONPATH environment variable, as shown here. Alternatively, you can also set it in the .bash_profile script: export PYTHONPATH=/opt/PostgreSQL/9.2/Sky/lib64/python2.6/site-packages/ How to do it... We are going to perform the following sequence of steps to set up replication between two different databases using Londiste. First, create the two databases between which replication has to occur: createdb node1 createdb node2 Populate the node1 database with data using the pgbench utility: pgbench -i -s 2 -F 80 node1 Add any primary key and foreign keys to the tables in the node1 database that are needed for replication. Create the following .sql file and add the following lines to it: Vi /tmp/prepare_pgbenchdb_for_londiste.sql -- add primary key to history table ALTER TABLE pgbench_history ADD COLUMN hid SERIAL PRIMARY KEY;   -- add foreign keys ALTER TABLE pgbench_tellers ADD CONSTRAINT pgbench_tellers_branches_fk FOREIGN KEY(bid) REFERENCES pgbench_branches; ALTER TABLE pgbench_accounts ADD CONSTRAINT pgbench_accounts_branches_fk FOREIGN KEY(bid) REFERENCES pgbench_branches; ALTER TABLE pgbench_history ADD CONSTRAINT pgbench_history_branches_fk FOREIGN KEY(bid) REFERENCES pgbench_branches; ALTER TABLE pgbench_history ADD CONSTRAINT pgbench_history_tellers_fk FOREIGN KEY(tid) REFERENCES pgbench_tellers; ALTER TABLE pgbench_history ADD CONSTRAINT pgbench_history_accounts_fk FOREIGN KEY(aid) REFERENCES pgbench_accounts; We will now load the .sql file created in the previous step and load it into the database: psql node1 -f /tmp/prepare_pgbenchdb_for_londiste.sql We will now populate the node2 database with table definitions from the tables in the node1 database: pg_dump -s -t 'pgbench*' node1 > /tmp/tables.sql psql -f /tmp/tables.sql node2 Now starts the process of replication. We will first create the londiste.ini configuration file with the following parameters in order to set up the root node for the source database, node1: Vi londiste.ini   [londiste3] job_name = first_table db = dbname=node1 queue_name = replication_queue logfile = /home/postgres/log/londiste.log pidfile = /home/postgres/pid/londiste.pid In the next step, we are going to use the londiste.ini configuration file created in the previous step to set up the root node for the node1 database, as shown here: [postgres@localhost bin]$ ./londiste3 londiste3.ini create-root node1 dbname=node1   2014-12-09 18:54:34,723 2335 WARNING No host= in public connect string, bad idea 2014-12-09 18:54:35,210 2335 INFO plpgsql is installed 2014-12-09 18:54:35,217 2335 INFO pgq is installed 2014-12-09 18:54:35,225 2335 INFO pgq.get_batch_cursor is installed 2014-12-09 18:54:35,227 2335 INFO pgq_ext is installed 2014-12-09 18:54:35,228 2335 INFO pgq_node is installed 2014-12-09 18:54:35,230 2335 INFO londiste is installed 2014-12-09 18:54:35,232 2335 INFO londiste.global_add_table is installed 2014-12-09 18:54:35,281 2335 INFO Initializing node 2014-12-09 18:54:35,285 2335 INFO Location registered 2014-12-09 18:54:35,447 2335 INFO Node "node1" initialized for queue "replication_queue" with type "root" 2014-12-09 18:54:35,465 2335 INFO Don We will now run the worker daemon for the root node: [postgres@localhost bin]$ ./londiste3 londiste3.ini worker 2014-12-09 18:55:17,008 2342 INFO Consumer uptodate = 1 In the next step, we will create a slave.ini configuration file in order to make a leaf node for the node2 target database: Vi slave.ini [londiste3] job_name = first_table_slave db = dbname=node2 queue_name = replication_queue logfile = /home/postgres/log/londiste_slave.log pidfile = /home/postgres/pid/londiste_slave.pid We will now initialize the node in the target database: ./londiste3 slave.ini create-leaf node2 dbname=node2 –provider=dbname=node1 2014-12-09 18:57:22,769 2408 WARNING No host= in public connect string, bad idea 2014-12-09 18:57:22,778 2408 INFO plpgsql is installed 2014-12-09 18:57:22,778 2408 INFO Installing pgq 2014-12-09 18:57:22,778 2408 INFO   Reading from /var/lib/pgsql/9.3/Sky/share/skytools3/pgq.sql 2014-12-09 18:57:23,211 2408 INFO pgq.get_batch_cursor is installed 2014-12-09 18:57:23,212 2408 INFO Installing pgq_ext 2014-12-09 18:57:23,213 2408 INFO   Reading from /var/lib/pgsql/9.3/Sky/share/skytools3/pgq_ext.sql 2014-12-09 18:57:23,454 2408 INFO Installing pgq_node 2014-12-09 18:57:23,455 2408 INFO   Reading from /var/lib/pgsql/9.3/Sky/share/skytools3/pgq_node.sql 2014-12-09 18:57:23,729 2408 INFO Installing londiste 2014-12-09 18:57:23,730 2408 INFO   Reading from /var/lib/pgsql/9.3/Sky/share/skytools3/londiste.sql 2014-12-09 18:57:24,391 2408 INFO londiste.global_add_table is installed 2014-12-09 18:57:24,575 2408 INFO Initializing node 2014-12-09 18:57:24,705 2408 INFO Location registered 2014-12-09 18:57:24,715 2408 INFO Location registered 2014-12-09 18:57:24,744 2408 INFO Subscriber registered: node2 2014-12-09 18:57:24,748 2408 INFO Location registered 2014-12-09 18:57:24,750 2408 INFO Location registered 2014-12-09 18:57:24,757 2408 INFO Node "node2" initialized for queue "replication_queue" with type "leaf" 2014-12-09 18:57:24,761 2408 INFO Done We will now launch the worker daemon for the target database, that is, node2: [postgres@localhost bin]$ ./londiste3 slave.ini worker 2014-12-09 18:58:53,411 2423 INFO Consumer uptodate = 1 We will now create the configuration file, that is pgqd.ini, for the ticker daemon: vi pgqd.ini   [pgqd] logfile = /home/postgres/log/pgqd.log pidfile = /home/postgres/pid/pgqd.pid Using the configuration file created in the previous step, we will launch the ticker daemon: [postgres@localhost bin]$ ./pgqd pgqd.ini 2014-12-09 19:05:56.843 2542 LOG Starting pgqd 3.2 2014-12-09 19:05:56.844 2542 LOG auto-detecting dbs ... 2014-12-09 19:05:57.257 2542 LOG node1: pgq version ok: 3.2 2014-12-09 19:05:58.130 2542 LOG node2: pgq version ok: 3.2 We will now add all the tables to the replication on the root node: [postgres@localhost bin]$ ./londiste3 londiste3.ini add-table --all 2014-12-09 19:07:26,064 2614 INFO Table added: public.pgbench_accounts 2014-12-09 19:07:26,161 2614 INFO Table added: public.pgbench_branches 2014-12-09 19:07:26,238 2614 INFO Table added: public.pgbench_history 2014-12-09 19:07:26,287 2614 INFO Table added: public.pgbench_tellers Similarly, add all the tables to the replication on the leaf node: [postgres@localhost bin]$ ./londiste3 slave.ini add-table –all We will now generate some traffic on the node1 source database: pgbench -T 10 -c 5 node1 We will now use the compare utility available with the londiste3 command to check the tables in both the nodes; that is, both the source database (node1) and destination database (node2) have the same amount of data: [postgres@localhost bin]$ ./londiste3 slave.ini compare   2014-12-09 19:26:16,421 2982 INFO Checking if node1 can be used for copy 2014-12-09 19:26:16,424 2982 INFO Node node1 seems good source, using it 2014-12-09 19:26:16,425 2982 INFO public.pgbench_accounts: Using node node1 as provider 2014-12-09 19:26:16,441 2982 INFO Provider: node1 (root) 2014-12-09 19:26:16,446 2982 INFO Locking public.pgbench_accounts 2014-12-09 19:26:16,447 2982 INFO Syncing public.pgbench_accounts 2014-12-09 19:26:18,975 2982 INFO Counting public.pgbench_accounts 2014-12-09 19:26:19,401 2982 INFO srcdb: 200000 rows, checksum=167607238449 2014-12-09 19:26:19,706 2982 INFO dstdb: 200000 rows, checksum=167607238449 2014-12-09 19:26:19,715 2982 INFO Checking if node1 can be used for copy 2014-12-09 19:26:19,716 2982 INFO Node node1 seems good source, using it 2014-12-09 19:26:19,716 2982 INFO public.pgbench_branches: Using node node1 as provider 2014-12-09 19:26:19,730 2982 INFO Provider: node1 (root) 2014-12-09 19:26:19,734 2982 INFO Locking public.pgbench_branches 2014-12-09 19:26:19,734 2982 INFO Syncing public.pgbench_branches 2014-12-09 19:26:22,772 2982 INFO Counting public.pgbench_branches 2014-12-09 19:26:22,804 2982 INFO srcdb: 2 rows, checksum=-3078609798 2014-12-09 19:26:22,812 2982 INFO dstdb: 2 rows, checksum=-3078609798 2014-12-09 19:26:22,866 2982 INFO Checking if node1 can be used for copy 2014-12-09 19:26:22,877 2982 INFO Node node1 seems good source, using it 2014-12-09 19:26:22,878 2982 INFO public.pgbench_history: Using node node1 as provider 2014-12-09 19:26:22,919 2982 INFO Provider: node1 (root) 2014-12-09 19:26:22,931 2982 INFO Locking public.pgbench_history 2014-12-09 19:26:22,932 2982 INFO Syncing public.pgbench_history 2014-12-09 19:26:25,963 2982 INFO Counting public.pgbench_history 2014-12-09 19:26:26,008 2982 INFO srcdb: 715 rows, checksum=9467587272 2014-12-09 19:26:26,020 2982 INFO dstdb: 715 rows, checksum=9467587272 2014-12-09 19:26:26,056 2982 INFO Checking if node1 can be used for copy 2014-12-09 19:26:26,063 2982 INFO Node node1 seems good source, using it 2014-12-09 19:26:26,064 2982 INFO public.pgbench_tellers: Using node node1 as provider 2014-12-09 19:26:26,100 2982 INFO Provider: node1 (root) 2014-12-09 19:26:26,108 2982 INFO Locking public.pgbench_tellers 2014-12-09 19:26:26,109 2982 INFO Syncing public.pgbench_tellers 2014-12-09 19:26:29,144 2982 INFO Counting public.pgbench_tellers 2014-12-09 19:26:29,176 2982 INFO srcdb: 20 rows, checksum=4814381032 2014-12-09 19:26:29,182 2982 INFO dstdb: 20 rows, checksum=4814381032 How it works... The following is an explanation of the steps performed in the preceding section: Initially, in step 1, we create two databases, that is node1 and node2, that are used as the source and target databases, respectively, from a replication perspective. In step 2, we populate the node1 database using the pgbench utility. In step 3 of the preceding section, we add and define the respective primary key and foreign key relationships on different tables and put these DDL commands in a .sql file. In step 4, we execute these DDL commands stated in step 3 on the node1 database; thus, in this way, we force the primary key and foreign key definitions on the tables in the pgbench schema in the node1 database. In step 5, we extract the table definitions from the tables in the pgbench schema in the node1 database and load these definitions in the node2 database. We will now discuss steps 6 to 8 of the preceding section. In step 6, we create the configuration file, which is then used in step 7 to create the root node for the node1 source database. In step 8, we will launch the worker daemon for the root node. Regarding the entries mentioned in the configuration file in step 6, we first define a job that must have a name, so that distinguished processes can be easily identified. Then, we define a connect string with information to connect to the source database, that is node1, and then we define the name of the replication queue involved. Finally, we define the location of the log and pid files. We will now discuss steps 9 to 11 of the preceding section. In step 9, we define the configuration file, which is then used in step 10 to create the leaf node for the target database, that is node2. In step 11, we launch the worker daemon for the leaf node. The entries in the configuration file in step 9 contain the job_name connect string in order to connect to the target database, that is node2, the name of the replication queue involved, and the location of log and pid involved. The key part in step 11 is played by the slave, that is the target database—to find the master or provider, that is source database node1. We will now talk about steps 12 and 13 of the preceding section. In step 12, we define the ticker configuration, with the help of which we launch the ticker process mentioned in step 13. Once the ticker daemon has started successfully, we have all the components and processes setup and needed for replication; however, we have not yet defined what the system needs to replicate. In step 14 and 15, we define the tables to the replication that is set on both the source and target databases, that is node1 and node2, respectively. Finally, we will talk about steps 16 and 17 of the preceding section. Here, at this stage, we are testing the replication that was set up between the node1 source database and the node2 target database. In step 16, we generate some traffic on the node1 source database by running pgbench with five parallel database connections and generating traffic for 10 seconds. In step 17, we check whether the tables on both the source and target databases have the same data. For this purpose, we use the compare command on the provider and subscriber nodes and then count and checksum the rows on both sides. A partial output from the preceding section tells you that the data has been successfully replicated between all the tables that are part of the replication set up between the node1 source database and the node2 destination database, as the count and checksum of rows for all the tables on the source and target destination databases are matching: 2014-12-09 19:26:18,975 2982 INFO Counting public.pgbench_accounts 2014-12-09 19:26:19,401 2982 INFO srcdb: 200000 rows, checksum=167607238449 2014-12-09 19:26:19,706 2982 INFO dstdb: 200000 rows, checksum=167607238449   2014-12-09 19:26:22,772 2982 INFO Counting public.pgbench_branches 2014-12-09 19:26:22,804 2982 INFO srcdb: 2 rows, checksum=-3078609798 2014-12-09 19:26:22,812 2982 INFO dstdb: 2 rows, checksum=-3078609798   2014-12-09 19:26:25,963 2982 INFO Counting public.pgbench_history 2014-12-09 19:26:26,008 2982 INFO srcdb: 715 rows, checksum=9467587272 2014-12-09 19:26:26,020 2982 INFO dstdb: 715 rows, checksum=9467587272   2014-12-09 19:26:29,144 2982 INFO Counting public.pgbench_tellers 2014-12-09 19:26:29,176 2982 INFO srcdb: 20 rows, checksum=4814381032 2014-12-09 19:26:29,182 2982 INFO dstdb: 20 rows, checksum=4814381032 Summary This article demonstrates the high availability and replication concepts in PostgreSQL. After reading this chapter, you will be able to implement high availability and replication options using different techniques including streaming replication, Slony-I replication and replication using Longdiste. Resources for Article: Further resources on this subject: Running a PostgreSQL Database Server [article] Securing the WAL Stream [article] Recursive queries [article]

In this article by Scott Gallagher, author of the book Mastering Docker – Second Edition, we will cover the places you store your containers, such as Docker Hub and Docker Hub Enterprises. We will also cover Docker Registry that you can use to run your own local storage for the Docker containers. We will review the differences between them all and when and how to use each of them. It will also cover how to set up automated builds using web hooks as well as the pieces that are all required to set them up. Lastly, we will run through an example of how to set up your own Docker Registry. Let's take a quick look at the topics we will be covering in this article: Docker Hub Docker Hub Enterprise Docker Registry Automated builds (For more resources related to this topic, see here.) Docker Hub In this section, we will focus on that Docker Hub, which is a free public option, but also has a private option that you can use to secure your images. We will focus on the web aspect of Docker Hub and the management you can do there. The login page is like the one shown in the following screenshot: Dashboard After logging into the Docker Hub, you will be taken to the following landing page. This page is known as the Dashboard of Docker Hub. From here, you can get to all the other sub pages of Docker Hub. In the upcoming sections, we will go through everything you see on the dashboard, starting with the dark blue bar you have on the top. Exploring the repositories page The following is the screenshot of the Explore link you see next to Dashboard at the top of the screen: As you can see in the screenshot, this is a link to show you all the official repositories that Docker has to offer. Official repositories are those that come directly from Docker or from the company responsible for the product. They are regularly updated and patched as needed. Organizations Organizations are those that you have either created or have been added to. Organizations allow you to layer on control, for say, a project that multiple people are collaborating on. The organization gets its own setting such as whether to store repositories as public or private by default, changing plans that will allow for different amounts of private repositories, and separate repositories all together from the ones you or others have. You can also access or switch between accounts or organizations from the Dashboard just below the Docker log, where you will typically see your username when you log in. This is a drop-down list, where you can switch between all the organizations you belong to. The Create menu The Create menu is the new item along the top bar of the Dashboard. From this drop-down menu, you can perform three actions: Create repository Create automated build Create organization A pictorial representation is shown in the following screenshot: The Settings Page Probably, the first section everyone jumps to once they have created an account on the Docker Hub—the Settings page. I know, that's what I did at least. The Account Settings page can be found under the drop-down menu that is accessed in the upper-right corner of the dashboard on selecting Settings. The page allows you to set up your public profile; change your password; see what organization you belong to, the subscriptions for e-mail updates you belong to, what specific notifications you would like to receive, what authorized services have access to your information, linked accounts (such as your GitHub or Bitbucket accounts); as well as your enterprise licenses, billing, and global settings. The only global setting as of now is the choice between having your repositories default to public or private upon creation. The default is to create them as public repositories. The Stars page Below the dark blue bar at the top of the Dashboard page are two more areas that are yet to be covered. The first, the Stars page, allows you to see what repositories you yourself have starred. This is very useful if you come across some repositories that you prefer to use and want to access them to see whether they have been updated recently or whether any other changes have occurred on these repositories. The second is a new setting in the new version of Docker Hub called Contributed. In this section, there will be a list of repositories you have contributed to outside of the ones within your Repositories list. Docker Hub Enterprise Docker Hub Enterprise, as it is currently known, will eventually be called Docker Subscription. We will focus on Docker Subscription, as it's the new and shiny piece. We will view the differences between Docker Hub and Docker Subscription (as we will call it moving forward) and view the options to deploy Docker Subscription. Let's first start off by comparing Docker Hub to Docker Subscription and see why each is unique and what purpose each serves: Docker Hub Shareable image, but it can be private No hassle of self-hosting Free (except for a certain number of private images) Docker Subscription Integrated into your authentication services (that is, AD/LDAP) Deployed on your own infrastructure (or cloud) Commercial support Docker Subscription for server Docker Subscription for server allows you to deploy both Docker Trusted Registry as well as Docker Engine on the infrastructure that you manage. Docker Trusted Registry is the location where you store the Docker images that you have created. You can set these up to be internal only or share them out publicly as well. Docker Subscription gives you all the benefits of running your own dedicated Docker hosted registry with the added benefits of getting support in case you need it. Docker Subscription for cloud As we saw in the previous section, we can also deploy Docker Subscription to a cloud provider if we wish. This allows us to leverage our existing cloud environments without having to roll our own server infrastructure up to host our Docker images. The setup is the same as we reviewed in the previous section; but this time, we will be targeting our existing cloud environment instead. Docker Registry In this section, we will be looking at Docker Registry. Docker Registry is an open source application that you can run anywhere you please and store your Docker image in. We will look at the comparison between Docker Registry and Docker Hub and how to choose among the two. By the end of the section, you will learn how to run your own Docker Registry and see whether it's a true fit for you. An overview of Docker Registry Docker Registry, as stated earlier, is an open source application that you can utilize to store your Docker images on a platform of your choice. This allows you to keep them 100% private if you wish or share them as needed. The registry can be found at https://docs.docker.com/registry/. This will run you through the setup and the steps to follow while pushing images to Docker Registry compared to Docker Hub. Docker Registry makes a lot of sense if you want to roll your own registry without having to pay for all the private features of Docker Hub. Next, let's take a look at some comparisons between Docker Hub and Docker Registry, so you can make an educated decision as to which platform to choose to store your images. Docker Registry will allow you to do the following: Host and manage your own registry from which you can serve all the repositories as private, public, or a mix between the two Scale the registry as needed based on how many images you host or how many pull requests you are serving out All are command-line-based for those that live on the command line Docker Hub will allow you to: Get a GUI-based interface that you can use to manage your images A location already set up on the cloud that is ready to handle public and/or private images Peace of mind of not having to manage a server that is hosting all your images Automated builds In this section, we will look at automated builds. Automated builds are those that you can link to your GitHub or Bitbucket account(s) and, as you update the code in your code repository, you can have the image automatically built on Docker Hub. We will look at all the pieces required to do so and, by the end, you'll be automating all your builds. Setting up your code The first step to create automated builds is to set up your GitHub or Bitbucket code. These are the two options you have while selecting where to store your code. For our example, I will be using GitHub; but the setup will be the same for GitHub and Bitbucket. First, we set up our GitHub code that contains just a simple README file that we will edit for our purpose. This file could be anything as far as a script or even multiple files that you want to manipulate for your automated builds. One key thing is that we can't just leave the README file alone. One key piece is that a Dockerfile is required to do the builds when you want it to for them to be automated. Next, we need to set up the link between our code and Docker Hub. Setting up Docker Hub On Docker Hub, we are going to use the Create drop-down menu and select Create Automated Build. After selecting it, we will be taken to a screen that will show you the accounts you have linked to either GitHub or Bitbucket. You then need to search and select the repository from either of the locations you want to create the automated build from. This will essentially create a web hook that when a commit is done on a selected code repository, then a new build will be created on Docker Hub. After you select the repository you would like to use, you will be taken to a screen similar to the following one: For the most part, the defaults will be used by most. You can select a different branch if you want to use one, say a testing branch if you use one before the code may go to the master branch. The one thing that will not be filled out, but is required, is the description field. You must enter something here or you will not be able to continue past this page. Upon clicking Create, you will be taken to a screen similar to the next screenshot: On this screen, you can see a lot of information on the automated build you have set up. Information such as tags, the Dockerfile in the code repository, build details, build settings, collaborators on the code, web hooks, and settings that include making the repository public or private and deleting the automated build repository as well. Putting all the pieces together So, let's take a run at doing a Docker automated build and see what happens when we have all the pieces in place and exactly what we have to do to kick off this automated build and be able to create our own magic: Update the code or any file inside your GitHub or Bitbucket repository. Upon committing the update, the automated build will be kicked off and logged in Docker Hub for that automated repository. Creating your own registry To create a registry of your own, use the following command: $ docker-machine create --driver vmwarefusion registry Creating SSH key... Creating VM... Starting registry... Waiting for VM to come online... To see how to connect Docker to this machine, run the following command: $ docker-machine env registry export DOCKER_TLS_VERIFY="1" export DOCKER_HOST="tcp://172.16.9.142:2376" export DOCKER_CERT_PATH="/Users/scottpgallagher/.docker/machine/machines/ registry" export DOCKER_MACHINE_NAME="registry" # Run this command to configure your shell: # eval "$(docker-machine env registry)" $ eval "$(docker-machine env registry)" $ docker pull registry $ docker run -p 5000:5000 -v <HOST_DIR>:/tmp/registry-dev registry:2 This will specify to use version 2 of the registry. For AWS (as shown in example from https://hub.docker.com/_/registry/): $ docker run -e SETTINGS_FLAVOR=s3 -e AWS_BUCKET=acme-docker -e STORAGE_PATH=/registry -e AWS_KEY=AKIAHSHB43HS3J92MXZ -e AWS_SECRET=xdDowwlK7TJajV1Y7EoOZrmuPEJlHYcNP2k4j49T -e SEARCH_BACKEND=sqlalchemy -p 5000:5000 registry:2 Again, this will use version 2 of the self-hosted registry. Then, you need to modify your Docker startups to point to the newly set up registry. Add the following line to the Docker startup in the /etc/init.d/docker file: -H tcp://127.0.0.1:2375 -H unix:///var/run/docker.sock --insecureregistry <REGISTRY_HOSTNAME>:5000 Most of these settings might already be there and you might only need to add --insecure-registry <REGISTRY_HOSTNAME>:5000: To access this file, you will need to use docker-machine: $ docker-machine ssh <docker-host_name> Now, you can pull a registry from the public Docker Hub as follows: $ docker pull debian Tag it, so when we do a push, it will go to the registry we set up: $ docker tag debian <REGISTRY_URL>:5000/debian Then, we can push it to our registry: $ docker push <REGISTRY_URL>:5000/debian We can also pull it for any future clients (or after any updates we have pushed for it): $ docker pull <REGISTRY_URL>:5000/debian Summary In this article, we dove deep into Docker Hub and also reviewed the new shiny Docker Subscription as well as the self-hosted Docker Registry. We have gone through the extensive review of each of them. You learned of the differences between them all and how to utilize each one. In this article, we also looked deep into setting up automated builds. We took a look at how to set up your own Docker Hub Registry. We have encompassed a lot in this chapter and I hope you have learned a lot and will like to put it all into good use. Resources for Article: Further resources on this subject: Docker in Production [article] Docker Hosts [article] Hands On with Docker Swarm [article]

(For more resources on JavaScript, see here.) The Error object An Error is a generic exception, and it accepts an optional message that provides details of the exception. We can use the Error object by using the following syntax: new Error(message); // message can be a string or an integer Here's an example that shows the Error object in action. The source code for this example can be found in the file error-object.html. <html><head><script type="text/javascript">function factorial(x) { if(x == 0) { return 1; } else { return x * factorial(x-1); } } try { var a = prompt("Please enter a positive integer", ""); if(a < 0){ var error = new Error(1); alert(error.message); alert(error.name); throw error; } else if(isNaN(a)){ var error = new Error("it must be a number"); alert(error.message); alert(error.name); throw error; } var f = factorial(a); alert(a + "! = " + f); } catch (error) { if(error.message == 1) { alert("value cannot be negative"); } else if(error.message == "it must be a number") { alert("value must be a number"); } else throw error; } finally { alert("ok, all is done!"); } </script> </head> <body> </body> </html> You may have noticed that the structure of this code is similar to the previous examples, in which we demonstrated try, catch, finally, and throw. In this example, we have made use of what we have learned, and instead of throwing the error directly, we have used the Error object. I need you to focus on the code given above. Notice that we have used an integer and a string as the message argument for var error, namely new Error(1) and new Error("it must be a number"). Take note that we can make use of alert() to create a pop-up window to inform the user of the error that has occurred and the name of the error, which is Error, as it is an Error object. Similarly, we can make use of the message property to create program logic for the appropriate error message. It is important to see how the Error object works, as the following built-in objects, which we are going to learn about, work similarly to how we have seen for the Error object. (We might be able to show how we can use these errors in the console log.) The RangeError object A RangeError occurs when a number is out of its appropriate range. The syntax is similar to what we have seen for the Error object. Here's the syntax for RangeError: new RangeError(message); message can either be a string or an integer. <html><head><script type="text/javascript">try { var anArray = new Array(-1); // an array length must be positive}catch (error) { alert(error.message); alert(error.name);}finally { alert("ok, all is done!");}</script></head><body></body></html> We'll start with a simple example to show how this works. Check out the following code that can be found in the source code folder, in the file rangeerror.html: When you run this example, you should see an alert window informing you that the array is of an invalid length. After this alert window, you should receive another alert window telling you that The error is RangeError, as this is a RangeError object. If you look at the code carefully, you will see that I have deliberately created this error by giving a negative value to the array's length (array's length must be positive). The ReferenceError object A ReferenceError occurs when a variable, object, function, or array that you have referenced does not exist. The syntax is similar to what you have seen so far and it is as follows: new ReferenceError(message); message can either be a string or an integer. As this is pretty straightforward, I'll dive right into the next example. The code for the following example can be found in the source code folder, in the file referenceerror.html. <html><head><script type="text/javascript">try { x = y; // notice that y is not defined // an array length must be positive}catch (error) { alert(error); alert(error.message); alert(error.name);}finally { alert("ok, all is done!");}</script></head><body></body></html> Take note that y is not defined, and we are expecting to catch this error in the catch block. Now try the previous example in your Firefox browser. You should receive four alert windows regarding the errors, with each window giving you a different message. The messages are as follows: ReferenceError: y is not defined y is not defined ReferenceError ok, all is done If you are using Internet Explorer, you will receive slightly different messages. You will see the following messages: [object Error] message y is undefined TypeError ok, all is done The TypeError object A TypeError is thrown when we try to access a value that is of the wrong type. The syntax is as follows: new TypeError(message); // message can be a string or an integer and it is optional An example of TypeError is as follows: <html><head><script type="text/javascript">try { y = 1 var test = function weird() { var foo = "weird string"; } y = test.foo(); // foo is not a function}catch (error) { alert(error); alert(error.message); alert(error.name);}finally { alert("ok, all is done!");}</script></head><body></body></html> If you try running this code in Firefox, you should receive an alert box stating that it is a TypeError. This is because test.foo() is not a function, and this results in a TypeError. JavaScript is capable of finding out what kind of error has been caught. Similarly, you can use the traditional method of throwing your own TypeError(), by uncommenting the code. The following built-in objects are less used, so we'll just move through quickly with the syntax of the built-in objects.

In this article by Raydelto Hernandez, the author of the book Building Android games with Cocos2d-x, we will talk about the Cocos2d-x game engine, which is widely used to create Android games. The launch of the Apple App Store back in 2008 leveraged the reach capacity of indie game developers who since its occurrence are able to reach millions of users and compete with large companies, outperforming them in some situations. This reality led the trend of creating reusable game engines, such as Cocos2d-iPhone, which is written natively using Objective-C by the Argentine iPhone developer, Ricardo Quesada. Cocos2d-iPhone allowed many independent developers to reach the top charts of downloads. (For more resources related to this topic, see here.) Picking an existing game engine is a smart choice for indies and large companies since it allows them to focus on the game logic rather than rewriting core features over and over again. Thus, there are many game engines out there with all kinds of licenses and characteristics. The most popular game engines for mobile systems right now are Unity, Marmalade, and Cocos2d-x; the three of them have the capabilities to create 2D and 3D games. Determining which one is the best in terms of ease of use and availability of tools may be debatable, but there is one objective fact, which we can mention that could be easily verified. Among these three engines, Cocos2d-x is the only one that you can use for free no matter how much money you make using it. We highlighted in this article's title that Cocos2d-x is completely free. This was emphasized because the other two frameworks also allow some free usage; nevertheless, both of these at some point require a payment for the usage license. In order to understand why Cocos2d-x is still free and open source, we need to understand how this tool was born. Ricardo, an enthusiastic Python programmer, often participated in game creation challenges that required participants to develop games from scratch within a week. Back in those days, Ricardo and his team rewrote the core engine for each game until they came up with the idea of creating a framework to encapsulate core game capabilities. These capabilities could be used on any two-dimensional game to make it open source, so contributions could be received worldwide. This is why Cocos2d was originally written for fun. With the launch of the first iPhone in 2007, Ricardo led the development of the port of the Cocos2d Python framework to the iPhone platform using its native language, Objective-C. Cocos2d-iPhone quickly became popular among indie game developers, some of them turning into Appillionaires, as Chris Stevens called these individuals and enterprises that made millions of dollars during the App Store bubble period. This phenomenon made game development companies look at this framework created by hobbyists as a tool to create their products. Zynga was one of the first big companies to adopt Cocos2d as their framework to deliver their famous Farmville game to iPhone in 2009. This company has been trading on NASDAQ since 2011 and has more than 2,000 employees. In July 2010, a C++ port of the Cocos2d iPhone called Cocos2d-x, was written in China with the objective of taking the power of this framework to other platforms, such as the Android operating system, which by that time was gaining market share at a spectacular rate. In 2011, this Cocos2d port was acquired by Chukong Technologies, the third largest mobile game development company in China, who later hired the original Cocos2d-IPhone author to join their team. Today, Cocos2d-x-based games dominate the top grossing charts of Google Play and the App Store, especially in Asia. Recognized companies and leading studios, such as Konami, Zynga, Bandai Namco, Wooga, Disney Mobile, and Square Enix are using Cocos2d-x in their games. Currently, there are 400,000 developers working on adding new functionalities and making this framework as stable as possible. These include engineers from Google, ARM, Intel, BlackBerry, and Microsoft who officially support the ports of their products, such as Windows Phone, Windows, Windows Metro Interface, and they're planning to support Cocos2d-x for the Xbox in this year. Cocos2d-x is a very straightforward engine that requires a little learning to grasp it. I teach game development courses at many universities using this framework; during the first week, the students are capable of creating a game with the complexity of the famous title Doodle Jump. This can be easily achieved because the framework provides us all the single components that are required for our game, such as physics, audio handling, collision detection, animation, networking, data storage, user input, map rendering, scene transitions, 3D rendering, particle systems rendering, font handling, menu creation, displaying forms, threads handling, and so on. This abstracts us from the low-level logic and allows us to focus on the game logic. Summary In conclusion, if you are willing to learn how to develop games for mobile platforms, I strongly recommend you to learn and use the Cocos2d-x framework because it is easy to use, is totally free, is an open source. This means that you can better understand it by reading its source, you could modify it if needed, and you have the warranty that you will never be forced to pay a license fee if your game becomes a hit. Another big advantage of this framework is its highly available documentation, including the Packt Publishing collection of Cocos2d-x game development books. Resources for Article: Further resources on this subject: Moving the Space Pod Using Touch [article] Why should I make cross-platform games? [article] Animations in Cocos2d-x [article]

One of the trending topics in the tech world this year is conversational interfaces. Everyone from big tech companies to start-ups are putting their bets on it. There is an explosion of virtual assistants and chat bots, and the number is increasing every week. What is conversational interface and why are companies investing in it? In this post we will learn about it and build a simple bot using the Facebook Messenger Platform. Conversational Interface A conversational interface is an interface that allows the users to interact with computers using natural language, instead of commands or clicks and taps. It can be voice-based like Siri, Ok Google, or Microsoft's Cortana, or it can be text-based like many Slack or Facebook Messenger bots. Instead of building apps for different platforms, device types, and sizes, companies can now provide a way to interact with their services using voice or text, without the need to download an app or learn how to use it. Facebook Messenger Bot Many apps like Slack, Telegraph, and Skype now offer platforms to build virtual assistants or bots. But with more than 1 billion users, Facebook Messenger is definitely the most popular choice. A Facebook Messenger bot works in the following way: The user sends a message to the bot using their Facebook Messenger account. The Facebook Messenger Platform will receive and post the message to the bot's server using a webhook. The bot server will receive the message, process it, and send a response back to the Facebook Messenger Platform using HTTP POST. Facebook will forward the message to the user. Let's build a simple Facebook Messenger Bot. Creating a Facebook App and Page First we need to create a Facebook Page and a Facebook app: Create a Facebook account, if you do not have one. Create a Facebook Page. This page will work as the identity of your bot. Create a Facebook App. Enter a display name and contact e-mail, and select a category for your bot. After creating the app, click on 'Add product' and then select the 'Messenger' option. Click on the 'Get started' button, and under the 'Token Generation' section, select the page you just created. This will generate a Page Access Token, which the bot server will need to communicate with the Facebook Messenger Platform. Copy this token for later use. You also need to set up webhook so that Facebook can forward the message to the bot server, but before that, let's build the server. Building the Bot Server You can use any language or framework to build the bot server. For this example, I am using Node.js. Let's start by creating an Express.js hello world app. Before we start, we need Node.js and npm installed. Follow the instructions on the Node.js website if you do not have these installed already: Create a new directory for your application, and inside of the app directory, create the package.json file by using the npm init command. Install Express and add it as a dependency using the npm install --save express command. Create a file called index.js with the following code: const app = require('express')(); app.set('port', (process.env.PORT || 5000)); app.get('/', function (req, res) { res.send('Helle World'); }); app.listen(app.get('port'), function () { console.log('App running on port', app.get('port')) }); Our hello world app is done. Run the node index.js command and open http://localhost:5000 in your browser. If all is good, you will see 'Hello World' text on the page. Now that we have the basic app set up, let's add the code to be able to communicate with the Facebook Messenger Platform: Install body-parser and request npm modules using the npm install --save body-parser request command. Body-parser is a middleware to parse incoming request bodies and request is an HTTP client. Now update index.js with the following code. Replace <fb-page-access-token> with the page access token generated before. You will need the validation token while setting up the webhook. Feel free to change it to anything you like: 'use strict' const app = require('express')(); const bodyParser = require('body-parser'); const request = require('request'); const VALIDATION_TOKEN = 'super_secret_validation_token'; const PAGE_ACCESS_TOKEN = '<fb-page-access-token>'; // replace <fb-page-access-token> with your page access token app.set('port', (process.env.PORT || 5000)); app.use(bodyParser.urlencoded({ extended: false })); app.use(bodyParser.json()); app.get('/', function (req, res) { res.send('Helle World'); }); app.get('/webhook', function (req, res) { if (req.query['hub.mode'] === 'subscribe' && req.query['hub.verify_token'] === VALIDATION_TOKEN) { res.status(200).send(req.query['hub.challenge']); } else { res.sendStatus(403); } }); app.post('/webhook/', function (req, res) { // Iterate over each entry req.body.entry.forEach(function (entry) { // Iterate over each messaging event entry.messaging.forEach(function (event) { let sender = event.sender.id if (event.message && event.message.text) { callSendAPI(sender, `Message received ${event.message.text}`); } }); res.sendStatus(200); // Respond with 200 to notify the message is received by the webhook successfully }); }); function callSendAPI(sender, text) { request({ url: 'https://graph.facebook.com/v2.6/me/messages', qs: { access_token: PAGE_ACCESS_TOKEN }, method: 'POST', json: { recipient: { id: sender }, message: { text: text }, } }, function (error, response, body) { if (error) { console.error('Unable to send message', error); } }); } app.listen(app.get('port'), function () { console.log('App running on port', app.get('port')) }); Here we have defined two new service endpoints. The first one responds to the GET request and will be used for verification when we set up the webhook. The second endpoint will respond to the POST request. This will process the incoming message and send a response using the callSendAPI method. For now, the server will simply send back the message received from the user. You can deploy the server anywhere you like. Because of its simplicity, I am going to use Heroku for this tutorial. Deploying the App to Heroku If you do not have an account, sign up on heroku.com and then install Heroku CLI. Initialize a git repository and commit the code to easily push it to Heroku. Use git init to initialise and git add . followed by git commit -m "Initial commit" to commit the code. Create a Procfile file with the following content. Procfile declares what commands to run on heroku. web: node index.js. Run the heroku create command to create and set up a new app on Heroku. Now you can run the app locally using the heroku local command. To deploy the app on Heroku, use git push heroku master and use heroku open to open the deployed app in the browser. Setting up the Messenger Webhook Now that the server is ready, we can set up the webhook: Go back to the app page; under the Messenger settings page, click on the 'Setup Webhooks' button. Enter the bot server webhook url (https:///webhook), and the validation token, and select all the options under subscription fields. We are ready to test our bot. Go to the Facebook page you created, click on message, and send a message to the bot. If everything is good, you will see a response from the bot. The bot is working, but we can update it to send a different response based on the user message. Let's create a simple bot for Game of Thrones fans, where if a user types in one of the great house's names, the bot will respond with their house words. Update index.js with the following code: 'use strict' const app = require('express')(); const bodyParser = require('body-parser'); const request = require('request'); const VALIDATION_TOKEN = 'super_secret_validation_token'; const PAGE_ACCESS_TOKEN = '<fb-page-access-token>'; // replace <fb-page-access-token> with your page access token const HOUSE_WORDS = { STARK: 'Winter is Coming', LANNISTER: 'Hear Me Roar!', BARATHEON: 'Ours is the Fury', TULLY: 'Family, Duty, Honor', GREYJOY: 'We Do Not Sow', MARTELL: 'Unbowed, Unbent, Unbroken', TYRELL: 'Growing Strong', FREY: 'We Stand Together', } app.set('port', (process.env.PORT || 5000)); app.use(bodyParser.urlencoded({ extended: false })); app.use(bodyParser.json()); app.get('/', function (req, res) { res.send('Helle World'); }); app.get('/webhook', function (req, res) { if (req.query['hub.mode'] === 'subscribe' && req.query['hub.verify_token'] === VALIDATION_TOKEN) { res.status(200).send(req.query['hub.challenge']); } else { res.sendStatus(403); } }); app.post('/webhook/', function (req, res) { // Iterate over each entry req.body.entry.forEach(function (entry) { // Iterate over each messaging event entry.messaging.forEach(function (event) { let sender = event.sender.id if (event.message && event.message.text) { callSendAPI(sender, getHouseWord(event.message.text)); } }); res.sendStatus(200); // Respond with 200 to notify the message is received by the webhook successfully }); }); function callSendAPI(sender, text) { request({ url: 'https://graph.facebook.com/v2.6/me/messages', qs: { access_token: PAGE_ACCESS_TOKEN }, method: 'POST', json: { recipient: { id: sender }, message: { text: text }, } }, function (error, response, body) { if (error) { console.error('Unable to send message', error); } }); } function getHouseWord(text) { const house = Object.keys(HOUSE_WORDS) .find(function (houseName) { return text.toUpperCase().indexOf(houseName) !== -1; }); return house ? HOUSE_WORDS[house] : 'No house word found :('; } app.listen(app.get('port'), function () { console.log('App running on port', app.get('port')) }); Now if you type one of the family names defined in our code, the bot will respond with the family words. So if you type 'stark', the bot will respond 'Winter is Coming'. App Review While you and other page admins can chat with the bot, to make it publicly available, you have to submit the bot for review. This can be done from the app page's 'App Review' option. Fill out the form, and once approved, your bot will be accessible by all Facebook Messenger users. Although our bot is working, it is not truly conversational. It only responds to a fix set of commands. To build a smarter bot, we need natural language processing. Natural language processors convert natural language text or audio to a format that computers can understand. There are many services that allow us to write smarter bot engines, and one of them is Wit. Wit was acquired by Facebook in 2015. You can read more about Wit and how to integrate it with your bot here. I hope you enjoyed this post. If you have built any bot on Facebook or any other platform, please share it with us in the comment section below. Author: Amit Kothari is a full-stack software developer based in Melbourne, Australia. He has 10+ years experience in designing and implementing software, mainly in Java/JEE. His recent experience is in building web applications using JavaScript frameworks like React and AngularJS, and backend micro services / REST API in Java. He is passionate about lean software development and continuous delivery.

In this article by Jason Kneen, the author of the book Appcelerator Titanium Smartphone Application Development Cookbook - Second Edition, we'll cover the following recipes: Reading data from remote XML via HTTPClient Displaying data using a TableView Enhancing your TableViews with custom rows Filtering your TableView with the SearchBar control Speeding up your remote data access with Yahoo! YQL and JSON Creating a SQLite database Saving data locally using a SQLite database Retrieving data from a SQLite database Creating a "pull to refresh" mechanism in iOS (For more resources related to this topic, see here.) As you are a Titanium developer, fully understanding the methods available for you to read, parse, and save data is fundamental to the success of the apps you'll build. Titanium provides you with all the tools you need to make everything from simple XML or JSON calls over HTTP, to the implementation of local relational SQL databases. In this article, we'll cover not only the fundamental methods of implementing remote data access over HTTP, but also how to store and present that data effectively using TableViews, TableRows, and other customized user interfaces. Prerequisites You should have a basic understanding of both the XML and JSON data formats, which are widely used and standardized methods of transporting data across the Web. Additionally, you should understand what Structured Query Language (SQL) is and how to create basic SQL statements such as Create, Select, Delete, and Insert. There is a great beginners' introduction to SQL at http://sqlzoo.net if you need to refer to tutorials on how to run common types of database queries. Reading data from remote XML via HTTPClient The ability to consume and display feed data from the Internet, via RSS feeds or alternate APIs, is the cornerstone of many mobile applications. More importantly, many services that you may wish to integrate into your app will probably require you to do this at some point or the other, so it is vital to understand and be able to implement remote data feeds and XML. Our first recipe in this article introduces some new functionality within Titanium to help facilitate this need. Getting ready To prepare for this recipe, open Titanium Studio, log in and create a new mobile project. Select Classic and Default Project. Then, enter MyRecipes as the name of the app, and fill in the rest of the details with your own information, as you've done previously. How to do it... Now that our project shell is set up, let's get down to business! First, open your app.js file and replace its contents with the following: // this sets the background color of the master View (when there are no windows/tab groups on it) Ti.UI.setBackgroundColor('#000'); // create tab group var tabGroup = Ti.UI.createTabGroup(); var tab1 = Ti.UI.createTab({ icon:'cake.png', title:'Recipes', window:win1 }); var tab2 = Ti.UI.createTab({ icon:'heart.png', title:'Favorites', window:win2 }); // // add tabs // tabGroup.addTab(tab1); tabGroup.addTab(tab2); // open tab group tabGroup.open(); This will get a basic TabGroup in place, but we need two windows, so we create two more JavaScript files called recipes.js and favorites.js. We'll be creating a Window instance in each file to do this we created the window2.js and chartwin.js files. In recipes.js, insert the following code. Do the same with favorites.js, ensuring that you change the title of the Window to Favorites: //create an instance of a window module.exports = (function() { var win = Ti.UI.createWindow({ title : 'Recipes', backgroundColor : '#fff' }); return win; })(); Next, go back to app.js, and just after the place where TabGroup is defined, add this code: var win1 = require("recipes"); var win2 = require("favorites"); Open the recipes.js file. This is the file that'll hold our code for retrieving and displaying recipes from an RSS feed. Type in the following code at the top of your recipes.js file; this code will create an HTTPClient and read in the feed XML from the recipe's website: //declare the http client object var xhr = Ti.Network.createHTTPClient(); function refresh() { //this method will process the remote data xhr.onload = function() { console.log(this.responseText); }; //this method will fire if there's an error in accessing the //remote data xhr.onerror = function() { //log the error to our Titanium Studio console console.log(this.status + ' - ' + this.statusText); }; //open up the recipes xml feed xhr.open('GET', 'http://rss.allrecipes.com/daily.aspx?hubID=79'); //finally, execute the call to the remote feed xhr.send(); } refresh(); Try running the emulator now for either Android or iPhone. You should see two tabs appear on the screen, as shown in the following screenshot. After a few seconds, there should be a stack of XML data printed to your Appcelerator Studio console log. How it works… If you are already familiar with JavaScript for the Web, this should make a lot of sense to you. Here, we created an HTTPClient using the Ti.Network namespace, and opened a GET connection to the URL of the feed from the recipe's website using an object called xhr. By implementing the onload event listener, we can capture the XML data that has been retrieved by the xhr object. In the source code, you'll notice that we have used console.log() to echo information to the Titanium Studio screen, which is a great way of debugging and following events in our app. If your connection and GET request were successful, you should see a large XML string output in the Titanium Studio console log. The final part of the recipe is small but very important—calling the xhr object's send() method. This kicks off the GET request; without it, your app would never load any data. It is important to note that you'll not receive any errors or warnings if you forget to implement xhr.send(), so if your app is not receiving any data, this is the first place to check. If you are having trouble parsing your XML, always check whether it is valid first! Opening the XML feed in your browser will normally provide you with enough information to determine whether your feed is valid or has broken elements. Displaying data using a TableView TableViews are one of the most commonly used components in Titanium. Almost all of the native apps on your device utilize tables in some shape or form. They are used to display large lists of data in an effective manner, allowing for scrolling lists that can be customized visually, searched through, or drilled down to expose child views. Titanium makes it easy to implement TableViews in your application, so in this recipe, we'll implement a TableView and use our XML data feed from the previous recipe to populate it with a list of recipes. How to do it... Once we have connected our app to a data feed and we're retrieving XML data via the XHR object, we need to be able to manipulate that data and display it in a TableView component. Firstly, we will need to create an array object called data at the top of our refresh function in the recipes.js file; this array will hold all of the information for our TableView in a global context. Then, we need to disseminate the XML, read in the required elements, and populate our data array object, before we finally create a TableView and set the data to be our data array. Replace the refresh function with the following code: function refresh() { var data = []; //empty data array //declare the http client object var xhr = Ti.Network.createHTTPClient(); //create the table view var tblRecipes = Ti.UI.createTableView(); win.add(tblRecipes); //this method will process the remote data xhr.onload = function() { var xml = this.responseXML; //get the item nodelist from our response xml object var items = xml.documentElement.getElementsByTagName("item"); //loop each item in the xml for (var i = 0; i < items.length; i++) { //create a table row var row = Ti.UI.createTableViewRow({ title: items.item(i).getElementsByTagName("title").item(0).text }); //add the table row to our data[] object data.push(row); } //end for loop //finally, set the data property of the tableView to our //data[] object tblRecipes.data = data; }; //open up the recipes xml feed xhr.open('GET', 'http://rss.allrecipes.com/daily.aspx?hubID=79'); //finally, execute the call to the remote feed xhr.send(); } The following screenshot shows the TableView with the titles of our recipes from the XML feed: How it works... The first thing you'll notice is that we are taking the response data, extracting all the elements that match the name item, and assigning it to items. This gives us an array that we can use to loop through and assign each individual item to the data array object that we created earlier. From there, we create our TableView by implementing the Ti.UI.createTableView() function. You should notice almost immediately that many of our regular properties are also used by tables, including width, height, and positioning. In this case, we did not specify these values, which means that by default, the TableView will occupy the screen. A TableView has an extra, and important, property—data. The data property accepts an array of data, the values of which can either be used dynamically (as we have done here with the title property) or be assigned to the subcomponent children of a TableRow. As you begin to build more complex applications, you'll be fully understanding just how flexible table-based layouts can be. Summary In this article, we covered fundamental methods of implementing remote data access over HTTP. As you are a Titanium developer, we had also understand the available methods to build a successful app. More importantly, many services that you may wish to integrate into your app will probably require you to do this at some point or the other, so it is vital to understand and be able to implement remote data feeds and XML Resources for Article: Further resources on this subject: Mobile First Bootstrap [article] Anatomy of a Sprite Kit project [article] Designing Objects for 3D Printing [article]

Let’s continue on from Part 1, and add a new table view to our app. In our storyboard, let’s add a table view controller by searching in the bottom right and dragging. Next, let’s add a button to our main view controller that will link to our new table view controller. Similar to what we did with the web view, Ctrl + click on this button and drag it to the newly created table view controller.Upon release, choose push. Now, let’s make sure everything works properly. Hit the large play button and click on Table View. You should now be taken to a blank table: Let’s populate this table with some text. Go to File ->  New ->  File  and choose a Cocoa Touch Class. Let’s call this file TableViewController, and make this a subclass of UITableViewController in the Swift language. Once the file is saved, we’ll be presented with a file with some boilerplate code.  On the first line in our class file, let’s declare a constant. This constant will be an array of strings that will be inserted into our table: let tableArray: NSArray = ["Apple", "Orange", "Banana", "Grape", "Kiwi"] Let’s modify the function that has this signature: func tableView(tableView: UITableView!, numberOfRowsInSection section: Int) -> Int This function returns the number of rows in our table view. Instead of setting this to zero, let’s change this to ten. Next, let’s uncomment the function that has this signature: override func numberOfSectionsInTableView(tableView: UITableView!) -> Int This function controls how many sections we will have in our table view. Let’s modify this function to return 1.  Finally, let’s add a function that will populate our cells: override func tableView(tableView: UITableView!, cellForRowAtIndexPath indexPath: NSIndexPath!) -> UITableViewCell! { let cell: UITableViewCell = UITableViewCell(style: UITableViewCellStyle.Subtitle, reuseIdentifier: "MyTestCell") cell.textLabel.text = tableArray.objectAtIndex(indexPath.row) as NSString return cell }  This function iterates through each row in our table and sets the text value to be equal to the fruits we declared at the top of the class file. The final file should look like this: class TableViewController: UITableViewController { let tableArray: NSArray = ["Apple", "Orange", "Banana", "Grape", "Kiwi"] override func viewDidLoad() { super.viewDidLoad() } override func didReceiveMemoryWarning() { super.didReceiveMemoryWarning() // Dispose of any resources that can be recreated. } // MARK: - Table view data source override func numberOfSectionsInTableView(tableView: UITableView!) -> Int { // #warning Potentially incomplete method implementation. // Return the number of sections. return 1 } override func tableView(tableView: UITableView!, numberOfRowsInSection section: Int) -> Int { // #warning Incomplete method implementation. // Return the number of rows in the section. return tableArray.count } override func tableView(tableView: UITableView!, cellForRowAtIndexPath indexPath: NSIndexPath!) -> UITableViewCell! { let cell: UITableViewCell = UITableViewCell(style: UITableViewCellStyle.Subtitle, reuseIdentifier: "MyTestCell") cell.textLabel.text = tableArray.objectAtIndex(indexPath.row) as NSString return cell } } Finally, we need to go back to our storyboard and link to our custom table view controller class. Select the storyboard, click on the table view controller, choose the identity inspector and fill in TableViewController  for the custom class. If we click the play button to build our project and then click on our table view button, we should see our table populated with names of fruit: Adding a map view Click on the Sample Swift App icon in the top left of the screen and then choose Build Phases. Under Link Binary with Libraries, click the plus button and search for MapKit. Once found, click Add: In the story board, add another view controller. Search for a MKMapView and drag it into the newly created controller. In the main navigation controller, create another button named Map View, Ctrl + click + drag to the newly created view controller, and upon release choose push: Additionally, choose the Map View in the storyboard, click on the connections inspector, Ctrl + click on delegate and drag to the map view controller. Next, let’s create a custom view controller that will control our map view. Go to File -> New -> File and choose Cocoa Touch. Let’s call this file MapViewController and inherit from UIViewController. Let’s now link our map view in our storyboard to our newly created map view controller file. In the storyboard, Ctrl + click on the map view and drag to our Map View Controller to create an IBOutlet variable. It should look something like this: @IBOutlet var mapView: MKMapView! Let’s add some code to our controller that will display the map around Apple’s campus in Cupertino, CA. I’ve looked up the GPS coordinates already, so here is what the completed code should look like: import UIKit import MapKit class MapViewController: UIViewController, MKMapViewDelegate { @IBOutlet var mapView: MKMapView! override func viewDidLoad() { super.viewDidLoad() let latitude:CLLocationDegrees = 37.331789 let longitude:CLLocationDegrees = -122.029620 let latitudeDelta:CLLocationDegrees = 0.01 let longitudeDelta:CLLocationDegrees = 0.01 let span:MKCoordinateSpan = MKCoordinateSpan(latitudeDelta: latitudeDelta, longitudeDelta: longitudeDelta) let location:CLLocationCoordinate2D = CLLocationCoordinate2DMake(latitude, longitude) let region: MKCoordinateRegion = MKCoordinateRegionMake(location, span) self.mapView.setRegion(region, animated: true) // Do any additional setup after loading the view. } override func didReceiveMemoryWarning() { super.didReceiveMemoryWarning() // Dispose of any resources that can be recreated. } } This should now build, and when you click on the Map View button, you should be able to see a map showing Apple’s campus at the center of the screen.  About the Author Ryan is a software engineer and electronic dance music producer currently residing in San Francisco, CA. Ryan started up as a biomedical engineer but fell in love with web/mobile programming after building his first Android app. You can find him on GitHub @rloomba.

Dive deeper into the world of AI innovation and stay ahead of the AI curve! Subscribe to our AI_Distilled newsletter for the latest insights. Don't miss out – sign up today!👋 Hello ,Welcome back to a new issue of AI_Distilled - your guide to the key advancements in AI, ML, NLP, and GenAI. Let's dive right into an industry expert’s perspective to sharpen our understanding of the field's rapid evolution. "In the near future, anyone who's online will be able to have a personal assistant powered by artificial intelligence that's far beyond today's technology."  - Bill Gates, Co-Founder, Microsoft. In a recent interview, Gates minced no words when he said software is still “pretty dumb” even in today’s day and age. The next 5 years will be crucial, he believes, as everything we know about computing in our personal and professional lives is on the brink of a massive disruption. Even everyday things as simple as phone calls are due for transformation as evident from Samsung unveiling the new 'Galaxy AI' and real-time translate call feature. In this issue, we’ll talk about Google exploring massive investment in AI startup Character.AI, Microsoft's GitHub Copilot user base surging to over a million, OpenAI launching data partnerships to enhance AI understanding, and Adobe researchers’ breakthrough AI that transforms 2D images into 3D models in 5 seconds.We’ve also got you your fresh dose of AI secret knowledge and tutorials. Explore how to scale multimodal understanding to long videos, navigate the landscape of hallucinations in LLMs, read a practical guide to enhancing RAG system responses, how to generate Synthetic Data for Machine Learning and unlock the power of low-code GPT AI apps.  📥 Feedback on the Weekly EditionWe've hit 6 months and 38K subscribers in our AI_Distilled newsletter journey — thanks to you!  The best part? Our emails are opened by 60% of recipients each week. We're dedicated to tailoring them to enhance your Data & AI practice. Let's work together to ensure they fully support your AI efforts and make a positive impact on your daily work.Share your thoughts in a quick 5-minute survey to shape our content. As a big thanks, get our bestselling "The Applied Artificial Intelligence Workshop" in PDF.  Let's make AI_Distilled even more awesome! 🚀 Jump on in! Complete the Survey. Get a Packt eBook for Free!Writer’s Credit: Special shout-out to Vidhu Jain for their valuable contribution to this week’s newsletter content!  Cheers,  Merlyn Shelley  Editor-in-Chief, Packt  SignUp | Advertise | Archives⚡ TechWave: AI/GPT News & Analysis🔳 Google Explores Massive Investment in AI Startup Character.AI: Google is reportedly in discussions to invest 'hundreds of millions' in Character.AI, an AI chatbot startup founded by ex-Google Brain employees. The investment is expected to deepen the collaboration between the two entities, leveraging Google's cloud services and Tensor Processing Units (TPUs) for model training. Character.AI, offering virtual interactions with celebrities and customizable chatbots, targets a youthful audience, particularly those aged 18 to 24, constituting 60% of its web traffic.  🔳 AI Actions Empowers AI Platforms with Zapier Integration: AI Actions introduces a tool enabling AI platforms to seamlessly run any Zapier action, leveraging Zapier's extensive repository of 20,000+ searches and actions. The integration allows natural language commands to trigger Zapier actions, eliminating obstacles like third-party app authentication and API integrations. Supported on platforms like ChatGPT, GPTs, Zapier, and customizable solutions, AI Actions provides flexibility for diverse applications. 🔳 Samsung Unveils 'Galaxy AI' and Real-Time Translate Call Feature: Samsung declares its commitment to AI with a preview of "Galaxy AI," a comprehensive mobile AI experience that combines on-device AI with cloud-based AI collaborations. The company introduced an upcoming feature, "AI Live Translate Call," embedded in its native phone app, offering real-time audio and text translations on the device during calls. Set to launch early next year, Galaxy AI is anticipated to debut with the Galaxy S24 lineup.  🔳 Google Expands Collaboration with Anthropic, Prioritizing AI Security and Cloud TPU v5e Accelerators: In an intensified partnership, Google announces its extended collaboration with Anthropic, focusing on elevated AI security and leveraging Cloud TPU v5e chips for AI inference. The collaboration, dating back to Anthropic's inception in 2021, highlights their joint efforts in AI safety and research. Anthropic, utilizing Google's Cloud services like GKE clusters, AlloyDB, and BigQuery, commits to Google Cloud's security services for model deployment. 🔳 Microsoft's GitHub Copilot User Base Surges to Over a Million, CEO Nadella Reports: Satya Nadella announced a substantial 40% growth in paying customers for GitHub Copilot in the September quarter, surpassing one million users across 37,000 organizations. Nadella highlights the rapid adoption of Copilot Chat, utilized by companies like Shopify, Maersk, and PWC, enhancing developers' productivity. The Bing search engine, integrated with OpenAI's ChatGPT, has facilitated over 1.9 billion chats, demonstrating a growing interest in AI-driven interactions. Microsoft's Azure revenue, including a significant contribution from AI services, exceeded expectations, reaching $24.3 billion, with the Azure business rising by 29%.  🔳 Dell and Hugging Face Join Forces to Streamline LLM Deployment: Dell and Hugging Face unveil a strategic partnership aimed at simplifying the deployment of LLMs for enterprises. With the burgeoning interest in generative AI, the collaboration seeks to address common concerns such as complexity, security, and privacy. The companies plan to establish a Dell portal on the Hugging Face platform, offering custom containers, scripts, and technical documentation for deploying open-source models on Dell servers.  🔳 OpenAI Launches Data Partnerships to Enhance AI Understanding: OpenAI introduces Data Partnerships, inviting collaborations with organizations to develop both public and private datasets for training AI models. The initiative aims to create comprehensive datasets reflecting diverse subject matters, industries, cultures, and languages, enhancing AI's understanding of the world. Two partnership options are available: Open-Source Archive for public datasets and Private Datasets for proprietary AI models, ensuring sensitivity and access controls based on partners' preferences. 🔳 Iterate Unveils AppCoder LLM for Effortless AI App Development: California-based Iterate introduces AppCoder LLM, a groundbreaking model embedded in the Interplay application development platform. This innovation allows enterprises to generate functional code for AI applications effortlessly by issuing natural language prompts. Unlike existing AI-driven coding solutions, AppCoder LLM, integrated into Iterate's platform, outperforms competitors, producing better outputs in terms of functional correctness and usefulness.  🔳 Adobe Researchers Unveil Breakthrough AI: Transform 2D Images into 3D Models in 5 Seconds: A collaborative effort between Adobe Research and Australian National University has resulted in a groundbreaking AI model capable of converting a single 2D image into a high-quality 3D model within a mere 5 seconds. The Large Reconstruction Model for Single Image to 3D (LRM) utilizes a transformer-based neural network architecture with over 500 million parameters, trained on approximately 1 million 3D objects. This innovation holds vast potential for industries like gaming, animation, industrial design, AR, and VR.  🔮 Expert Insights from Packt Community Synthetic Data for Machine Learning - By Abdulrahman Kerim Training ML models Developing an ML model usually requires performing the following essential steps: Collecting data. Annotating data. Designing an ML model. Training the model. Testing the model. These steps are depicted in the following diagram: Fig – Developing an ML model process. Now, let’s look at each of the steps in more detail to better understand how we can develop an ML model. Collecting and annotating data The first step in the process of developing an ML model is collecting the needed training data. You need to decide what training data is needed: Train using an existing dataset: In this case, there’s no need to collect training data. Thus, you can skip collecting and annotating data. However, you should make sure that your target task or domain is quite similar to the available dataset(s) you are planning to deploy. Otherwise, your model may train well on this dataset, but it will not perform well when tested on the new task or domain. Train on an existing dataset and fine-tune on a new dataset: This is the most popular case in today’s ML. You can pre-train your model on a large existing dataset and then fine-tune it on the new dataset. Regarding the new dataset, it does not need to be very large as you are already leveraging other existing dataset(s). For the dataset to be collected, you need to identify what the model needs to learn and how you are planning to implement this. After collecting the training data, you will begin the annotation process. Train from scratch on new data: In some contexts, your task or domain may be far from any available datasets. Thus, you will need to collect large-scale data. Collecting large-scale datasets is not simple. To do this, you need to identify what the model will learn and how you want it to do that. Making any modifications to the plan later may require you to recollect more data or even start the data collection process again from scratch. Following this, you need to decide what ground truth to extract, the budget, and the quality you want. This content is from the book “Synthetic Data for Machine Learning” written by Abdulrahman Kerim (Oct 2023). Start reading a free chapter or access the entire Packt digital library free for 7 days by signing up now. To learn more, click on the button below. Read through the Chapter 1 unlocked here...  🌟 Secret Knowledge: AI/LLM Resources🤖 Scaling Multimodal Understanding to Long Videos: A Comprehensive Guide: This guide provides a step-by-step explanation of the challenges associated with modeling diverse modalities like video, audio, and text. Learn about the Mirasol3B architecture, which efficiently handles longer videos, and understand the coordination between time-aligned and contextual modalities. The guide also introduces the Combiner, a learning module to effectively combine signals from video and audio information.  🤖 Mastering AI and ML Workloads: A Guide with Cloud HPC Toolkit: This post, authored by Google Cloud experts, delves into the convergence of HPC systems with AI and ML, highlighting their mutual benefits. They provide instructions on deploying clusters, utilizing preconfigured partitions, and utilizing powerful tools such as enroot and Pyxis for container integration. Discover the simplicity of deploying AI models on Google Cloud with the Cloud HPC Toolkit, fostering innovation and collaboration between HPC and AI communities. 🤖 Mastering the GPT Workflow: A Comprehensive Guide to AI Language Model: From understanding the basics of GPT's architecture and pre-training concept to unraveling the stages of the GPT workflow, including pre-training, fine-tuning, evaluation, and deployment, this guide provides a step-by-step walkthrough. Gain insights into ethical considerations, bias mitigation, and challenges associated with GPT models. Delve into future developments, including model scaling, multimodal capabilities, explainable AI enhancements, and improved context handling.  🤖 Navigating the Landscape of Hallucinations in LLMs: A Comprehensive Exploration: Delve into the intricate world of LLMs and the challenges posed by hallucinations in this in-depth blog post. Gain an understanding of the various types of hallucinations, ranging from harmless inaccuracies to potentially harmful fabrications, and their implications in real-world applications. Explore the root factors leading to hallucinations, such as overconfidence and lack of grounded reasoning, during LLM training.  🤖 Unveiling the Core Challenge in GenAI: Cornell University's Insightful Revelation: Cornell University researchers unveil a pivotal threat in GenAI, emphasizing the crucial role of "long-term memory" and the need for a vector database for contextual retrieval. Privacy issues emerge in seemingly secure solutions, shedding light on the complex challenges of handling non-numerical data in advanced AI models. 🔛 Masterclass: AI/LLM Tutorials👉 Unlocking the Power of Low-Code GPT AI Apps: A Comprehensive Guide. Explore how AINIRO.IO introduces the concept of "AI Apps" by seamlessly integrating ChatGPT with CRUD operations, enabling natural language interfaces to databases. Dive into the intricacies of creating a dynamic AI-based application without extensive coding, leveraging the Magic cloudlet to generate CRUD APIs effortlessly. Explore the significant implications of using ChatGPT for business logic in apps, offering endless possibilities for user interactions. 👉 Deploying LLMs Made Easy with ezsmdeploy 2.0 SDK: This post provides an in-depth understanding of the new capabilities, allowing users to effortlessly deploy foundation models like Llama 2, Falcon, and Stable Diffusion with just a few lines of code. The SDK automates instance selection, configuration of autoscaling, and other deployment details, streamlining the process of launching production-ready APIs. Whether deploying models from Hugging Face Hub or SageMaker Jumpstart, ezsmdeploy 2.0 reduces the coding effort required to integrate state-of-the-art models into production, making it a valuable tool for data scientists and developers. 👉 Enhancing RAG System Responses: A Practical Guide: Discover how to enhance the performance of your Retrieval-Augmented Generation (RAG) systems in generative AI applications by incorporating an interactive clarification component. This post offers a step-by-step guide on improving the quality of answers in RAG use cases where users present vague or ambiguous queries. Learn how to implement a solution using LangChain to engage in a conversational dialogue with users, prompting them for additional details to refine the context and provide accurate responses.  👉 Building Personalized ChatGPT: A Step-by-Step Guide. In this post, you'll learn how to explore OpenAI's GPT Builder, offering a beginner-friendly approach to customize ChatGPT for various applications. With the latest GPT update, users can now create personalized ChatGPT versions, even without technical expertise. The tutorial focuses on creating a customized GPT named 'EduBuddy,' designed to enhance the educational journey with tailored learning strategies and interactive features. 🚀 HackHub: Trending AI Tools💮 reworkd/tarsier: Open-source utility library for multimodal web agents, facilitating interaction with GPT-4(V) by visually tagging interactable elements on a page.  💮 recursal/ai-town-rwkv-proxy: Allows developers to locally run a large AI town using the RWKV model, a linear transformer with low inference costs. 💮 shiyoung77/ovir-3d: Enables open-vocabulary 3D instance retrieval without training on 3D data, addressing the challenge of obtaining diverse annotated 3D categories.  💮 langroid/langroid: User-friendly Python framework for building LLM-powered applications through a Multi-Agent paradigm. 💮 punica-ai/punica: Framework for Low Rank Adaptation (LoRA) to incorporate new knowledge into a pretrained LLM with minimal storage and memory impact. 

Have you ever wanted an easy way to connect your Arduino to your phone? The Blend Micro from RedBearLab is an Arduino-compatible development board that includes a Bluetooth Low-Energy (BLE) module for connecting with phones and computers. With BLE you can make a quick connection between your phone and Arduino to exchange simple messages like sensor readings or commands for the Arduino to execute. Blend Micro top and bottom showing Bluetooth module on the top side. ATMega32u4 (Arduino) microcontroller on the bottom and on-board antenna. The Blend Micro is a rather small development board - much smaller than a normal-sized Arduino. This makes it great for portable devices - just the kind we might like to connect to our phone! The larger Blend is available in the full-size Arduino format, if you have shields you’d like to connect. Bluetooth Low-Energy offers a lot of advantages over older versions of Bluetooth, particularly for battery-powered devices that aren’t always transmitting data. Recent iOS / Android devices and laptops with Bluetooth 4.0 should work (there’s a list of compatible devices on the Blend Micro page). I’ve been using the Blend Micro with my iPhone 5. Even on a breadboard it’s small enough to be portable. Coin cell battery pack on the right. Getting set up for development is unfortunately a bit complicated. To use the normal Arduino IDE you have to download an older version of Arduino (I’m using 1.0.6), install a few libraries, and patch a file inside the Arduino application itself (details here). Luckily that only has to be done once. A potentially easier way to get started is to use the online programming environment Codebender (quickstart instructions). One hitch with Codebender is you may need to manually press the reset button on the Blend Micro while programming (this isn’t required when programming using the normal Arduino IDE). If the Blend Micro is actively connected via Bluetooth, closing the connection on your phone or other device before programming seems to help. Once you’re set up for development programming the board is relatively straightforward. Blinking an LED from your Arduino is cool. How about blinking an LED on an Arduino, controlled by your phone? You can load the SimpleControls example onto your Blend Micro and the BLE Controller app onto your phone (iOS, Android). Connecting to the Blend Micro is simple - with the app open just tap "Scan" and your Blend Micro should be shown in the list of discovered devices. There’s no pairing step (required by previous Bluetooth versions) so connecting is easy. The BLE Controller app lets you control a few pins on the Arduino and receive data back, all without needing any more hardware on your phone. Pretty slick! Having the user interface to our device on our phone allows us to show a lot of information since there’s a lot of screen real estate. Since we already have our phone with us, why carry another screen for our portable device? I’m currently working on a wearable light logger that will record the intensity and color of the ambient light that people experience throughout their day. The wearable device is part of my Light Catchers project that collects our "light histories" into a public light sculpture. The wearable will have an Arduino-compatible micro-controller, RGB light sensor and data storage. For prototyping the wearable I’ve been using the Blend Micro to get a real-time view of the light sensor data on my phone so I can see how the sensor reacts to different lighting conditions. Sending live color readings under blue light. I started with the SimpleControls example and adapted it to send the RGB data from the light sensor. You can see the full code that runs on the Blend in my RGBLE.ino sketch. Sending the light sensor data was fairly straight forward. Let’s have a quick look at the code that’s needed to send data over BLE. Color display on the iPhone. RSSI is Bluetooth signal strength. Inside our setup function we can set the name of our BLE device. This name will show up when we scan for the device on our phone. Then we start the BLE library. void setup() { // ... // Set your BLE device name here, max. length 10 ble_set_name("RGB Sensor"); // Init. and start BLE library. ble_begin(); // ... } Inside our loop function, we can check if data is available, and read the bytes that were sent from the phone. void loop() { // ... // If data is ready while(ble_available()) { // read out command and data byte data0 = ble_read(); The RGB sensor that I’m using reads each color channel as a 10-bit value. Since the data won’t fit in an 8-bit byte the value is stored as two bytes. Sending a byte over the BLE connection is as simple as calling ble_write. I send each byte of the two-byte value separately using a little math with the shift operator (>>). I only take a reading and send the data if there is an active BLE connection. // Check if we’re connected if (ble_connected()) { // Take a light reading // ... // Send reading as bytes. ble_write(r >> 8); ble_write(r); At the end of our loop function the library needs to do some work to handle the Bluetooth data. // Allow BLE library to send/receive data ble_do_events(); } // end loop The app I run on my iPhone is a customized version of the Simple Controls sample app. My app shows the received color values on-screen. RedBearLab has sample code for various platforms available on the RedBearLab github page. For prototyping my device having an on-screen display with debugging controls is great. The small size of the Blend Micro makes it well suited for prototyping my wearable device. Range seems to be fairly limited (think inside a room rather than between rooms) but I haven’t done anything to optimize the antenna placement, so your mileage may vary. Color sensor prototype "in the field" on a sunny day at Tempelhofer Feld in Berlin. Battery life seems quite promising. I’m running my prototype off two 3V lithium coin cells and get several hours of life even before doing power optimization. Some Arduino boards have a power LED that’s always on while the board is powered. That LED might draw 20mA of current, which is a lot when you consider that good coin cells might provide 240mAh of current in the best case (typical datasheet). With the Blend Micro it’s easy to turn off all the onboard LEDs (see the RGBLE sketch for details). I measured the current consumption of my prototype around 14-16mA, with peaks around 20mA when starting the Bluetooth connection to my phone. It’s impressive to be sending data over the air using less power than you might use to light an LED! Accurately measuring the power consumption can be tricky since the radio transmissions can happen in short bursts. Probably the topic of another post! Other than some initial difficulty setting up the development environment programming with the Blend Micro is pretty smooth. Connecting your Arduino to your phone over a low power radio link opens up a lot of possibilities when you consider that your phone probably has a large touchscreen, cellular Internet connection, GPS and more. Once you try an Arduino that can wirelessly talk to your phone and computer, you always want it to do that! Resources RedBearLab Blend Micro RGBLE Arduino sketch (We Are) Light Catchers - wearable light logger About the Author Michael Ang is a Berlin-based artist and engineer working at the intersection of art, engineering and human experience. His works use technology to enhance our understanding of natural phenomena, modulate social interaction, and bridge the divide between the virtual and physical. His Light Catchers workshops and public installations will take place in Germany for the International Year of Light 2015.

In this article by Jérôme Jaglale, author of the book Spring Cookbook , we will cover the following recipes: Installing Java, Maven, Tomcat, and Eclipse on Mac OS Installing Java, Maven, Tomcat, and Eclipse on Ubuntu Installing Java, Maven, Tomcat, and Eclipse on Windows Creating a Spring web application Running a Spring web application Using Spring in a standard Java application (For more resources related to this topic, see here.) Introduction In this article, we will first cover the installation of some of the tools for Spring development: Java: Spring is a Java framework. Maven: This is a build tool similar to Ant. It makes it easy to add Spring libraries to a project. Gradle is another option as a build tool. Tomcat: This is a web server for Java web applications. You can also use JBoss, Jetty, GlassFish, or WebSphere. Eclipse: This is an IDE. You can also use NetBeans, IntelliJ IDEA, and so on. Then, we will build a Springweb application and run it with Tomcat. Finally, we'll see how Spring can also be used in a standard Java application (not a web application). Installing Java, Maven, Tomcat, and Eclipse on Mac OS We will first install Java 8 because it's not installed by default on Mac OS 10.9 or higher version. Then, we will install Maven 3, a build tool similar to Ant, to manage the external Java libraries that we will use (Spring, Hibernate, and so on). Maven 3 also compiles source files and generates JAR and WAR files. We will also install Tomcat 8, a popular web server for Java web applications, which we will use throughout this book. JBoss, Jetty, GlassFish, or WebSphere could be used instead. Finally, we will install the Eclipse IDE, but you could also use NetBeans, IntelliJ IDEA, and so on. How to do it… Install Java first, then Maven, Tomcat, and Eclipse. Installing Java Download Java from the Oracle website http://oracle.com. In the Java SE downloads section, choose the Java SE 8 SDK. Select Accept the License Agreement and download the Mac OS X x64 package. The direct link to the page is http://www.oracle.com/technetwork/java/javase/downloads/jdk8-downloads-2133151.html. Open the downloaded file, launch it, and complete the installation. In your ~/.bash_profile file, set the JAVA_HOME environment variable. Change jdk1.8.0_40.jdk to the actual folder name on your system (this depends on the version of Java you are using, which is updated regularly): export JAVA_HOME="/Library/Java/JavaVirtualMachines/ jdk1.8.0_40.jdk/Contents/Home" Open a new terminal and test whether it's working: $ java -versionjava version "1.8.0_40"Java(TM) SE Runtime Environment (build 1.8.0_40-b26)Java HotSpot(TM) 64-Bit Server VM (build 25.40-b25, mixed mode) Installing Maven Download Maven from the Apache website http://maven.apache.org/download.cgi. Choose the Binary zip file of the current stable version: Uncompress the downloaded file and move the extracted folder to a convenient location (for example, ~/bin). In your ~/.bash_profile file, add a MAVEN HOME environment variable pointing to that folder. For example: export MAVEN_HOME=~/bin/apache-maven-3.3.1 Add the bin subfolder to your PATH environment variable: export PATH=$PATH:$MAVEN_HOME/bin Open a new terminal and test whether it's working: $ mvn –vApache Maven 3.3.1 (12a6b3...Maven home: /Users/jerome/bin/apache-maven-3.3.1Java version: 1.8.0_40, vendor: Oracle CorporationJava home: /Library/Java/JavaVirtualMachines/jdk1.8.0_...Default locale: en_US, platform encoding: UTF-8OS name: "mac os x", version: "10.9.5", arch... … Installing Tomcat Download Tomcat from the Apache website http://tomcat.apache.org/download-80.cgi and choose the Core binary distribution. Uncompress the downloaded file and move the extracted folder to a convenient location (for example, ~/bin). Make the scripts in the bin subfolder executable: chmod +x bin/*.sh Launch Tomcat using the catalina.sh script: $ bin/catalina.sh runUsing CATALINA_BASE:   /Users/jerome/bin/apache-tomcat-7.0.54...INFO: Server startup in 852 ms Tomcat runs on the 8080 port by default. In a web browser, go to http://localhost:8080/ to check whether it's working. Installing Eclipse Download Eclipse from http://www.eclipse.org/downloads/. Choose the Mac OS X 64 Bit version of Eclipse IDE for Java EE Developers. Uncompress the downloaded file and move the extracted folder to a convenient location (for example, ~/bin). Launch Eclipse by executing the eclipse binary: ./eclipse There's more… Tomcat can be run as a background process using these two scripts: bin/startup.shbin/shutdown.sh On a development machine, it's convenient to put Tomcat's folder somewhere in the home directory (for example, ~/bin) so that its contents can be updated without root privileges. Installing Java, Maven, Tomcat, and Eclipse on Ubuntu We will first install Java 8. Then, we will install Maven 3, a build tool similar to Ant, to manage the external Java libraries that we will use (Spring, Hibernate, so on). Maven 3 also compiles source files and generates JAR and WAR files. We will also install Tomcat 8, a popular web server for Java web applications, which we will use throughout this book. JBoss, Jetty, GlassFish, or WebSphere could be used instead. Finally, we will install the EclipseIDE, but you could also use NetBeans, IntelliJ IDEA, and so on. How to do it… Install Java first, then Maven, Tomcat, and Eclipse. Installing Java Add this PPA (Personal Package Archive): sudo add-apt-repository -y ppa:webupd8team/java Refresh the list of the available packages: sudo apt-get update Download and install Java 8: sudo apt-get install –y oracle-java8-installer Test whether it's working: $ java -versionjava version "1.8.0_40"Java(TM) SE Runtime Environment (build 1.8.0_40-b25)...Java HotSpot(TM) 64-Bit Server VM (build 25.40-b25… Installing Maven Download Maven from the Apache website http://maven.apache.org/download.cgi. Choose the Binary zip file of the current stable version:   Uncompress the downloaded file and move the resulting folder to a convenient location (for example, ~/bin). In your ~/.bash_profile file, add a MAVEN HOME environment variable pointing to that folder. For example: export MAVEN_HOME=~/bin/apache-maven-3.3.1 Add the bin subfolder to your PATH environment variable: export PATH=$PATH:$MAVEN_HOME/bin Open a new terminal and test whether it's working: $ mvn –vApache Maven 3.3.1 (12a6b3...Maven home: /home/jerome/bin/apache-maven-3.3.1Java version: 1.8.0_40, vendor: Oracle Corporation... Installing Tomcat Download Tomcat from the Apache website http://tomcat.apache.org/download-80.cgi and choose the Core binary distribution.   Uncompress the downloaded file and move the extracted folder to a convenient location (for example, ~/bin). Make the scripts in the bin subfolder executable: chmod +x bin/*.sh Launch Tomcat using the catalina.sh script: $ bin/catalina.sh run Using CATALINA_BASE:   /Users/jerome/bin/apache-tomcat-7.0.54 ... INFO: Server startup in 852 ms Tomcat runs on the 8080 port by default. Go to http://localhost:8080/ to check whether it's working. Installing Eclipse Download Eclipse from http://www.eclipse.org/downloads/. Choose the Linux 64 Bit version of Eclipse IDE for Java EE Developers.   Uncompress the downloaded file and move the extracted folder to a convenient location (for example, ~/bin). Launch Eclipse by executing the eclipse binary: ./eclipse There's more… Tomcat can be run as a background process using these two scripts: bin/startup.sh bin/shutdown.sh On a development machine, it's convenient to put Tomcat's folder somewhere in the home directory (for example, ~/bin) so that its contents can be updated without root privileges. Installing Java, Maven, Tomcat, and Eclipse on Windows We will first install Java 8. Then, we will install Maven 3, a build tool similar to Ant, to manage the external Java libraries that we will use (Spring, Hibernate, and so on). Maven 3 also compiles source files and generates JAR and WAR files. We will also install Tomcat 8, a popular web server for Java web applications, which we will use throughout this book. JBoss, Jetty, GlassFish, or WebSphere could be used instead. Finally, we will install the Eclipse IDE, but you could also use NetBeans, IntelliJ IDEA, and so on. How to do it… Install Java first, then Maven, Tomcat, and Eclipse. Installing Java Download Java from the Oracle website http://oracle.com. In the Java SE downloads section, choose the Java SE 8 SDK. Select Accept the License Agreement and download the Windows x64 package. The direct link to the page is http://www.oracle.com/technetwork/java/javase/downloads/jdk8-downloads-2133151.html.   Open the downloaded file, launch it, and complete the installation. Navigate to Control Panel | System and Security | System | Advanced system settings | Environment Variables…. Add a JAVA_HOME system variable with the C:Program FilesJavajdk1.8.0_40 value. Change jdk1.8.0_40 to the actual folder name on your system (this depends on the version of Java, which is updated regularly). Test whether it's working by opening Command Prompt and entering java –version. Installing Maven Download Maven from the Apache website http://maven.apache.org/download.cgi. Choose the Binary zip file of the current stable version:   Uncompress the downloaded file. Create a Programs folder in your user folder. Move the extracted folder to it. Navigate to Control Panel | System and Security | System | Advanced system settings | Environment Variables…. Add a MAVEN_HOME system variable with the path to the Maven folder. For example, C:UsersjeromeProgramsapache-maven-3.2.1. Open the Path system variable. Append ;%MAVEN_HOME%bin to it.   Test whether it's working by opening a Command Prompt and entering mvn –v.   Installing Tomcat Download Tomcat from the Apache website http://tomcat.apache.org/download-80.cgi and choose the 32-bit/64-bit Windows Service Installer binary distribution.   Launch and complete the installation. Tomcat runs on the 8080 port by default. Go to http://localhost:8080/ to check whether it's working. Installing Eclipse Download Eclipse from http://www.eclipse.org/downloads/. Choose the Windows 64 Bit version of Eclipse IDE for Java EE Developers.   Uncompress the downloaded file. Launch the eclipse program. Creating a Spring web application In this recipe, we will build a simple Spring web application with Eclipse. We will: Create a new Maven project Add Spring to it Add two Java classes to configure Spring Create a "Hello World" web page In the next recipe, we will compile and run this web application. How to do it… In this section, we will create a Spring web application in Eclipse. Creating a new Maven project in Eclipse In Eclipse, in the File menu, select New | Project…. Under Maven, select Maven Project and click on Next >. Select the Create a simple project (skip archetype selection) checkbox and click on Next >. For the Group Id field, enter com.springcookbook. For the Artifact Id field, enter springwebapp. For Packaging, select war and click on Finish. Adding Spring to the project using Maven Open Maven's pom.xml configuration file at the root of the project. Select the pom.xml tab to edit the XML source code directly. Under the project XML node, define the versions for Java and Spring. Also add the Servlet API, Spring Core, and Spring MVC dependencies: <properties> <java.version>1.8</java.version> <spring.version>4.1.5.RELEASE</spring.version> </properties>   <dependencies> <!-- Servlet API --> <dependency>    <groupId>javax.servlet</groupId>    <artifactId>javax.servlet-api</artifactId>    <version>3.1.0</version>    <scope>provided</scope> </dependency>   <!-- Spring Core --> <dependency>    <groupId>org.springframework</groupId>    <artifactId>spring-context</artifactId>    <version>${spring.version}</version> </dependency>   <!-- Spring MVC --> <dependency>    <groupId>org.springframework</groupId>    <artifactId>spring-webmvc</artifactId>    <version>${spring.version}</version> </dependency> </dependencies> Creating the configuration classes for Spring Create the Java packages com.springcookbook.config and com.springcookbook.controller; in the left-hand side pane Package Explorer, right-click on the project folder and select New | Package…. In the com.springcookbook.config package, create the AppConfig class. In the Source menu, select Organize Imports to add the needed import declarations: package com.springcookbook.config; @Configuration @EnableWebMvc @ComponentScan (basePackages = {"com.springcookbook.controller"}) public class AppConfig { } Still in the com.springcookbook.config package, create the ServletInitializer class. Add the needed import declarations similarly: package com.springcookbook.config;   public class ServletInitializer extends AbstractAnnotationConfigDispatcherServletInitializer {    @Override    protected Class<?>[] getRootConfigClasses() {        return new Class<?>[0];    }       @Override    protected Class<?>[] getServletConfigClasses() {        return new Class<?>[]{AppConfig.class};    }      @Override    protected String[] getServletMappings() {        return new String[]{"/"};    } } Creating a "Hello World" web page In the com.springcookbook.controller package, create the HelloController class and its hi() method: @Controller public class HelloController { @RequestMapping("hi") @ResponseBody public String hi() {      return "Hello, world."; } } How it works… This section will give more you details of what happened at every step. Creating a new Maven project in Eclipse The generated Maven project is a pom.xml configuration file along with a hierarchy of empty directories: pom.xml src |- main    |- java    |- resources    |- webapp |- test    |- java    |- resources Adding Spring to the project using Maven The declared Maven libraries and their dependencies are automatically downloaded in the background by Eclipse. They are listed under Maven Dependencies in the left-hand side pane Package Explorer. Tomcat provides the Servlet API dependency, but we still declared it because our code needs it to compile. Maven will not include it in the generated .war file because of the <scope>provided</scope> declaration. Creating the configuration classes for Spring AppConfig is a Spring configuration class. It is a standard Java class annotated with: @Configuration: This declares it as a Spring configuration class @EnableWebMvc: This enables Spring's ability to receive and process web requests @ComponentScan(basePackages = {"com.springcookbook.controller"}): This scans the com.springcookbook.controller package for Spring components ServletInitializer is a configuration class for Spring's servlet; it replaces the standard web.xml file. It will be detected automatically by SpringServletContainerInitializer, which is automatically called by any Servlet 3. ServletInitializer extends the AbstractAnnotationConfigDispatcherServletInitializer abstract class and implements the required methods: getServletMappings(): This declares the servlet root URI. getServletConfigClasses(): This declares the Spring configuration classes. Here, we declared the AppConfig class that was previously defined. Creating a "Hello World" web page We created a controller class in the com.springcookbook.controller package, which we declared in AppConfig. When navigating to http://localhost:8080/hi, the hi()method will be called and Hello, world. will be displayed in the browser. Running a Spring web application In this recipe, we will use the Spring web application from the previous recipe. We will compile it with Maven and run it with Tomcat. How to do it… Here are the steps to compile and run a Spring web application: In pom.xml, add this boilerplate code under the project XML node. It will allow Maven to generate .war files without requiring a web.xml file: <build>    <finalName>springwebapp</finalName> <plugins>    <plugin>      <groupId>org.apache.maven.plugins</groupId>      <artifactId>maven-war-plugin</artifactId>      <version>2.5</version>      <configuration>       <failOnMissingWebXml>false</failOnMissingWebXml>      </configuration>    </plugin> </plugins> </build> In Eclipse, in the left-hand side pane Package Explorer, select the springwebapp project folder. In the Run menu, select Run and choose Maven install or you can execute mvn clean install in a terminal at the root of the project folder. In both cases, a target folder will be generated with the springwebapp.war file in it. Copy the target/springwebapp.war file to Tomcat's webapps folder. Launch Tomcat. In a web browser, go to http://localhost:8080/springwebapp/hi to check whether it's working.   How it works… In pom.xml the boilerplate code prevents Maven from throwing an error because there's no web.xml file. A web.xml file was required in Java web applications; however, since Servlet specification 3.0 (implemented in Tomcat 7 and higher versions), it's not required anymore. There's more… On Mac OS and Linux, you can create a symbolic link in Tomcat's webapps folder pointing to the.war file in your project folder. For example: ln -s ~/eclipse_workspace/spring_webapp/target/springwebapp.war ~/bin/apache-tomcat/webapps/springwebapp.war So, when the.war file is updated in your project folder, Tomcat will detect that it has been modified and will reload the application automatically. Using Spring in a standard Java application In this recipe, we will build a standard Java application (not a web application) using Spring. We will: Create a new Maven project Add Spring to it Add a class to configure Spring Add a User class Define a User singleton in the Spring configuration class Use the User singleton in the main() method How to do it… In this section, we will cover the steps to use Spring in a standard (not web) Java application. Creating a new Maven project in Eclipse In Eclipse, in the File menu, select New | Project.... Under Maven, select Maven Project and click on Next >. Select the Create a simple project (skip archetype selection) checkbox and click on Next >. For the Group Id field, enter com.springcookbook. For the Artifact Id field, enter springapp. Click on Finish. Adding Spring to the project using Maven Open Maven's pom.xml configuration file at the root of the project. Select the pom.xml tab to edit the XML source code directly. Under the project XML node, define the Java and Spring versions and add the Spring Core dependency: <properties> <java.version>1.8</java.version> <spring.version>4.1.5.RELEASE</spring.version> </properties>   <dependencies> <!-- Spring Core --> <dependency>    <groupId>org.springframework</groupId>    <artifactId>spring-context</artifactId>    <version>${spring.version}</version> </dependency> </dependencies> Creating a configuration class for Spring Create the com.springcookbook.config Java package; in the left-hand side pane Package Explorer, right-click on the project and select New | Package…. In the com.springcookbook.config package, create the AppConfig class. In the Source menu, select Organize Imports to add the needed import declarations: @Configuration public class AppConfig { } Creating the User class Create a User Java class with two String fields: public class User { private String name; private String skill; public String getName() {    return name; } public void setName(String name) {  this.name = name; } public String getSkill() {    return skill; } public void setSkill(String skill) {    this.skill = skill; } } Defining a User singleton in the Spring configuration class In the AppConfig class, define a User bean: @Bean public User admin(){    User u = new User();    u.setName("Merlin");    u.setSkill("Magic");    return u; } Using the User singleton in the main() method Create the com.springcookbook.main package with the Main class containing the main() method: package com.springcookbook.main; public class Main { public static void main(String[] args) { } } In the main() method, retrieve the User singleton and print its properties: AnnotationConfigApplicationContext springContext = new AnnotationConfigApplicationContext(AppConfig.class);   User admin = (User) springContext.getBean("admin");   System.out.println("admin name: " + admin.getName()); System.out.println("admin skill: " + admin.getSkill());   springContext.close(); Test whether it's working; in the Run menu, select Run.   How it works... We created a Java project to which we added Spring. We defined a User bean called admin (the bean name is by default the bean method name). In the Main class, we created a Spring context object from the AppConfig class and retrieved the admin bean from it. We used the bean and finally, closed the Spring context. Summary In this article, we have learned how to install some of the tools for Spring development. Then, we learned how to build a Springweb application and run it with Tomcat. Finally, we saw how Spring can also be used in a standard Java application.