Besides its core content management functionality, Drupal can also feed content into its framework from other web applications, including Flickr, Twitter, Google, Amazon, Facebook, Mollom, and many more. This communication between Drupal and other web portals is what makes Drupal a feature-rich content management framework capable of supporting multiple methods of feeding content into its database and site structure. For example, as a Drupal developer, you can feed content into your Drupal site using aggregation or RSS feeds. The Drupal FeedAPI (Application Programming Interface) module allows you to take RSS or XML URLs from external websites and add these feeds to your Drupal site. This is one robust method of getting content from other web applications and sites.
How do we take content from all of these different web applications and share the content with a Drupal site? This is becoming highly important now due to the wealth of rich content management applications that are both on the market and also available in the open source community. For example, how can we take all of the images we upload to our Flickr site and share those images with users on our Drupal site? In this book, we'll look in detail at the Drupal Services module, a contributed module that helps you to speed up your connections to web services. This module will allow us to integrate your Drupal site with external applications by using interfaces, such as XMLRPC, JSON, JSON-RPC, REST, SOAP, and AMF. These interfaces will allow your Drupal site to interact with and provide web services.
In this chapter, you will learn the basics of web services and Drupal, including:
What are web services and why are web services useful?
Why do we use web services in Drupal?
How does Drupal 6 use web services?
Standards compliance when using web services in Drupal
Drupal as a service consumer and as a service provider
Let's begin our discussion of what web services are and how they work with Drupal. To get started, we need to define some of the larger concepts and the Drupal concepts that we'll be talking about.
In order for our Drupal site to communicate and interact with other web applications, such as Flickr, Amazon, Mollom, or Twitter, we need to use standard communication protocols in the web development world called web services. Web service protocols will allow applications that reside on external websites and servers to interact and communicate with our Drupal website that is running on our own server. Web services will also allow our Drupal site to pass along content and data to external web applications existing on remote servers.
When we define web services, we need to point out that this type of communication provides for interoperability. This means that a web service communication can happen between two completely different environments but still work because we use standard protocols to make it happen.
Web services allow us to call another application on a remote server. A good analogy to this is using your cell phone to call a friend or colleague. You have one type of cell phone using one type of cell phone service. You call your colleague's cell phone. They have another type of cell with a different service provider, but the call goes through and is successful because the two services communicate with each other using standard cell phone communication protocols.
The web service call happens through coded protocols that are translated into a language and standard protocol that both computers can understand. Generally, this is done by using the XML language to translate the programmed request into the other external applications. Web applications have a standard in which they can usually read XML files. XML is a text-based format, so nearly all computer systems and applications can work with the XML format.
The web services protocol also uses a concept called remoting or Remote Procedure Calling (RPC) that allows one application to initiate or "call" a function inside of an application on a remote server. Our Drupal site can communicate with an application on a remote server and call a function in that application. For example, we might want to make our Drupal website call and interact with our Flickr photo gallery, or we may want to take all of our Drupal home page content and push it over to our Facebook account. We can do both of these actions using the web service protocols.
The computer that contains the application â that we will communicate with â can be anywhere in the world. It could be sitting on a server in the US, Europe, Asia, South America, or somewhere else.
As mentioned above, the base foundation for web services is a protocol or code called XML. For our Drupal site residing on our server, to talk and interact with a website or application on another server, we need to use XML, which is a language commonly understood between different applications. Our site and server understands XML as does the application we want to communicate with. We can do this over the standard
http protocol for website communication, as
http is the most standard protocol for Internet communication. The reason we use XML for communication between the applications and the sites is because XML replaces the proprietary function (whether the function is in RPC or another programming language or interface) and formats it into the standard XML code format. This allows applications to understand each other easily.
An analogy to this is: if we have two people, one from Germany and the other from France, speaking to one another, and neither person knows the other's language but both of them know English, then they must speak in English, as it's a language they both understand and can easily communicate in. It's a similar situation when XML is used to translate a web service's function into a commonly understood format.
So first we need to send the function call to a remote application. Our calling application or website creates the XML document that will represent the programmed function we want to execute. The XML is then transmitted over HTTP to the remote application and it can then be interpreted and understood by the remote application. The remote application then executes the function based on the XML formatting.
Here's a diagram that outlines a simple web service request and response. This is a request sent from our Drupal site (client) over
http to an external server to request data. The data exists on the server (remote) in the form of a URI (Uniform Resource Identifier) item. The response is sent back to our Drupal site through XML.
Let's look briefly at one web service protocol and technology, and define it. As mentioned before, there are many technologies you can use to implement web services. REST (Representational State Transfer) is one such technology. The reason REST is a preferred technology within the web development and Drupal community is due to its flexibility and standards. REST allows us to do the following when we initiate a web service using its protocol:
Use a standard method such as XML as our message format
Send the message over standard protocol such as HTTP
Provide or connect to specific resources where each resource (image, document, page, and node) is given a unique resource identifier (a URI)
We can take this concept and try it out on our Drupal site by writing some PHP code that makes an HTTP request to another web application resource. For example, we may want to make a call to a Picasa photo gallery and feed a select number and type of photos back to our Drupal site and display the photos in a Drupal node on our site. The request targets this specific resource by making a
GET request to the URI of the resource. The application we are communicating with sends a response back to us in XML format. That XML can then be integrated into our Drupal site using a module, for example. The request might be made to a user's Flickr or Picasa photo gallery. The request gets returned to our Drupal site as XML and we parse this XML into our Drupal site and the user's photos or gallery then get displayed on our site.ï»¿ï»¿
This is just one protocol example. We'll discuss in detail about the other protocols in the later chapters.
Greg Hines of pingVision provides a good introductory resource on REST and Drupal in the document titled RESTful Web Services and Drupal. The document is available on pingVision's website as a PDF download from: http://pingvision.com/files/restful_web_services_and_drupal.pdf
As discussed in the REST protocol's example, web services and Drupal's use of web services follow specific standards. In order to maintain as much interoperability and flexibility as possible, all of the protocols used respond for the most part using XML as the standard response mechanism and format.
Additionally, all the communication between services, in our example between a client and a server, happens over HTTP (the standard web protocol). This is a uniform protocol that is used for transport and communication of the service. All transports take place uniformly using
GET, POST, PUT, and
DELETE requests, for example.
The HTTP requests are stateless, meaning that the request over HTTP happens once at one given moment and is isolated from all other activated requests. So the request stands alone. If it succeeds, it gets a response. If it fails, it gets no response from the server or application it's communicating with. The request can be repeated an infinite number of times.
Finally, all of the resources we try and access are those that we are sending to another application using a unique resource identifier (URI) to identify and define what they are. So images on our site have unique identifiers as well as those residing in another web application. Each of these unique identifiers allows for addresses or locations for each node or file in question. So each resource in a web service's communication has an address. Each resource has one URI and each address has one URI. Some examples of this would be the following locations on my Drupal site:
Another reason we want to be standards compliant, when writing or working with web services, is for simplicity. We do not need any special tools to program web services as long as we follow these standards. We can use the web application modules and PHP, and stick to these coding standards and protocols.ï»¿
Web services are useful for a number of reasons, especially when it comes to Drupal and Drupal's relationship and interaction with other web content management systems and applications. The web has a huge number of web applications, so web developers and content developers can pass their content to the web browsers and make it available to the web visitors. This is why the Internet is useful to us. We can go to a website and view the content. Whenever we do that, we're looking at content that is proprietary to a specific web application. In Drupal, our content is in the form of nodes, for example. We may want to share these nodes with other websites that are non-Drupal, such as a Wordpress-powered site.
Web services are useful because they present us with an architecture where a resource on a site (an image, textual content, such as a node ID or block ID, a video or audio file) is given a unique identifier. For example, in Drupal, every node has an ID. Every file you upload to a Drupal site also has a unique path to it.
This is extremely useful since all applications share this common semantic standard. We name things similarly on all of our web applications. We can then leverage this by writing code in PHP, for example, the one that calls these resources. The application server that houses the resource then responds to our request using an XML document.
With web services, we can take our Drupal content and share this content with other web applications and, essentially, with the web at large. Our content is no longer just our content and it is not specific to our Drupal website. It can be shared and integrated. Drupal's codebase is PHP-based and many of the popular web applications being used today, including Wordpress, Joomla!, and Flickr, are also PHP-based. So we have a common programming language we can work with and use to integrate these applications.
Here are some concrete examples. Perhaps your Human Resources Department wants to integrate its job postings and applications with another web application such as Monster.com. Web services can allow this to happen. Your office's payroll department may want to connect to its bank account in order to pass data from the payroll reports over to its bank reporting mechanism. Web services can allow this to happen. You may want to take all of the photos you upload to your Drupal site in image galleries built with the Views module, and take these photos and send them to Flickr so that they automatically show up in your Flickr account or on Flickr's public site. Web services can make this happen.
This leads to another advantage of using web services with Drupal and why we would choose to use Drupal in the first place. Instead of having to upload our photos twice â once to our Drupal site and then repeating the procedure to our Flickr site â web services allows us to upload the images to our Drupal site once and then automatically send that data over to Flickr without having to upload one (or even a batch of images) again. It saves us time and speeds up the entire process of generating web-based content.
Additionally, there may be applications we want to use in our Drupal site, for example applications where we want to consume content without having to code again. We can just reuse these applications using the web services protocols and get this application content into our Drupal site. So we can consume web services. Examples of this would be converting currency on our site, feeding weather reports and other weather data into our site, feeding natural disaster scientific data into our site from services that provide it, feeding language translation services, feeding music podcasts, and more. Instead of having to reprogram this type of content, we can grab it from another web application and show it automatically on our site using web services.
Simply put, this opens up a method of easily sharing data and content resources between applications and machines that are running on different platforms and architecture. We have opened up a gold mine of capabilities here because we can talk to applications that run different software from our Drupal site and on different computing platforms.
Drupal can use web services following any of the protocols mentioned earlier, including XML-RPC, REST, and SOAP. Drupal can consume web services by requesting data from other web applications using RSS and XML-formatted requests. As a web developer, you can write your own service code in Drupal using PHP. You can also use the Services module as well as other service-specific contributed modules to create these web service requests. In this next section, we're going to look at both these examples. First, we'll see how Drupal works as a service consumer, where basically it is a client requesting data from an external server.
We'll also look at how Drupal can provide services using the Services module, RSS, AMFPHP, and XML-RPC. All of these protocols will be explained in detail in the later chapters.
Let's outline some brief examples of how Drupal consumes content and data from other web applications, including Mollom, Flickr, and Facebook. We're going to look at these applications in more detail later in the book, but we'll introduce them here and show some basic examples.
You can configure your Drupal site to consume various web services by using contributed Drupal modules for each specific task or application you are trying to consume. Drupal can consume services from applications that will help your website prevent spam, integrate photos, integrate taxonomy and tags, and enhance your Drupal free tagging and autotagging abilities, and integrate with applications such as Facebook and Twitter.
Mollom is a web service that will help you to block spam on your Drupal site. It's a separate application that runs as a web service. Drupal can connect to the Mollom web service through a contributed module called Mollom. The contributed module project page is available at http://drupal.org/project/mollom
Mollom will offer you CAPTCHA options for your Drupal site as well as prevent and block comment spam and Drupal node form spam, including any spam that might populate your nodes through content type forms, story, page, and forum forms. It will prevent user registration from being compromised and prevent fake users from signing up on your site.
The Mollom project was developed and is maintained by Drupal's founder, Dries Buytaert, and a team of developers very familiar with Drupal, so the integration between the two applications is seamless. Mollom is included in the Acquia Drupal packaged installation, so if you use Acquia Drupal you will already have the Mollom module and service integrated into your Drupal site. If you run a Drupal 6 installation independent of the Acquia package, you'll need to install the Mollom contributed module to make the service interaction work. You can read the entire Mollom client API documentation on the Mollom website at http://mollom.com/files/mollom-client-api.pdf.
The API documentation provides a huge amount of detail on how the service works, but simply put, it uses the XML-RPC interface. So as explained earlier in this chapter, Mollom uses Remote Procedure Call protocol, which itself uses XML to encode calls as its service mechanism. The Mollom API notes that any XML-RPC call to its service should follow the HTTP/1.0 standard. The documentation also mentions that any client (our Drupal site in our case) that makes a RPC call to the Mollom service needs to only make these requests from valid Mollom servers, and using a valid public and private key encryption for the specific website making the calls. This means that the communication is encrypted between your Drupal site and the Mollom application.
mollom.getServerList- this requests which Mollom servers can handle the call coming in from the Drupal site
mollom.checkContent- this asks Mollom whether the request is legitimate
mollom.sendFeedback- this tells the Mollom application that the spam message was indeed spam
mollom.getImageCaptcha- this asks Mollom to generate an image CAPTCHA
mollom.getAudioCaptcha- this asks Mollom to generate an audio CAPTCHA
mollom.checkCaptcha- asks Mollom to verify the result of a CAPTCHA
mollom.getStatistics- asks Mollom to send statistics
mollom.verifyKey- asks Mollom to return a status value
These calls are routed from your Drupal site over to a Mollom server each time your Drupal site needs to check whether a specific content post is spam or not.
Another interesting concept here is that Mollom actually provides a higher availability backup server that a Drupal user can sign up for. This server would then kick in and work if the other Mollom application servers have failed. So Mollom also provides a fallback, but it will cost you to sign up for it. It's not a free service.
So you can see here that the Mollom-contributed module allows you, as a Drupal site manager, easy access to set up this web service. We'll look at the code and backend of this configuration in more detail later, but for now this introduces us to how a Drupal contributed module can allow our site connections to a web service.
The Auto Tagging-contributed module allows you to auto-tag your site's content using a web services-based interface. The services interface provides you with the tag contexts to use to tag your content. This module allows for integration with the popular OpenCalais web service as well as the tagthe.net and Yahoo! Terms Extraction services. OpenCalais is a web service provided by Thomson Reuters that allows Drupal developers to access a huge variety of tagging and terms that are continuously updated and added to. It's basically a stockpile of tags and terms that you can utilize and integrate with your Drupal site content. You can use Auto Tagging module to make the connection to OpenCalais. More details about the Auto Tagging module are available on its Drupal.org project page at http://drupal.org/project/autotagging
Also check OpenCalais project for other important details at http://www.opencalais.com/.
To utilize these web services, you first need to install and enable the Auto Tagging module on your Drupal site. Once enabled, and depending on the web service you decide to use, you will configure a category/vocabulary that will be populated with the terms from the service. For example, if you were going to integrate with OpenCalais, you would first create a taxonomy vocabulary for OpenCalais tags. You would associate this vocabulary with the content types you want to tag on your site. The services will read your content that you want to tag and then apply tags automatically based on the content of your node.
If you are using OpenCalais, you'll need to first create an API key with the OpenCalais application. Then you'll add this API key to your module configuration.
This is a great module to use if you are looking to auto-tag content on your site using common and popular tags that are being culled, based on other web content using these tag-based web services. It's another example of how you don't have to reinvent the wheel or the application when you are building your site. You do not need to create tags. You can simply use tags that have already been generated for popular web content.
The Flick and Flickr API modules allow Drupal to consume and access photos that are posted on the Flickr website. In order to use this web service, with Drupal functioning as the consumer, we'll need to set up a Flickr API key so that we can use this key in our configuration in our Drupal site. You will become used to this process when setting up Drupal as a web service consumer. In order for your Drupal site to communicate with the web service and use its functionality, you'll need to sign up for API keys for many of these modules and configurations. We'll look at how the Flickr API works in much more detail in Chapter 3, Drupal and Flickr
The Apache Solr Search Integration module takes your Drupal site and integrates it with the Apache Solr Search web service. There is more information about the Apache Solr project at http://lucene.apache.org/solr/.
You would use this module if you want to add a more robust and enhanced Search functionality to your Drupal site besides the core Drupal search module. The service provides many extra features and better performance than the core Drupal search. You can have specific searching on content authors, taxonomy, and CCK fields, for example. This is called faceted search (http://en.wikipedia.org/wiki/Faceted_browser).
The module provides XML files that you need to have installed in order to make the web services work. The module also depends on your Drupal core search framework being in place, so you can run both the core Drupal search and the Apache Solr search in tandem, or just run one or the other. But the core search needs to be installed. The Drupal.org website lists many related projects that you can integrate with this module and the web service.
Drupal for Facebook (fb)
The Drupal for Facebook module is actually a larger scale module that allows you to program applications that run on Facebook and/or on your Drupal site but provides Facebook mechanisms. You can code up applications that run on your Drupal site and consume Facebook data â these are Facebook Connect-style applications â using the standards that this web service provides.
Drupal provides web services using a variety of methods and protocols. Some of these protocols are supported by using core modules and code that provide RSS- and XML-based feeds; and contributed modules, including the Services module that supports various service protocols. Drupal also supports protocols including AMFPHP XML-RPC. We'll look at each of them briefly in this section.
The Services module is the latest and newest version of the web services-contributed Drupal module. The Services module is a standard solution that allows for the integration of external web applications with your Drupal site. This module supports service callbacks used with standard service protocols, such as JSON, JSON-RPC, REST, SOAP, AMF, and more. The Services module allows your Drupal site to communicate and provide web services via these multiple interfaces using the same callback programming. So the module provides a large amount of flexibility and standards that you can use when programming web services to work with your Drupal site.
We'll be discussing this module in detail in Chapter 5, Drupal and Multimedia Web Services, but here's a very brief introduction to what the module can do:
Contains an API that allows other modules the ability to create web services
Contains server API that allows modules to create servers such as REST and SOAP
Includes test API and test pages
Provides the ability to manage API keys easily
File, Menu, Node, System, Taxonomy, User, and Views services included
As mentioned earlier, the Services module allows you to plug web services' API keys into its configuration so that you can set up a communication with various web service applications. An API key works similarly to a username, allowing you to access the applications securely by adding your specific API key or code. Many times an API key comes with a secret passcode that you will also add to the module's configuration. So when you sign up for a Twitter Developer's account to utilize and configure the Twitter module in Drupal, you'll be given an API key and secret code that you'll need to add to your module's configuration page. Many of the modules that we'll look at in this book use this method of API key and code.
The Services module provides a detailed handbook and documentation on Drupal.org at http://drupal.org/handbook/modules/services.
Drupal comes installed with core RSS functionality and support. Your main Drupal home page can have an RSS feed implemented on it if you post Story nodes to your home page. You can also create RSS feeds for any node or block in your Drupal site using the Views module to set up attached feeds. So, Drupal provides a very flexible environment for allowing other external web applications access to your content feeds. Many contributed modules also come installed with a default RSS feed. Using the core Drupal functionality for RSS feeds and also core modules such as Aggregator, you can post RSS feeds in RSS, RFF, or Atom format. These formats are all XML-based, again supporting and adhering to the web service standard.
In addition, each term on your Drupal site (using Taxonomy core module) displays an RSS feed. For example, on my site, I have a Featured term at this unique identifier at:
This term also has a feed attached to it at:
This feed shows all of the nodes (node title and teaser) for any content tagged with the featured term. So, Drupal provides many ways for other applications to call for content. Other web apps can call our site and request these feeds and this feed's content. Drupal can also act as a client here and call feeds from other web applications using a module such as the FeedAPI module: http://drupal.org/project/feedapi.
AMFPHP is an open source PHP-based implementation of the Action Message Format (AMF), which allows for ActionScript objects to be sent to server-based services. This allows for web client applications built in Flash, Flex, and AIR to communicate with PHP-based web applications such as Drupal. There is more introductory detail about AMFPHP on the AMFPHP website at http://www.amfphp.org/ï»¿.
Drupal can use AMFPHP through a contributed module called AMFPHP. This module (http://drupal.org/project/amfphp) provides support for integrating the AMFPHP protocol with the Services module in Drupal. So this is a contributed module that allows for a bridge between AMFPHP and Services. In order to use this service and module, you need to have the Drupal Services module installed and you need to install AMFPHP (version 1.9 beta 2) on your server. With this in place, Drupal can act as an AMFPHP-based client and provide Drupal integration with Flash and Flex applications.
Drupal supports the XML-RPC protocol natively. XML-Remote Procedure Call is one of the basic and simplest web service architectures. It uses XML to encode the function calls it makes and it makes these calls over HTTP. XML-RPC was created in 1998 by Dave Winer of UserLand Software. More about XML-RPC can be viewed on the main XML-RPC website at http://www.xmlrpc.com/..
There is also a good introduction to XML-RPC on Wikipedia at http://en.wikipedia.org/wiki/XML-RPC.
In your Drupal site, you can view the main
xmlrpc.php file, which is located in your root Drupal site folder. The code in this file looks like this:
<?php // $Id: xmlrpc.php,v 1.15 2005/12/10 19:26:47 dries Exp $ /** * @file * PHP page for handling incoming XML-RPC requests from clients. */ include_once './includes/bootstrap.inc'; drupal_bootstrap(DRUPAL_BOOTSTRAP_FULL); include_once './includes/xmlrpc.inc'; include_once './includes/xmlrpcs.inc'; xmlrpc_server(module_invoke_all('xmlrpc'));
This PHP file provides the default core code for handling any XML-RPC request or call that is attempted on your site by an external web application. You will notice that it includes the Drupal bootstrap as well as two
xmlrpc-specific include files that are located in your
/includes core folder. It also invokes the
xml_rpc server using a
In Drupal, the XML-RPC request is sent from a client (your Drupal site) to an external host or server. It's a similar type of call as the REST protocol explained earlier. If your Drupal site is acting as the server receiving the request from an external client, then your Drupal site provides the
xmlrpc.php file to handle this incoming request. This file handles the incoming call. The incoming request will most likely be formatted in XML and look something like this:
<?xml version="1.0"?> <methodCall> <methodName>examples.getBlogName</methodName> <params> <param> <value><i4>4</i4></value> </param> </params> </methodCall>
This request is calling for a user blog on your Drupal site with the method call of
.getBlogName. It's asking for a specific blog value of
4. This is just an example of the semantics of the code.
<?xml version="1.0"?> <methodResponse> <params> <param> <value><string>Trevor's Blog</string></value> </param> </params> </methodResponse>
For a very detailed introduction to XML-RPC, refer to Chapter 19, XML-RPC, of "Pro Drupal Development", John K. VanDyk, APress. We will also return to a more detailed discussion of XML-RPC and how to write your own XML-RPC code in Chapter 4, Drupal and Amazon. For now, it's enough to understand that your Drupal site does support XML-RPC by default, using the
xmlrpc.php file and code.
In this chapter, we've introduced the concepts and functionality of web services and how they interact with your Drupal site and applications. Here's a brief recap of what we learned in this chapter:
We learned what web services are and how they work using a set of standard protocols, including HTTP, Uniform Resource Indicators, and the XML format.
We discussed the various protocols and interfaces that web services take the form of when including the REST, SOAP, XML-RPC, and AMFPHP interfaces. We talked about how Drupal can integrate and use these protocols.
We mapped out how a web service protocol works and functions, specifically looking at the REST protocol and also at XML-RPC, which comes native to your Drupal core install. Drupal supports web services in its core configuration using XML-RPC.
We looked in detail at how Drupal uses web services, both as a consumer of services and a provider of services.
As a consumer of web services, we looked at a number of contributed modules that Drupal uses to interface with web services when Drupal is the client in the client/server relationship. These include Mollom, Auto Tagging, Flickr, Apache Solr, and Facebook.
We looked at how Drupal can serve as a web services provider by looking briefly at the Services module, RSS, AMFPHP, and XML-RPC.
We looked in detail at some code examples of how Drupal uses XML-RPC, including the code for sending a call to another server or application, and for receiving the response back from the web service.
In Chapter 2, Consuming Web Services in Drupal, we will start taking a deeper and more detailed look at Drupal web services as we discuss how Drupal consumes web services using the SOAP client module. We'll also take a detailed look at the SOAP protocol.