Sakai Web Services: Connecting to the Enterprise (Part 2)

by Alan Mark Berg Michael Korcuska | June 2009 | Open Source

In the previous part of the article by Alan Mark Berg and Michael Korcuska, we saw how web browsers talk to servers using Protocols and we had a look the currently available web services. In this part, we will create our first web service, first client and then discuss about Entity Broker.

Sakai and SOAP

Sakai SOAP web services piggyback on top of the Apache Axis project. Creating basic Sakai web services is programmer-friendly because Apache Axis removes many of the hard chores. All you have to do is create a Java class in a text file under the /web-apps/sakai-axis directory and any public method is automatically compiled into a service with a WSDL file automatically generated for it, ready for discovery by the client program. The compilation of the web service occurs after creation or modification and is triggered by the next incoming request. What is helpful is that when you make a typo or other mistake, the server displays the compilation error as a web page at the URL of the broken service, as shown in the next screen grab. Notice that the line number and type of error are included. The combination of text processing with a rich set of services to call on, plus the fact that it is not necessary to restart the server every time you compile, makes for rapid development cycles.

Sakai Web Services: Connecting to the Enterprise (Part 2)

My first web service

To create your first web service, you can add the file /web-apps/sakai-axis/MyTest.jws with the following contents to a running demonstration instance of Sakai:

public class MyTest{
public String YouSaid(String message){
return "You said: "+message;
}
}

Then, typing in http://localhost:8080/sakai-axis/MyTest.jws?wsdl will return a corresponding WSDL file similar to the figure below. Notice that it would take a human perhaps 30 minutes to generate the file and the computer took milliseconds.

Sakai Web Services: Connecting to the Enterprise (Part 2)

My first client

For the programmers among you, the following piece of Perl code consumes the service:

#!/usr/bin/perl
use SOAP::Lite;
my $host='http://localhost:8080';
my $soap = SOAP::Lite -> proxy("$host/sakai-axis/MyTest.jws?wsdl");
my $result =$soap->YouSaid("WHAT!");
print $result->result()."nn";

The returned result is:

You said: WHAT!

The SOAP Lite module interprets the WSDL file and after that, you can name the web service method directly in the code with the correct number of parameters. This feature results in code that is much more readable and thus maintainable. Changing the variable $host changes the server location. Changing the service and method requires the little nudge of modifying lines 4 and 5.

A more realistic client example

Sakai web services will not let you perform any action without fulfilling two prerequisites: the first is you need to set the property webservices.allowlogin=true in sakai/sakai.properties, and the second is that the client code needs to obtain a session in the form of a returned random string from a login service, and then use this string as part of any calls you make to other services. If the client code tries to perform any action without logging in, the server returns an error message.

The login service requires a username and password and it is very important to note that in production, you are expected to run the client code over an SSL/TLS connection.

The following piece of Perl code gets a session ID and then uses it as part of a second web service call to the addNewUser method, which, as you would expect from the name, then creates a new user in Sakai.

#!/usr/bin/perl
use SOAP::Lite;
my $host='http://localhost:8080';
my $soap = SOAP::Lite

-> proxy("$host/sakai-axis/SakaiLogin.jws?wsdl");
my $result =$soap->login("admin","admin");
my $sessionid=$result->result();
$soap = SOAP::Lite

-> proxy("$host/sakai-axis/SakaiScript.jws?wsdl");
$soap->addNewUser( $sessionid, 'alanberg', 'Alan', 'Berg',
'berg@xx.nl', '', 'useruser');

if ($result->fault) {
print "Error";
} else {
print "Successn";
}

Even if you find the Perl code unreadable, the point of the example is to show how only few lines of coding are required for an enterprise to hook into Sakai.

Entity Broker

Over time, more and more tools and services are included with Sakai. Therefore, there is an ever-expanding set of data, such as courses, users, polls, forums, grade books, assessments, and new data structures, available for integration.

It would be handy indeed if instead of needing to write custom web services per new entity, a tool programmer could call a service, write, and register his or her data for exposure. The kernel would then become responsible for the end delivery and the RESTful web services. Because the programmer does not have to deal as much with the details as before the services existed, the structure reduces the duplication of code and effort and increases maintainability, quality, and scalability, and generally eases the programmer's burden. Further, if by default, the entities are exposed as MIME types HTML, .JSON, .XML, you can write rich web-based applications and widget sets that consume the .JSON and .XML formats from the data within Sakai.

The Entity Broker is one such service that allows code to find and get at important data in Sakai and easily manipulate that data from within Java objects. To accommodate the ever-changing set of requirements, the data needs to have some uniform parts to it, such as an ID and an associated URL, and it needs to have the ability to register its existence to a central service. If the data has this kind of a structure, it is called an entity, the original technical details of which you can find in the source code under /reference/docs/architecture/sakai_entity.doc.

You can find the Java-specific details of the Entity Broker on Confluence (http://confluence.sakaiproject.org/confluence/display/SAKDEV/Entity+Provider+and+Broker).

Unless you are a hardcore Sakai kernel programmer, it really is not important to understand the hidden and subtle details. You just need to know how to find out which services exist and how to do business with those services.

Finding descriptions of services

For the demonstration instance, the Entity Broker services exist under the /direct URL space. To view a human-readable description of all the services, visit http://localhost:8080/direct/desc. The following figure is the description of services available on one of the Sakai QA servers.

Sakai Web Services: Connecting to the Enterprise (Part 2)

To zoom into the description of the user service, use the following demonstration URL: http://localhost:8080/direct/user/describe.

On different tag versions of Sakai, different services exist. However, every available entity is described by the same URL structure: http://hostname:port/direct/entity_prefix/describe.

It is helpful to read the specific description page for each entity, as Entity Broker empowers the programmer to add custom actions. The describe page next is for the user entity.

Sakai Web Services: Connecting to the Enterprise (Part 2)

Notice that custom actions currently exist and the server returns data in either XML or in the JSON format.

Before logging on to the demonstration instance of Sakai, first visit the URL http://localhost:8080/direct/user/current. Notice that the returned HTML page tells you in a 400 HTTP status error message that there is no current user to get user information about. This makes sense, as you have not logged in. After you log in and revisit the page, the server still does not return the user information. Instead, an error occurs, as HTML is not one of the supported return formats for this entity. The JSON format is, and to obtain your current information in JSON format, simply visit: http://localhost:8080/direct/user/current.json

Authenticating

At this point in the section, if you have everything set up properly to run TCPMON and watch the request and responses generated, then running the example code mentioned next will allow you to see how REST works in practice.

For a client-side application to create a new user, it must first obtain a session via a post to the URL http://localhost:8080/direct/session/new with the variables _username and _password set, as described in http://localhost:8080/direct/session/describe. The server returns the session ID in the form of one of the header values, EntityId, which the script then passes on in any sent requests. You can also pass the sessionId as sakai.session=sessionId as a header or in the URL. You can also use a cookie with the same values included.

To create a user, the client application will need to post to the user service with at minimum the eid (Enterprise ID) variable set. Note that the user/describe URL explains which name and value pairs are valid.

A client-side coding example

For the programming-inclined, I include the following listing that creates a session as user admin, with the password admin, and then creates a user in Sakai with eid=its_alive, firstName=The and lastName=Monster. For the sake of brevity, there is no programmatic error checking.

#!/usr/bin/perl -w
use strict;
use LWP::UserAgent;
use HTTP::Request::Common;

my $host='http://localhost:8080';
my $credential = "_username=admin&_password=admin";
my $user='eid=its_alive&firstName=The&lastName=Monster';
my $userAgent = LWP::UserAgent->new();
my $response = $userAgent->request(POST "$host/direct/session/new",
Content_Type => 'application/x-www-form-urlencoded', Content => $credential);
my $entityid= $response->header('EntityId');
print "Session: $host/direct/session/$entityidn";
$response = $userAgent->request(POST "$host/direct/user/new",
Content_Type => 'application/x-www-form-urlencoded', Content => $user);
$entityid= $response->header('EntityId');

print "User [json format]: $host/direct/user/$entityid.jsonn";
print "User [XML format]: $host/direct/user/$entityid.xmln";

On running, the output of the script should look similar to the following:

Session: http://localhost:8080/direct/session/770588c7-9a58-46f6-8d47-
7c92cab93759

User [json format]: http://localhost:8080/direct/user/c9ab941f-3fac-4827-
ad00-c4f98cf9ad5e.json

User [XML format]: http://localhost:8080/direct/user/c9ab941f-3fac-4827-ad00-
c4f98cf9ad5e.xml

Once you have written one client script, any new scripts are going to be quite similar. Expect an ever-expanding set of client scripts to be included in the contrib section, waiting for new organizations to pick them up.

Interview with Entity Broker author Aaron Zeckoski

Who is Aaron Zeckoski and what is his relationship with Sakai?

I am a developer and Senior Research Engineer for CARET (Centre for Applied Research in Educational Technologies), University of Cambridge. I am responsible for webapp and service development. I have worked in academic computing for about seven years, maintaining development documentation for Sakai and running training for Sakai developers. I am an inaugural Sakai Fellow and test-driven development advocate, DSpace and Sakai committer.

Can you tell us a little bit about the functionality you have been involved in coding into Sakai in general?

Many bug fixes and patches for various boring things, Integration works at various universities. Development tools like the Sakai App Builder, ReflectUtils, and GenericDAO. Tools like Evaluation and BlogWow. Data feeds for the UX project.

What was your motivation for writing Entity Broker?

I needed a way to generate clean URLs into Sakai tools and wanted to make Sakai development and integration with core services easier for the average developer. Further, I wanted a more standard way to handle REST and data input and output in Sakai. I wanted to make external (non-Java) Sakai development easier.

Why did you choose to use RESTful services over SOAP services?

REST is easier for the average developer to understand and it integrates and works with anything without much effort. It is also much easier to use with things like Javascript/AJAX.

Have you any future plans for Entity Broker

Add more support for standards (OpenSearch 1.1 URL support was just added) and output formats (RSS, ATOM are on the radar), integration with GWT, make it more modular so it can be used in projects like K2 and DSpace 2.

WSRP

Portals such as uPortal (http://www.uportal.org) aggregate information from various systems into channels that are part of one view for the user. A typical university may include an accumulation of the newest emails, RSS feeds for up and coming events, and links into important systems such as Sakai and the library systems. An institute can enforce a single corporate look and feel through a portal and empower the end user to transverse efficiently through their most current personalized information.

In the Java world, programmers can package channels into little applications that interact in a standard way with the portal system. These standard packages are called portlets and the interactions are standardized via JSR-168 (http://jcp.org/aboutJava/communityprocess/review/jsr168/). This standardization allows the portlets to be shared between different commercial and non-commercial portals and enables organizations to avoid locking in to a particular vendor's solution.

The issue with JSR-168 portlets is that the standardization constrains the range of events the portlet can react to and consequently makes the user experience less rich.

Portlets reside on the portal and, traditionally, get their own external data from RSS feeds or under the water via web services.

Sakai is thoroughly RSS enabled. For example, visit the main page of your demonstration server with the URL http://localhost:8080/portal/rss and you will see an RSS-rendered version of the main page. After logging in and visiting the page again, you will get to see more details. For a PDA-compliant page, visit http://localhost:8080/portal/pda.

Having all the portlet code on the portal system makes for a lot of code in one place and this is a serious risk for later trouble in terms of performance, code duplication, maintainability, and connecting to external data sources consistently. The web services for Remote Portlets WSRP (http://oasis-open.org/committee/wsrp) service allows a Portal to call WSRP-enabled portlets remotely directly from the portal via web services. On the portal side, all you would need now is a connector that an administrator can then configure to target a specific service.

Building a viable Portal system has knock-on effects on the background systems. If users hit the Portal heavily, and potentially the whole of an organizations population, then also expect a considerable increase in usage on the secondary systems. The deploying organization needs to preemptively strengthen legacy systems. Further, end users naturally expect to safely follow links from the various feeds directly into the associated background application. End users do not expect to have to log in more than once and only through the portal. If enacted, Single Sign On through mechanisms such as CAS (http://www.ja-sig.org/products/cas/) or Shibboleth (http://shibboleth.internet2.edu/) is viable. Uniform provisioning of user accounts across the full spectrum of linked-to applications is also a concern.

For Sakai, it makes sense to expose to a portal user a list of what is new in the user's courses, schedules, the Message of the Day, and other facets of the daily interaction between learners and Sakai. Whenever possible, it is a good idea for system integrators to use current standards to do so.

Activating the services within Sakai requires downloading and installing an extra web application (Servlet) that runs within a specific Sakai instance and delivers the WSRP producer services. The location of the most up-to-date README is https://source.sakaiproject.org/svn/wsrp/trunk/producer/README.txt. The code is based on the WSRP4J framework http://portals.apache.org/wsrp4j.

As the code is not included as part of the enterprise core tool set, your organization will have to fully test any significant deployments.

In summary to this section, there is code available to connect a Portal to Sakai via WSRP-based web services. However, you need to test the code before you deploy it in production.

Summary

By placing a text file with a few lines of Java in the right location in Sakai, a programmer can create new web services rapidly. Many client-side libraries remove the need to understand the underlying complexities of the protocols involved.

The Entity Broker exposes managed data (entities) within Sakai, such as the representation of users and sites by RESTful web services. You can discover currently available services by visiting http://host/direct.

It is possible to connect Sakai to Portal systems via the WSRP standard.

About the Author :


Alan Mark Berg

Alan Mark Berg Bsc. MSc. PGCE, has for the last twelve years been the lead developer at the Central Computer Services at the University of Amsterdam. In his famously scarce spare time, he writes. Alan has a degree, two masters degrees, and a teaching qualification. He has also co-authored two books about Sakai (http://sakaiproject.org), a highly successful open source learning management platform used by many millions of students around the world. Alan has also won a Sakai Fellowship.

In previous incarnations, Alan was a technical writer, an Internet/Linux course writer, a product line development officer, and a teacher. He likes to get his hands dirty with the building and gluing of systems. He remains agile by ruining various development and acceptance environments.

Michael Korcuska

Michael Korcuska is the Executive Director of the Sakai foundation and has nearly 20 years of experience in technology-enabled education and training. Prior to joining Sakai, Michael served as Chief Operating Officer for ELT, Inc., a leading compliance-training provider. He has also held leadership positions at DigitalThink (now Convergys Learning Solutions) and Cognitive Arts, an award winning custom e-learning developer. Michael got his start in technology-based learning at Stanford University's Courseware Authoring Tools Lab and Apple Computer's Multimedia Lab in the late 1980s. He holds an M.S. in Computer Science from Northwestern University (where he studied and worked at the Institute for the Learning Sciences) and B.S. in Symbolic Systems from Stanford University. He usually lives in Berkeley, California, with his wife and two children although his writing for this book was done during a year living in Paris, France.

Books From Packt

Flex 3 with Java
Flex 3 with Java

Moodle 1.9 for Teaching 7-14 Year Olds: Beginner's Guide
Moodle 1.9 for Teaching 7-14 Year Olds: Beginner's Guide

Drools JBoss Rules 5.0 Developer's Guide
Drools JBoss Rules 5.0 Developer's Guide

Plone 3 Theming
Plone 3 Theming

Scratch 1.3: Beginner’s Guide
Scratch 1.3: Beginner’s Guide

Moodle 1.9 Multimedia
Moodle 1.9 Multimedia

Seam 2.x Web Development
Seam 2.x Web Development

Grails 1.1 Web Application Development
Grails 1.1 Web Application Development

 


Code Download and Errata
Packt Anytime, Anywhere
Register Books
Print Upgrades
eBook Downloads
Video Support
Contact Us
Awards Voting Nominations Previous Winners
Judges Open Source CMS Hall Of Fame CMS Most Promising Open Source Project Open Source E-Commerce Applications Open Source JavaScript Library Open Source Graphics Software
Resources
Open Source CMS Hall Of Fame CMS Most Promising Open Source Project Open Source E-Commerce Applications Open Source JavaScript Library Open Source Graphics Software