Processing Twitter and New York Times APIs with ASP.NET Ajax on Microsoft CDN

Exclusive offer: get 50% off this eBook here
Learning SQL Server 2008 Reporting Services

Learning SQL Server 2008 Reporting Services — Save 50%

A step-by-step book and eBook to getting the most of Microsoft SQL Server Reporting Services 2008

$29.99    $15.00
by Jayaram Krishnaswamy | November 2009 | AJAX Microsoft

In this article by Dr. Jayaram Krishnaswamy, we look at two APIs, the Twitter API and the New York Times API and see how we can access data on them using the JavaScript libraries available on the Microsoft Edge Content Network also know as Microsoft Content Delivery Network (CDN). While jQuery is used to access API retrieved data on the intranet web server, the Microsoft AJAX client templates are used in displaying them.

APIs (Application Programming Interface) are application-to-application programming interfaces that support harvesting information on the web using the known web standards. These APIs are provided by the entities who wish to expose parts of their resources that a third party can use. The APIs run transparent to the user and exposes just what they want to expose, with some providing access to material for public consumption with others giving access to resources based on authentication. In a sense they may be called a basic form of SAAS.

Amazon.com, Google etc have exposed their APIs for some time. Twitter and New York Times have also exposed their API's which can be used to do some digging into the information contained in them, a kind of web mining. Many others such as Netflix have provided their own APIs described on their web sites.

What is Twitter API?

Twitter API is provided by the Social Networking and Micro-blogging service. Twitter API adheres to the web standards and one can talk to Twitter using HTTP. You can just about access anything on the Twitter web site. One example of creating a Microsoft SQL Server Report using Twitter API is available here - Tweets with Reporting Services, wherein the response from the Twitter API was in XML format. JSON (JavaScript Object Notation) is another format in which data is returned when an API call is made. In this article we will be looking at API call that returns a JSON response.

Twitter exposes a large number methods through their API's such as API's for Search, Timeline, Status, User, Direct Message, Friendship and many more. As previously mentioned the responses will be in XML or JSON. Also while some APIs may take parameters others may not.

The Twitter API used in this tutorial

We will be looking at trends in Twitter API exposed by the url, http://search.twiiter.com/trends.format. We will be using the GET method and we will expect a JSON response. Since the volume of traffic may overwhelm, the calls that you can make to this in an hour are limited (also known as rate limiting) but not critical for the demo in this tutorial.

Here is a typical call to the trends method on the Twitter API. Herein we will search for trends on the Twitter site and expect a response in JSON, if we use json instead of Format in the next URL address.

Instead of:
http://search.twitter.com/trends.Format
type-in, the following for URL address,
http://search.twitter.com/trends.json

When you plug the above in a web Brower you would get a response trends.json which you may save to your hard drive or, use it in any way you like. The next quoted text is what you get in response (note that this is what I got on Saturday 31, 2009 and what you get will be different), the content of the file trends.json you saved to your computer. Note that presently you get about top ten trends from this API call.

{"as_of":"Sat, 31 Oct 2009 20:44:46 +0000","trends":[{"name":"Happy Halloween", 
"url":"http://search.twitter.com/search?q=%22Happy+Halloween%22+OR+%22Feliz+
Halloween%22"},{"name":"#nxzerosetechaves","url":"http://search.twitter.com/
search?q=%23nxzerosetechaves"},{"name":"Danyl","url":"http://search.twitter.com/
search?q=Danyl"},{"name":"#HappyHalloween","url":"http://search.twitter.com/
search?q=%23HappyHalloween"},{"name":"#potterday","url":"http://search.twitter.com/
search?q =%23potterday"},{"name":"X Factor","url":"http://search.twitter.com/
search?q=%22X+ Factor%22"},{"name":"It's Halloween","url":"http://search.twitter.
com/search?q=%22It %27s+Halloween%22+OR+%22Its+Halloween%22"},{"name":"Trick",
"url":"http://search.twitter.com/search?q=Trick+OR+%23trick"},{"name":"Paranormal
Activity","url":"http://search.twitter.com/search?q=%22Paranormal+Activity%22"},
{"name":"This Is It","url":"http://search.twitter.com/search?q=%22This+Is+It%22"}]}

First of all what you see returned is a JSON object. If you are new to JSON review this article on my blog. The various elements that you see such as 'name', 'url' etc are fields in the response that are all described in the API documentation(look for Return Values). Some of the API calls can return a ton of information and you will have to know the API method so that you can correctly parse this data. Another thing you would notice is that the JSON object you get out is a nested object with many levels. You may need a JSON Parser to get a clearer picture of this nesting and I recommend using the online parser at this site. Using the above site, the JSON Object would appear as shown (only a portion is shown).

Microsoft AJAX and jQuery

New York Times API

New York Times made available to the developers sometime in the middle of October 2008 APIs that can search New York Times for various kinds of information . Just like in Twitter there are a large number of APIs that you can use such as: Article Search; Best Sellers; Campaign Finance; Congress; and many others. Interested users can get on to this resource by signing up here requesting what APIs they would like to use. After signing up, New York Times would provide keys for the APIs that you want to access. It is important therefore, that the call should include the keys provided to you. For example, I received keys to access the following resources: Movie Reviews, Article Search, Best Sellers and Times Newswire. The key for the Movies Reviews API appears as shown here (the one shown here has been doctored and will not work).

Movie Reviews API	 
Key: b57378910b9fd80ecc73461547c93e8a:10:50673441

Using the New York Times API

It is a valuable resource since you can get for example with the Article Search API access to more than 2.8 million articles from 1981. Using this is quite simple, just paste the URL shown below into the address box of your browser. Note that the key shown here is fake (but of correct format).

http://api.nytimes.com/svc/search/v1/article?query=India&facets=
publication_year&api-key=6c208890a4880093c30020be8fe17a40:0:50633441

This will display in the browser the JSON object that is returned as shown. You can use the previously mentioned site to parse it for more friendly display.

{"facets" : {"publication_year" : [{"count" : 2724 , "term" : "2008"} , 
{"count" : 2345 , "term" : "2006"} , {"count" : 2311 , "term" : "2009"} ,
{"count" : 2282 , "term" : "2007"} , {"count" : 2144 , "term" : "2002"} ,
{"count" : 2111 , "term" : "2001"} , {"count" : 1988 , "term" : "2005"} ,
{"count" : 1951 , "term" : "2004"} , {"count" : 1921 , "term" : "1985"} ,
{"count" : 1798 , "term" : "2003"} , {"count" : 1761 , "term" : "1999"} ,
{"count" : 1720 , "term" : "2000"} , {"count" : 1642 , "term" : "1998"} ,
{"count" : 1442 , "term" : "1984"} , {"count" : 1382 , "term" : "1986"}]} ,
"offset" : "0" , "results" :
[{"body" : "BARSUR, India — At the edge of the Indravati River, hundreds
of miles from the nearest international border, India effectively ends.
Indian paramilitary officers point machine guns across the water.
The dense jungles and mountains on the other side belong to Maoist rebels
dedicated to overthrowing the government. "That is their liberated" ,
"byline" : "By JIM YARDLEY" , "date" : "20091101" , "title" :
"Maoist Rebels Widen Deadly Reach Across India" , "url" : "http://www.
nytimes.com/2009/11/01/world/asia
/01maoist.html"} ,.........(there is more of this but abbreviated here)

Response Format

As you can see the responses to the API calls return JSON objects in general of the form shown belo<!--[if gte mso 9]> Normal 0 false false false EN-US X-NONE X-NONE MicrosoftInternetExplorer4 <![endif]--><!--[if gte mso 9]> <![endif]-->

<!--[if gte mso 10]> <! /* Style Definitions */ table.MsoNormalTable {mso-style-name:"Table Normal"; mso-tstyle-rowband-size:0; mso-tstyle-colband-size:0; mso-style-noshow:yes; mso-style-priority:99; mso-style-qformat:yes; mso-style-parent:""; mso-padding-alt:0in 5.4pt 0in 5.4pt; mso-para-margin:0in; mso-para-margin-bottom:.0001pt; mso-pagination:widow-orphan; font-size:11.0pt; font-family:"Calibri","sans-serif"; mso-ascii-font-family:Calibri; mso-ascii-theme-font:minor-latin; mso-fareast-font-family:"Times New Roman"; mso-fareast-theme-font:minor-fareast; mso-hansi-font-family:Calibri; mso-hansi-theme-font:minor-latin; mso-bidi-font-family:"Times New Roman"; mso-bidi-theme-font:minor-bidi;} --> <!--[endif]-->w (this one is of the form returned by the Twiiter API). What we propose to do is to use jQuery's GetJSON() method to get the JSON Objects and use Microsoft AJAX JavaScript files to display the data on the web page. Both jQuery javascript files and Microsoft ASP.NET AJAX files are both available on the Microsoft ECN (CDN). The GetJSON() method as well as the Microsoft ASP.NET AJAX templates can be easily implemented in the Visual Studio 2008 IDE. Alternatively Microsoft AJAX can also be used to retrieve data from the web sites. In this article the GetJSON() method will be used.

{"x":{"y":[{"a1":"b1", "c1":"d1"}, {"a2":"b2", "c2":"d2"}]},....
"f":"g",....}
Learning SQL Server 2008 Reporting Services A step-by-step book and eBook to getting the most of Microsoft SQL Server Reporting Services 2008
Published: March 2009
eBook Price: $29.99
Book Price: $49.99
See more
Select your format and quantity:

Accessing the APIs from the intranet

When you use the APIs from your browser the APIs respond in different ways. The Twitter API will respond by allowing you to download the JSON file which you can persist to a file on your local intranet. The New York Times API on the other hand will display the JSON object directly in your browser which you can persist by saving it with the JSON extension to your local intranet server. However, these APIs cannot be directly reached by the intranet due to cross-domain browser policy (The same origin web security model). One may use other constructs such as plug-ins that empower the browser to overstep the same domain policy, subspacing, etc. The article does not cover these aspects. In trying out these demos the user can access these APIs via browsers and persist them to their file system.

Displaying data from retrieved from the APIs

As previously mentioned data will be retrieved using the API URLs which are then persisted to the intranet server. The web page created using Visual Studio is then used to display data where jQuery retrieves the persisted data from the intranet which then gets displayed by the Microsoft ASP.NET Client templates. To get an initial exposure to using these tools review this article.

Data from Twitter API

The next listing created using the Visual Studio 2008 shows the how the data is retrieved and displayed. One of the main things to notice is that the parsing of the JSON object to correctly display data has to be first determined. The treeview provides a hint as to how this may be accomplished. Depending on whether it is a JSON array or a JSON object the parsing will be slightly different. Also the JSON file in the Visual Studio IDE should not have line breaks. It should be preferably in an unbroken continuous line.

Listing 1: Response from Twitter API displayed using Microsoft AJAX Client library

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" 
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">

<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<title></title>
<script src='//dgdsbygo8mp3h.cloudfront.net/sites/default/files/blank.gif' data-original="http://ajax.microsoft.com/ajax/jquery/jquery-1.3.2.js"
type="text/javascript"></script>

<style type="text/css"> .sys-template { display:none; } </style>
<script type="text/javascript"
src='//dgdsbygo8mp3h.cloudfront.net/sites/default/files/blank.gif' data-original="http://ajax.microsoft.com/ajax/beta/0909/MicrosoftAjax.debug.js">
</script>
<script type="text/javascript"
src='//dgdsbygo8mp3h.cloudfront.net/sites/default/files/blank.gif' data-original="http://ajax.microsoft.com/ajax/beta/0909/MicrosoftAjaxTemplates.debug.js">
</script>
<script>
$(document).ready(function() {
$("#n1").css("color", "red");
$("#n2").css("color", "green");

});
</script>
<script type="text/javascript">

var retval;
var url_json = "http://localhost/AJAXEnabledWebSite2/TwitterType2.json";

jQuery.getJSON(url_json, function(json) {

retval = json;
cdn();
});

function cdn() {

$create(Sys.UI.DataView, {data: retval}, null, null, $get('SList'));

}
</script>


</head>

<body xmlns:dataview="javascript:Sys.UI.DataView"
xmlns:sys="javascript:Sys">
<h1>Twitter Stuff</h1>

<table id="SList" class="sys-template"
border="1" style="background-color:#B0E0E6" >
<tr> <td>{{as_of}}</td>
<td id="n1">{{trends[0].name }}</td>
<td id="n2">{{trends[0].url}}</td>
</tr>
</table>
</body>
</html>

In the above listing, the 1st three scripts are those available on the Microsoft CDN. The fourth script does the formatting for the HTML elements via jQuery. In the fifth script, the GetJSON() method is called to retrieve the persisted response from the Twitter API call mentioned earlier. The persisted JSON object is in file TwitterType2.json (it is just a text file in VS 2008 with a json extension). The function cdn() creates the template for display creating a dataview using the return value from the getJSON call. The parsed elements from the JSON object are displayed in the table. The name space attribute in the body are important and the class attribute in the table makes the initial template hidden from the user (see the style information in the <head/> section. Data fields retrieved from the javascript $Create() are pushed to the markup blocks {{}}. Please refer to an earlier article on this site. After building the project the HTML page can be displayed using the default browser as shown. The page did not display using the Opera browser (version 9.1) but displayed as shown in Mozilla Firefox, IE 8.0, Safari, Chrome and even in my one time favorite Netscape 8.1.

Microsoft AJAX and jQuery

Data from New York Times

The New York Times API is called as previously mentioned which gives access to India related articles as of the date the call was made. The json object returned follows the general pattern as shown here. This is just the pattern to which the data adheres. The pattern is different from the one returned from Twitter.

{"x":{"y":["a1":"b1","a2","b2"}]},"p":"q","m":["n1":"valn1","n2":"valn2",
"n3":"valn3"],"ts":[|"v","w":"z"],"r":"s"}

The next listing shown here for the sake of completeness is very similar to the one used with the Twitter API except for the json file it accesses and the display of parsed content. You are also shown how to pick the data you want to display in this listing. The IndiaArticles.json is the name of the json data returned from the API call persisted to the intranet. The parsing of the data is very similar to the earlier Twitter example.

Listing 1: Response from New York Times API displayed using Microsoft AJAX Client library

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" 
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">

<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<title></title>
<script src='//dgdsbygo8mp3h.cloudfront.net/sites/default/files/blank.gif' data-original="http://ajax.microsoft.com/ajax/jquery/jquery-1.3.2.js"
type="text/javascript"></script>

<style type="text/css"> .sys-template { display:none; } </style>
<script type="text/javascript"
src='//dgdsbygo8mp3h.cloudfront.net/sites/default/files/blank.gif' data-original="http://ajax.microsoft.com/ajax/beta/0909/MicrosoftAjax.debug.js">
</script>
<script type="text/javascript"
src='//dgdsbygo8mp3h.cloudfront.net/sites/default/files/blank.gif' data-original="http://ajax.microsoft.com/ajax/beta/0909/MicrosoftAjaxTemplates.debug.js">
</script>
<script>
$(document).ready(function() {
$("#n1").css("color", "red");
$("#n2").css("color", "green");
});
</script>
<script type="text/javascript">

var retval;
var url_json = "http://localhost/AJAXEnabledWebSite2/IndiaArticles.json";

jQuery.getJSON(url_json, function(json) {

retval = json;
cdn();
});

function cdn() {

$create(Sys.UI.DataView, {data: retval}, null, null, $get('SList'));

}
</script>


</head>

<body xmlns:dataview="javascript:Sys.UI.DataView"
xmlns:sys="javascript:Sys">
<h1>New York Times Articles</h1>

<table id="SList" class="sys-template"
border="1" style="background-color:#B0E0E6" >
<tr><td>{{facets}}</td>
<td id="n1">{{facets.publication_year[0] }}</td>
<td>{{facets.publication_year.count}} | {{facets.publication_year.term}}</td>
<td>{{offset}}</td>
<td>{{results[0].body}}</td>
<td>{{results[0].byline}}</td>
<td>{{results[0].date}}</td>
<td>{{results[0].title}}</td>
<td id="l1">{{results[0].url}}</td>
</tr>
</table>
</body>
</html>

The browsed html page appears as shown in theOpera 10.0 browser.

Microsoft AJAX and jQuery

Summary

The article described accessing data on Twitter and New York Times using jQuery and the Microsoft ASP.NET 3.5 AJAX client libraries. Both jQuery and the Microsoft AJAX are available on the Microsoft Edge Content Network.

Learning SQL Server 2008 Reporting Services A step-by-step book and eBook to getting the most of Microsoft SQL Server Reporting Services 2008
Published: March 2009
eBook Price: $29.99
Book Price: $49.99
See more
Select your format and quantity:

About the Author :


Jayaram Krishnaswamy

Jayaram Krishnaswamy studied at the Indian Institute of Science in Bangalore India and Madras University in India and taught at the Indian Institute of Technology in Madras. He went to Japan on a Japanese Ministry of Education Research scholarship to complete his PhD in Electrical Engineering from Nagoya University. He was a Post-Doctoral Fellow at Sydney University in Australia; a Government of India Senior Scientific Officer at the Indian Institute of Science in Bangalore and Indian Institute of Technology at Kanpur; a Visiting Scientist at the Eindhoven Institute of Technology in the Netherlands; a visiting Professor of Physics at the Federal University in Brazil; an Associate Research Scientist at a government laboratory in São Jose dos Campos in São Paulo, Brazil; a visiting scientist at the National Research Council in Ottawa, Canada before coming to USA in 1985. He has also taught and worked at the Colorado State University in Fort Collins and North Carolina State University in Raleigh, North Carolina. He worked with Northrop Grumman Corporation on a number of projects related to high energy electron accelerators and Free Electron Lasers. These projects were undertaken at the Brookhaven National Laboratory in Long Island and in the Physics Department at Princeton University. He has over 80 publications in refereed and non-refereed publications and 8 issued patents. He is fluent in Japanese and Portuguese and lives in Honolulu, Hawaii, USA.

He has been working in IT-related fields since 1997. He was once a Microsoft Certified Trainer in Networking and a Siebel Certified developer. He has worked with several IT related companies, such as the Butler International in their Siebel practice, with several IBM sub-contractors and smaller companies. Presently he is active in writing technical articles in the IT field to many online sites such as http://CodeProject.com, http://APSFree.com, http://DevShed.com, http://DevArticles.com, http://OfficeUsers.org, http://ASPAlliance.com, Egghead Café, http://SSWUG.org, Packt Article Network, http://databasedev.co.uk, http://cimaware.com, and many others. Between 2006 and 2010 he wrote more than 400 articles mostly related to database and web related technologies covering Microsoft, Oracle, Sybase, ColdFusion, Sun, and other vendor products.

He has written four books all published by Packt related to Microsoft Database and Application Development: SQL Server Integration Services Using Visual Studio 2005, Learning SQL Server Reporting Services 2008, Microsoft SQL Azure; Enterprise Application Development, and Microsoft Visual Studio Lightswitch Business Application Development. He regularly writes for his four blogs on Blogger; http://Hodentek.blogspot.com, http://HodentekHelp.blogspot.com, http://HodentekMobile.blogspot.com, and http://HodentekMSSS.blogspot.com. He received the 2011 Microsoft Community Contributor award.

Contact Jayaram Krishnaswamy

Books From Packt


jQuery 1.3 with PHP
jQuery 1.3 with PHP

ICEfaces 1.8: Next Generation Enterprise Web Development
ICEfaces 1.8: Next Generation Enterprise Web Development

Microsoft Dynamics AX 2009 Programming: Getting Started
Microsoft Dynamics AX 2009 Programming: Getting Started

Learning jQuery 1.3
Learning jQuery 1.3

Backbase 4.4.x RIA Development
Backbase 4.4.x RIA Development

Microsoft Office Live Small Business: Beginner’s Guide [RAW]
Microsoft Office Live Small Business: Beginner’s Guide [RAW]

Drupal 6 JavaScript and jQuery
Drupal 6 JavaScript and jQuery

PHP and script.aculo.us Web 2.0 Application Interfaces
PHP and script.aculo.us Web 2.0 Application Interfaces


No votes yet

Post new comment

CAPTCHA
This question is for testing whether you are a human visitor and to prevent automated spam submissions.
K
M
V
g
F
2
Enter the code without spaces and pay attention to upper/lower case.
Code Download and Errata
Packt Anytime, Anywhere
Register Books
Print Upgrades
eBook Downloads
Video Support
Contact Us
Awards Voting Nominations Previous Winners
Judges Open Source CMS Hall Of Fame CMS Most Promising Open Source Project Open Source E-Commerce Applications Open Source JavaScript Library Open Source Graphics Software
Resources
Open Source CMS Hall Of Fame CMS Most Promising Open Source Project Open Source E-Commerce Applications Open Source JavaScript Library Open Source Graphics Software