The John Smith problem

Different people, locations, and concepts can have the same orthographic representation but be distinct. There are multiple instances of "John Smith", "Paris", and "bank" in the world, and a proper cross-document coreference system should be able to handle it. For the case of concepts such as "bank" (a river bank versus a financial bank), the term of art is word-sense disambiguation. This recipe will demonstrate one approach to the problem that Baldwin developed back in the day with Amit Bagga for person disambiguation.

Getting ready

The code for this recipe closely follows the clustering tutorial at http://alias-i.com/lingpipe/demos/tutorial/cluster/read-me.html but changes it to more closely fit the original Bagga-Baldwin work. There is a fair amount of code but nothing very complicated. The source is in src/com/lingpipe/cookbook/chapter7/JohnSmith.java.

The class starts with the standard panoply of NLP tools for tokenization, sentence detection, and named-entity detection...