Testing a HELP System in a Java Application

Swing Extreme Testing

June 2008


The Extreme approach to complete Java application testing


{literal}As more and more features get added to your software, the Help system for the software becomes extensive. It could probably contains hundreds of HTML files plus a similar number of images. We could always face the problems listed below.

  • There could be broken links in the help index.
  • Some files may not be listed in the index, therefore they can not be read by the customers.
  • Some of the contextual help buttons could show the wrong topic.
  • Some of the HTML files can contain broken links or incorrect image tags.
  • Not all of the file titles would match their index entry.

Another problem could occur when the user does a free text search of the help system. The result of such a search is a list of files, each represented by its title. In our system,  documents could have the title "untitled".

Testing a HELP System in a Java Application

In fact, the JavaHelp 2.0 System User's Guide contains the recommendation

"To avoid confusion, ensure that the <TITLE> tag corresponds to the title used in the table of contents."

Given that customers mostly use the Help system when they are already frustrated by our software, we should always see to it that such errors do not exist in our help system. To do this, we will write a tool, HelpGenerator, that generates some of the boilerplate XML in the help system and checks the HTML and index files for the problems listed above. We will also build tools for displaying and testing the contextual help. We've re-engineered and improved these tools and present them in this article.

In this article we are assuming familiarity with the JavaHelp system. Documentation and sample code for JavaHelp can be found at: http://java.sun.com/products/javahelp.


A JavaHelp package consists of:

  • A collection of HTML and image files containing the specific Help information to be displayed.
  • A file defining the index of the Help topics. Each index item in the file consists of the text of the index entry and a string representing the target of the HTML file to be displayed for that index entry, for example:
    <index version="1.0">
    <indexitem text="This is an example topic."target="Topic">
    <indexitem text="This is an sub-topic."target="SubTopic"/>
  • A file associating each target with its corresponding HTML file (or more generally, a URL)—the map file. Each map entry consists of the target name and the URL it is mapped to, for example:
  • <map version="1.0">
    <mapID target="Topic" url="Topic.html"/>
    <mapID target="SubTopic" url="SubTopic.html"/>
  • A HelpSet file (by default HelpSet.hs) which specifies the names of the index and map files and the folder containing the search database.

Our software will normally have a main menu item to activate the Help and, in addition, buttons or menu items on specific dialogs to activate a Help page for a particular topic, that is, "context-sensitive" Help.

What Tests Do We Need??

At an overall structural level, we need to check:

  • For each target referred to in the index file, is there a corresponding entry in the map file? In the previous example, the index file refers to targets called Topic and SubTopic. Are there entries for these targets in the map file?
  • For each URL referred to in the map file, is that URL reachable? In the example above, do the files Topic.html and SubTopic.html exist?
  • Are there HTML files in our help package which are never referred to?
  • If a Help button or menu item on some dialog or window is activated, does the Help facility show the expected topic?
  • If the Help search facility has been activated, do the expected search results show? That is, has the search database been built on the latest versions of our Help pages?

At a lower level, we need to check the contents of each of the HTML files:

  • Do the image tags in the files really point to images in our help system?
  • Are there any broken links?

Finally, we need to check that the contents of the files and the indexes are consistent:

  • Does the title of each help page match its index?

To simplify these tests, we will follow a simple naming pattern as follows:

We adopt the convention that the name of each HTML file should be in CamelCase format (conventional Java class name format) plus the .html extension. Also, we use this name, without the extension, as the target name. For example, the target named SubTopic will correspond to the file SubTopic.html.

Furthermore, we assume that there is a single Java package containing all the required help files, namely, the HTML files, the image files, the index file, and the map file. Finally, we assume a fixed location for the Help search database.

With this convention, we can now write a program that:

  • Generates the list of available targets from the names of the HTML files.
  • Checks that this list is consistent with the targets referred to in the index file.
  • Checks that the index file is well-formed in that:
    • It is a valid XML document.
    • It has no blank index entries.
    • It has no duplicate index entries.
    • Each index entry refers to a unique target.
  • Generates the map file, thereby guaranteeing that it will be consistent with the index file and the HTML files.

The class HelpGenerator in the package jet.testtools.help does all this,and, if there are no inconsistencies found, it generates the map file. If an inconsistency or other error is found, an assertion will be raised. HelpGenerator also performs the consistency checks at the level of individual HTML files. Let's look at some examples.

An HTML File That is Not Indexed

Here is a simple help system with just three HTML files:

Testing a HELP System in a Java Application

The index file, HelpIndex.xml, only lists two of the HTML files:

<index version="1.0">
<indexitem text="This is an example topic."
<indexitem text="This is an example sub-topic."

When we run HelpGenerator over this system (we'll see how to do this later in this article), we get an assertion with the error message
The Help file: TopicWithoutTarget.html was not referenced in the Index file: HelpIndex.xml.

An index item for which there is no HTML file

If we take the help system in the previous example, but remove the file TopicWithoutTarget.html (which was not indexed) and the file ExampleSubTopic.html (which did have an index item) then we are left with a system in which the index file references a non-existent HTML file. Applying HelpGenerator to this system gives an assertion with the error message The entry: ExampleSubTopic in the Index file:HelpIndex.xml did not have a HTML file.

Broken links

Consider this simple help system. The following screenshot shows a help system in which some of the links in the HTML files are broken.

Testing a HELP System in a Java Application

Running HelpGenerator over this system produces an error that indicates which files have broken links and which links are actually broken:
The file BrokenLinks.html has these broken links:
The file SingleBrokenLink.html has this broken link:

Incorrectly titled help pages

As a final example, consider a help system consisting of three HTML files A.html, B.html, and C.html such that

  • A.html has an empty title tag.
  • B.html has title "Topic B".
  • C.html has title "untitled".

Further, suppose that our index file is as follows:

<index version="1.0">
<indexitem text="Topic A" target="A"/>
<indexitem text="Topic B" target="B"/>
<indexitem text="Topic C" target="C"/>

The HelpGenerator tool applied to this system fails with an error message that indicates the problem, which is:
"Title'' does not match index text 'Topic A' for file A.html
Title 'untitled' does not match index text 'Topic C' for file C.html.

Creating and Testing Context-Sensitive Help

Each component of an application should include a facility for showing the Help system with the topic most relevant to that component selected. For example, the LabWizard Attribute Manager screen has a context-sensitive Help button:

Testing a HELP System in a Java Application

Activating this button shows the Attributes chapter of the Help system:

Testing a HELP System in a Java Application

HelpGenerator generates an enumeration, Help.java, which makes it very easy to include this kind of context-sensitive help in our application screens. For example, the code to build the Help button in the Attribute Editor shown previously is:

private Box createButtonBar() {
JButton closeButton = new JButton( actions.close );
Box box = Box.createHorizontalBox();
box.add( Box.createHorizontalGlue() );
box.add( closeButton );
box.add( Box.createHorizontalStrut( 5 ) );
box.add( createHelpButton( Help.Attribute ) );
return box;

The highlighted line here creates the button and adds it to the button box at the bottom of the dialog. The code to create the button itself is:

public static JButton createHelpButton( final Help topic ) {
JButton helpButton = us.createButton( SimpleUIMessages.HELP );
helpButton.addActionListener( new ActionListener() {
public void actionPerformed( ActionEvent e ) {
HelpGenerator.helpActionFor( topic ).actionPerformed( e );
} );
return helpButton;

The first line of this method uses the UserStrings class  to create the button, and the second line adds an ActionListener to the button that actually shows the Help browser with correct topic shown. Here is the HelpGenerator method that creates the ActionListener:

public static ActionListener helpActionFor( Help topic ) {
if (broker == null) {//broker is a javax.help.HelpBroker
broker.setCurrentID( topic.toString() );
return new CSH.DisplayHelpFromSource( broker );

The enumeration Help.java lists all of the available help topics. For example, a Help system containing just the HTML files ExampleTopic.html and ExampleSubTopic.html would be represented as:

package book.testtools.help;
public enum Help {

This enumeration is automatically generated by HelpGenerator.main() method, provided that all the consistency checks described earlier pass.

Using this methodology we can be sure that activating a Help button or menu item will bring up a Help topic. However, without inspecting the source code, how can we check whether the correct topic has been associated with this Help button or menu item?

Our unit tests can be based on the HelpGeneratorTest method:

public static void checkHelpIsShowingForTopic( Help topic )

This method reads the index file to determine the index item for the specified topic, finds the frame showing the help facility's index JTree, and then checks whether this index item is in fact selected. If so, then we can be sure from the other HelpGenerator checks that the appropriate HTML file is displayed. If the required index item is not selected then an assertion is raised.

As an example of how easy these unit tests are to write, here is the test for the Help button in the Attribute Editor unit test:

public boolean helpTest() {
Help.Attribute );
return true;

In this test, the first line calls the init() method that populates and displays an Attribute Editor. The second line uses a mnemonic to activate the Help button. The third line uses the HelpGeneratorTest method to check that the JavaHelp browser is showing and is displaying the correct topic.

More details of HelpGenerator and HelpGeneratorTest can be found in the packages jet.testtools.help and jet.testtools.help.test.

Executing HelpGenerator

To execute HelpGenerator, two parameters need to be provided: the name of the package we are using to provide the help functionality, and the name of the corresponding source directory, for example:

private String helpPackageName = "jet.run.help";
private String helpDirectoryName = "C:jetrunhelp";
HelpGenerator.main( new String[]{helpPackageName, helpDirectoryName});

As HelpGenerator creates a production class, it should be executed with each software build. Here is our Ant task for doing this. Note that we rebuild the search database at the same time so that it is always up to date:

<target name="GenerateHelp" description="Generate Help info.">
<java classname="jet.testtools.help.HelpGenerator" fork="true"
maxmemory="384m" dir="${src}">
<path refid="classpath" />
<arg value="jet.testtools.help"/>
<arg path="${src}/jet/testtools/help"/>
<java classname="com.sun.java.help.search.Indexer" fork="true"
maxmemory="384m" >
<arg line="${helpsrc}/*.html"/>
<arg line="-db ${helpdst}/JavaHelpSearch"/>
<arg line="-c ${src}/IndexerConfig.txt"/>
<path refid="classpath" />

In addition, HelpGenerator can be executed whenever a new Help page is added, enabling the corresponding topic to be used by a Help button or menu item.


By using HelpGenerator and its associated test class, we can be sure that our help system has no broken indexes, has no missed topics, and shows the right topic for each context. As well, we will know that our help files contain no broken links, no missing images, and are correctly titled. We can also use HelpGenerator and HelpGeneratorTest to greatly simplify the implementation and testing of our application's context-sensitive help.{/literal}

Books to Consider

comments powered by Disqus