Manual, Generic, and Ordered Tests using Visual Studio 2008

Exclusive offer: get 50% off this eBook here
Software Testing with Visual Studio Team System 2008

Software Testing with Visual Studio Team System 2008 — Save 50%

A comprehensive and concise guide to testing your software applications with Microsoft's Visual Studio Team System 2008 with this book and eBook

$29.99    $15.00
by N Satheesh Kumar Subashni. S | December 2008 | .NET

It is not always possible to conduct the automated tests. There are situations where we may need the manual tests. Manual testing is a document or a text file, which contains a set of steps for the tester to follow. Manual testing can be conducted in between automated tests or anywhere between a series of different tests. For example, after some sets of automated tests, we may have to check log files or configuration information or settings based on the tests executed to continue with the remaining automated tests. We can group all these tests together, order them, and create an ordered test to execute the tests in an order.

In some cases, we may need to execute the tests that are not created by using Visual Studio but required for this current application. In that case, we go for the generic test, which acts as a wrapper for the tests written by some third-party tool and executes that test inside Visual Studio IDE. Once it is wrapped, it is executed normally like any other test inside Visual Studio.

This article by Subashni.S and N Satheesh Kumar talks about the manual, generic, and ordered test types in detail. We will go through the steps to create and manage these tests types in VSTS.

The following screenshot describes a simple web application, which has a page for the new user registration. The user has to provide the necessary field details. After entering the details, the user will click on the Register button provided in the web page to submit all the details so that it gets registered to the site. To confirm this to the user, the system will send a notification with a welcoming email to the registered user. The mail is sent to the email address provided by the user.

 Software Testing with Visual Studio Team System 2008

In the application shown in the above screenshot, the entire registration process cannot be automated for testing. For example, the email verification and checking the confirmation email sent by the system will not be automated as the user has to go manually and check the email. This part of the manual testing process will be explained in detail in this article.

Manual tests

Manual testing, as described earlier, is the simplest type of testing carried out by the testers without any automation tool. This test may contain a single or multiple tests inside. Manual test type is the best choice to be selected when the test is too difficult or complex to automate, or if the budget allotted for the application is not sufficient for automation.

Visual Studio 2008 supports two types of manual tests file types. One as text file and the other as Microsoft Word.

Manual test using text format

This format helps us to create the test in the text format within Visual Studio IDE. The predefined template is available in Visual Studio for authoring this test. This template provides the structure for creating the tests. This format has the extension of .mtx. Visual Studio servers act as an editor for this test format.

For creating this test in Visual Studio, either create a new test project and then add the test or select the menu option Test | New Test... and then choose the option to add the test to a new project. Now create the test using the menu option and select Manual Test (Text Format) from the available list as shown in the screenshot below. You can see the list Add to Test Project drop–down, which lists the different options to add the test to a test project.

If you have not yet created the test project and selected the option to create the test, the drop-down option selected will create a new test project for the test to be added. If you have a test project already created, then we can also see that project in the list to get this new test added to the project. We can choose any option as per our need. For this sample, let us create a new test project in C#. So the first option from the drop-down of Add to Test Project would be selected in this case. After selecting the option, provide the name for the new test project the system will ask for. Let us name it TestingAppTest project.

 Software Testing with Visual Studio Team System 2008

Now you can see the project getting created under the solution and the test template is also added to the test project as shown next. The template contains the detailed information for each section. This will help the tester or whoever is writing the test case to write the steps required for this test.

 Software Testing with Visual Studio Team System 2008

Now update the test case template created above with the test steps required for checking the email confirmation message after the registration process. The test document also contains the title for the test, description, and the revision history for the changes made to the test case.

 Software Testing with Visual Studio Team System 2008

Before executing the test and looking into the details of the run and the properties of the test, we will create the same test using Microsoft Word format as described in the next section.

Manual test using Microsoft Word format

This is similar to the manual test that was created using text format, except that the file type is Microsoft Word with extension .mht. While creating the manual test choose the template Manual Test (Word format) instead of the Manual Test (Text Format) as explained in the previous section. This option is available only if Microsoft Word is installed in the system. This will launch the Word template using the MS Word installed (version 2003 or later) in the system for writing the test details as shown in the following screenshot. The Word format helps us to have richer formatting capabilities with different fonts, colors, and styles for the text with graphic images and tables embedded for the test.

 Software Testing with Visual Studio Team System 2008

This document not only provides the template but also the help information for each and every section so that the tester can easily understand the sections and write the test cases. This help information is provided in both the Word and Text format of the manual tests.

In the test document seen in previous screenshot, we can fill the Test Details, Test Target, Test Steps, and Revision History similar to the one we did for the text format. The completed test case test document will look like this:

 Software Testing with Visual Studio Team System 2008

Save the test details and close the document. Now we have both formats of manual tests in the project. Open the Test View window or the Test List Editor window to see the list of tests we have in the project. It should list two manual tests with their names and the project to which the tests are associated with. The tests shown in the Test View window looks like the one shown here:

 Software Testing with Visual Studio Team System 2008

The same tests list shown by the Test List Editor would look like the one shown below. The additional properties like test list name, the project name the test belongs to, is also shown in the list editor. There are options for each test either to run or get added to any particular list.

 Software Testing with Visual Studio Team System 2008

Manual tests also have other properties, which we can make use of during testing. These properties can be seen in the Properties window, which can be opened by choosing the manual test either in the Test View or in the Test List Editor windows by right-clicking the test and selecting the Properties option. The same window can also be opened by choosing the menu option View | Properties window. Both formats of manual testing have the same set of properties.

 Software Testing with Visual Studio Team System 2008

Some of these properties are editable while some are read-only, which will be set by the application based on the test type. Some properties are directly related to TFS. The VSTFS is the integrated collaboration server, which combines team portal, work item tracking, build management, process guidance, and version control into a unified server.

Software Testing with Visual Studio Team System 2008 A comprehensive and concise guide to testing your software applications with Microsoft's Visual Studio Team System 2008 with this book and eBook
Published: December 2008
eBook Price: $29.99
Book Price: $49.99
See more
Select your format and quantity:

Property

Is Editable

Description

Associated Work Item

This is editable if part of VSTFS

We can associate this test with the work item in the VSTFS; the work item could be a task against which we are creating this test. So we can associate this test to the corresponding work item in TFS and easily map both. There are other different types of work items such as defects, requirements, issues, and reviews; we can associate this test with any type of work item for which we are creating this tests. The work item of type Task in TFS would look like this:

 Software Testing with Visual Studio Team System 2008

This property also provides a dialog window, which helps us to filter the available work items in TFS; based on the work items listed, we choose the work item to which this test has to be associated:

 Software Testing with Visual Studio Team System 2008

After selecting the work item you can see the work item ID is selected as the property value.

Deployment Items

Editable

Files or folders that need to be deployed along with this test; we need to give the full path of the files or folders that we are going to deploy. When ... button is pressed it opens the String Collection Editor dialog box. In the string collection editor, enter the path of the files and folders.Each line of text is considered as a separate string.

 Software Testing with Visual Studio Team System 2008

Description

Editable

This property is used for giving the detailed description of the test

Host Data

Editable

This explains the custom data to pass to the host adapter; this is very useful in deploying the test under a different host. By by default, the test is hosted under VSTestHost.exe

Host Type

Editable

This property explains the test execution environment. It contains the value Default, which means the default test host environment (VSTestHost.exe).

ID

Read-only

This property gives us the unique name of the test; it gives the full path of the manual test, that is, where it resides.

Iteration

This is editable if part of VSTFS

This property explains which iteration in the software project life cycle these tests belongs to; these values are based on the iterations set in the TFS. This property will provide a dialog with the list of iterations in TFS. From this, we can select the iteration for which we want to associate this test.

 Software Testing with Visual Studio Team System 2008

Non-runnable Error

Read-only

This property explains the reason of the manual test that cannot be executed in this test run. Even though a test is included in a test project, if this test cannot be executed in this test for some reason, then we can enter the reason in this property

Property

Is Editable

Description

Owner

Editable

This gives the name of the owner for the test

Priority

Editable

This property explains the priority of the test; this is an integer value explaining the relative importance of the test and tells which test needs to be executed first

Project

Read-only

This is the test project name to which the test belongs

Project Area

This is Editable if part of VSTFS

This property maps to the Area Path in the VSTFS. This property also provides the dialog similar to the Iteration with list of Areas created in TFS; from the list we can select the area path for this test

Project Relative Path

Read-only

This property tells the file name of the project where the test resides; the path is relative to the location of the solution

Solution

Read-only

This property denotes the name of the solution that contains the test project to which this test belongs

Test Enabled

Editable

This property allows us to make a test disabled for this test run; if you set this property as False this test won't be available for execution

Test Name

Read-only

To specify the name of the test.

Test Storage

Read-only

This property gives us the complete path of this test in the hard disk; for manual tests, this property is the same as the test ID property

Test Type

Read-only

This property tells us the type of the test as to whether the test is manual, or load, or web, or generic, or ordered. In this case, the value displayed is Manual; using this property we can filter to see all the tests that belong to the manual test type

Timeout

Read-only

This property tells us about the timeout value for the test, that is, it will specify how long the test can take to run before it is marked as failed, and aborted. For a manual test, this property is Infinite a as manual test doesn't have a timeout value

Software Testing with Visual Studio Team System 2008 A comprehensive and concise guide to testing your software applications with Microsoft's Visual Studio Team System 2008 with this book and eBook
Published: December 2008
eBook Price: $29.99
Book Price: $49.99
See more
Select your format and quantity:

After selecting the properties for the selected tests, the property window would look like the one shown below. These properties are the same for a manual test with the Word format.

 Software Testing with Visual Studio Team System 2008

Some of these properties are also useful in grouping or ordering the tests in the Test View window and the Test List Editor. Open the Test View window and on the column header of the window right-click and select the Add/Remove Columns... option.

 Software Testing with Visual Studio Team System 2008

This selection would open a dialog with all the properties of the tests from which we can select the properties to be listed in the Test View or the Test List Editor window.

 Software Testing with Visual Studio Team System 2008

On clicking OK, we can see the selected properties listed in the Test View. Now we can use these values to sort the tests listed in the grid.

 Software Testing with Visual Studio Team System 2008

Running a manual test

We are now ready with the test project, and manual test cases in different formats by setting the properties. We should now start with testing the test application by running the manual tests. Let us consider that the tests for creating the new user account using sample web application is complete, and now we should test the email confirmation sent to the email address provided during the account creation.

We can run the tests from the Test View window or the Test List Editor window. In case of multiple tests, we can use the Test List Editor so that we can select all the tests at once and run it. In case of Test View, we can run one test at a time. Select the test and right-click and select the Run Selection option or after selecting the tests choose the menu option Test | Run | Tests in Current Context or All Tests in Solution. This will start running the test. In this process the first message given by the system would be:

 Software Testing with Visual Studio Team System 2008

Click OK to accept the message, and upon accepting the message you would see another message saying the test is ready for execution.

 Software Testing with Visual Studio Team System 2008

When this message is accepted, the actual manual test will be shown to the tester. Based on the details provided in the test, the tester has to follow the steps mentioned in the test. In this example, the tester has to open the mail client that was provided during the registration process and check whether notification mail is there in the inbox.

Before actually completing the test, let us look at the Test Results window. Open the window using the menu option Test | Windows | Test Results. By default, it will show the result window with the current test status. Since we have started the test but not completed it yet, the status will be shown pending. You can check the status of all the tests run so far by changing the option in the Test Results window and selecting the option All in the Select Run drop-down.

 Software Testing with Visual Studio Team System 2008

In the Test Results window shown above, we can see different statuses for the tests run. They are also shown with different images to represent the status:

  • Aborted: This one denotes that the test has been aborted during the run.
  • Failed: This status denotes that the test run has failed because of some reason.
  • Passed: This is the status for a successful run of the test with an expected result.
  • Pending: This means that the test is still in progress. It has neither completed, nor failed or aborted. It is waiting for the tester or system to complete the test.

The individual test can be run from this window by selecting the test result from the Test Result drop-down. Choose the checkbox option against the test result and rerun the test using the Run option in the toolbar.

 Software Testing with Visual Studio Team System 2008

In case if the manual test is open and waiting for the tester to complete it, Test run list will show the test as pending. It means that the test is open and waiting for the tester to complete it as it is a manual test. As we have selected the test, which is the manual test in text format, we can see the test window look like this:

 Software Testing with Visual Studio Team System 2008

The window in the preceding screenshot is the same that we created during the design time of the manual test in text except the Comments, Select result, and Apply options. These are the options for submitting the result of the manual test performed by the tester. For example, in the test result shown in the preceding screenshot, you can see the checkbox Fail selected and the comment says that the email is not received. It means that the test has failed. So the tester has chosen the option Fail and put in the comments about the result from this test. After entering all the details, the tester has to complete the test by clicking the apply button on top left corner of the test window. Once the Apply button is pressed, the test window becomes read-only, and the test is completed. Now the manual test in the text format is executed with the result Failed. We can also see the status in the Test Results window.

The next manual test format in Word is almost similar to the text format, the only difference being the test opens in Microsoft Word, where we can have additional formatting and documentation features. The test run of the manual test in Word would look like this:

 Software Testing with Visual Studio Team System 2008

Manual tests are used for tracking the test steps, history, and the results. The actual testing of the application is done by the tester. This indirectly means that these tests cannot be run remotely, or by using the command line execution, as it always requires human intervention to carry out the testing. In case this is required, the tester has to note the steps and then go to the remote machine and execute the test as per the steps and then come back and apply with the results.

Ordered tests

We have created Unit tests, Web tests, Load tests, and other different types of tests. We might have a situation where some tests are dependent on others. For example, the Unit test should be run first before moving on to the other tests. In the previous examples, we have seen that the new user registration sends the confirmation email to the user. So the manual test for checking the email confirmation message is fully dependent on the Web test, which sends the email to the registered user. The ordered tests are very useful in placing or running the tests in order.

This section explains how to create the ordered test, and how to place the tests in order and run the same.

Creating an ordered test

There are different ways of creating ordered tests similar to the other tests:

  1. Select the test project from the Solution Explorer, right-click and select Add | Ordered Test and then select ordered test from the list of different types of tests. Save the ordered test by choosing the File | Save option.
  2. Select the menu option Test then select New Test..., which opens a dialog with different test types. Select the test and choose the test project from the Add to Test Project List and click on OK.

Now the ordered test will get created under the test project and the ordered test window is shown to select the existing tests in the project and order them as required.

 Software Testing with Visual Studio Team System 2008

The above window shows different options for ordering the tests.

The first line is the status bar, which shows the number of tests selected for the ordered test.

The Select test list to view drop-down has the option to choose the display of tests in the available test lists. This drop-down has the default /All Loaded Tests, which displays all the tests under the project. The other options in the drop-down are Lists of Tests and Tests Not in a List. The List of Tests will display the test lists created using the Test List Editor. It is easier to include the number of tests grouped together and order them. The next option Tests Not in a List will display the available tests, which are not part of any test lists.

The Available tests list displays all the tests from the project based on the option chosen in the drop-down.

The Selected tests contains the tests that are selected from the available tests list to be placed in order.

The two right and left arrows are used for selecting and unselecting the tests from the Available tests list to the Selected tests list. We can also select multiple tests by pressing the Ctrl key and selecting the tests.

The up-down arrows on the right of the Selected tests list are used for moving up or down the tests in the Selected tests list.

The last option Continue after failure checkbox at the bottom of the window is to override the default behavior of the ordered tests aborting the execution after the failure of any test. The tests in the order are dependent on the previous tests run result. If the option Continue after failure is unchecked, and if any test in the order fails, then all remaining tests will get aborted. In case the tests are not dependent, we can check this option and override the default behavior to allow the application to continue running the remaining tests in order.

Properties of an ordered test

Ordered tests have properties similar to the other test types, in addition to some specific properties. To view the properties, select the ordered test in Test View or Test List Editor window and right-click and select the Properties option. The Properties dialog box displays the available properties for the ordered test.

 Software Testing with Visual Studio Team System 2008

The above screenshot depicts that most of the properties are the same as the properties of the other test types. We can associate this test with the TFS work items, iterations, and area.

Executing an ordered test

An ordered test can be run like any other test. Open the Test View window or the Test List Editor and select the ordered test from the list, then right-click and choose the Run Selection option from Test View or Run Checked Tests from the Test List Editor. Once the option is selected, we can see the tests running one after the other in the same order in which they are placed in the ordered test. After the execution of the ordered tests, the Test Results window will show the status of the ordered test. If any of the tests in the list fails, then the ordered test status would be Failed. The summary of the status of all the tests in the ordered test is shown below the toolbar. The sample test application had five tests in the ordered tests, but two of them failed.

 Software Testing with Visual Studio Team System 2008

Clicking the Test run failed hyperlink in the status bar will show a detailed view of the test run summary:

 Software Testing with Visual Studio Team System 2008

The Test Results window also provides detailed information about the tests run so far. To get these details, choose the test from the Test Results window and then right-click and choose the option View Test Results Details, which will open the details window and the common results information such as test name, result, duration of the test run, start time, and end time.

 Software Testing with Visual Studio Team System 2008

The details window also displays the status of each and every test run within the ordered test. In addition it also displays the duration for each test run, name, owner, and type of test in the list. Even though the second test in the list fails, the other tests continue to execute as if the Continue after failure option were checked.

Generic tests

Generic tests are ways to integrate third-party tests into Visual Studio. There are situations where we would be dependent on third-party tests because of business requirements or because those tests were created when Visual Studio was not available in the organization. The generic tests act as wrappers for executing these third-party tests within the boundary of Visual Studio. Once it is wrapped, we can execute these generic tests like any other test in Visual Studio IDE.

The third-party tests should adhere to the following conditions to be categorized under the generic tests in Visual Studio:

  1. We must be able to execute the third-party tests from the command line.
  2. The third-party tool must return a Boolean value of either True or False when executed in the command line.
  3. It is preferred that the third-party test tool writes an XML file, which contains the list of individual tests. The XML file is the summary results file, which contains the details of all the tests run.

Creating a generic tests

This is similar to any other test in Visual Studio. Either use the Test menu and choose the options of creating a new test or right-click on the test project in the Solution Explorer and add a generic test, or open the Test View window and then right-click from there and add a new test. Then provide a name for the generic test. For this example, name it GenericTestforThirdPartyTest. A new window opens to set the values or parameters for the generic test.

 Software Testing with Visual Studio Team System 2008

You can see from the form that all the values we have to enter are for executing another test application from the command line by passing parameters. For a command line execution , we may have to set the environment variables, execution parameters, set the working directory, copy or deploy some files, and set the output directory and the file. All these details can be set using the generic test.

Parameters for Generic Test

 

Description

 

Specify an existing program

 

This is the name and path of the third-party test application to be executed at the command line; it is the name of the executable application. We can also use the browse button to the right of the text box to find the application and select it.

 

Command-line arguments to pass to the generic test

 

This is the place to specify the command line parameters required for the third-party test application; these parameters totally depend on the testing tool's expected value.

 

Additional files to deploy with this generic test

 

In some cases, there might be other files required for this test execution; add those files or remove the selected files in the list using the option to the right of the text box.

 

Environment variables

 

If the test application used any environment variables for the execution, we are required to set those environment variables for the execution; using this option we can set those environment variables to be used by the test application.

Working directory

This is to set the current working directory in the command line before we actually run the test application in the command line.

Redirect standard output and standard error to test result

While executing the test application, instead of displaying all the results at the command line, we can redirect those results to the output file, just as we do during the normal command line commands.

Exit test run if run duration (in milliseconds)exceeds

This is to limit the wait time for Visual Studio to move on to the next test in the list, or quit; these numbers denotes milliseconds and the default is 60 minutes.

Summary results file

This is helpful in case the third-party tests application can write the test results to an XML file; this is the name and path of the XML file in which the output results should be written. If the number of tests in the test application is more in number, then it will be easy to track the result of these individual tests by having the results in the XML file. Not only the result but also detailed information of the test result would be written to this file.

The following is an example of a generic test that executes the Test.exe, which is a third-party test application capable of writing the output to the XML file. The command-line parameter for this application is also provided along with the supporting file to be deployed, which is the Readme.txt file. You can see the Output.xml, which is to store the output details of the test by the Test.exe.

 Software Testing with Visual Studio Team System 2008

Summary results file

When we execute the above generic test, the third-party Test.exe will get executed at the command line. The generic test by Visual Studio will get the result back from the third-party Test.exe application, which is a single test. But we do not know how many tests are executed internally within the test, and it is not easy to track the results of all the tests of the third-party application using the generic test. But Visual Studio supports the third-party application with a summary results file, which can be used by the application to write the details of the internal test results.

The third-party applications can make use of the class file, which can be generated by using the schema file provided by Visual Studio. The schema file is located at the Visual Studio command-line. If Visual Studio is installed in the default c: drive, then the path would be:

C:Program FilesMicrosoft Visual Studio 9.0XmlSchemas
SummaryResult.xsd

The class file can be generated from this schema file using the xsd.exe utility on any .NET supported languages. The following is an example for generating the default SummaryResult.cs class file out of an XSD file. The output folder should exist before the command can be run. The c:temp is the output folder used in the following sample:

Xsd SummaryResult.xsd /c /l:cs /out:c:temp
 Software Testing with Visual Studio Team System 2008

The class file is the C# file as we have specified C# as the language in the command-line parameter as /l:cs . The generated output file would be like this:

//------------------------------------------------------------------------------
// <auto-generated>
// This code was generated by a tool.
// Runtime Version:2.0.50727.1433
//
// Changes to this file may cause incorrect behavior and will be lost if
// the code is regenerated.
// </auto-generated>
//------------------------------------------------------------------------------
using System.Xml.Serialization;
//
// This source code was auto-generated by xsd, Version=2.0.50727.1432.
//
/// <remarks/>
[System.CodeDom.Compiler.GeneratedCodeAttribute("xsd", "2.0.50727.1432")]
[System.SerializableAttribute()]
[System.Diagnostics.DebuggerStepThroughAttribute()]
[System.ComponentModel.DesignerCategoryAttribute("code")]
[System.Xml.Serialization.XmlTypeAttribute(AnonymousType=true)]
[System.Xml.Serialization.XmlRootAttribute(Namespace="", IsNullable=false)]
public partial class SummaryResult {
private string testNameField;
private testResultType testResultField;
private string errorMessageField;
private string detailedResultsFileField;
private SummaryResultInnerTest[] innerTestsField;
/// <remarks/>
public string TestName {
get {
return this.testNameField;
}
set {
this.testNameField = value;
}
}
/// <remarks/>
public testResultType TestResult {
get {
return this.testResultField;
}
set {
this.testResultField = value;
}
}
/// <remarks/>
public string ErrorMessage {
get {
return this.errorMessageField;
}
set {
this.errorMessageField = value;
}
}
/// <remarks/>
public string DetailedResultsFile {
get {
return this.detailedResultsFileField;
}
set {
this.detailedResultsFileField = value;
}
}
/// <remarks/>
[System.Xml.Serialization.XmlArrayItemAttribute("InnerTest",
IsNullable=false)]
public SummaryResultInnerTest[] InnerTests {
get {
return this.innerTestsField;
}
set {
this.innerTestsField = value;
}
}
}
/// <remarks/>
[System.CodeDom.Compiler.GeneratedCodeAttribute("xsd",
"2.0.50727.1432")]
[System.SerializableAttribute()]
public enum testResultType {
/// <remarks/>
Aborted,
/// <remarks/>
Error,
/// <remarks/>
Inconclusive,
/// <remarks/>
Failed,
/// <remarks/>
NotRunnable,
/// <remarks/>
NotExecuted,
/// <remarks/>
Disconnected,
/// <remarks/>
Warning,
/// <remarks/>
InProgress,
/// <remarks/>
Pending,
/// <remarks/>
PassedButRunAborted,
/// <remarks/>
Completed,
/// <remarks/>
Passed,
}
/// <remarks/>
[System.CodeDom.Compiler.GeneratedCodeAttribute("xsd",
"2.0.50727.1432")]
[System.SerializableAttribute()]
[System.Diagnostics.DebuggerStepThroughAttribute()]
[System.ComponentModel.DesignerCategoryAttribute("code")]
[System.Xml.Serialization.XmlTypeAttribute(AnonymousType=true)]
public partial class SummaryResultInnerTest {
private string testNameField;
private testResultType testResultField;
private string errorMessageField;
private string detailedResultsFileField;
/// <remarks/>
public string TestName {
get {
return this.testNameField;
}
set {
this.testNameField = value;
}
}
/// <remarks/>
public testResultType TestResult {
get {
return this.testResultField;
}
set {
this.testResultField = value;
}
}
/// <remarks/>
public string ErrorMessage {
get {
return this.errorMessageField;
}
set {
this.errorMessageField = value;
}
}
/// <remarks/>
public string DetailedResultsFile {
get {
return this.detailedResultsFileField;
}
set {
this.detailedResultsFileField = value;
}
}
}

The third-party tool can make use of the above class file to write the test result details, or the test application should take care of writing the test result details into the XML file based on the XML schema used. The results output XML file should look like this:

<?xml version="1.0" encoding="utf-8" ?>
<SummaryResult>
<TestName>Third party test Application</TestName>
<TestResult>Failed</TestResult>
<InnerTests>
<InnerTest>
<TestName>Export the Data to Excel</TestName>
<TestResult>Passed</TestResult>
<ErrorMessage></ErrorMessage>
<DetailedResultsFile></DetailedResultsFile>
</InnerTest>
<InnerTest>
<TestName>Import Data from Excel</TestName>
<TestResult>Failed</TestResult>
<ErrorMessage>Data Not in Expected format</ErrorMessage>
<DetailedResultsFile>D:TestingTrace.txt</DetailedResultsFile>
</InnerTest>
</InnerTests>
</SummaryResult>

In the previous example, we can see that there are two different tests within a single test. One is to export the data to Excel, which is passed, and the other test is to import the details from Excel, which is failed. The second test which failed writes detailed information about the test result to the text file. Writing into the log file should be taken care of by the third-party test application in the required format.

Now the Test Results window for this generic test would show the result as Failed as shown in the following screenshot:

 Software Testing with Visual Studio Team System 2008

Select the result in the Test Results window, then right-click and select View Test Results Details. This would show all the summary and detailed results based on the XML output as shown here:

 Software Testing with Visual Studio Team System 2008

You can notice the Inner Test Results grid where it shows the results of all the inner tests conducted by the third-party test application. If you look at the Test Results Details column, we can see the Trace.txt file for the second failed test. Now select the second row in the Inner Test Results grid. On selecting the row, the text file Trace.txt opens and the details would be shown in the Summary File area, which is nothing but the detailed information written by the application about the failed test:

 Software Testing with Visual Studio Team System 2008

The Summary File information is just the content of the text file given in the DetailedResultsFile, in the XML output file.

Properties of a generic test

Besides having some properties common with the other tests, generic tests also have some specific properties. To view the properties, select the generic test in Test View or Test List Editor window and right-click to select the Properties option. The Properties dialog box is then displayed with the available properties for the generic test:

 Software Testing with Visual Studio Team System 2008

Using the Properties window we can associate a test to a work item, an iteration, or a project area defined in the TFS. We can also set the input file as arguments, specify deployment items, and set the maximum duration for the test run and many other properties. Most of the properties such as deployment items, description, owner, priority, work item, iteration, and solution are common for all test types, whereas properties such as arguments and target executable are specific to the generic test type.

About the Author :


N Satheesh Kumar

N. Satheesh Kumar has Bachelor’s Degree in Computer Science Engineering and has around twelve years of experience in software development life cycle, project, and program management. He started his career developing software applications using Borland software products in a company based in India and then moved to the United Arab Emirates and continued developing applications using Borland Delphi and customizing Great Plain Dynamics (now known as Microsoft Dynamics) for an automobile sales company. He moved back to India and spent three years designing and developing application software using Microsoft products for a top Multinational Company in India and then spent couple of years in Project Management and Program Management activities. Now he works as a Technical Architect for a top retail company based in the United States. He works with the latest Microsoft technologies and has published many articles on LINQ and other features of .NET. He is also the author of the book LINQ Quickly in Microsoft .NET.

Contact N Satheesh Kumar

Subashni. S

Subashni S holds a Bachelor’s Degree in Computer Science engineering and has around 15 years of experience in software development and testing life cycle, project, and program management. She is a certified PMP (Project Management Professional), CSTM (Certified Software Test Manager), and ITIL V3 Foundation certified. She started her career as a DBA in Oracle 8i technology, and later developed many software applications using Borland software products for a multinational company based in Chennai, and then moved to Bangalore. She is presently working for a multinational company, in the area of Project Management for developing and testing projects. She is also currently working for one of the top multinational companies headquartered at Austin, Texas.

She has also authored Software Testing using Visual Studio Team System 2008 and Software Testing using Visual Studio 2010 for Packt Publishing.

Books From Packt

ASP.NET 3.5 Social Networking
ASP.NET 3.5 Social Networking

ASP.NET 3.5 Application Architecture and Design
ASP.NET 3.5 Application Architecture and Design

ASP.NET Data Presentation Controls Essentials
ASP.NET Data Presentation Controls Essentials

Microsoft Visual C++ Windows Applications by Example
Microsoft Visual C++ Windows Applications by Example

Entity Framework Tutorial
Entity Framework Tutorial

DotNetNuke Skinning Tutorial
DotNetNuke Skinning Tutorial

BlackBerry Enterprise Server for Microsoft® Exchange
BlackBerry Enterprise Server for Microsoft® Exchange

Microsoft AJAX Library Essentials: Client-side ASP.NET AJAX 1.0 Explained
Microsoft AJAX Library Essentials: Client-side ASP.NET AJAX 1.0 Explained

No votes yet

Post new comment

CAPTCHA
This question is for testing whether you are a human visitor and to prevent automated spam submissions.
H
t
H
c
L
f
Enter the code without spaces and pay attention to upper/lower case.
Code Download and Errata
Packt Anytime, Anywhere
Register Books
Print Upgrades
eBook Downloads
Video Support
Contact Us
Awards Voting Nominations Previous Winners
Judges Open Source CMS Hall Of Fame CMS Most Promising Open Source Project Open Source E-Commerce Applications Open Source JavaScript Library Open Source Graphics Software
Resources
Open Source CMS Hall Of Fame CMS Most Promising Open Source Project Open Source E-Commerce Applications Open Source JavaScript Library Open Source Graphics Software