(For more resources related to this topic, see here.)
To manage the increased number of tests and to ensure their execution and quick and effective analysis, robot framework can be used which is a tool written in python which can be used to write different tests that may call other testing tools/software which perform actual tests and report back to the test framework. You should use robot framework because of various advantages that it offers, some of which is detailed in the following section:
- XTest compliant
- Library support
Robot framework tests are uniform and readable from the end user's perspective and even if different technologies are involved in different tests for the same software, the test structure (since it is written in robot framework) for all the tests is consistent across the test project.
Robot framework is also xtest compliant, which means that it follows the same set of testing jargon and process as in other popular testing tools like JUnit, NUnit, and so on; also the developer associated with writing tests in robot framework has the least of surprises.
Robot framework is supported by a vide variety of libraries for third party tools and given its robust community, it is easy to create a custom library as well.
Not only the tests are uniform, but they may also be written according to the needs of the testing project. It is not uncommon to extend robot framework using keywords and associate tests to various keywords. This reduces complexity and brings about changes in writing tests. Test writing strategies like simple, keyword driven or behavior driven tests can be created and the same test can be written in many interesting ways.
Robot framework is network friendly tool, and can also be used remotely to collect and share information. It also works with various network protocols straight out of box and presents advantages of a mature tool that can communicate and control tests present remotely.
Owing to the open source nature(licensed under Apache version 2.0) of the robot framework, it is possible to see how it has been created and not get limited to its binary. For large scale customization, the source can directly be tweaked to perform work as desired.
Being a python based tool, it can work in python and python based environments such as jyton or ironpython which ensures that tests can be executed for newer application platforms like the java and .Net natively. The installations of various environments can be obtained directly or by building the source code .
As mentioned by the build script, by using appropriate python instance to call the install.py, robot framework can be installed in different environments where the different executable are generated pertaining to the environment.
For python environment, following executable are generated: pybot and rebot.
For java environment, following executable are generated: jybot and jyrebot.
For .net environment, following executable are generated: ipybot and ipyrebot.
Note that in all three environments, the test structure and syntax is the same. What differs is the target applications which could be clr/jvm dependant and running robot framework on the same environment offers it the flexibility to access the software under test (SUT) more clearly.
Test Naming Conventions
In order to name tests, robot framework is very peculiar; it uses the configuration file and folder names to determine the execution order and test naming.
For example, consider the following test hierarchy:
Running the pybot in the application folder will result in the following:
The testsuite will be named after the root of test configuration, in this case, tests.
Next, the Test1.txt will result in a nested testsuite, Test1 within testsuite tests. Similarly the testsuite another test will be nested under the tests/other tests testsuites.
In order to organize the results, it is important to ensure proper naming. So, instead of using spaces, underscore '_' should be used. In order to get proper ordering irrespective of the test names, prefixing the test configuration with numbers can be done.
This test hierarchy will therefore result in more understandable tests as well as the execution order of the tests can easily be predetermined.
As mentioned, the test hierarchy is similar to any other XUnit based testing solution.
Here, note that you can go on inserting test suites with themselves, which is different from other testing tool hierarchies. This approach gives more flexibility over the tests and introduces a new level of nesting.
Running tests in robot framework is also very descriptive and the results of the tests can instantly be seen.
In a similar manner, the results of the test can be viewed not only in the form of a detailed report, but also in the form of a log.
The test results are also saved in an XML format where the tests can be recreated/merged/compared with other tests at a later date. This is achieved through the rebot command which can recreate the test structure in the generated report and log files through reading the metadata present in the XML file.
Through this article, you can clearly see the benefits of grouping and structuring the tests using robot framework and allowing for scalability in the tests by making the tests encapsulated within one another through the use of nested test suites. Robot Framework will not only integrated with your testing tool/solution, but also act as an assistant for your tests and manage, run and report their results back to you in a concise manner.
Resources for Article:
- Cross-browser-distributed testing [Article]
- Testing Backbone.js Application [Article]
- Python Testing: Installing the Robot Framework [Article]