In this article by Matteo Pescarin, the author of Learning Yii Testing, we will get introduced to Codeception. Not everyone has been exposed to testing. The ones who actually have are aware of the quirks and limitations of the testing tools they've used. Some might be more efficient than others, and in either case, you had to rely on the situation that was presented to you: legacy code, hard to test architectures, no automation, no support whatsoever on the tools, and other setup problems, just to name a few. Only certain companies, because they have either the right skillsets or the budget, invest in testing, but most of them don't have the capacity to see beyond the point that quality assurance is important. Getting the testing infrastructure and tools in place is the immediate step following getting developers to be responsible for their own code and to test it.
(For more resources related to this topic, see here.)
Even if testing is something not particularly new in the programming world, PHP always had a weak point regarding it. Its history is not the one of a pure-bred programming language done with all the nice little details, and only just recently has PHP found itself in a better position and started to become more appreciated.
Because of this, the only and most important tool that came out has been PHPUnit, which was released just 10 years ago, in 2004, thanks to the efforts of Sebastian Bergmann.
PHPUnit was and sometimes is still difficult to master and understand. It requires time and dedication, particularly if you are coming from a non-testing experience. PHPUnit simply provided a low-level framework to implement unit tests and, up to a certain point, integration tests, with the ability to create mocks and fakes when needed.
Although it still is the quickest way to discover bugs, it didn't cover everything and using it to create large integration tests will end up being an almost impossible task.
On top of this, PHPUnit since version 3.7, when it switched to a different autoloading mechanism and moved away from PEAR, caused several headaches rendering most of the installations unusable.
Other tools developed since mostly come from other environments and requirements, programming languages, and frameworks. Some of these tools were incredibly strong and well-built, but they came with their own way of declaring tests and interacting with the application, set of rules, and configuration specifics.
A modular framework rather than just another tool
Clearly, mastering all these tools required a bit of understanding, and the learning curve wasn't promised to be the same among all of them.
So, if this is the current panorama, why create another tool if you will end up in the same situation we were in before?
Well, one of the most important things to be understood about Codeception is that it's not just a tool, rather a full stack, as noted on the Codeception site, a suite of frameworks, or if you want to go meta, a framework for frameworks.
Codeception provides a uniform way to design different types of test by using as much as possible the same semantic and logic, a way to make the whole testing infrastructure more coherent and approachable.
Outlining concepts behind Codeception
Codeception has been created with the following basic concepts in mind:
Easy to read: By using a declarative syntax close to the natural language, tests can be read and interpreted quite easily, making them an ideal candidate to be used as documentation for the application. Any stakeholder and engineer close to the project can ensure that tests are written correctly and cover the required scenarios without knowing any special lingo. It can also generate BDD-style test scenarios from code test cases.
Easy to write: As we already underlined, every testing framework uses its own syntax or language to write tests, resulting in some degree of difficulty when switching from one suite to the other, without taking into account the learning curve each one has. Codeception tries to bridge this gap of knowledge by using a common declarative language. Further, abstractions provide a comfortable environment that makes maintenance simple.
Easy to debug: Codeception is born with the ability to see what's behind the scenes without messing around with the configuration files or doing random print_r around your code.
On top of this all, Codeception has also been written with modularity and extensibility in mind, so that organizing your code is simple while also promoting code reuse throughout your tests.
But let's see what's provided by Codeception in more detail.
Types of tests
As we've seen, Codeception provides three basic types of test:
Unit tests
Functional tests
Acceptance tests
Each one of them is self-contained in its own folder where you can find anything needed, from the configuration and the actual tests to any additional piece of information that is valuable, such as the fixtures, database snapshots, or specific data to be fed to your tests.
In order to start writing tests, you need to initialize all the required classes that will allow you to run your tests, and you can do this by invoking codecept with the build argument:
$ cd tests
$ ../vendor/bin/codecept build
Building Actor classes for suites: functional, acceptance, unit
FunctionalTester includes modules: Filesystem, Yii2
FunctionalTester.php generated successfully. 61 methods added
AcceptanceTester includes modules: PhpBrowser
AcceptanceTester.php generated successfully. 47 methods added
UnitTester includes modules:
UnitTester.php generated successfully. 0 methods added
$
The codecept build command needs to be run every time you modify any configuration file owned by Codeception when adding or removing any module, in other words, whenever you modify any of the .suite.yml files available in the /tests folder.
What you have probably already noticed in the preceding output is the presence of a very peculiar naming system for the test classes.
Codeception introduces the Guys that have been renamed in Yii terminology as Testers, and are as follows:
AcceptanceTester: This is used for acceptance tests
FunctionalTester: This is used for functional tests
UnitTester: This is used for unit tests
These will become your main interaction points with (most of) the tests and we will see why. By using such nomenclature, Codeception shifts the point of attention from the code itself to the person that is meant to be acting the tests you will be writing.
This way we will become more fluent in thinking in a more BDD-like mindset rather than trying to figure out all the possible solutions that could be covered, while losing the focus of what we're trying to achieve.
Once again, BDD is an improvement over TDD, because it declares in a more detailed way what needs to be tested and what doesn't.
AcceptanceTester
AcceptanceTester can be seen as a person who does not have any knowledge of the technologies used and tries to verify the acceptance criteria that have been defined at the beginning.
If we want to re-write our previously defined acceptance tests in a more standardized BDD way, we need to remember the structure of a so-called user story. The story should have a clear title, a short introduction that specifies the role that is involved in obtaining a certain result or effect, and the value that this will reflect. Following this, we will then need to specify the various scenarios or acceptance criteria, which are defined by outlining the initial scenario, the trigger event, and the expected outcome in one or more clauses.
Let's discuss login using a modal window, which is one of the two features we are going to implement in our application.
Story title – successful user login
I, as an acceptance tester, want to log in into the application from any page.
Scenario 1: Log in from the homepage
I am on the homepage.
I click on the login link.
I enter my username.
I enter my password.
I press submit.
The login link now reads "logout (<username>)" and I'm still on the homepage.
Scenario 2: Log in from a secondary page
I am on a secondary page.
I click on the login link.
I enter my username.
I enter my password.
I press Submit.
The login link now reads "logout (<username>)" and I'm still on the secondary page.
As you might have noticed I am limiting the preceding example to successful cases.
The preceding story can be immediately translated into something along the lines of the following code:
// SuccessfulLoginAcceptanceTest.php
$I = new AcceptanceTester($scenario);
$I->wantTo("login into the application from any page");
// scenario 1
$I->amOnPage("/");
$I->click("login");
$I->fillField("username", $username);
$I->fillField("password", $password);
$I->click("submit");
$I->canSee("logout (".$username.")");
$I->seeInCurrentUrl("/");
// scenario 2
$I->amOnPage("/");
$I->click("about");
$I->seeLink("login");
$I->click("login");
$I->fillField("username", $username);
$I->fillField("password", $password);
$I->click("submit");
$I->canSee("logout (".$username.")");
$I->amOnPage("about");
As you can see this is totally straightforward and easy to read, to the point that anyone in the business should be able to write any case scenario (this is an overstatement, but you get the idea).
Clearly, the only thing that is needed to understand is what the AcceptanceTester is able to do: The class generated by the codecept build command can be found in tests/codeception/acceptance/AcceptanceTester.php, which contains all the available methods. You might want to skim through it if you need to understand how to assert a particular condition or perform an action on the page. The online documentation available at http://codeception.com/docs/04-AcceptanceTests will also give you a more readable way to get this information.
Don't forget that at the end AcceptanceTester is just a name of a class, which is defined in the YAML file for the specific test type:
$ grep class tests/codeception/acceptance.suite.yml
class_name: AcceptanceTester
Acceptance tests are the topmost level of tests, as some sort of high-level user-oriented integration tests. Because of this, acceptance tests end up using an almost real environment, where no mocks or fakes are required. Clearly, we would need some sort of initial state that we can revert to, particularly if we're causing actions that modify the state of the database.
As per Codeception documentation, we could have used a snapshot of the database to be loaded at the beginning of each test. Unfortunately, I didn't have much luck in finding this feature working. So later on, we'll be forced to use the fixtures. Everything will then make more sense.
When we will write our acceptance tests, we will also explore the various modules that you can also use with it, such as PHPBrowser and Selenium WebDriver and their related configuration options.
FunctionalTester
As we said earlier, FunctionalTester represents our character when dealing with functional tests.
You might think of functional tests as a way to leverage on the correctness of the implementation from a higher standpoint.
The way to implement functional tests bears the same structure as that of acceptance tests, to the point that most of the time the code we've written for an acceptance test in Codeception can be easily swapped with that for a functional test, so you might ask yourself: "where are the differences?"
It must be noted that the concept of functional tests is something specific to Codeception and can be considered almost the same as that of integration tests for the mid-layer of your application.
The most important thing is that functional tests do not require a web server to run, and they're called headless: For this reason, they are not only quicker than acceptance tests, but also less "real" with all the implications of running on a specific environment. And it's not the case that the acceptance tests provided by default by the basic application are, almost, the same as the functional tests.
Because of this, we will end up having more functional tests that will cover more use cases for specific parts of our application.
FunctionalTester is somehow setting the $_GET, $_POST and $_REQUEST variables and running the application from within a test. For this reason, Codeception ships with modules that let it interact with the underlying framework, be it Symfony2, Laravel4, Zend, or, in our case, Yii 2.
In the configuration file, you will notice the module for Yii 2 already enabled:
# tests/functional.suite.yml
class_name: FunctionalTester
modules:
enabled:
- Filesystem
- Yii2
# ...
FunctionalTester has got a better understanding of the technologies used although he might not have the faintest idea of how the various features he's going to test have been implemented in detail; he just knows the specifications.
This makes a perfect case for the functional tests to be owned or written by the developers or anyone that is close to the knowledge of how the various features have been exposed for general consumption.
The base functionality of the REST application, exposed through the API, will also be heavily tested, and in this case, we will have the following scenarios:
I can use POST to send correct authentication data and will receive a JSON containing the successful authentication
I can use POST to send bad authentication data and will receive a JSON containing the unsuccessful authentication
After a correct authentication, I can use GET to retrieve the user data
After a correct authentication, I will receive an error when doing a GET for a user stating that it's me
I can use POST to send my updated hashed password
Without a correct authentication, I cannot perform any of the preceding actions
The most important thing to remember is that at the end of each test, it's your responsibility to keep the memory clean: The PHP application will not terminate after processing a request. All requests happening in the same memory container are not isolated.
If you see your tests failing for some unknown reason when they shouldn't, try to execute a single test separately.
UnitTester
I've left UnitTester for the end as it's a very special guy. For all we know, until now, Codeception must have used some other framework to cover unit tests, and we're pretty much sure that PHPUnit is the only candidate to achieve this. If any of you have already worked with PHPUnit, you will remember the learning curve together with the initial problem of understanding its syntax and performing even the simplest of tasks.
I found that most developers have a love-and-hate relationship with PHPUnit: either you learn its syntax or you spend half of the time looking at the manual to get to a single point. And I won't blame you.
We will see that Codeception will come to our aid once again if we're struggling with tests: remember that these unit tests are the simplest and most atomic part of the work we're going to test. Together with them come the integration tests that cover the interaction of different components, most likely with the use of fake data and fixtures.
If you're used to working with PHPUnit, you won't find any particular problems writing tests; otherwise, you can make use of UnitTester and implement the same tests by using the Verify and Specify syntax.
UnitTester assumes a deep understanding of the signature and how the infrastructure and framework work, so these tests can be considered the cornerstone of testing.
They are super fast to run, compared to any other type of test, and they should also be relatively easy to write.
You can start with adequately simple assertions and move to data providers before needing to deal with fixtures.
Other features provided by Codeception
On top of the types of tests, Codeception provides some more aids to help you organize, modularize, and extend your test code.
As we've seen, functional and acceptance tests have a very plain and declarative structure, and all the code and the scenarios related to specific acceptance criteria are kept in the same file at the same level and these are executed linearly.
In most of the situations, as it is in our case, this is good enough, but when your code starts growing and the number of components and features become more and more complex, the list of scenarios and steps to perform an acceptance or functional test can be quite lengthy.
Further, some tests might end up depending on others, so you might want to start considering writing more compact scenarios and promote code reuse throughout your tests or split your test into two or more tests.
If you feel your code needs a better organization and structure, you might want to start generating CEST classes instead of normal tests, which are called CEPT instead.
A CEST class groups the scenarios all together as methods as highlighted in the following snippet:
<?php
// SuccessfulLoginCest.php
class SuccessfulLoginCest
{
public function _before(CodeceptionEventTestEvent $event) {}
CodeceptionEventTestEvent $event
public function _fail(CodeceptionEventTestEvent $event) {}
// tests
public function loginIntoTheApplicationTest(AcceptanceTester $I)
{
$I->wantTo("login into the application from any page");
$I->amOnPage("/");
$I->click("login");
$I->fillField("username", $username);
$I->fillField("password", $password);
$I->click("submit");
$I->canSee("logout (".$username.")");
$I->seeInCurrentUrl("/");
// ...
}
}
?>
Any method that is not preceded by the underscore is considered a test, and the reserved methods _before and _after are executed at the beginning and at the end of the list of tests contained in the test class, while the _fail method is used as a cleanup method in case of failure.
This alone might not be enough, and you can use document annotations to create reusable code to be run before and after the tests with the use of @before <methodName> and @after <methodName>.
You can also be stricter and require a specific test to pass before any other by using the document annotation @depends <methodName>.
We're going to use some of these document annotations, but before we start installing Codeception, I'd like to highlight two more features: PageObjects and StepObjects.
The PageObject is a common pattern amongst test automation engineers. It represents a web page as a class, where its DOM elements are properties of the class, and methods instead provide some basic interactions with the page. The main reason for using PageObjects is to avoid hardcoding CSS and XPATH locators in your tests. Yii provides some example implementation of the PageObjects used in /tests/codeception/_pages.
StepObject is another way to promote code reuse in your tests: It will define some common actions that can be used in several tests. Together with PageObjects, StepObjects can become quite powerful. StepObject extends the Tester class and can be used to interact with the PageObject. This way your tests will become less dependent on a specific implementation and will save you the cost of refactoring when the markup and the way to interact with each component in the page changes.
For future reference, you can find all of these in the Codeception documentation in the section regarding the advanced use at http://codeception.com/docs/07-AdvancedUsage together with other features, like grouping and an interactive console that you can use to test your scenarios at runtime.
Summary
In this article, we got hands-on with Codeception and looked at the different types of tests available.
Resources for Article:
Further resources on this subject:
Building a Content Management System [article]
Creating an Extension in Yii 2 [article]
Database, Active Record, and Model Tricks [article]
Read more
United States
Great Britain
India
Germany
France
Canada
Russia
Spain
Brazil
Australia
Singapore
Canary Islands
Hungary
Ukraine
Luxembourg
Estonia
Lithuania
South Korea
Turkey
Switzerland
Colombia
Taiwan
Chile
Norway
Ecuador
Indonesia
New Zealand
Cyprus
Denmark
Finland
Poland
Malta
Czechia
Austria
Sweden
Italy
Egypt
Belgium
Portugal
Slovenia
Ireland
Romania
Greece
Argentina
Netherlands
Bulgaria
Latvia
South Africa
Malaysia
Japan
Slovakia
Philippines
Mexico
Thailand
