Bill from Georgetown writes:
Just curious, what made you choose Behat for [WP-CLI] testing rather than PHPUnit like WordPress core, EDD and a lot of other big WordPress plugins?
Is it just for easier readability of tests, or is there more to it?
I have a few projects that need unit testing and am trying to decide what tool to use.
Thanks Bill! Happy to take a swing at answering your question.
But first, let’s take a little step back to understand what is a Behat and what is a PHPUnit.
Behat is “an open source Behavior Driven Development framework for PHP”. Behavior driven development (BDD) is different than test driven development (TDD) in that BDD tests the overall behavior of an application, as opposed to individual units of functionality. For instance, the Behat test for wp option get looks something like this:
When I run `wp option add str_opt 'bar'` Then STDOUT should not be empty When I run `wp option get str_opt` Then STDOUT should be: """ bar """
These Behat assertions test the behavior of the application (WP-CLI) because they perform the tests in the same way a user might actually use WP-CLI. For each assertion, the entirety of WP-CLI is loaded and executed, in addition to the command being tested. BDD makes it much easier to say “Yes, WP-CLI is working as the user expects it to work.”
On the other hand, PHPUnit is “a programmer-oriented testing framework for PHP”. PHPUnit is better suited for unit tests, and some forms of integration tests. WordPress has a PHPUnit-based test suite you can use in your own plugin. Writing unit tests (and TDD generally) is different than BDD because you’re writing tests for a more limited scope of the codebase. For instance, the PHPUnit test for get_option() doesn’t reload the entire codebase each time, it only tests the function itself:
function test_the_basics() {
$key = rand_str();
$key2 = rand_str();
$value = rand_str();
$value2 = rand_str();
$this->assertFalse( get_option( 'doesnotexist' ) );
$this->assertTrue( add_option( $key, $value ) );
$this->assertEquals( $value, get_option( $key ) );
}
Unit tests are typically more precise. Because they execute less of the codebase, you can run hundreds or thousands of unit tests in the time it takes to run one integration test.
Back to the original question: why did I choose Behat over PHPUnit for WP-CLI? Easy answer: I didn’t, scribu did!
In hindsight, I’ve been very happy with the decision because I think Behat / BDD has helped ensure a high degree of quality for WP-CLI. But, if I were to be known for any postulate, it would be:
Tests only get written when they’re easy to write and run.
I’ve typically opted for PHPUnit (and Mocha) tests for plugins and themes because running automated tests against code rendering in a browser requires using a headless browser in your test suite, which adds complexity, which means tests are less likely to be written.