One thing I love most about WP-CLI is its Behat-based test suite. In fact, if you consider WP-CLI successful at all, I'd attribute said success to the test suite.
Having a great test suite ensures exceptional build quality because:
- It’s easy to write new tests, which means they actually get written.
- The tests interface with a command in the same manner as users interface with a command, and they describe how the command is expected to work in human-readable terms.
WordPress should have amazing integration tests too.
Historically, PhantomJS has been the de-facto standard for headless browser testing. PhantomJS is also Yet Another Abandoned Open Source project, and generally a pain to deal with. But we're in luck! Headless Chrome shipped in Chrome 59. It's pretty amazing. For your next side project, try out the Puppeteer library.
Now that we're over the headless browser hurdle, implementing automated browser testing is simply a matter of:
- Picking a framework. Behat is one option (see WordHat for an example), and Codeception is another. I like Behat, but Codeception is based on PHPUnit which might be a perk.
- Setting up a Docker container to provide a WordPress install in an isolated environment. For ease of use, Chrome headless could be provided in another container.
- Writing some tests. Prior to writing tests though, it would be helpful to plan out all major UX flows we want to cover with tests.
In fact, I'd argue that integration tests will be key for managing breakage when Gutenberg lands in WordPress core.
What would be really neat is if the test suite supported importing some base "state", such that it'd be possible to easily run the same test against dozens of scenarios. This would let us perform an experiment where run the test suite against every plugin in the WordPress.org plugin directory.