“Unit testing” a WordPress widget

I recently finished developing my very first WordPress widget. It’s called GR Progress, and it shows books from your Goodreads shelves together with their reading progress (which, to my knowledge, no other Goodreads or WordPress widgets can do). In this post I want to talk about how I went about “unit testing” the widget.

This post provides a lot of code samples, some slightly simplified. If you want the full context, you can find the files at GitHub.

Like most times I sit down and code something entirely new, I wasn’t initially doing test-​driven development. TDD requires you to at least have the slightest idea of how you want to solve a problem, and I was just wildly experimenting back and forth, throwing rocks at the wall to see what sticks.

Naturally, the code quickly became too complex to know at a glance whether things still worked. I was constantly doing a lot of refactoring, and I soon really started to miss having unit tests I could run all the time to check for regressions. In my own experimental projects, that’s how I know the time has come to add tests.

But how should I test the widget? I had never before tested PHP code. Furthermore, most of my properties and methods were encapsulated to make it clear to myself what was implementation details and what was the interfaces I could use between classes, and I didn’t want to test specific implementation details anyway – they were apt to change significantly in both the short and long term.

Testing the widget HTML

The only thing I really cared about was the final output, i.e., the HTML that is produced. Specifically, I wanted to make sure that 1) the widget output was valid HTML, and 2) the output HTML contained everything it should.

That still left me with the choice of whether to test just the back-​end stuff of my widget, or to test the full WordPress integration. Testing just the widget back-​end would mean I have to mock out internal WordPress functions, such as setting and getting data from temporary storage. That would mean I couldn’t test the interplay between multiple widgets in the WordPress options storage – not without perfect knowledge of how WordPress stores multi-​widget options and replicating it myself, which would be kind of counter-​productive. Besides, I wanted to test that I had used the WordPress API correctly.

So I decided to have tests that integrated the widget in a test WordPress installation and rendered the widget (which is actually really easy), and checked the widget output for the correct information. What I ended up doing was a mix between unit tests (testing small, specific concepts) and integration test (testing the whole system), in the sense that I have many small tests that test one specific thing, but each test render the full widget HTML (i.e., run most of the codebase from input to output) using specific input settings and check if the output is correct. This may be what’s called (black-​box) functional testing, though I haven’t been able to find a very good or consistent definition of what functional testing really is.

To help me get started, I ran WP-​CLI, which basically just dumped a few files into my source folder: a WordPress installation script, a PHPUnit bootstrap script, a PHPUnit configuration file (I’ve made some edits), a travis.yml (I’ve added some WordPress and PHP versions to the file) for easy CI testing on Travis, and a sample test case (which I threw away).

When creating test cases for WordPress plugins, you normally inherit from WP_UnitTestCase. However, all of my tests would require very similar functionality, and would in the simplest case consist of two basic operations: 1) Given a specific set of settings, get the corresponding widget output HTML, and 2) check that the HTML contains the correct information. Here are a few examples of checking for correct information in my widget:

• Check that the widget contains the correct books in the correct order
• Check that the books have the correct author names
• Check that the books have or doesn’t have progress bars (depending on input settings and whether the books actually have reading progress on Goodreads)
• Check that these progress bars contain or do not contain certain strings (for testing if the correct progress is displayed, and for testing the “time since last update” functionality)
• etc.

To reduce duplication, I made my own test case framework class which inherited from WP_UnitTestCase, and then I made my test cases inherit from my own framework class instead of WP_UnitTestCase. For example, here is the framework method(s) for getting the widget HTML given an input array of settings to override:

/**
 * Returns the HTML used for rendering the widget.
 * @param array $overrideSettings Key-value pairs of settings to override
 * @param boolean $deleteCachedHTML If true, the widget HTML cache is deleted
 * @return string
 */
public function getWidgetHTML($overrideSettings = []) {
  $settings = $this->getSettingsWithOverride($overrideSettings);

  $widget = new gr_progress_cvdm_widget();

  ob_start();
  $widget->widget($this->DEFAULT_ARGS, $settings);
  $html = ob_get_clean();

  return $html;
}

private function getSettingsWithOverride($overrideSettings) {
  $settings = $this->DEFAULT_SETTINGS;
  foreach ($overrideSettings as $k => $v) {
    $this->assertArrayHasKey(
      $k, $settings,
      "Attempted to override non-existent setting $k");
    $settings[$k] = $v;
  }
  return $settings;
}

And here is the framework method for checking whether the widget contains books in a specific order (using PHP Simple HTML DOM Parser, which is also used in the actual widget back-end):

/**
 * Asserts that the book titles in the given HTML contains the
 * substrings given in $bookNameSubstrings, in order.
 * @param string[] $bookNameSubstrings
 * @param string $html
 */
public function assertBooksOnShelf($bookNameSubstrings, $html) {
  $dom = str_get_html($html);
  $bookTitlesActual = [];
  foreach ($dom->find('.bookTitle') as $bookTitleElement) {
    $bookTitlesActual[] = $bookTitleElement->plaintext;
  }

  $this->assertCount(
    count($bookNameSubstrings),
    $bookTitlesActual,
    'Shelf does not contain expected number of books');

  for ($i = 0; $i < count($bookNameSubstrings); $i++) {
    $this->assertContains(
      $bookNameSubstrings[$i],
      $bookTitlesActual[$i],
      "Wrong book on index $i");
  }
}

This means I can write many of my tests in a neat and compact way – in this case, simply one line for getting the widget HTML, and another for checking if the correct books appear on the in the correct order (though I’ve split the lines here since my blog is not as wide as my IDE):

public function testCorrectBooksOnTwoSimultaneousShelves() {
  $html = $this->getWidgetHTML(
    ['shelfName' => 'to-read',
     'sortBy' => 'position',
     'sortOrder' => 'a']);
  $this->assertBooksOnShelf(
    ['The Name of the Wind',
     'The Eye of the World',
     'His Dark Materials'],
    $html);
}

Mixing production and testing code… Sorry!

“But wait”, you say, being an unusually observant reader. “This plugin fetches data from Goodreads. Are you really doing that every time you run the tests?”

No, I’m most decidedly not. Firstly, I don’t want my tests to fail if Goodreads has problems. Secondly, my tests would take a whole minute to run instead of 5 seconds, which is a bad thing because your tests should be so quick that you willingly run them all the time. Thirdly, it would breach (at least momentarily) the Goodreads API Terms of Service by performing any given API call more than once per second.

I could most likely mock the Goodreads fetching calls. But mocking seemed to require some additional libraries, and I had already spent a lot of time on this plugin – I wanted to get back to actually reading books. I ended up doing something very simple and not really recommended since it mixes production code with testing code: Factoring out the Goodreads fetching functionality into its own static utility class, and including static properties which can be toggled at runtime to disable Goodreads fetching (getting data from local files instead), or even force a fetch failure based on the URL (to make sure I handle failures correctly). Here’s the static method that’s called to perform the fetching:

public static function fetch($url) {
  if (!empty(self::$fail_if_url_matches) &&
      preg_match(self::$fail_if_url_matches, $url)) {
    $result = false;
  } elseif (self::$test_local) {
    $result = self::fetchFromFile($url);
  } else {
    $result = self::fetchFromGoodreads($url);
  }

  return $result;
}

Note that fetchFromFile() has a fallback to fetchFromGoodreads() if the local file doesn’t exist – it then calls the Goodreads API normally and saves the response to a file to be used the next time. The local files are checked into the Git repository and are considered part of the source. That way, it’s easy to run tests on Travis or on other machines without having to fetch anything from Goodreads. And if something in the Goodreads API response should change, I simply delete the local files and they will be re-​created when I run the tests.

I also created a dedicated Goodreads user for testing this plugin. That way, I can have full control of which books are on which shelves, what the reading progress is, etc.

It would be possible to mock away everything and return dummy data, but the Goodreads API has a lot of quirks and bugs that I need to handle correctly, so it’s safer and much easier to just test using the actual data returned by Goodreads.

Testing admin form and validating user input

Finally, a very important step is testing how the widget (or plugin) settings are updated. For WordPress widgets, the method you override to update the settings (called update()) takes two arrays as input (the previous widget settings and the new widget settings from the admin widget configuration form) and returns the new widget settings. This is where you validate and filter the user input, which as everyone doing web development should know, is of paramount importance for security reasons.

My test case framework has a method which more or less wraps the widget’s update() method, and takes as input the settings you want to override (it makes use of getSettingsWithOverride() provided earlier):

public function getNewSettings($overrideSettings = []) {
  $settings = $this->getSettingsWithOverride($overrideSettings);

  $widget = new gr_progress_cvdm_widget();

  $newSettings = $widget->update($settings, $this->DEFAULT_SETTINGS);
  return $newSettings;
}

This allows me to write the following simple method for checking whether a setting with a given input value is transformed to a given output value before being saved:1

public function assertSettingSavedAs($setting, $inputValue, $expectedOutput) {
  $settings = $this->getNewSettings([$setting => $inputValue]);
  $this->assertSame($expectedOutput, $settings[$setting]);
}

And that allows me to write the following very simple checks for text inputs: 1) the setting actually works, 2) HTML is escaped, and 3) leading and trailing whitespace is removed:

public function assertSettingTextIsSaved($setting) {
  $this->assertSettingSavedAs($setting, "foobar", "foobar");
}

public function assertSettingHTMLEscaped($setting) {
  $this->assertSettingSavedAs(
    $setting,
    "<script>evil()</script>foobar",
    "<script>evil()</script>foobar");
}

public function assertSettingWhitespaceRemoved($setting) {
  $this->assertSettingSavedAs($setting, " foobar ", "foobar");
}

In fact, I have sufficiently many text inputs that I ended up making another helper method to collect these three checks:

public function assertPassesAllTextInputTests($setting) {
  $this->assertSettingTextIsSaved($setting);
  $this->assertSettingHTMLEscaped($setting);
  $this->assertSettingWhitespaceRemoved($setting);
}

This makes my actual tests trivially small:

public function test_title() {
  $this->assertPassesAllTextInputTests('title');
}

Other settings which are not simple text inputs require a bit more thought, but are just as small thanks to assertSettingSavedAs():

public function test_userid_numeric() {
  $this->assertSettingSavedAs('userid', '55769144', '55769144');
  $this->assertSettingSavedAs('userid', '7691', '7691');
}

public function test_userid_numericAndName() {
  $this->assertSettingSavedAs('userid', '55769144-abcd', '55769144');
}

public function test_sortBy() {
  $this->assertSettingSavedAs('sortOrder', 'a', 'a');
  $this->assertSettingSavedAs('sortOrder', 'd', 'd');
  $this->assertSettingSavedAs('sortOrder', 'foobar', 'd');
}

Testing that it works well with popular themes

Finally, I installed the 10-​ish most popular WordPress Themes on my development installation and checked that the widget looked fine on each of them. I had to make some changes to the stylesheet here and there to fix visual problems on a few themes, so I’m glad I checked.

And that’s it, really. No eloquent punchline, no summary. I just wanted to throw this out there to help others in the same situation, and to sort out my own thoughts.

Possibly helpful further reading

• Unit Tests for WordPress Plugins
• Introduction to WordPress Unit Testing
• Writing Unit Tests for WordPress