Visual Regression Testing of Stencil Web Components with Happo.io

Happo + Stencil = TLF

Visual regression testing (VRT) automatically monitors your web application to ensure no unintended visual changes have been made. As web developers, we are constantly making updates to our UI. Whether we’re adding a new feature, fixing a bug, improving some logic, or refactoring code, we often run the risk of accidentally changing the appearance of something unrelated. If we modify a CSS selector, or a component that’s used in multiple locations, it can be easy to miss some of the resulting ripples, and traditional testing techniques like unit and integration tests don’t offer a lot of assistance. 

Fortunately, there are two, newer techniques available that take different approaches. One technique is to use snapshot testing, which verifies that our HTML structure hasn’t changed. Alternatively, to get a pixel-to-pixel comparison of how our content actually appears, we can use VRT. There are multiple solutions available, and if you’re using React to build your UI, then you might have heard about Happo.io.

Our Story Begins

Indeed, this is how we began using Happo at Manifold. It was designed for testing React components, and creating new screenshots is just as easy as writing a little bit of React code. But where Happo really shines it its GitHub integration! This allows you to run Happo when you make a pull request. If Happo detects any differences, it can block your PR from being merged until you either approve the changes or fix them.

We were doing lots of React development at the time, so this worked really well for us, but when we began developing our Marketplace-as-a-Service, we needed to create a tiny, embeddable UI. So we shifted to using the Stencil framework to create Web Components that our customers could easily drop into their own applications. 

At this point, we had to determine if Happo would still fit the bill, or if a different solution would work better for us. But the great thing about Web Components is that they’re part of web platform itself, and it just so happens that Happo allows us to take screenshots of plain HTML and JavaScript! So as long as we can get our web components registered in the environment that Happo runs in, we should be able to append them to the document and take screenshots, and everything will Just Work™️!

At first, though, it didn’t. But with a little bit of support and a whole lot of learning about how Stencil and Happo both work, we were able to achieve even more than I expected. Indeed, it turned out that Happo and Stencil were accidentally as perfect for each other as PB&J. Let me tell you why, and show you how! What follows is the journey that I took - including a couple of stumbling points - to get Happo working beautifully with our Stencil components — even the ones that make real API calls! 

The Plain Basics

One nice thing about Web Components is that they can be added to the page in the same way as any HTML element. The Happo docs indicate that you can use the DOM API to append HTML elements to the document, and that Happo will then take a screenshot of the content you appended. So it follows that you should be able to do the same for Web Components. Before we can do this though, we have to make sure to configure Happo correctly, as shown below:

```js

// .happo.js
module.exports = {
  type: 'plain',
  // ...
};

```

By using a value of ‘plain’ for the `type` field, we are telling Happo that our tests are using plain HTML and JavaScript instead of React. With that in place, let’s see a basic example of adding a paragraph to a document!

```js

export const paragraphText = () => {
  const p = document.createElement('p');
  p.innerText = "Hello, I'm a simple paragraph!";
  document.body.appendChild(p);
};

```

Here we’ve created a paragraph element, assigned it some text content, and appended it to the document body. If we run Happo, we should see the following screenshot included in our report:


A Custom Twist

Since this works so well, perhaps the obvious next step is to just try it with a Web Component, but there are some preliminary steps that I knew I’d need to take in order to register our components in the CustomElementRegistry. Stencil components are written using JSX and other goodies that make it much easier to write Web Components, so a build step is required to compile them to vanilla Web Components. The compilation process also produces a defineCustomElements function that defines all of our Web Components in the registry. So we’ll want to run our build step before running Happo, and then register our elements as part of the Happo setup. The first part of that implies that we should add a script to `package.json` like this:

```js

"scripts": {  
  "happo": "stencil build && npm run set-environment-variables && happo run",
}

```

Defining our Happo task this way will ensure that our components are compiled and that you’ve set any necessary environment variables, like HAPPO_API_KEY and HAPPO_API_SECRET, before running your visual regression tests. The next step is to configure Happo to run a setup script before it starts, which will allow us to register our custom elements. We’ll do this by modifying the Happo configuration.

```js
// .happo.js
module.exports = {
  type: 'plain',
  setupScript: path.resolve(__dirname, 'happoSetup.js'),
  // ...
};

```

Here we’ve added a `setupScript` field that points to a file within the same folder called `happoSetup.js`. We’re telling Happo to execute this script before running the test suite. The contents of the setup script should look like this:

```js

// happoSetup.js
import { defineCustomElements } from './dist/loader';

defineCustomElements(window);

```

This defines our Web Components in the registry, so we should be able to use them in our Happo examples just like regular HTML elements. At this point in my journey, the next obvious step was to create a Happo test that looks like this:

```js

export const basic = () => {
  const toast = document.createElement('manifold-toast');
  toast.textContent = 'Basic';

  document.body.appendChild(toast);
};

```

I bubbled with excitement as I ran my Happo task and waited for the report to be generated. Then the bubbles popped. It didn’t work at all! Instead of a report, I saw errors in my console. 

Please Send Halp!

I reached out to Henric Trotzig, the creator of Happo, for support. Making Happo work for Web Components was really important to Henric, as he demonstrated many times throughout this experience by helping to debug and making tweaks to his service. In this case, he needed to expose another configuration option.

```js

// .happo.js
module.exports = {
  type: 'plain',
  prerender: false,
  setupScript: path.resolve(__dirname, 'happoSetup.js'),
};

```
Adding a prerender key with a value of `false` to the configuration got my Happo suite to run and produce reports! I could finally see my `manifold-toast` component screenshot, but there was one more problem: the screenshot was cut off so that the bottom part of the element was not included in the image, as shown below.

This behavior was sporadic, so it was hard to make sense of it. Again we turned to Henric for support. After some debugging, he had an explanation. When a Happo example function finishes executing, Happo measures the content to determine how big the image should be before taking the screenshot. It turned out that in some cases, there was no visible content on the page at the time that Happo made this calculation! This happened because Stencil wasn’t quite finished rendering, which is actually a result of a very helpful feature of Stencil: a parent component will often wait for an asynchronous child component to finish before it renders.

For example, perhaps you’ve defined a Stencil component that renders an `img` tag. Stencil knows that the image source is retrieved asynchronously, so it will actually wait to render your Web Component until that image has been fetched and can be displayed. And it’s turtles all the way down with this scheme, so even if that image tag is rendered by a grandchild or a great-grandchild, the top level component still won’t render until the image is ready. Fortunately for us, Stencil components expose a function called `componentOnReady`; this returns a promise that resolves when a component has finally rendered all of its contents! To make our screenshots more reliable, we can return this promise from our example function.

```js

// manifold-toast.happo.ts
export const basic = () => {
  const toast = document.createElement('manifold-toast');
  toast.textContent = 'Basic';

  document.body.appendChild(toast);

  return toast.componentOnReady();
};

```
This works because Happo supports asynchronous functions; if you return a promise from your example function, Happo will wait for this promise to resolve before making any calculations. Now we’re full speed ahead! Behold, a screenshot that shows this example, as well as the same `manifold-toast` component in some different states:

Our journey up to this point has definitely contained some gotcha moments, but we were able to fill all the gaps with some help from Henric coupled with a deeper understanding of how Stencil components work. And it just so happens that this `componentOnReady` functionality was the key to allowing us to test components that make real API calls!

Full Stack VRT

A common pattern we see in component based programming is to split a single component into two components that are sometimes referred to as smart and dumb components, or alternatively, containers and presentational components. In this separation, we have an outer component that makes an API call, performs some state management, and renders another component that’s only responsible for displaying the data. This technique is often employed to get around the difficulties of testing components that connect to an API. When we introduced Happo into our suite of testing tools, we experienced this challenge ourselves, and wound up making this same separation to overcome it. We later found that we didn’t have to do this at all!

The problem that we faced was really a timing issue. Happo runs in a real browser, so there are no obstacles to making real API requests. But we couldn’t find a reliable way to ensure that we’d waited long enough for the API call to complete before taking the screenshot, so the image would often be captured while the component was still loading. It took us a while to put two and two together, but in the end, `componentOnReady` came to the rescue yet again.

The trick here is to leverage Stencil’s asynchronous lifecycle methods. When you return a promise from Stencil’s `componentWillLoad` lifecycle, it won’t render your component until the promise resolves. This is similar to how Stencil treats image elements, but in this case, you decide whether or not to opt in. If you want to display some loading state while your request is in progress, then you can skip the asynchronous lifecycle methods altogether. But if you don’t want your component to display anything until all the data has been fetched, then this technique is for you! And it’s just the ticket for our use case, because `componentOnReady` will resolve after our component is fully rendered with all of the API data. That means that we can capture it with Happo without guessing at acceptable wait times!

As an example, let’s take a look at the manifold-plan-selector. This accepts a unique product label from our catalog, and displays an interface that allows a user to select which plan they want. The first step is to make the lifecycle asynchronous by returning a promise:

```js
  componentWillLoad() {
    return this.fetchPlans(this.productLabel);
  }

```

Now that this is async, we can write a Happo test and be certain that it won’t take a screenshot until all of the plans are displayed. Here’s an example that displays the plan selector for our JawsDB service:

```js

export const jawsDB = async () => {
  const conn = document.createElement('manifold-connection');
  document.body.appendChild(conn);

  const selector = document.createElement('manifold-plan-selector');
  selector.productLabel = productJawsDB.label;

  document.body.appendChild(selector);

  await selector.componentOnReady();
};

```

This example produces the screenshot shown below, using real data from our API. This is an enormous win!

Final Thoughts

Although Happo was designed to test React components, it wound up being a better fit for Stencil than any of us could have hoped for. Coupling Happo’s ability to wait for a promise along with the `componentOnReady` function exposed by Stencil components allowed us to achieve a bonafide full stack screenshot. It’s hard to overstate how big of a deal this is! The challenge we overcame is practically ubiquitous, leading UI developers everywhere towards some combination of extensive mocking and employing patterns of separation that serve little purpose beyond testability. We certainly needed a little help from Henric along the way, and we had to learn a whole lot about how Stencil and Happo both work to get to where we are, but it’s all smooth sailing from here.

Nonetheless, it’s important to call out the things that you might not easily cover with VRT. Its main purpose is to show what your components look like in different states, and make sure that they don’t accidentally change. Testing for side effects that occur when users interact with the components is not really what it’s for, so you should still be writing other kinds of tests for that. Furthermore, the full stack testing that I demonstrated only touched the public portion of our API; much of our GraphQL API is protected by authentication, so to test components that display private data, we might still want to use mocking, or figure out a way to authenticate with some kind of test account. So clearly, not all problems have been solved here, but we’ve achieved some truly incredible things.

Recent posts

Related posts

New
Create and manage pricing plans with the new Plan Builder. Read our blog post
Explore Plan Builder