Engineering Lab 05

What the release QA matrix actually checks

Question: What does representative browser QA actually verify before release?

Finding: The workflow exercises representative routes at desktop and mobile sizes, scrolls the pages, records runtime failures, and enforces Lighthouse thresholds on a smaller benchmark set.

Evidence source

What this note is grounded in

These labels identify the implementation or verification surface used for the observation. They are not a claim that the result generalizes to every website.

01

Playwright route matrix

02

Desktop and mobile viewports

03

Lighthouse thresholds

A release checklist is easy to write and easy to ignore.

A browser test is more useful because the application has to produce the expected result.

The current sitewide quality workflow gives us a concrete matrix to inspect.

The browser viewports

The representative browser pass runs at two viewport sizes:

  • 1440 by 900 for desktop
  • 390 by 844 for mobile

The mobile pass is important because a page that looks stable on a large desktop can still expose horizontal overflow, broken fixed positioning, image problems, or navigation behavior at a narrow width.

The test also requests reduced motion.

That makes the static layout easier to evaluate without depending on reveal effects or decorative animation.

The route matrix

The browser workflow samples several route families rather than checking only the homepage.

The current matrix includes examples from:

  • homepage
  • service pages
  • location pages
  • portfolio index and project pages
  • contact
  • blog index and article
  • about

That is deliberate.

A route family can break while the homepage remains perfect. Dynamic portfolio and content pages deserve direct coverage because they use different templates and data.

What happens on each route

The test opens the page and waits for the initial document to load.

It then scrolls through the page in steps.

That forces more of the actual experience to execute, including lazy-loaded images and elements that appear farther down the document.

After that, the workflow checks several invariants.

Structural failures

The route fails if it does not contain exactly one H1 or exactly one main landmark.

Those rules are simple, but they catch template regressions that are easy to miss visually.

The workflow also measures horizontal overflow. A page that extends beyond the viewport by more than a trivial amount is treated as broken.

Runtime failures

The browser records:

  • unexpected console errors
  • page-level JavaScript errors
  • failed local responses
  • images that completed loading with zero natural width

Those checks matter because a production build can compile while a component still fails at runtime.

A bad image path, client-side exception, or broken internal request may never appear in the compiler output.

Screenshots are evidence, not the only test

Representative routes are also captured as screenshots.

That gives a human reviewer useful visual evidence, but screenshots are not the quality gate by themselves.

The automated checks still inspect document structure, runtime behavior, overflow, and broken resources.

A page can look fine in a static image while logging errors in the background.

The Lighthouse set is narrower

Running Lighthouse on every route would make the workflow much more expensive and noisy.

Instead, the current quality pass uses a representative group including the homepage, a core SEO service page, major location pages, a portfolio page, and contact.

The thresholds are:

  • SEO: 100
  • accessibility: at least 95
  • best practices: at least 95
  • performance: at least 90

The workflow also records diagnostics such as FCP, LCP, Speed Index, Total Blocking Time, CLS, render-blocking resources, unused JavaScript, main-thread work, and total payload.

Why performance is not set to 100

A perfect synthetic performance score is not the product.

Different pages have different media, content, and third-party requirements.

The 90 threshold functions as a release floor for the representative set while the diagnostic metrics tell us what the page is spending.

If a useful business feature reduces a synthetic score slightly but the experience remains fast and stable, the decision should be evaluated in context.

What this workflow does not prove

Representative QA is not exhaustive proof that every possible device, browser extension, network, and user path is perfect.

It is a repeatable release signal.

The value comes from making common regressions expensive to ignore.

A page that introduces horizontal overflow, a broken image, a duplicate H1, or a client exception now has to get through a system designed to notice those problems.

Why publish this

A technical agency should be able to explain what "QA" means.

This matrix gives that word a concrete meaning.

The public Release Quality Gate describes the standard. This Lab note exposes the mechanics behind it.

Related documentation

Go from the observation to the standard

Delivery & Operations

Why regression prevention is part of the engineering, not just QA

Why regression prevention belongs in the engineering system itself, with repeatable checks that catch breakage before visual QA has to rediscover it.

Technical SEO

Why we validate SEO in the build instead of relying on a checklist

Why we inspect generated production HTML for metadata, canonicals, schema, links, images, robots state, and other SEO regressions before release.

Performance

Why we treat Lighthouse as a diagnostic, not a scoreboard

A high Lighthouse score is useful evidence, but the real goal is a site that renders quickly, stays stable, works across devices, and does not hide real problems behind a single number.