Building a quality gate for an AI website audit.

Checking a page before publication

I designed Quality Gate to help a publishing team check accessibility and SEO before release. It coordinated two specialist agents, combined their findings and returned a PASS, FAIL or INCOMPLETE verdict with links to the available reports.

I wrote the coordinating instruction and both specialist agents. I also built a tool to render JavaScript-heavy pages so the agents could audit the page’s content rather than an empty application shell.

Orchestrating without a workflow engine

Workflow Agents were unavailable in the assessment environment. A workflow would normally call the specialists in order, pass their results along and decide what happens when one fails. Without it, the coordinating instruction had to do that work itself, so I wrote it as a procedure rather than a request.

The instruction set thresholds up front and called each specialist once, allowing a retry only for unusable output. It then merged their scores by a fixed rule. Missing or unusable scores were normalised to null, which led to INCOMPLETE. To make the instruction discoverable, I listed the phrases that should invoke it and directed the model to use it for those requests instead of searching for or creating another agent with the same name.

What counts as a pass

The assessment policy required an accessibility score of at least 80 and an SEO score of at least 75. Passing depended on both thresholds, so a strong SEO result could not make up for an accessibility result below the threshold. The weighted total was reported alongside the verdict but never decided it.

On the final run, Logitech’s software page scored 89 for accessibility and 67 for SEO, and failed on SEO alone. Both specialists had found its missing H1 independently.

A PASS met the assessment’s scoring rules; it did not establish WCAG conformance. Automated accessibility checks still need human evaluation.

What each agent hands back

Each specialist had two readers: the orchestrator needed structured results, and a person needed a readable report. The accessibility checker put its findings in a PDF and returned just two lines: a score and the report link. The SEO checker also produced a PDF, and returned its section findings and prioritised fixes in strict JSON. It validated that JSON before answering.

ACCESSIBILITY_SCORE: 72
PDF_LINK: <report link>

{
  "seo_score": 90,
  "sections": {…},
  "fixes": […],
  "pdf": "<report link>"
}

Accessibility  72 / 100
SEO            90 / 100
Weighted total 81
Data quality   ok, ok
Status         FAIL
Top fixes      1 … n
Reports        2 links

Designing against invented results

Most of the rules in these instructions stop an agent from filling a gap with something plausible.

1. The orchestrator names no accessibility violations.

   It received a score and a link, so it points to the report rather than describing findings it never read.

2. Unreadable output counts as missing.

   SEO output that was not valid JSON was treated as unusable. The orchestrator never tried to extract meaning from prose.

3. Lighthouse is a reference point.

   Each specialist scored the page against its own rubric. A Lighthouse score could sit beside that score but never replace it.

4. Scores come from fixed deductions.

   Accessibility failures cost points by WCAG level; SEO was split across five weighted sections. A section that listed a fix could not receive full marks.

5. No evidence, no score.

   When the page could not be read well enough to judge, the specialist marked its score as unavailable. The orchestrator treated that as null and reported INCOMPLETE.

Rendering the page for the audit

Rendered Audit Bridge is a Node.js service with two tools. The renderer opens a page in Chromium, then returns an HTML excerpt and structured fields such as headings, image attributes and metadata. The other tool wraps Google’s PageSpeed Insights API, returning compact Lighthouse category scores.

Each specialist had fallbacks. If rendering failed, each one stepped down to a plain HTML fetch and then page text, stopping at the first that worked; the accessibility checker could fall back further, to a screenshot. Text alone could not support an SEO score, so that path returned findings with a null score.

Diagnosing runaway token use

Individual runs repeatedly consumed around 1.6 million tokens while I was developing the workflow. Full HTML and large audit responses were passing through the agent pipeline. I fixed it in two places. The tool returned capped excerpts, extracted fields and compact scores. The SEO instruction limited reading to the page head and a small slice of the body. Both specialists were told to avoid full PageSpeed payloads, and the accessibility checker could retry a tool only once. The orchestrator received compact results and report links instead of the full reports.

Making truncated input visible

Capping the response introduced another problem: an agent could treat something missing from the excerpt as missing from the website. The tool therefore reports truncation, original and excerpt lengths, and the limits applied. The SEO instruction carried the same rule: a tag absent from a truncated excerpt was reported as not found in the excerpt, and a missing critical tag earned one targeted follow-up fetch.

{
  "rendered": true,
  "truncated": true,
  "body_limit_kb": 8
}

What I proposed next

The local Playwright renderer ran throughout the assessment. I also built a second renderer, using Puppeteer and serverless Chromium on Vercel, and it hit Chromium compatibility problems. The fix looked straightforward, but with a firm deadline and the report and presentation still to prepare, I disabled it and kept the local renderer.

Each specialist made its own render request. My proposed next steps were one shared render for both specialists, parallel checks, orchestration in Workflow Agents, and a finished cloud deployment.

Explore the repository Related assessment: investigating a GraphQL schema