# Progressive Threshold Management for Repositories With Existing Debt

A first full scan of a four-year-old application does not produce a bug list, it produces a number: 4,812 violations across 214 routes, spread over 63 distinct rule ids, with color-contrast alone accounting for 2,190 of them. No team fixes that in a sprint, and a gate that fails on any violation fails on every pull request, which means it gets removed within a week. This guide is part of CI/CD Integration & Automated Quality Gating, and it covers the two mechanisms that make a gate useful on day one anyway — a fingerprinted baseline of known findings and a numeric budget per rule or area — plus the comparison logic that separates a new violation from an old one and the ratchet that lowers the ceiling as findings get fixed.

# Problem Statement

The gate has to answer one question on every pull request: did this change make accessibility worse? Answering it requires distinguishing the 4,812 findings that were already there from the one that arrived in this diff, and there are only two ways to do that. Either the pipeline holds a record of every known finding and checks membership, or it holds a count and checks arithmetic. Those are the baseline and the budget, and they fail in opposite directions.

A baseline is precise and brittle. It names each finding, so the gate can say “aria-required-children on the order-history table is new” instead of “serious violations went from 96 to 97”. The cost is that a finding has to be recognised across commits, and the only handle the scanner gives is a CSS selector generated from the DOM at scan time — div:nth-child(3) > ul > li:nth-child(7) > button. Wrap that list in a flex container and every selector under it changes, so a refactor that fixed nothing reports 40 fixed findings and 40 new ones on the same nodes. Fingerprinting is what stops that.

A budget is robust and blunt. It is a number, it never churns, it survives any refactor, and it tells the team nothing about what changed. Worse, a budget expressed as a single total is actively exploitable: a developer can delete a region violation on a footer and introduce a label violation on a checkout field, and the total stays at 4,812, so the gate passes while the product gets meaningfully worse for a screen-reader user. Budgets are only safe when they are keyed — per rule, per impact, or per area of the repository.

In practice a working setup uses both. The budget is the day-one gate because it takes an afternoon to build and cannot produce false failures; the baseline arrives once the team wants the gate to name the finding it is complaining about; and the ratchet is what keeps either of them from becoming a permanent allowance that nobody ever lowers.

Key implementation targets:

• A committed baseline file listing every known finding by fingerprint, generated once from a clean scan of the default branch.
• A fingerprint function that survives cosmetic DOM change: rule id, nearest stable ancestor, accessible name — never an nth-child chain.
• A budget file keyed by rule id (and optionally by area) with a per-key ceiling, reviewed like any other config change.
• A comparison step that classifies every finding in a run as new, fixed or unchanged, and fails only on new.
• A scheduled ratchet that lowers each ceiling to the lowest count recently observed, so fixed findings cannot silently be re-spent.
• Reporting that makes the budget diff visible in the pull request, so raising a ceiling is a conversation rather than a commit nobody notices.

# Prerequisites

• Node.js 20.x or newer — the scripts below use node:crypto, top-level await and native ESM.
• axe-core 4.10 or newer, driven by @axe-core/playwright 4.10, so results.violations[].nodes[].target is available.
• A repeatable production-mode build served on a fixed port, because a dev-server overlay injects its own DOM and pollutes the baseline.
• A route list checked into the repository (a11y/routes.json), not crawled at scan time — a crawler that finds a different page set produces phantom new and fixed findings.
• Write access to .github/workflows/ and permission to add a required status check in branch protection.
• A bot token with contents: write and pull-requests: write for the ratchet job to open its own pull request.
• A CODEOWNERS entry on a11y/budget.json and a11y/baseline.json so a raised ceiling needs a specific reviewer.

# Baseline Versus Budget

Before writing either file, be clear about which question each one answers, because teams routinely build a baseline, discover it churns, and conclude that progressive gating does not work. The two artifacts are not competing implementations of the same idea; they store different things and catch different regressions.

There is a third artifact teams reach for that is worth naming so it can be rejected: the suppression list, usually a set of axe rule ids passed to disableRules or a list of CSS selectors passed to exclude. A suppression is invisible in reports, it applies to code that has not been written yet, and it silently protects new violations of the same rule. A baseline entry expires the moment the node is fixed; a disabled rule never expires. Keep exclude for third-party iframes the team genuinely does not control, and put everything else in the baseline where it stays countable.

# 1. Generating the Initial Baseline

The baseline is generated from the default branch, never from a feature branch, and never from a partial run. A baseline built from a scan that crashed halfway through records 60% of the findings, and the remaining 40% arrive as “new” on the next pull request that touches anything. Guard against that explicitly: assert the route count and fail the generator if any route errored.

Run the scan through Playwright so route waiting, authentication and the settled-DOM signal are the same in generation and enforcement. The scanner emits one JSON file per route into a11y/runs/current/, and a second script folds those into the baseline.

// a11y/scan.mjs — usage: node a11y/scan.mjs
// Writes one result file per route into a11y/runs/current/ plus a manifest.
import { mkdir, writeFile, rm } from 'node:fs/promises';
import { readFileSync } from 'node:fs';
import { chromium } from 'playwright';
import AxeBuilder from '@axe-core/playwright';

const BASE = process.env.A11Y_BASE_URL ?? 'http://127.0.0.1:4173';
const routes = JSON.parse(readFileSync('a11y/routes.json', 'utf8'));
const outDir = 'a11y/runs/current';

await rm(outDir, { recursive: true, force: true });
await mkdir(outDir, { recursive: true });

const browser = await chromium.launch();
const context = await browser.newContext({
  // Freeze anything that would make two scans of one commit differ.
  viewport: { width: 1280, height: 900 },
  colorScheme: 'light',
  locale: 'en-GB',
  timezoneId: 'UTC',
  reducedMotion: 'reduce', // stops mid-animation contrast readings
});

const manifest = { scannedAt: new Date().toISOString(), routes: [], errors: [] };

for (const route of routes) {
  const page = await context.newPage();
  try {
    await page.goto(`${BASE}${route.path}`, { waitUntil: 'load' });
    await page.locator(route.readySelector ?? 'main').waitFor({ timeout: 15_000 });
    await page.waitForFunction(() => !document.querySelector('[aria-busy="true"]'));
    const results = await new AxeBuilder({ page })
      .withTags(['wcag2a', 'wcag2aa', 'wcag21a', 'wcag21aa', 'wcag22aa'])
      .analyze();
    const file = `${route.id}.json`;
    await writeFile(`${outDir}/${file}`, JSON.stringify(results));
    manifest.routes.push({ id: route.id, path: route.path, file });
  } catch (error) {
    manifest.errors.push({ id: route.id, message: String(error.message).slice(0, 200) });
  } finally {
    await page.close();
  }
}

await browser.close();
await writeFile(`${outDir}/manifest.json`, JSON.stringify(manifest, null, 2));

if (manifest.errors.length > 0) {
  console.error(`${manifest.errors.length} route(s) failed to scan; refusing to continue.`);
  process.exit(2); // exit 2 = the scan itself is untrustworthy, not an a11y failure
}
console.log(`Scanned ${manifest.routes.length} route(s) into ${outDir}`);

Note the exit code split, which matters more than it looks: 2 means the measurement failed and 1 is reserved for a genuine accessibility regression. A pipeline that returns 1 for both trains reviewers to treat every red accessibility job as an infrastructure flake. The wider convention for these codes is worked through in the auto-fail versus warning workflows guide.

Generation then reduces each route’s violations array to fingerprints. Keep the human-readable fields — rule id, impact, anchor, name, one route — in the file alongside the hash, because a baseline nobody can read is a baseline nobody will prune.

// a11y/baseline-generate.mjs — usage: node a11y/baseline-generate.mjs
import { readFileSync, writeFileSync } from 'node:fs';
import { collectFindings } from './findings.mjs';

const findings = collectFindings('a11y/runs/current');
const byPrint = new Map();

for (const finding of findings) {
  const existing = byPrint.get(finding.fingerprint);
  if (existing) existing.count += 1;
  else byPrint.set(finding.fingerprint, { ...finding, count: 1 });
}

const baseline = {
  generatedAt: new Date().toISOString(),
  fingerprintVersion: 3, // bump when the fingerprint recipe changes
  total: findings.length,
  entries: [...byPrint.values()].sort((a, b) =>
    a.rule.localeCompare(b.rule) || a.fingerprint.localeCompare(b.fingerprint),
  ),
};

writeFileSync('a11y/baseline.json', JSON.stringify(baseline, null, 2) + '\n');
console.log(`Baseline: ${baseline.total} findings, ${baseline.entries.length} fingerprints`);

Sorting the entries deterministically is what makes the file reviewable. An unsorted baseline produces a 4,000-line diff every time it is regenerated, and a 4,000-line diff is approved without being read.

# 2. The Fingerprint Function

A fingerprint has to identify the same defect across commits while distinguishing two different defects on the same page. Three inputs get that balance right: the rule id, the path of stable ancestors above the node, and the node’s accessible name. Everything else the DOM offers — position among siblings, generated class names, hashed ids, inline styles — is churn.

“Stable ancestor” means an element whose presence is a product decision rather than a layout decision: a landmark, a labelled region, a <form>, a <dialog>, a <table>, or an element the team has explicitly marked with data-a11y-anchor. Layout wrappers are skipped entirely, which is precisely why inserting a flex container changes nothing.

The anchor chain and the accessible name both have to be computed in the page, where the DOM and the accessibility tree exist. Collect them immediately after analyze(), in the same browser context, and store them on each node before the results leave the page.

// a11y/annotate.mjs — run inside the page, right after AxeBuilder().analyze()
// Adds { anchor, name, shape } to every violation node so Node can hash it.
export const annotateInPage = (violations) =>
  violations.map((violation) => ({
    ...violation,
    nodes: violation.nodes.map((node) => {
      const el = document.querySelector(node.target.join(' '));
      if (!el) return { ...node, anchor: 'unresolved', name: '', shape: '' };

      // Stable = the element exists because of a product decision, not a layout one.
      const STABLE = 'main,nav,header,footer,aside,form,dialog,table,fieldset,' +
        '[data-a11y-anchor],[role="region"],[role="dialog"],[role="tabpanel"]';
      const parts = [];
      for (let cur = el.parentElement; cur; cur = cur.parentElement) {
        if (!cur.matches(STABLE)) continue; // layout wrappers never reach the hash
        const label = cur.getAttribute('aria-label')
          ?? cur.getAttribute('data-a11y-anchor')
          ?? cur.querySelector(':scope > caption, :scope > legend')?.textContent?.trim()
          ?? '';
        parts.unshift(label ? `${cur.localName}:${label}` : cur.localName);
      }

      // Accessible name via the same computation a screen reader would use.
      const name = (el.ariaLabel || el.getAttribute('aria-label') || el.textContent || '')
        .replace(/\s+/g, ' ').trim().slice(0, 60);

      return {
        ...node,
        anchor: parts.slice(-4).join('>') || 'document', // 4 levels is enough context
        name,
        // Shape is the invariant part of the element itself, never its classes.
        shape: [el.localName, el.getAttribute('role') ?? '', el.getAttribute('type') ?? '']
          .join('|'),
      };
    }),
  }));

Hashing happens in Node, where the recipe can be versioned. The fingerprintVersion field is not decoration: changing the recipe invalidates every entry, so the comparison step must refuse to run against a baseline generated by a different version rather than reporting 4,812 new findings.

// a11y/findings.mjs
import { createHash } from 'node:crypto';
import { readFileSync, readdirSync } from 'node:fs';

export const FINGERPRINT_VERSION = 3;

// Route id is deliberately excluded for shared chrome: a header defect must not
// count 214 times. Rules tagged page-scoped keep the route in the hash.
const PAGE_SCOPED = new Set(['document-title', 'html-has-lang', 'landmark-one-main',
  'page-has-heading-one', 'bypass', 'region']);

export function fingerprint({ rule, anchor, name, shape, route }) {
  const scope = PAGE_SCOPED.has(rule) ? route : '';
  const material = [FINGERPRINT_VERSION, rule, anchor, shape, name, scope].join('::');
  return createHash('sha1').update(material).digest('hex').slice(0, 12);
}

export function collectFindings(dir) {
  const manifest = JSON.parse(readFileSync(`${dir}/manifest.json`, 'utf8'));
  const out = [];
  for (const entry of manifest.routes) {
    const results = JSON.parse(readFileSync(`${dir}/${entry.file}`, 'utf8'));
    for (const violation of results.violations) {
      for (const node of violation.nodes) {
        const finding = {
          rule: violation.id,
          impact: node.impact ?? violation.impact ?? 'minor',
          anchor: node.anchor ?? 'document',
          name: node.name ?? '',
          shape: node.shape ?? '',
          route: entry.path,
        };
        out.push({ ...finding, fingerprint: fingerprint(finding) });
      }
    }
  }
  return out;
}

export function readDirNames(dir) {
  return readdirSync(dir).filter((f) => f.endsWith('.json') && f !== 'manifest.json');
}

Two deliberate compromises are worth stating plainly. First, the accessible name is part of the hash, so rewording a button’s label reports the old finding as fixed and a new one in its place — annoying, but the alternative (dropping the name) merges every unlabelled icon button in a toolbar into one entry and hides three of the four. Second, the anchor chain is truncated to four levels; deeper chains encode component nesting that changes for reasons nobody cares about. Both choices trade a little churn for a lot less ambiguity, and the churn shows up as a matched pair in the comparison output where a reviewer can recognise it.

# 3. The Comparison Step

The comparison is a set operation on multisets, because the same fingerprint can legitimately occur several times — twelve unlabelled buttons in one table, all with the same anchor and empty name. Treating the baseline as a set instead of a multiset means eleven of those twelve go unnoticed forever.

// a11y/compare.mjs — usage: node a11y/compare.mjs
// exit 0 = no new findings, 1 = new findings, 2 = baseline unusable
import { readFileSync, writeFileSync } from 'node:fs';
import { collectFindings, FINGERPRINT_VERSION } from './findings.mjs';

const baseline = JSON.parse(readFileSync('a11y/baseline.json', 'utf8'));
if (baseline.fingerprintVersion !== FINGERPRINT_VERSION) {
  console.error(`Baseline is v${baseline.fingerprintVersion}, code is ` +
    `v${FINGERPRINT_VERSION}. Regenerate the baseline on the default branch.`);
  process.exit(2);
}

const remaining = new Map(baseline.entries.map((e) => [e.fingerprint, e.count]));
const current = collectFindings('a11y/runs/current');

const added = [];
let unchanged = 0;

for (const finding of current) {
  const left = remaining.get(finding.fingerprint) ?? 0;
  if (left > 0) {
    remaining.set(finding.fingerprint, left - 1); // consume one allowance
    unchanged += 1;
  } else {
    added.push(finding); // no allowance left: this occurrence is new
  }
}

// Anything with allowance still unconsumed no longer occurs in the app.
const fixed = baseline.entries
  .filter((e) => (remaining.get(e.fingerprint) ?? 0) > 0)
  .map((e) => ({ ...e, count: remaining.get(e.fingerprint) }));

const report = { added, fixed, unchanged, baselineTotal: baseline.total };
writeFileSync('a11y/comparison.json', JSON.stringify(report, null, 2) + '\n');

console.log(`unchanged ${unchanged} · fixed ${fixed.reduce((s, f) => s + f.count, 0)} ` +
  `· new ${added.length}`);
for (const f of added.slice(0, 25)) {
  console.log(`NEW  ${f.impact.padEnd(8)} ${f.rule.padEnd(24)} ${f.anchor} — "${f.name}"`);
}
process.exit(added.length > 0 ? 1 : 0);

Read the two failure signatures in that output. A count of 29 new against 52 fixed, all with the same rule ids, is fingerprint churn from a refactor — regenerate the baseline in the same pull request and note it in the description. A count of 29 new against 0 fixed, concentrated in one rule on one route, is a real regression in the diff. The distinction is the reason the script prints both numbers on one line instead of just failing.

# 4. Committing the Budget File

The budget is the second gate and the one that catches what the baseline cannot: an occurrence that happens to hash to an existing fingerprint, and, more importantly, the slow trade where fixed findings are replaced with new ones of a different rule. Key it by rule id and give each rule an impact so a raise can be reviewed in proportion to what it costs users.

{
  "fingerprintVersion": 3,
  "policy": { "newRuleDefault": 0, "totalCeiling": 4812 },
  "rules": {
    "color-contrast":        { "max": 2190, "impact": "serious" },
    "link-name":             { "max": 412,  "impact": "serious" },
    "aria-required-children":{ "max": 118,  "impact": "critical" },
    "label":                 { "max": 96,   "impact": "critical" },
    "region":                { "max": 214,  "impact": "moderate" },
    "heading-order":         { "max": 331,  "impact": "moderate" },
    "image-alt":             { "max": 87,   "impact": "critical" },
    "duplicate-id-aria":     { "max": 44,   "impact": "minor" }
  }
}

newRuleDefault: 0 is the load-bearing line. Any rule id absent from the file has a ceiling of zero, so upgrading axe-core — which regularly adds rules — surfaces the new rule as a failure that has to be triaged rather than absorbed silently into a total. totalCeiling exists only as a backstop against a scan that somehow doubles; it is never the primary check, for the reason the next figure makes concrete.

Enforcement is a short script, but the review process around the file matters more than the code. Put a11y/budget.json in CODEOWNERS under the accessibility group, and have the pull-request job comment the diff of every ceiling that moved, in both directions. A raised ceiling should be a visible, attributable decision with a linked ticket; a lowered one is worth celebrating in the same comment.

// a11y/budget-check.mjs — usage: node a11y/budget-check.mjs
import { readFileSync } from 'node:fs';
import { collectFindings } from './findings.mjs';

const budget = JSON.parse(readFileSync('a11y/budget.json', 'utf8'));
const counts = new Map();
for (const f of collectFindings('a11y/runs/current')) {
  counts.set(f.rule, (counts.get(f.rule) ?? 0) + 1);
}

const breaches = [];
for (const [rule, count] of [...counts].sort((a, b) => b[1] - a[1])) {
  const max = budget.rules[rule]?.max ?? budget.policy.newRuleDefault;
  const impact = budget.rules[rule]?.impact ?? 'unknown';
  if (count > max) breaches.push({ rule, count, max, impact, over: count - max });
}

const total = [...counts.values()].reduce((a, b) => a + b, 0);
if (total > budget.policy.totalCeiling) {
  breaches.push({ rule: 'TOTAL', count: total, max: budget.policy.totalCeiling,
    impact: 'n/a', over: total - budget.policy.totalCeiling });
}

for (const b of breaches) {
  console.error(`OVER BUDGET  ${b.rule} ${b.count}/${b.max} (+${b.over}, ${b.impact})`);
}
// Slack that a ceiling was undershot: the ratchet will claim it later.
const slack = [...counts].filter(([r, c]) => (budget.rules[r]?.max ?? 0) > c).length;
console.log(`${breaches.length} breach(es); ${slack} rule(s) below ceiling`);
process.exit(breaches.length > 0 ? 1 : 0);

The day-one version of this file — counts per impact level rather than per rule, no fingerprints at all — is the fastest thing to ship and is worked through end to end in setting up progressive accessibility thresholds in CI. Once the repository has more than one team in it, a single set of ceilings starts to hide regressions in new code behind legacy allowances, which is the point at which the budget gets keyed by path as well as by rule, described in per-directory accessibility budgets in legacy code.

# 5. The Ratchet Job

Every ceiling in that file is a debt the team has agreed to carry, and left alone it will be carried forever. When color-contrast drops from 2,190 to 2,140 because someone fixed a token, the ceiling still says 2,190, and the next fifty regressions are free. The ratchet closes that gap mechanically: a scheduled job re-reads recent observed counts, takes the minimum, and proposes it as the new ceiling in a pull request.

Three parameters keep it from becoming a source of spurious failures. The window (how many recent runs to consider) has to be long enough that one unusually clean run cannot set a floor the branch cannot reach again. The grace factor allows a small margin above the observed minimum for rules with genuinely variable counts — anything involving virtualised lists or lazy images. And the job must propose rather than commit, because a ceiling change is a policy change.

// a11y/ratchet.mjs — usage: node a11y/ratchet.mjs (writes a11y/budget.json in place)
import { readFileSync, writeFileSync } from 'node:fs';

const budget = JSON.parse(readFileSync('a11y/budget.json', 'utf8'));
// a11y/history.jsonl: one { commit, countsByRule } line appended per default-branch run.
const history = readFileSync('a11y/history.jsonl', 'utf8')
  .trim().split('\n').map((l) => JSON.parse(l));

const WINDOW = 10;          // consider the last ten default-branch runs
const MIN_SAMPLES = 6;      // never ratchet a rule seen fewer than six times
const GRACE = { 'color-contrast': 1.02, 'image-alt': 1.05 }; // known-variable rules

const window = history.slice(-WINDOW);
const changes = [];

for (const [rule, entry] of Object.entries(budget.rules)) {
  const observed = window.map((run) => run.countsByRule[rule]).filter(Number.isInteger);
  if (observed.length < MIN_SAMPLES) continue;
  const floor = Math.min(...observed);
  const proposed = Math.ceil(floor * (GRACE[rule] ?? 1));
  if (proposed < entry.max) {
    changes.push({ rule, from: entry.max, to: proposed });
    entry.max = proposed;
  }
}

budget.policy.totalCeiling = Object.values(budget.rules).reduce((s, r) => s + r.max, 0);
writeFileSync('a11y/budget.json', JSON.stringify(budget, null, 2) + '\n');

for (const c of changes) console.log(`RATCHET ${c.rule}: ${c.from} → ${c.to}`);
console.log(`${changes.length} ceiling(s) lowered across ${window.length} runs`);
process.exit(changes.length > 0 ? 0 : 3); // 3 = nothing to propose, skip the PR step

The rules for choosing the window, the grace factor and the escape hatch for a legitimate feature that adds findings mid-sprint are the whole subject of ratcheting violation budgets down each sprint, including the workflow that opens the pull request and the label that pauses the ratchet for one iteration.

# Pipeline Integration

Two jobs, two different triggers. The pull-request job scans, compares against the baseline, checks the budget, and blocks; the default-branch job scans, appends to a11y/history.jsonl, and never blocks, because a merge that has already happened cannot be gated. Only the first is a required status check.

name: a11y-progressive-gate
on:
  pull_request:
    branches: [main]
  push:
    branches: [main]
concurrency:
  group: a11y-gate-${{ github.ref }}
  cancel-in-progress: true
permissions:
  contents: read
  pull-requests: write
jobs:
  gate:
    runs-on: ubuntu-24.04
    timeout-minutes: 30
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: '20'
          cache: npm
      - run: npm ci
      - run: npx playwright install --with-deps chromium
      - run: npm run build
      - name: Serve the production build
        run: npx serve -s dist -l 4173 &   # background; scan waits on the port itself
      - name: Scan every route
        run: node a11y/scan.mjs            # exit 2 here means the scan is untrustworthy
        env:
          A11Y_BASE_URL: http://127.0.0.1:4173
      - name: Compare against the baseline
        if: github.event_name == 'pull_request'
        run: node a11y/compare.mjs
      - name: Check the per-rule budget
        if: github.event_name == 'pull_request'
        run: node a11y/budget-check.mjs
      - name: Append to the ratchet history
        if: github.event_name == 'push'
        run: node a11y/history-append.mjs && git push origin HEAD:main
      - name: Comment the classification on the PR
        if: always() && github.event_name == 'pull_request'
        run: node a11y/comment.mjs a11y/comparison.json
        env:
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
      - uses: actions/upload-artifact@v4
        if: always()
        with:
          name: a11y-progressive-${{ github.run_id }}
          path: |
            a11y/runs/current/
            a11y/comparison.json
          retention-days: 30

Three details in that workflow are easy to get wrong. concurrency keyed on github.ref rather than head_ref covers both triggers with one group. The artifact retains the raw per-route results for 30 days, which is what makes “was this finding really there last week” answerable without re-running an old commit. And the history append pushes directly to main from the post-merge job — small, append-only, and the reason the ratchet has anything to read. Feeding the same counts into a longer-lived store for trend charts is covered in tracking accessibility violation trends across sprints.

For the pull-request comment, print the three numbers first and the list of new findings second, capped at 25 lines with a link to the artifact for the rest. A comment that dumps 400 findings gets collapsed by the reviewer and never expanded again.

# Troubleshooting and Flaky-Test Mitigation

Every finding reports as new after a dependency upgrade. Check fingerprintVersion first — if the recipe changed, the comparison should have exited 2 rather than reporting 4,812 new findings, so the version guard is missing or the constant was not bumped. If the version matches, the more likely cause is an axe-core upgrade that renamed rule ids (aria-required-children splitting, duplicate-id being retired), which changes the first hash input for every affected node. Regenerate the baseline in the same pull request as the upgrade, and keep those two changes out of any functional diff.

The count for one rule oscillates by a handful of findings between runs. This is almost always a scan-timing problem rather than a fingerprinting problem. Lazy images that have not loaded produce no image-alt finding; a virtualised table renders a different number of rows depending on how fast the runner is; a skeleton placeholder with grey-on-grey text produces color-contrast findings that vanish once real content paints. Wait on an application-owned signal — the aria-busy check in the scan script is the minimum — and give the affected rules a grace factor in the ratchet rather than chasing a single stable number.

A pull request that only reformats JSX fails with dozens of new findings. The anchor chain is picking up something cosmetic. Look at the anchor field on the new findings: if it contains a div at all, the STABLE selector list has a match it should not have. Add explicit data-a11y-anchor attributes to the components that own those regions; the attribute is cheap, it is self-documenting, and it makes the fingerprint immune to whatever the layout does next.

The gate passes locally and fails in CI with contrast findings only. Fonts. A runner without the application’s webfonts falls back to a system face with different metrics and, in some pipelines, different rendered weight, which changes computed contrast at the edges. Install the fonts in the container or run the scan in the same image locally, and never generate a baseline on a developer machine — generate it in CI, in the container the gate uses.

A single route times out intermittently and the whole job exits 2. That is the intended behaviour, but it should be rare. Set the per-route wait on a selector the application controls rather than a network state, retry the route once inside the scan loop before recording an error, and keep the route list in the repository so a flaky route can be quarantined with a comment and a ticket instead of silently disappearing from the scan.

Two teams disagree about a raised ceiling. This is a process failure showing up as a merge conflict in a11y/budget.json. Resolve it by keying the budget by area so the two teams stop editing the same integers, and by making the ratchet job the only automated writer of the file.

# Common Pitfalls

• Generating the baseline from a feature branch or a laptop, which bakes in local font rendering, dev-server overlays and a route set nobody else has.
• Treating the baseline as a set rather than a multiset, so eleven of twelve identical unlabelled buttons in one table are permanently invisible to the gate.
• Putting an nth-child selector or the raw node.html string in the fingerprint, guaranteeing that the first refactor produces hundreds of matched fixed-and-new pairs.
• Gating on a single total instead of per-rule ceilings, which lets a critical label violation be paid for by deleting a moderate region warning.
• Omitting newRuleDefault: 0, so an axe-core minor upgrade that adds three rules quietly widens the allowance instead of raising a triage ticket.
• Letting the ratchet commit straight to the default branch, which turns a policy change into an unreviewed automated push and destroys trust in the whole mechanism the first time it sets an unreachable floor.
• Regenerating the baseline as a convenient way to make a red job green, with no note in the pull request description — the single fastest way to end up with a gate that measures nothing.
• Keeping suppressed rules in disableRules rather than as baseline entries, so the count of known debt is wrong and the debt is unfixable-by-construction.

# FAQ

Should a team run a baseline, a budget, or both? Both, but not on the same day. Ship the budget first because it is a dozen lines of comparison logic and cannot produce a false failure, and it immediately stops the count from growing. Add the baseline when reviewers start asking which violation the job is complaining about, because that question is unanswerable from counts alone. Once both exist, the budget catches trades between rules and the baseline names individual regressions.

How large can the baseline file get before it becomes a problem? A 4,812-finding baseline with the human-readable fields is about 380 KB and adds roughly two seconds to the comparison, which is negligible next to a browser scan. The real limit is reviewability, not size: past a few thousand entries nobody reads a regeneration diff, so keep entries sorted deterministically, keep the route out of the hash for shared chrome, and split the file per area once more than one team edits it.

What happens when axe-core renames or splits a rule? The rule id is the first input to the fingerprint, so every finding for that rule reports as one fixed and one new, and the budget entry for the old id becomes dead while the new id falls to newRuleDefault. Handle it as a deliberate migration: upgrade axe-core in an isolated pull request, regenerate the baseline, rename the budget key, and bump fingerprintVersion if the recipe itself changed. Never mix a scanner upgrade with a functional change.

Can the ratchet make the build fail without anyone changing code? Only if it commits directly, which is why it opens a pull request instead. A proposed ceiling that turns out to be unreachable is a red check on the ratchet’s own pull request, which gets closed or edited; the default branch never sees it. The window and minimum-sample settings exist to make that outcome rare rather than to make it impossible.

How does this interact with a gate that only scans changed pages? Badly, if the two are combined naively: a partial scan produces no findings for unscanned routes, and the comparison would report every one of them as fixed. Restrict baseline comparison and ratchet history to full scans, and let a diff-aware job run as a fast advisory check on its own. That split — a complete scan for the accounting, a partial scan for speed — is the only version that keeps both numbers honest.

# Related

• Setting Up Progressive Accessibility Thresholds in CI — the day-one gate: counts per impact level in a committed JSON file.
• Ratcheting Violation Budgets Down Each Sprint — the scheduled job, the window, the grace factor and the escape hatch.
• Per-Directory Accessibility Budgets in Legacy Code — one budget file, different ceilings per path glob.
• Pull Request Gating & Branch Policies — making the comparison job a required status check that cannot be bypassed.
• Auto-Fail vs Warning Workflows — which breaches should block a merge and which should only annotate it.