Testing my AI accessibility skill

Accessibility in a world of AI

AI has the potential to help improve digital accessibility by making it easier to raise standards.

However, AI can amplify the good and the bad.

The scale of its potential influence is growing quickly. GitHub reports that developers pushed 986 million commits in 2025, and that a new developer joins the platform roughly every second. This growth cannot be attributed to AI alone, but GitHub describes AI-assisted development as an increasingly routine part of how software is produced.

Greater output does not automatically mean higher standards. The WebAIM Million, published in March 2026, found automatically detectable WCAG failures on 95.9% of one million home pages, up from 94.8% in 2025. The average number of detected errors rose from 51 to 56.1 per page.

WebAIM identifies automated and AI-assisted coding as one possible influence on this change, alongside increasingly complex pages and greater reliance on third-party frameworks. The report does not establish that AI caused the decline, but it demonstrates why increasing the speed of production without equivalent quality controls presents a risk.

AI is therefore better understood as an amplifier than an assurance of quality. When accessibility is built into requirements, design systems, prompts, testing and human oversight, AI can help good practice spread. Without those foundations, it can produce inaccessible experiences faster and at a much greater scale.

Skills

Skills act as reference points for AI, to execute actions in a certain way. They allow humans to quickly reference these pre-made actions, when asking AI to complete a task. Or agents to regularly re-use when operating.

You can write them directly in AI platforms, or upload markdown files into them, to save within the tool.

Once they're in the tool, they normally come up in auto-suggests when you type '/' or select from a menu. The specifics vary between platforms though — the file format, where it lives and how you invoke it are not standardised.

About my accessibility 'skill'

By combining my own accessibility baselines with documentation from my accessible design system, and AI-assisted analysis of the WCAG source material checked against those baselines, I used GPT to generate a reusable skill.

You can download a copy of the accessibility skill and apply it to your own work.

It can be used to audit work or to generate accessible solutions.

Testing the skill effectiveness

I wanted to see whether using the skill worked.

I wrote a brief for AI to build a contact page for a fictional company renting assistive robots to older people who need help at home. I made the page deliberately awkward — a radio group, a postcode lookup, a checkbox group, a date and time, a message with a character counter, a file upload and two consent checkboxes — so there would be plenty for an audit to get hold of.

I set three conditions:

1. WCAG 2.2 AA required, skill invoked.
2. WCAG 2.2 AA required, no skill. A genuine effort from general knowledge.
3. Accessibility not mentioned, no skill. The control — just “build me a contact page”.

Test prototypes

• Prototype 1 — requirement stated, skill invoked
• Prototype 2 — requirement stated, no skill
• Prototype 3 — the control, and knowingly inaccessible

The results

I have split the findings into three tiers, because merging them is how accessibility results get overstated. A WCAG failure is a failure. A pass that depends on an exception is still a pass. A best-practice observation is neither, however sensible it may be.

What axe found

Axe cannot tell prototype 1 from prototype 2. Both return zero detected violations in every state. If an automated check were the whole of your quality gate, the skill would look as though it had changed nothing.

Tier one: conformance failures

Prototype 2’s single failure is 1.4.11 Non-text Contrast: eight text input borders at 2.96:1 against a 3:1 requirement. The border is the only thing distinguishing those fields from the panel behind them, so it is required to identify the control, and it misses. Narrowly, but it misses.

Tier two: conforms, through an exception

Neither of these is a failure. WCAG explicitly permits small targets where the spacing between them is sufficient, and landmarks are a documented sufficient technique for bypassing repeated blocks. But conforming by exception is more fragile than conforming by design: change the layout and the spacing exception can disappear, and a sighted keyboard user who does not use landmark navigation gets no bypass at all.

Tier three: best practice

The control cannot announce anything twice because it announces nothing at all. And the last two rows apply to every version, including the best one: the skill did not think to mark optional fields explicitly, or to question a two-column form. Best practice is where the skill helps most, and also where it still has gaps.

So the honest summary is this. On conformance alone, the skill is worth one Level AA criterion over simply asking for accessibility. Most of what it adds sits in tiers two and three — conforming by design rather than by exception, and getting the advisory details right. That is a more modest claim than the raw finding count suggested, and I think a more useful one: conformance is a floor, not a standard of quality.

How much axe missed

On the control, axe found 21 of the 37 issue instances that review found, and every one was a colour-contrast problem. Counted by distinct success criteria rather than instances, it found two of nine.

Those two numbers are worth sitting with. 57 per cent by instance. 22 per cent by criterion. Same page, same tool, same run. Deque’s automated coverage study reports that axe detects around 57 per cent of accessibility issues, having deliberately moved from measuring success criteria to measuring issue volume — the older criteria-based estimates sat at 20 to 30 per cent. My single page reproduces both figures almost exactly, which suggests the disagreement is not really about how capable the tool is. It is about what you choose to count.

These are the failures axe found nothing of:

• A file upload that no keyboard user can reach, because it is a div with a click handler wrapping a hidden input.
• A mobile menu button with no tab stop, making the navigation unreachable below 980 pixels.
• Focus outlines removed globally, leaving nine of twenty-two tab stops with no visible indicator.
• A form that announces nothing on failed submission, and loses focus to the document body on success.
• No autocomplete on name, email, phone or postcode.
• Every input border at 1.38:1, where the border is what identifies the field.

In this test, every issue reported by axe was a colour-contrast problem. It did not identify several interaction, focus-management and announcement problems that could prevent someone from completing the form. Automated testing remains valuable, but a clean result is not evidence that a page conforms to WCAG or is usable by disabled people.

What the control got wrong

If you don't include accessibility, as a standard, at all, you can expect issues.

• The file upload is a div with a click handler wrapping a hidden file input. A keyboard user cannot attach a file at all.
• The mobile menu button is also a div, with no tab stop. On a narrow screen, keyboard users cannot open the navigation.
• Focus outlines are removed globally and never replaced. Nine of twenty-two tab stops show nothing at all.
• Submitting an invalid form announces nothing. No invalid state, no live region, focus left on the button. The errors are red text and that is all.
• On success the form is hidden and focus falls back to the document body. A screen reader user is never told the message was sent.
• Twenty text nodes below 4.5:1. Every input border at 1.38:1.
• No skip link, and no autocomplete on name, email, phone or postcode.

What the second version missed

It got most things right. One conformance failure, and a handful of advisory points.

• A failure. Eight control borders at 2.96:1 against a 3:1 requirement. The text colours had been checked properly; the border grey was picked by eye.
• Advisory. Three cases of content likely to be announced twice: an error summary that is both a live region and a focus target, a character counter that is both a status region and part of the field description, and radio hints that sit inside the label and are also referenced by it. WCAG does not prohibit this, and the exact behaviour varies by browser and screen reader pairing.
• Advisory. Fourteen targets under 24 pixels. These conform through the spacing exception, but sizing them properly would survive a layout change.
• Advisory. A sticky header, mitigated with scroll padding. Nothing was obscured in the conditions I tested, but I did not test above 200% zoom, which is where sticky headers usually cause trouble.
• Advisory. A maximum length attribute on the message field, which silently truncates pasted text. Reasonable people differ on this one.

What the skill added

One of these closed a conformance gap. The rest are judgement calls that I happen to agree with, which is not the same thing.

• Contrast ratios calculated and recorded in the stylesheet, rather than judged by eye. This is the one that avoided the 1.4.11 failure.
• Every target 24 pixels by size rather than by exception, so the layout can change without the conformance changing with it.
• No content set up to be announced twice.
• A two-tone focus ring that holds 3:1 on light and dark backgrounds alike.
• The sticky header removed rather than mitigated. A design decision rather than an accessibility gain, but it removes a whole category of risk instead of managing it.
• The maximum length attribute dropped, so pasted text is never silently truncated. A trade-off, not an improvement in every case.
• Error states that clear as the user corrects them. Helpful in principle, though I did not test how the change is announced.
• Written reasoning for every criterion marked not applicable, rather than silence. This is the part I would keep even if nothing else survived.

What this doesn’t show

Certain limitations to the evaluation.

• Not manually tested with voice-control software.
• Not manually tested with screen magnification software.
• Not tested with people with disabilities.
• Not tested with a wide range of screen readers, only VoiceOver.
• The evaluation was without a pre-existing design system.
• Predominantly tested, using Claude, with minimal, additional human checks.

Where even the best version fell short

Two things the skill did not raise, on the version that otherwise performed best.

• The form is laid out in two columns. That complicates the reading order, particularly for people using high levels of browser zoom, who may not see a field to the left or right and may reasonably expect to scroll down to find the next one.
• Required fields are marked with an asterisk. Marking both required and optional fields removes the ambiguity, and keeps the sighted and non-sighted experience the same.

Neither is a WCAG failure. Both are the sort of thing a person notices and a rule does not.

The short version

In this experiment, simply asking AI to meet WCAG 2.2 AA prevented many of the obvious accessibility failures produced when accessibility was not mentioned.

The accessibility skill did not improve the automated axe result: both accessibility-led prototypes returned zero detected violations. Its value became apparent during deeper review, where the version created without the skill contained several issues involving control contrast, repeated announcements and target size.

This is one small test rather than proof that a skill will always produce accessible work. It does suggest that specific, reusable accessibility instructions can provide stronger safeguards than a general request for accessibility alone.

Neither approach replaces expert review or testing with disabled people. AI can help apply an accessibility baseline, but people remain responsible for defining that baseline, checking the result and learning whether it works in practice.