Lean UX case studyAI agent designSolo build
AccessWatch: designing an AI agent for accessibility triage
A solo-designed and solo-shipped tool that uses an AI agent to audit websites for accessibility issues. This case study covers the parts of the UX process that actually applied to a one-person, AI-native build — framing, decisions, and what changed once real usage hit real infrastructure.
01
Problem framing
Manual accessibility auditing doesn't fit how most teams actually work. Automated scanners exist, but they output flat lists of rule violations with no sense of priority, no plain-language explanation, and no connection to where on the page the issue actually lives. The output is technically correct and practically unusable for anyone who isn't already an accessibility specialist.
The design question I set for myself wasn't "can an AI find accessibility issues" — that's a capability question, not a design one. It was: what does it take for a non-specialist to look at an audit result and immediately understand what's wrong, how bad it is, and what to do about it? That reframing is what shaped every interface decision that followed.
02
Who this is for
Primary user
A generalist developer or small-team lead who knows accessibility matters but doesn't have a dedicated accessibility specialist on staff. They need triage, not theory — what's broken, how urgent, what to fix first.
What they don't want
A wall of WCAG criterion codes with no context. A tool that requires them to already understand accessibility deeply in order to use it. A black box they can't verify or push back on.
Secondary audience
Anyone evaluating my own design + AI-systems thinking — recruiters and hiring managers reviewing how I reason about agent UX, not just visual polish.
No formal interviews or usability testing happened here — this was a solo, self-initiated build, not a client engagement. I'm naming that plainly rather than implying research that didn't happen. The persona above is informed reasoning from accessibility-tooling pain points I've encountered directly in prior UX work, not validated user data.
03
Key design decisions
Visual language
A terminal aesthetic, used as a trust signal
I chose a dark, monospace, log-streaming interface deliberately — not for visual flair, but because it reads as "showing real process" rather than "showing a polished result." For a tool whose entire value proposition is trustworthy analysis, watching the agent's reasoning stream in real time does more for credibility than a clean progress spinner would.
Severity triage
Critical / serious / moderate / minor, before anything else
The very first thing a user sees after a scan is a four-number severity breakdown, not a list. Triage is the actual job-to-be-done — someone with limited time needs to know "is this bad" before they need to know "what exactly is wrong."
Spatial grounding
Numbered markers on a real screenshot
A flat list of "missing alt text" violations is abstract. Pinning each issue to a location on an actual screenshot of the real page — with a small zoomed crop per violation — turns the audit into something the user can immediately recognise against the product they actually own.
Closing the loop
Filing real GitHub issues, not just displaying results
An audit that lives only on a results screen gets ignored. Letting the agent file each violation as a labelled GitHub issue — with the fix suggestion in the body — moves the tool from "interesting demo" to "something that could plausibly enter a real workflow."
⚠ A scope decision worth naming explicitly
AccessWatch does not run a systematic check against the full WCAG 2.1 ruleset the way a tool like axe-core does. The agent identifies realistic, representative violations based on common accessibility failure patterns, not an exhaustive criterion-by-criterion pass. I made this trade-off deliberately: a true rule-engine audit needs a real headless browser environment, which a lightweight serverless backend can't run within reasonable time and cost limits. I chose to ship a credible, narrower agent rather than an incomplete attempt at full coverage — and I'd say exactly this to anyone who asks how thorough the audit actually is. A production version would pair this agent's triage layer with a real rule engine like axe-core running on a proper browser backend.
04
Iteration — what changed once it was real
Because this was built and shipped solo, "iteration" here means design decisions revised in direct response to what broke or what felt wrong once the tool was actually live — not iteration based on user feedback rounds. Two changes stand out:
Assumed
Auto-discovering pages to crawl would feel smart
The first version of multi-page support extracted links automatically from the homepage and picked a few to audit.
What broke
It surfaced an RSS feed and an unrelated blog post as "pages"
Technically working, practically wrong — the auto-selected pages weren't ones a real user would care about auditing, and there was no way to course-correct from the interface.
Changed to
Explicit, user-typed page paths
Less automated, more controllable. The user now decides exactly which pages matter, which also made the eventual "grouped by page" results view make sense.
Assumed
A flat severity score would communicate overall site health
A single 0–100 compliance number, calculated from raw violation counts.
What broke
Auditing more pages made the score collapse to zero
The formula penalised total violation count without accounting for how many pages produced them — auditing three pages instead of one made the site look three times worse for no real reason.
Changed to
A per-page-averaged score
The score now reflects severity density rather than raw count, so it stays meaningful regardless of how many pages someone chooses to audit.
05
What I'd validate next
None of this testing has happened yet — these are the specific methods I'd reach for, matched to the actual question each one answers, rather than generic "user testing" applied uniformly.
Guerrilla testing
Overall comprehension
Grab 5 developers who aren't accessibility specialists, watch them run a real audit cold, and see where they hesitate or misread something. Fast, cheap, and matched to an early-stage question.
First-click testing
Marker trust calibration
Show the screenshot with markers and ask what someone's first instinct is — do they read the dots as exact locations or as a rough hint? Tests whether the "positions are approximate" language actually changes the first read.
A/B testing
Aggregate score vs. no score
Run two variants — one with the single compliance score, one showing only the four severity counts with no aggregate — and measure which gets people to act on results faster or report higher trust.
Tree testing
Page-grouping structure
Test whether grouping violations by page (vs. a flat list sorted by severity) actually helps people locate a specific issue faster, or whether the grouping just looks organised without being more findable.
06
Reflection
The biggest shift in how I think about UX process came from building something where the "backend" wasn't abstract. When the interface needs to account for a 60-second processing budget, or explain what happened when its own infrastructure misfires, those become design constraints in the same way a small mobile viewport is a design constraint — except I had to go discover them myself by shipping, rather than reading them off a brief.
I'm framing this case study narrowly on purpose: problem framing, the decisions I actually made, and the iteration that came from real signal — not a padded six-phase process I didn't run. I'd rather show a smaller, honest process than a longer one that implies research I can't back up in an interview.
Problem framingPersona reasoningInteraction designDesigning for AI system limitationsScope trade-off decisionsIterative design from live signalUsability testing methodologyAI agent UX