Building accessibility tools on a truly open foundation

Brendan Works is a technical product and program manager in Seattle, focused on the paratransit services that help people with disabilities get to jobs, school, and medical appointments. He's seen up close how often digital tools fail those who most depend on them.

"When a booking tool is unusable or a screen reader can't navigate a schedule page, it's not simply an inconvenience," Works says. "It cuts someone off from getting to see their friends or getting to class."

Many of the users he's describing rely on assistive technology online. A person who has low vision might use browser zoom or high-contrast modes. Someone with a motor impairment might navigate entirely by keyboard, pressing Tab to move from one interactive element to the next.

When a site isn't built with these people in mind, the tools break down. Software exists to prevent this, but most of it has a fundamental limitation—it only inspects the underlying code and compares what it finds against standard accessibility guidelines. The trouble is that compliant code can still produce an unusable page.

Works wanted a tool that could catch those cases. On his own time, he created PointCheck, a web app that wires Molmo, MolmoWeb, and Olmo 3 together to evaluate web accessibility by what's rendered on screen—not just what's in the HTML.

A visual checker for accessibility

To close the perceptual gap, Works knew he needed two things: a way to drive a real browser the way a keyboard user would, and a way to inspect the resulting screen in pixel-level detail. He found both in open frameworks and models.

PointCheck is built on top of Playwright, an open browser automation tool that loads a target page in a background Chromium window. A script Works wrote pre-scans for common keyboard problems that prevent users from tabbing away from an element, broken link patterns, click handlers on elements that shouldn't be interactive, and tab-order values that override the natural reading sequence of the page.

PointCheck tabs through a webpage the way a user would, taking a screenshot at every step. For the first pass, Works chose MolmoWeb because it's purpose-built for visual pointing on web pages—each screenshot goes to the model with the prompt, "Find the element that currently has keyboard focus," and MolmoWeb returns its location as a pixel coordinate. PointCheck then sends the same region to Molmo with a direct query about what's on screen, including whether a focus ring is actually visible. The result is that PointCheck can confirm focus indicators are visually present on screen–not just defined somewhere in a stylesheet.

Focus indicators are the visual highlight – usually a colored ring or outline – that appears around a button or link when a person tabs to it. They're how keyboard-only users orient themselves on a page—the equivalent of seeing your cursor move. A focus indicator can pass every automated code check while being completely invisible against the background it's rendered on. For the millions who navigate by keyboard alone, that highlight is the difference between knowing you're on a "Buy Now" button and accidentally hitting "Cancel."

In PointCheck, keyboard navigation and focus visibility run as two separate tests under the hood, evaluating for different flaws. Works also built in zoom testing at 200% (a color-blindness simulation specifically for deuteranopia, the most common form) with contrast checks, form submission with invalid data, and a structural analysis of the page's markup. 

When all of PointCheck’s test modules finish, Olmo 3 writes a plain-English summary of the findings. Works chose Olmo because the reports needed to be useful to a product manager or a designer, not just an accessibility specialist—and Olmo 3 was strong enough at plain-language synthesis to do that.

The importance of transparency 

Works considered proprietary vision and text APIs for PointCheck, but decided against them.

"The whole point of PointCheck is that it should be fully self-hostable," he says. "You shouldn't have to send screenshots of your own pages to [a third party] to get an accessibility verdict. With Molmo and Olmo, you grab the weights, run it on your own infrastructure, and nothing leaves your environment."

Works built the first version of PointCheck in a single weekend. Getting Playwright to tab through pages and capture screenshots was straightforward. The harder parts were fitting three models onto a single A10G GPU without running out of VRAM and working through edge compatibility issues in the inference code.

"But the core concept worked quickly, which kept me going," Works says. "I'm a product person at heart. I like to finish things, get beyond a prototype. So I grabbed a domain, flipped the DNS, set up basic analytics, and made a logo."

Early reactions have come from people who've done accessibility work professionally. A former colleague focused on accessibility at Washington State's Department of Social and Health Services told Works, "I spent two years wishing this existed. Nothing beats turning off your monitor and navigating by keyboard, but that’s hard to scale."

To that end, Works sees PointCheck as a complement to existing tools, not a replacement—covering the ground that code-based inspection will never reach.

"What interests me is pushing further into the territory that's supposedly only coverable by human validation,” Works says. “I'm talking about the perceptual stuff, the things that require actually seeing the page."

Point check is a great example of the type of project that open-source models and tooling enable—we release weights, training data, and code because we believe the most transformative applications are often the ones we don't anticipate. On Global Accessibility Awareness Day, we're proud to have played a part in an effort to make the open web more navigable for everyone.

PointCheck is an independent project by Brendan Works and is not affiliated with his employer or Ai2. Works speaks in his personal capacity.