Assistive Technology User Testing: Beyond Automated ADA Scanning
Automated scanning told one story. Real assistive technology users told us something different. The barriers that matter most, the ones that prevent task completion are often the ones no scanner finds.
Project Overview
A national museum engaged Accessibility Innovations to conduct accessibility user testing on its digital properties. The organisation had run automated scans. It wanted to go further to understand what it actually feels like to navigate its platforms using the assistive technologies its visitors use daily.
Under Section 508 and the Americans with Disabilities Act, digital properties of federally funded or publicly accessible institutions must be accessible to people with disabilities. The Department of Justice has made clear that automated scanning alone is insufficient to demonstrate compliance. Manual testing and ideally, testing with real users of assistive technology is expected as part of a credible accessibility program.
The Challenge: Identifying Functional ADA Barriers
Automated scanning tools catch a defined category of barriers efficiently: missing alt text, colour contrast failures, certain structural issues. What they cannot determine is whether a screen reader user can actually complete a task, whether a keyboard-only user can reach every interactive element in a logical sequence, or whether a voice control user can navigate a complex interface without workarounds.
This organisation’s automated scans showed one conformance picture. The real picture was different. Critical task-completion barriers had gone undetected across months of regular automated scanning. Finding that difference required real people using real tools in real conditions.
Our Manual User Testing & Screen Reader Approach
We recruited, coordinated, and managed testing sessions with real assistive technology users, people who use these tools as part of their daily lives, not evaluators operating unfamiliar software under test conditions. Testing covered NVDA, VoiceOver, iOS Voice Control, Windows Magnifier, and keyboard-only navigation. We facilitated both moderated sessions, where a facilitator was present to observe and probe, and unmoderated sessions, which produced findings uninfluenced by a facilitator’s presence.
Every barrier identified was mapped to the specific WCAG 2.2 success criterion it violated, and where applicable to the corresponding Section 508 provision. We delivered a prioritised remediation report with screen captures, session transcripts, and developer-ready guidance. We followed the report with a developer workshop.
Project Snapshot
Industry
Museums & Cultural Institutions
Location
United States
Compliance Standard
ADA
Key Result
5 assistive technologies | WCAG + 508 mapped
Accessibility User Testing Results
Services Used
WCAG & Section 508 Gap Analysis
Developer Training
Legislation: Americans with Disabilities Act | Section 508 of the Rehabilitation Act
Working on a similar challenge?
Accessibility Innovations is a principal-led practice with over twelve years of delivery across federal, state, municipal, and private sector clients in the United State. Every engagement is led by a credentialed senior consultant. Our team holds CPWA, CPACC, and PMP credentials, and our work is backed by $5M errors and omissions insurance.
Whether you need to validate automated scan results with real assistive technology users, prepare for a DOJ compliance review, or understand how your digital properties perform for people who depend on screen readers and keyboard navigation, we would welcome the conversation.
Frequently Asked Questions (FAQs)
+
Why is automated scanning not enough to demonstrate ADA or Section 508 compliance?
Automated tools cannot fully test real user experiences or confirm whether people using assistive technology can complete important tasks successfully.
+
What assistive technologies do you use during user testing sessions?
We test with tools such as NVDA, VoiceOver, keyboard-only navigation, Voice Control, and screen magnifiers.
+
What is the difference between moderated and unmoderated accessibility testing sessions?
Moderated sessions include a facilitator guiding the process, while unmoderated sessions allow users to complete tasks independently in real-world conditions.
+
How are barriers identified during user testing mapped to WCAG success criteria?
Each issue is linked directly to the relevant WCAG requirement and documented with remediation guidance for developers.