Pros and Cons of Automated Web Testing

Automated Section 508 compliance web scanning software has become a popular tool for accessible code testing over the last decade. This blog will discuss some of the software’s pros and cons to determine if it’s the right tool for your organization.

Automated Testing Pros

Knowing where to start when it comes to Section 508 standards for the web can be overwhelming, especially if you are unfamiliar with the standards or do not have an accessibility team or web testers on staff. This is where using automated web testing software can be a good first step in testing websites before they are launched. Automated web testing can also be helpful if your organization must go through a high volume of webpages that can be difficult for manual testers to quickly turn around with manual testing. Some Federal agencies have hundreds of thousands of webpages, and going through each one would be a difficult task—even for an experienced team of Section 508 testers without automated assistance. Automated scanning software can also help organizations develop a reporting process for testing and identifying issues. With enough use and additional training, the software can help your organization begin to establish a Section 508 testing team that fully understands the importance of web accessibility.

Automated Testing Cons

While automated testing tools have improved significantly over the years, they still have some lingering issues and challenges. Automated tools will not solve all of your accessibility issues and passing an automated scan can give organizations a false sense of full compliance. While the tools do catch a lot of issues, they still require a trained tester to verify the results.  For example, these tools can verify if an <img> tag has an alternative attribute, but cannot verify if the alternative attribute correctly describes the image.  Conversely, the tools can also fail decorative images for not having a descriptive alternative attribute, even though one is not required. However, as long as the user keeps these types of challenges in mind, automated tools can still be beneficial, especially for teams that do not have the manpower to manually test all of the web applications.

There are other logistical challenges with automated tools, especially in the federal workplace. Security restrictions may require certain types of software to be approved prior to being hosted in an environment. This can be a time-consuming process. Some automated tools allow users to access them from a web browser and point them toward the site to scan. This is not an option that works in the federal workspace because external servers keep a record of the scanned pages and they may not meet the agency’s security requirements.  Users will need to verify the policies in place at their individual agency or organization before purchasing a tool to ensure it will work as intended.

Finally, automatic testing tools may not be customizable, which can make it difficult to align with federal standards like the Trusted Tester Success Criteria.  It is important to make sure the tool allows the user to modify and/or create rulesets to use when scanning web pages.