The Centers for Medicare & Medicaid Services (CMS) identified extensive variability in practice standards amongst their testing contractors, which include Application Development Organizations, Independent Verification and Validation (IV&V) organizations, and an Isolated Testing Facility that conducts stress and load testing. Test case preparation and execution were not being conducted to the same level of detail by individual testing contractor organizations. Likewise, testing outcomes and software defects were being reported differently. CMS desired to obviate this disparity by creating software testing standards of practice for all of their contractors that conduct software testing. The goal was to improve the overall quality of software applications/products and current testing practices throughout the product lifecycle, with the intent to ensure compliance with documented testing policies, standards and processes.
Titania established a strategic implementation plan to develop testing standards of practice. Titania understood that tenets of continuous quality improvement require the establishment of policies and standards that not only reflect current industry best practices, but that are also embraced by participating organizations and enforced by CMS management. These efforts had to be created from the “ground up” as no such prior guidance existed for software testing within the CMS community. Titania’s Subject Matter Experts (SMEs) initiated and led cross–contractor teams to assess current testing practices across the enterprise and build a unified set of policies and procedures using a common nomenclature, so testing practices were well articulated and understood. Titania’s SMEs led collaborative joint efforts participated by selected CMS testing contractors to review the standards and the proposed audit criteria by which success would be measured. To measure improvements in testing practices, a baseline of performance outcomes was established by conducting initial Gap Analyses before implementing formal Testing Audits, from which compliance would be determined.
A complete timeline and project plan was developed and implemented. A final analysis of the findings was shared with both CMS and testing contractors, proving transparency and cooperative efforts can lead to a single set of standards agreed to by all contractors. The goal established by CMS “to strengthen and evolve current full-lifecycle testing activities” was exceeded. A culture of transparency and continuous quality improvement was established for ongoing improvement efforts to flourish.