.png)
By Mason Cox, Associate Director of Computer Systems Quality Assurance (CSQA) Services
Here is a question for our quality community:
Is a Scripted Test Case a True Test to Determine your Systems are Accurate and Reliable?
The ‘test-step driven’ scripted test case is the industry standard documentation for Validation Testing, supported by evidence in the form of screenshots. This test method is established by defining a prescribed step-list of actions that ensure software performs as intended. The scripted test case is expected to be successful and not encounter failed steps, which would, if encountered, uncover issues within the system.
Though the industry standard method of software validation, the scripted test case is potentially not a real-world test. Dictating to testers step-by-step actions prescribes the test result which may not be a true pressure test of any given system. Risky blind spots may exist within Validation efforts that exclusively utilize scripted test cases; there is real potential to miss an issue that may occur outside of the dictated validation script.
Validation test architecture would move to a higher standard of quality if a new approach to testing system functionality with the unscripted test case is adopted and or integrated. This of course will need to work in conjunction with appropriate standards on proper documentation. A new approach could indeed be a higher quality pressure test, and in fact, require more streamlined documentation practices that result in a better Validation Effort that requires less overall time in generating documentation.
One Relatable Example:
A scripted test case for role-based System access will define the test steps to enter a specified incorrect password. This fails to allow a trained tester to understand the full extent of potentially limiting user access and encountering access issues not prescribed by the scripted steps within a scripted test case.
An unscripted test case engages the opportunity to enter any combination of special characters. This opportunity cannot be missed to encounter potential errors encountered by the trained tester. This scenario may be missed if a validation effort is dictated by a script.
Let the trained tester make their own assumptions and rules. These Validation actions are more likely to encounter bugs and system errors when not told what to do step-by-step.
There is of course, a need for strong caveat(s) at this stage. There are two (2) critical success factors in an un-scripted test case:
1. The Validation tester must, MUST be trained, experienced, and knowledgeable on all aspects of software testing and good documentation,
2. The unscripted test case must, MUST, be attributable to the human tester and properly documented & supported by real time, continuous supportive evidence.
In many cases an unscripted test case will provide a higher quality return on testing effort with less documentation and is a more realistic (true) example of the system’s ability to remain accurate and reliable under real world conditions.
If you are exploring how to implement a more streamlined approach to your Validation methodology, reach out to our global team of experts and leverage our deep CSQA expertise. hello@zigzagassociates.com
Mason Cox is a global expert in Computer System Quality Assurance (CSQA). His skills in designing Validation Test architecture, Data Management, and authoring original Validation Documents for Regulated Computer Systems development are unmatched in the industry.
He has deep knowledge of FDA 21 Part 11, GAMP5, and remote data collection and device hardware (ePRO/eCOA/Wearables). Mason was formerly a Senior Quality Assurance Validation Auditor in a global CRO with experience in: Information Technology (IT), Data Management (DM), Pharmacovigilance (PV), Clinical Operations, and Biostatistics as well as hosting and conducting over two hundred on-site and remote Sponsor, Vendor, and Internal Audits.