Software needs to be tested. Therefore, companies require tests, procedures and processes around quality that need to fit in with the flow of development, integration and deployment of applications. While continuous testing is dependent upon strategic automation, it is not synonymous with automated testing.
A continuous testing process encompasses other quality assurance (QA) activities that impact overall product quality. As such, continuous testing depends on four major pillars:
- User experience: Are users perceiving value in the application delivered?
- Code: Did developers create the application code correctly and securely?
- Pipeline automation: Can the application code flow across environments without manual intervention?
- Application quality: Did developers create the correct features?
Currently, many companies are unable to deliver these four pillars within their development cadence, because their QA remains shackled to old ideas and processes. As a result, companies fail to achieve the concept of releasing with confidence.
User experience is unquestionably the tie that binds customer loyalty or lack of same. Successful companies provide their product users with consistent quality experiences that at least meet their expectations and at best, exceed them. Users seek features, functionality, speed of use and delivery of relevant information in a timely manner.
This drives customer allegiance and endorses brand loyalty. In the digital age, user experience is as important as the products/goods businesses are selling.
Consequently, organisations are increasingly focused on meeting user requirements through faster delivery cycles, increased features and product improvements. This volume of change requires adoption of new technologies, processes and a supportive culture.
With regards to the latter, testing needs to shift ‘left and right’. Shifting testing right is essentially a method of continuously testing software while it is in a post-production environment. Also known as ‘testing in production’, this approach helps software developers uncover new, unexpected scenarios that may not have been detected within the development environment.
The shift left testing approach ensures businesses test early and often in the lifecycle, thereby moving left on the project timeline. As a result, testing is performed early to improve the quality of the process and features of the application and post release to determine user experience, application performance and track user behaviour – this in turn can drive product enhancements or solve user issues.
The following are some facts to take note of:
- 64% of total defect costs stem from the engineering and design phase. This means more than half of the time defects can be traced back to poorly-defined requirements. Late defect detection results in higher costs than when discovered earlier in the software delivery lifecycle (SDLC). (Quality Flaws: Issues and Challenges in Software Development. Hyderabad Business School, GITAM University, 2012)
- 70% of testing is still manual. Test teams are inundated by a myriad of mundane tasks. Teams spend too much time creating tests by hand, reading requirements, creating traceability matrices, identifying and creating test data, and more. (Bloor Research Spotlight Paper: Automated test case generation. Howard, Philip, September 2014)
- Only 10-20% of tests are covered, exposing software to defects. Because manual testing is tedious and inefficient, it can lead to both under-testing and over-testing. Some features are tested too little, while other features are tested too much, sometimes as much as 18 times over. (Implementation experience metrics collected by CA Technologies)
Smart testing is a cultural shift in an organisation. Along with cross-functional teams, agile testers are part developer, part tester and a bit of business analyst also.
The smart tester is involved in all aspects of the SDLC, from design to release. The agile tester relies on a tool chain that integrates, is cross-functional, and helps all team members to have clarity on what is being developed. Building this into the agile team will ensure ambiguous requirements, inconsistent workflows and misalignment between business and dev are corrected before the first line of code is written.
Continuous improvement is imperative to any application; however, knowing what, how and where to improve is as important.
It is the advanced engineering of requirements and test design automation that set up continuous testing for the agile environment.
Continuous improvement is imperative to any application; however, knowing what, how and where to improve is as important. Shifting testing right provides the business with insights into user behaviour and performance indices that are invaluable to enhance and improve the application.
Long, manual testing processes designed to be carried out when the application is already completed should be very familiar to most organisations; 22% of companies still don’t write tests until after development has completed its work. But waiting to test applications after the user interface is a recipe for disaster.
Poorly constructed APIs and process flows result in applications that fail to meet performance expectations. Early testing of workflows, APIs and overall performance of the application must be built into the SDLC from the outset. It is as important for these performance criteria to be as fully developed as it is for pure functionality.
Continuous testing is crucial. Users want software that performs perfectly and speedily every time. They have little patience with apps that do not deliver what is expected. Increasingly, organisations are moving towards a continuous testing model to deliver better software that more accurately reflects user needs. But enterprises are finding that the bottlenecks of older software delivery methods remain, delaying projects and impacting quality. Siloed test teams are often seen as a delivery bottleneck.
Poor code begins with low-quality requirements, which then carry over to the test design and execution process. Performed manually, traditional testing methods are too slow to keep pace with today’s development timetables.
Building software more quickly with fewer defects requires a new continuous testing approach. Testing must become an end-to-end, cross-functional operation, involving all teams throughout the development lifecycle. Continuous testing incorporates the methods of agile development to the testing and QA process, resulting in higher efficiency.
Automation is key and necessitates an end-to-end testing solution, coupled with existing integration and delivery processes. Only then can design flaws and manual processes be eliminated to facilitate true continuity throughout the development lifecycle.
The continuous testing approach helps ensure quality is baked in while requirements are being defined, and then validates each subsequent component at the development level before it enters the system. Thus, iterative QA commences from the beginning.
Defects and miscommunication from changing user needs can be resolved as they emerge, avoiding later rework and damage to the ultimate user experience.
The result of continuous testing is software that is built better from the start, resulting in faster delivery and lower costs.
In my final article in this series of three, I will discuss the rise of continuous testing accelerating application delivery and quality.
