The Next Big Debate: The Role of AI for Software Testing

First, why do we need AI for software testing? Currently, the focus seems to be primarily on automation testing within the realm of software testing. It's time to prepare for the next phase, where AI for software testing will play a crucial role. The development of automation testing is driven by software development models like Agile, aiming to deliver less faulty products to the market faster. Consequently, we've entered the era of Daily Dev with the rise of DevOps. DevOps is pushing organizations to accelerate the QA process further, reducing testing costs and enabling more optimal management.

So:

• Will AI-based testing technology be a dream or a nightmare for professional software testers?

• Can (and should) testing rely on AI?

• What are the real benefits and risks of AI?

To pique your interest further, here is a summary from Wolfgang Platz, Founder and Chief Product Officer, about the final debate at Accelerate SF (originally published on InfoWorld):

Boring or Exciting? The Evolution of Software Testing

Imagine this all-too-common scenario: Software testers are asked to check the functionality of a new application after developers assure that the "Dev" part is complete. They have a list of actions and manually click through each step—trying to follow instructions closely and systematically record the outcomes of each step in case something goes wrong. They do not truly understand the business logic from the user's perspective and rarely interact with developers or stakeholders. The main goal for these testers is to access X and verify that the output is Y and not Z.

There's no denying that this is very boring, but this isn’t truly software testing. It is certainly not the kind of software testing that demands providing a customer experience faster than the competition.

Digital Transformation - Impacting almost every business department nowadays, with some QA departments remaining unaffected. The resulting wave could wash away user boredom from manual validation. But it could also elevate the practice to a more engaging level, where testers become the main managers of the customer experience.

In the recent debate topic "The Great Testing Debate," we discussed the evolution of testing, touching on the emerging role of SDET and the future of TCoEs. We all agreed that the current issues present a great opportunity to reshape the role of testing as an innovation-driving discipline rather than one that hampers development. However, the devil is in the details. What exactly needs to change, and how do we get there?

Below are some interesting summaries of the debated viewpoints:

One Team, One Fight

Speaker Ander introduced a great mantra to capture how quality becomes everyone's responsibility: "When customers see a bug, they don’t blame the testers or the developers. They hold the company that released the software accountable."

Continuing with this theme, the speaker also shared an interesting approach to software quality as a process: "Think about School House Rock and how a bill becomes a law. This is how we should think about code. We need to consider everything that happens to our code as our software travels along the factory conveyor belt, all those differences included in the process. When we map this out, that’s when we can start optimizing the process. If a bug escapes from that process, it means an organizational failure that needs to be detected, analyzed, and addressed."

Diverse Needs AI for Software Testing

In this topic, we focused on moving from DevOps to opening up more options to fit different skill sets. For example, former manual testers can take different paths to contribute to the automation effort. Those who want to continue with automation testing can learn and experiment with models or Selenium to maintain the market in software testing. Others may decide to focus on test data management. Some may branch out and master DevOps practices and platforms. Accenture has about 40,000 testers. They all don’t have the same personalities and skills, making Accenture a much stronger company.

The Importance of Unit Testing

The next debate topic had speakers who were staunch supporters of unit testing. They required engineers to write unit tests for their code because it is much faster and cheaper than letting bugs slip through at higher levels. From current experience, many issues are found in production, and this situation can (and should) be stopped with better unit testing.

I agree that unit testing is valuable and necessary. As speakers noted, it forms a stable foundation in the Agile testing pyramid model. However, I don’t believe unit tests will capture the real issues that affect the end user’s experience. Especially in large enterprises, I have seen unit tests deteriorate over time to the point where no one pays attention to them anymore. I also notice that most issues reported by end users cannot be found at the unit level; they will require integration tests, system tests, or end-to-end tests by professional testers who truly understand the software.

This eventually became one of the few debated points we had to agree to disagree on.

Developers Test, But They Are Not Testers

I emphasize that unless you are a GAFA (Google, Apple, Facebook, Amazon), you cannot afford the luxury of hiring SDETs: Developers will devote themselves to testing. However, even if we can get developers to test, they are not the best choice for protecting the end user’s experience.

And yes, of course, developers can check their code and analyze, unit test, peer code review... Finding issues as early as possible. However, we should also consider that the best person at creating something does not mean they are the best at breaking it. These are clearly different disciplines. If you move into a skyscraper, do you want the final inspection to be done by the architect or a professional building inspector?

One speaker said, "QA is a mindset. To be a good QA, you have to be passionate about quality."

Shift Left

More and more organizations, including Accenture, Electric Cloud, and many Tricentis customers, are reinventing "Testing" to become "Quality Engineering." When testing is reactive, quality engineering is proactive. We not only catch bugs but also prevent them from entering the code in the first place.

We all agree that quality engineering means addressing quality issues much earlier in the process than most organizations do today. But the solution is not simply to apply techniques often labeled as "Shift Left Testing." It also involves early-stage reviews, with testers participating in those reviews. Furthermore, it may involve inviting testers to design phases, where they can establish strategies for building quality and testability from the start.

With this change, testers will be elevated in their roles to trusted advisors, quality ambassadors rather than doing the boring testing work of the past.