A few years ago, software development was linear – every team had a well scoped role. Product Managers decided requirements, engineers wrote code, QA tested and gave the green signal. Agile started to blur the boundaries, we started thinking of building products as a team and not in silos. Concepts such as ‘fail fast and learn’ needed all teams involved right from the beginning. Now with AI, features are being shipped faster and faster, but customer adoption remains low and in many cases is trending down. It’s become painfully obvious that end to end quality assurance cannot be treated as the final checkpoint before a release – it needs to be incorporated into every iteration.
We’ve all had the experience of shipping a feature with no major defects, but customers still complain that the feature is not intuitive and easy to use. QA ensured the requirements were met, found defects and ensured that technically, the feature behaves exactly as designed, but the customer’s experience still ending up being disappointing. This is the biggest drawback of the traditional QA process.
The big question that is being asked currently is whether testing is just the job of QA or should it be more inclusive? This sounds controversial, because isn’t QA what QA does? But in this world of weekly releases and vibe coding, is it fair to expect QA to be the only function that validates features being released? QA will always be accountable and responsible for quality, however ensuring that the product delivers the required value to the customer is not something that QA can do alone.
Product Manager talk to customers, analyze workflows and processes to identity problems, prioritize these problems and design features to solve these problems. However, all of this knowledge is rarely passed onto the QA teams. Requirements, use case scenarios, stories and acceptance criteria – these are all necessary, but unfortunately, they are not sufficient. These do not help answer questions like:
· Does this workflow solve the customer’s problem?
· Is the feature intuitive to use?
· Does the feature align with the customer’s business metrics?
· Can a customer naturally discover and use this capability?
The answers to these questions rarely can be found in a test plan, yet they often determine whether a feature succeeds or fails after shipping.
So what does this mean – should Product Managers execute test plans and regressions? Absolutely not. The answer is User Acceptance Testing (UAT).
Set-Up
To make UAT effective, teams need to establish a clear testing environment that closely mirrors the customer experience. Test data should represent real-world conditions and processes, along with the necessary hardware and software interfaces set up and functional. This helps in testing all scenarios including edge cases, workarounds, and dependencies that may not appear in requirements documents.
An important call out here: if possible, this test environment should always be reserved for UAT. I’ve been part of UATs where I’ve spent 30 mins executing a full workflow process and the feature breaks suddenly because a developer deployed a piece of code to conduct some quick tests of their own.
In many organizations that create software for industries such as healthcare, defense, etc. many employees are also former customers. These employees have experience in your customers’ workflows with all the problems and workarounds. Identifying a sub-set of these employees as your Subject Matter Experts (SMEs) group and asking them to participate in UAT can be very beneficial. These SMEs know and have lived the edge cases and the 1% situations, and in many cases also have experience with your competition’s products. Hence, their feedback can truly help make your product better. SMEs cannot be expected to participate in every sprint, but a dedicated UAT environment can enable them to test on their own time.
Cadence
In traditional development models, UAT often happens just before release when most of the features have been developed, and we are in the bug fixing stage. Unfortunately, by this stage feedback can be expensive, time consuming to incorporate, leading to feature releases that does not delight customers. Modern project teams should instead incorporate quick UATs in every sprint. When a sprint is completed, the product team must be expected and allowed to test the product. This helps identify design and implementation flaws early, when improvements and changes are still possible.
Objectives
The primary goal of UAT is not to find bugs. Yes, read that again – this is not a bug finding exercise. Bugs finding is still very much the purview of QA. UAT’s north star is to validate if the feature solves the identified customer problem in an intuitive and usable way. Before testing begins, Product Managers and QA should agree on the goals of each UAT cycle - especially the critical customer workflows that need to be evaluated. This includes defining realistic user scenarios, expected outcomes, and key questions that the team wants answered. They should focus on executing common workflows without relying on implementation knowledge and ask simple but important questions, as these observations uncover improvements that can never appear in a bug report.
Example, rather than focusing on whether a particular button works, the focus should be on asking the following questions:
· Is the button placement intuitive?
· Does the customer understand the purpose of the button easily?
· Would the customer know when to use the button?
· Are the inputs accepted relevant to that particular use case?
Results
The results of the UAT must be discussed as a team. All bugs identified are owned by the QA team, whereas it’s the Product Manager who must decide when the various improvements or design changes must be incorporated into the development process. In a truly agile environment, these tests will allow incremental improvements that lead to the delivery of a product that delivers value to customers.
Successful teams set up UAT as a necessary partnership between Product Management and QA. Together, they ensure that teams are not only building products that work correctly, but products that delight their customers. In an AI-driven world where speed of delivery continues to increase, this partnership is becoming less of a nice-to-have and more of a must-have.