What are risks, and how do they impact the quality of work

The sphere of testing is a complex and multi-level system that includes a vast number of diverse elements. In addition to the obvious basic components of testing, one of the less obvious but equally crucial aspects of the QA field is risk.

What is risk?

Risk is an active factor or a developing process factor that has the potential for a negative impact on the process. 

In simple terms, risk is something that can potentially complicate the situation. It's essential to distinguish between the concepts of risk and issues. Risk is something that might happen, while an issue is something that is already happening and has a negative impact. Thus, the main task regarding risks is to identify them in a timely manner, anticipate them, and take measures to mitigate their occurrence, ensuring that theoretical risks do not become actual problems.

In the field of testing, risks can generally be divided into two groups: product risks and project risks. The first group includes everything directly related to the testing object, such as:

• inaccuracy/superficiality of specifications or, conversely, overly complex and convoluted specifications;
• changes in specifications during development/testing;
• complications in the functional components of the product;
• security-related risks;
• data loss, and more.

Project risks, on the other hand, encompass all other risks:

• inadequate project effort estimation;
• risks related to sudden workforce reduction (layoffs/illness/other reasons for the sudden absence of employees);
• communication risks (slow response from the client to questions/clarifications, inadequate communication response from project team members, and others);
• testing infrastructure risks (sudden lack of internet connection or power supply for an extended period, unplanned unavailability of devices, and more);
• risks of ignoring risks (insufficiently realistic assessment of scenarios by each member of the testing team).

Furthermore, all risks in the testing field can be categorized as controllable or uncontrollable. Controllable risks are those theoretical negative manifestations that can be almost entirely mitigated. For instance, communication risk can be minimized in advance by defining mandatory communication rules within the team, providing additional communication channels, secondary sources of information, etc.

Uncontrollable risks, correspondingly, are those potential negative outcomes that cannot be directly controlled or are too challenging to control. For example, it's difficult to predict whether a developer will get sick during the project or whether there will be an unexpected power outage. Such risks can only be superficially covered, so create Plan B in case they occur. For instance, a backup system for key data or even a backup power supply system can be in place if a power outage is critical to the project.

Thus, the process of testing with consideration for potential negative manifestations, referred to as risk-based testing, is today one of the most important areas in the entire testing sphere. Starting from the analysis of testing documentation and ending with the final testing modules of the latest versions of the finished product, all testing processes need to be planned and implemented hand in hand with researching and analyzing the environment for the likelihood of all kinds of risks and developing measures to mitigate them.

The risk management process generally consists of the following stages:

1. Risk Identification.

The first step involves creating a list of everything that could potentially go wrong. QA experts help compile this list through requirement analysis, brainstorming, consultations with experts, previous project experience, and more. The main goal at this stage is to identify as many potential threats as possible.

2. Risk Assessment.

Once the list is compiled, the team proceeds to the next step of risk analysis. Responsible experts evaluate each potential threat based on two factors: the probability of occurrence and the potential damage it could cause. At this stage, a risk matrix is often used.

3. Risk Response Planning.

After risks are identified and assessed, a plan is developed on how to prevent these risks from turning into real problems. Based on the data obtained, the QA Lead or other responsible professionals can start developing a testing strategy. Modules, elements, or functions with significant risks of failure will be tested more thoroughly. The results of the assessment also determine how experienced professionals will work on a particular task.

4. Risk Monitoring.

At this stage, professionals actively monitor for new risks, reevaluate previously identified risks, and adjust the response plan as needed.

During the risk assessment stage, a matrix method is often used, which is a convenient and straightforward way to analyze the environment for the likelihood of risky situations. In this matrix, one axis represents the likelihood of an event occurring, and the other axis represents the potential impact on the project. This allows all risks to be divided into groups (see Illustration 1).

Certainly, depending on the project, such a matrix can be more detailed. In that case, higher levels of probability will be assigned to possible risks. For example: frequent, possible, random, rare, very rare. As for the degree of potential impact, it can be divided into catastrophic, critical, serious, acceptable, and insignificant.

In addition to matrix-based methods, there are other techniques for risk assessment and management: lightweight methods (PRAM, PRISMA) and more formal methods (FMEA, FTA).

Lightweight methods are somewhat more flexible and informal. For example, PRAM (Pragmatic Risk Analysis and Management) and PRISMA (Product Risk Management) methods successfully and easily combine risk-based and requirement-based strategies. These approaches generally use requirement specifications as input data but may involve stakeholders in the process. Lightweight risk analysis methods emphasize technical or commercial risks, considering the probability of risk occurrence and factors affecting that probability.

FTA (Fault Tree Analysis) models systematically identify the causes that could lead to defects in business processes or system failures. The analysis proceeds from the highest to the lowest level, visually resembling a tree-like structure with branching at each level. This method allows for a deep examination of even the smallest risk occurrences throughout the testing process.

FMEA (Failure Mode and Effect Analysis) is the most popular approach to risk-based testing. It involves analyzing the causes and consequences of system failures, identifying potential defects, and the reasons for their occurrence. Working with FMEA means the project team tries to identify all components, processes, and modules where failures might occur. These failures, with varying probabilities, can lead to a deterioration in the quality of the software.

When using the FMEA method during brainstorming, risks of the system are determined based on three indicators: severity (S), priority (P), and likelihood (L). Each potential risk occurrence is assigned values for these three indicators on a scale from 1 to 5, where 1 represents the highest importance/severity/impact, and 5 represents the lowest. Then, the Risk Priority Number (RPN) is calculated for each risk, which is the product of these three indicators: RPN = S*P*L.

Based on the obtained RPN value, the depth of risk testing is determined as follows:

• if the RPN value is 1-10, a comprehensive test with various positive and negative checks, including minor tests, is required;
• if the RPN value is 11-30, mandatory positive testing and basic negative checks of the main functionality should be performed;
• if the RPN value is 31-70, functional testing should be conducted at a superficial level;
• if the RPN value is above 70, testing is performed when time permits.

The advantages of the FMEA model lie in its transparent approach to assessing the testing scope, based on the risks associated with software use.

Risk-based testing allows companies to effectively manage limited resources. This approach helps QA teams optimize their work, prioritize correctly, identify critical errors earlier, and understand the level of test coverage at each stage of the software development lifecycle. Thus, the main advantages of risk-based testing include reduced development costs, shorter time-to-market, and improved product quality.