Methods for estimating testing time

Time is money, and finding out how much time and money it will take to test a product is one of the first steps when working with a QA team. 

Test Estimation

Test Estimation is the process of calculating the time it will take to complete a task. Estimating test efforts is one of the main and important tasks in test management.

During this process, the following moments are estimated:

• the required resources to perform any project tasks (this includes employees, equipment, etc.);
• time spent on the project;
• knowledge and experience of team members (a team with low testing skills will take longer to complete the project than one with more advanced ones);
• project budget. 

Today, there are numerous methods of project estimation. In this article, we will describe the most common estimation methods, as well as their advantages and disadvantages.

1. Work Breakdown Structure

This is a basic technique for dividing a project into smaller parts, with necessary time constraints. Since small tasks are much easier to estimate, they are distributed among team members. Each person takes responsibility for the task and delivers the result within a certain period of time. The WBS is the most powerful and effective method among the test time estimation methods.

2. Three-Point Estimation

The idea behind this method is to find the best and worst working conditions for your team. It's a statistical method, but it also breaks down tasks into subtasks (similar to the WBS technique). Each subtask should be divided into three possible scenarios:

1. You have a talented team and all the necessary resources, there are no problems, everything goes smoothly, and you can complete the task in, for example, 100 man-hours (hours of actual human work). This is the optimistic scenario (О).
2. You have a good team, enough resources, and almost everything is going right, sometimes with some problems, and you will complete the task in 150 man-hours. This is the normal scenario (N).
3. The team is inexperienced, most situations do not correspond to the expected scenarios, you will have to solve numerous problems, and the task will be completed in 200 man-hours. This is the pessimistic scenario (P).

This way, you have three values: O = 100, N = 150, P = 200.

Now you can calculate the average value for the test estimation (E) using the following formula:

E = (O+4*N+P)/6

E = (100 + 4 x 150 + 200)/6 = 150 man-hours

Since the average value can vary slightly, you need to calculate the standard deviation (SD) - the limits within which E can change. The formula looks as follows:

SD = (P – O)/6

SD = (200 – 100)/6 = 16,7 man-hours

The final estimate can be summarized as follows: the team needs 150 +/- 16.7 man-hours to complete the subtask.

3. Function Point Analysis

This method sets the time estimation depending on the complexity of the task (from 1 to 5). Each module is assigned points (FP – functional points) depending on its complexity. Simple tasks receive lower points, and complex ones – higher. The total effort is calculated using the next formula:

Total effort = total number of points x score assigned to one point

The score assigned to an item is a set value that the QA team leader determines based on the team's experience. It is calculated based on time, cost, or size.

For example, the score for FP = $100

The entire project is divided into three groups of modules:

• Complex modules (FP 5) – 2 pieces
• Average modules (FP 3) – 10 pieces
• Simple modules (FP 1) – 5 pieces

Total Effort = 45 x 100 = 4500$

This means that to complete the project, you need $4500

FPA is not an accurate method since it is connected with risks and can provide somewhat misguided results.

4. Delphi

This method is based on communication and relies on the opinion of the team's experts. The essence of it is that an expert group led by a manager discusses tasks and makes anonymous personal predictions (how many man-hours this task will take), explaining their opinions.

Usually, after the first round, the range of answers is quite wide. Then, the experts are asked to revise their answers based on the opinions of the other team members. It is possible to conduct several rounds until the range of answers is reduced and an average value can be calculated. The process is completed after a limited number of rounds if a consensus is reached or if the results are stable.

5. Agile

The methods described above are more suitable for Waterfall and V-model development. They use a bottom-up approach in which all details and requirements must be defined and all tasks must be estimated separately before planning a project schedule and budget.

More and more customers and suppliers prefer using the agile development methodology. Agile estimation is significantly different from traditional methods and uses a top-down approach.

In other words, first, you process the data that is currently available and provide a higher-level estimate that is sufficient to start the project. As the project progresses, new information becomes available and more details are revealed at lower levels. New knowledge is incorporated into the estimation, and values are specified and refined without stopping the development process. This approach allows the team to effectively estimate new features.

Kanban and Scrum are two popular methods for agile development.

Kanban is a visualization method. The entire workflow is organized on a Kanban board, which allows team members to track progress and see new features or difficulties as soon as they arise. In this way, the team can determine the appropriate workload.

Estimating the duration of an item is not necessary for this method. After completing one item, team members just proceed with the next one.

Scrum is a framework for process structuring. Using Scrum, developers can break their work into sprints. At the beginning of each sprint, they make plans, estimate size, prioritize, and predict releases. Working software is delivered at the end of each sprint.

6. Wall Estimation

There are two flexible estimation methods: planning poker and wall estimation. Planning poker is a great tool for estimating user stories, but with hundreds of them, it takes way too much time. Wall Estimation allows interested parties to prioritize among a large group of stories.

For the wall estimation, the first step is to print the user stories on cards. Then the development team and interested parties gather in a room with a large blank wall:

• The height of the wall indicates the priority. Stories at the top have a higher priority; stories at the bottom have a lower priority. Story priority can be based on ROI, business value, etc.
• Width is used for size (scope of work). Stories on the left are smaller; stories on the right are larger.

After stories are grouped by size, customers and interested parties can be involved to prioritize them. In the end, the wall should break down into approximately four quadrants:

• Stories in the upper left corner have a high priority and are small.
• Stories in the upper right corner have a high priority but are also large. These stories should be sorted out soon so that they can be included in future sprints.
• The lower left quadrant contains small stories that have a lower priority. They are likely to go to the bottom of the backlog.
• The lower right quadrant is filled with larger stories that also have lower priority (epics or themes). They will need to be broken down into smaller stories eventually, but only when they reach a higher priority.

Estimation rules

These are the rules to follow in estimating test time:

1. Conduct estimations based on software requirements.
2. Use data, experience, and knowledge from previous projects.
3. Consider possible internal and external risks (downtime, professional development, lack of resources, etc.).
4. Take into account the availability of resources, knowledge, and team skills.
5. Update the estimate if there are changes in the project.
6. Check the estimate by several specialists.

Therefore, the process of estimating test time is extremely important. However, one should also note that these are just model templates for estimating test time. The methods will likely be adjusted depending on the project.