Console game testing

According to Grand Review Research, the popularity of console games will increase in the coming years due to new and advanced audiovisual devices. It is predicted that the global market will reach $51.15 billion by 2027. The gaming experience will become better thanks to high-quality sound systems, displays, high-definition TVs with HDMI ports, and mostly wireless connection. Over the past two decades, gaming platforms have evolved immensely. With the improvement of hardware, each generation has produced more complex games. Some games have budgets reaching millions of dollars, competing with movie production. Many games with high budgets have been released with performance issues or poor design, which led to negative reviews and failure in sales. The reason was the lack of proper quality analysis. Therefore, game success depends both on the quality of the game and the user's satisfaction. 

According to The Guardian, these are some of the key trends that are expected to change the gaming experience:

• virtual reality;
• physically collaborative games;
• augmented reality;
• incremental console updates.

The process of creating games can be usually divided into two parts. First, there is the development side, where programmers and level designers take care of the game from scratch to ensure it functions smoothly. There is also the creative side, where artists, designers, and voice actors create visuals and narratives that engage players. The most forgotten group in the process is the game testers.

The most common belief is that all game testers focus solely on game playability and report bugs that interfere with the process. 

In reality, the combination of different approaches in the testing process and early detection of bugs is the key to a successful product release.

Platforms(Xbox Series S, PlayStation 5 та Nintendo Switch)

Xbox is a series of game consoles from the American multinational corporation Microsoft.

The first console was launched on November 15, 2001, and the latest one is currently available since November 10, 2020.

PlayStation is a fifth-generation game console developed by Sony Computer Entertainment. The console was released on December 3, 1994, in Japan, in the United States on September 9, 1995, and in Europe on September 29, 1995. The console is the first gaming system in the PlayStation line. The console became very popular, providing Sony with a breakthrough in the gaming industry, which was previously dominated by Nintendo and Sega.

Nintendo Switch is a hybrid game console developed by the Japanese company Nintendo. It is a combination of a home game console and a portable system that can be used as both. During the development phase, Nintendo Switch had the code name "Nintendo NX", which was first demonstrated in October 2016 and was released worldwide on March 3, 2017.

Testing process

As the popularity of video games grows, their testing becomes more and more important to ensure that they will be close to perfect by the moment of their release.  The QA team consists of specialists who not only look at the product as part of their work but also understand it from the user's point of view. 

The list of services provided by a console game tester:

• testing for console compliance (Sony PlayStation, Microsoft Xbox, Nintendo Switch);
• Functionality testing (pre-alpha, alpha, beta, and playtest)
• gameplay testing; 
• localization testing;
• load testing;
• stress testing;
• network testing;
• testing of PS VR integration.

When it comes to consoles, most tests are not performed on a regular system or consumer device. Developers and publishers are provided with special test equipment. The most significant tools are test or debug kits. The main difference between them and consumer units is the possibility to load games from a CD, USB drive, or hard disk. Unlike test systems, the function of setting the region of publication is absent in consumer units to prevent software piracy and gray market imports.

Test sets have the same technical characteristics and general appearance as consumer devices, even though they are often enhanced with additional ports and connectors for other test equipment. They also contain additional options such as automatic compliance checks, especially when it comes to data storage.

Developer kits are not usually used by game testers, instead, they are used by programmers to test lower levels. On top of the features, developer kits usually have higher hardware technical characteristics, particularly noticeable is the increase in the system memory. It allows evaluating the performance of an early version of the game without worrying about optimization. Development kits are usually larger and have a different design than test or consumer units.

Among the biggest obstacles games face on their way to the console market is the potentially difficult certification process. The product is tested by console manufacturers to ensure that every aspect of the game (text, instruction manuals, etc.) fully complies with their strict terminology, standards, and naming requirements. This compliance certification process is not only important to ensure that players understand the meaning behind the product, but is also a mandatory step in the release of all games for all consoles.

The result can only be guaranteed if the testers are fully trained, qualified, and use the most advanced tools and technologies. All new projects must be checked by the certification department, which monitors quality assurance and up-to-the-minute updates in industry compliance processes. Specialists who specialize in TRC (Technical Requirements List) and TCR (Technical Certification Requirements) are highly valued. It is necessary to work closely with the development team at every stage of the process, from early development to release and updates.  A complex regression testing process allows specialists to evaluate updates or new features without affecting the existing game content. In addition, the fact that consoles support different language combinations allows the release of games for different devices and audiences in different regions.

Functionality testing 

Functionality testing helps to detect various bugs in the game that affect its performance and UX.  At this stage, it is checked whether everything meets the specifications and if there are any major problems in the process, graphics, audiovisual effects, etc. Functional testing is the most common, but not the only one.

Localization testing

In addition to functionality testing, there is also localization testing, which focuses on the quality of all translated game content. During this process, specialists play the game from start to finish, going through every single detail and looking for signs of language errors. This means that every side quest, story scene, in-game menu, and error message must be tested. 

Compatibility testing

In the past, video games only ran on one platform. Today, they can be designed to run on a variety of devices and systems, whether it's iOS, Android, PlayStation, Nintendo, or Xbox. Compatibility testing ensures that a video game works flawlessly across platforms.

Console compliance testing 

Console compliance testing is a pivotal moment in the life cycle of a console game.

Testing in different environments, it is important to take into account how different all consoles are. Although they run games in the same way and seem to have similar functionality, they are fundamentally different machines. Each has its controller with its unique terminology which describes the way the components or names of elements should be described and displayed in the game.  

This kind of terminology accuracy testing can cover any range of hardware, software, or procedures. Console compliance testers are experts in the mandatory terminology defined by the manufacturer. They can recommend what steps developers and publishers need to take to ensure that their game passes the compliance check.

What are the consequences of non-compliance?

Games that do not pass the console manufacturer's compliance test are simply not released. Compliance testing follows a strict pass/fail basis. It leaves no room for ambiguity. Games without the correct terminology will not be allowed to be released on consoles until they fully comply with the requirements.

On top of that, it can be quite expensive to go through the retesting process, so meeting the requirements from the start is of utmost importance for a game to be released on time and within budget.

Regression testing

In the 1990s and early 2000s, games released to the market were not subject to any changes.  Now games can receive frequent updates and DLCS (downloadable content) from developers even after purchasing them. When these updates are released, regression testing is used to ensure that they do not break other components of the product.

Ad Hoc testing

Ad Hoc testing – is an informal test that doesn't have standard methods or steps. Specialists use a bug prediction technique to detect potential bugs in the future. 

By their nature, console games are deeper and have more levels and complexities. Designed for advanced gamers, console games can reach from 5-6 hours to 400+ hours of playtime. 

Checking a game includes not only design, gameplay mechanics, functionality, and usability. The tester should examine it for engagement, ability to hold attention, productivity, and sense of purpose.

Testing requirements

Since the console gaming industry is preparing to release a new generation of games with advanced features, testing requirements are also changing. The compliance process is very important. Quality control specialists recommend specific measures to ensure that games meet all console compliance requirements regardless of platform.

Faster/periodic releases

Today, developers release games faster thanks to agile development methodologies.  

Collaboration

During game development, the QA team participates in Scrum meetings with the development team and the customer. It provides feedback on the game's quality to improve its stability in the market.

Available infrastructure

Having the necessary devices for real-time testing can improve the performance and functionality testing results. This provides an accurate representation of how the game will work in real life and helps to find bugs faster.

Thinking outside the box

Straightforward play during testing is important, but it's not enough. In real life, the user may play in different ways, and the tester must be able to anticipate these steps. For this, the testers must be experienced gamers.

Conclusion

The current strength of the games industry is that there are now more platforms than ever to release the next big project. For developers and game publishers, this also means a better chance of success.  With this increased opportunity for global multi-platform distribution also comes additional responsibilities, such as console and operating system compliance.  Full compliance with all devices on which a game is released is mandatory. Before release, manufacturers must ensure that their requirements and standards are correctly implemented. This may include: 

• naming standards;
• capitalization of native brand terminology;
• spelling of button and control terminology, and much, much more.

Not meeting the standards can lead to a frustrating and prolonged launch process (the product cannot be released without compliance certification). It can also be costly for the game's budget and harm its overall success.