Q and A Testing
Legendary
https://youtu.be/Y_QNyVj8smk This here, is a link to a bit of game play from the game Legendary. This game is well known for it's bugs and glitches, and from the research I've done on it, this elevator glitch seems to be the most well known of them all (click link to see). This glitch happens right at the end of the game and prevents the player from completing the game. In the link, the character on screen runs into a lift and waits for it to move, as the lift moves up, the character falls through the floor of the map and just carries on falling because there is no longer a platform for it to land on. This bug was never actually fixed by the producers because they simply just didn't want to, although codes were released for the consumer to fix the glitch themselves. |
Image from game.
|
Dark Souls 1
Dark Souls as a game in general is a brilliant game, difficult but very good. There are minimal aspects of the game to complain about , but one thing that wasn't good about the game was the PC version. The game was prominently meant for consoles but was converted for PC gamers, this was what brought it down, the game didn't get optimised so the frame rate was terrible and the mouse didn't co-operate well and would minimise the screen whenever the mouse moved off the screen, this happened especially lots when you had dual monitors, you could be in the middle of a really hard battle and the screen would close, and because the game didn't pause, you would die a lot more and this is extremely aggravating. This only happened on PC, so if you were to play it on a console instead, you wouldn't encounter this problem. One way you could reduce the problem if you could definitely only have it on the PC, would be to buy a controller that plugs into your PC. |
Batman Arkham Knight - PC
Batman Arkham Knight has many glitches and bugs, here is a link I found to 10 of them: https://www.youtube.com/watch?v=sUnAQSnTWlQ Some of the glitches include rain inside the building, loading issues, textures not loading making the other characters in the game appear broken and look like robots that have been in accident, also objects such as the Bat-mobile are left floating in the air, making it unreachable. Also there have been incidents where Batman shoots his grapnel gun and lands on the edge of an invisible building and just hangs there. One of the main glitches that caused a lot of people problems, was the glitch that whenever you got into your tank, the frame rate would drop dramatically, making the game extremely difficult to play. To try and amend this Rocksteady, gave people refunds who asked, and they wouldn't ask many questions. They then tried to re-release it with lots of DLC, but the game was still broken so people were still not happy with the game at all. Steel Battalian Heavy Armour - Console Steel Battalian Heavy Armour had lots of problems and this was mostly due to the fact you had to use the Kinect. The Kinect always faces many problems, but with this game, if you even ever so slightly flinched on your couch it would ask you to re calibrate, or it would do the opposite and just not read your movements at all meaning you couldn't complete the game, or at least find it very very difficult. If the Kinect and all its problems didn't deter the player enough, the range on the guns was also abysmal, so it was near impossible to hit anything that wasn't within point blank range. As I say though, the big main problem with the game was the fact the game is heavily reliant on the Kinect, and the Kinect is just very unreliable. |
Main responsibilities of a tester and lead tester
QA Tester -
Game testers play a vital role within the games industry. They test, tune and debug the games, finding anything that isn't supposed to be there, and then they report it back to the designers and programmers so they can fix whatever is wrong, whether it's textures not loading, NPCs not working properly, objects not being solid (like buildings) or sounds not playing were they should etc. They test for bugs in the software from complete crashes to the tiniest of glitches. They also report back on how the game plays in general as they are basically the games first proper audience, so they will mention good things about the game as well as the bugs and glitches. To be a QA tester, you must know which issues are the most important and should prioritise them, because if a game braking bug got out, the game testers would be blaming and probably replaced if the bug was that bad. Testers work to deadlines and must understand production and marketing schedules so they can make sure they do everything on time. The testing itself consists of playing a game, or a level/build over and over again, Testing all aspects, jumping off every wall in the game, pressing all the buttons to see what happens. The work is very tedious and repetitive, but the testers have to keep testing the game, and usually by the time they're done, the fun of the game has worn off. |
Lead Testers -
The lead testers are involved in the planning, monitoring, and control of the testing activities and tasks that take place. At the start of a project, test leaders, along with other stakeholder, create the test objectives, organisational test policies, test strategies, and plans. They also estimate when the testing should be done and negotiate with management to get all the necessary resources they need. They recognise when test automation is appropriate, and if it is, they plan the effort, select the tools, and ensure the training of the team. (Test automation is the use of special software separate from the software being tested, to control the execution of tests and the comparison of actual outcomes with predicted ones.) Lead testers will also consult with other groups such as programmers, to help them out with their testing. They lead, monitor and guide the analysis, design, implementation and execution of the test cases, test procedures and test suites. They make sure that proper configuration management of the test-ware produced and traceability of the tests to the tests basis takes place. As the execution of tests comes closer, they make sure the test environment is put into place before the test execution and manage it throughout. They schedule the tests that are going to be tested, and then they monitor, measure, control and report on the test progress, the product quality status and the test results, adapting the test plan and compensating when needed to adjust to the changing conditions. As the test execution comes to an end, they write up summary reports on the test status. Overall, the lead tester basically controls everything that happens involving testing and keeps it organised. References: https://en.wikipedia.org/wiki/Test_automation |
Game defects and triggers (E-mag)
![Picture](/uploads/6/0/8/2/60827849/3159074.jpg?263)
Defect Typing
Defect types are what help classify bugs within game testing. Whilst testing a game, you will come across many errors if it hasn't been properly programmed, and this is where the game testers play a huge role in the development of a game. When the testers are testing for the bugs, they can find bugs ranging from small cosmetic bugs, to massive game and possibly console/PC crashers that completely prevent you from continuing the game past that certain point.
It is useful to classify defects into categories (Defect types) that reveal how the defect was introduced and how it can be found and even hot it can be avoided in the future. The Orthogonal Defect Classification (ODC) system, developed by IBM, was developed for this exact purpose. This system defines multiple categories of classification, depending on the development activity that is taking place. Each defect can be either the result of incorrect implementation or of code that is simply missing.
Defect types are what help classify bugs within game testing. Whilst testing a game, you will come across many errors if it hasn't been properly programmed, and this is where the game testers play a huge role in the development of a game. When the testers are testing for the bugs, they can find bugs ranging from small cosmetic bugs, to massive game and possibly console/PC crashers that completely prevent you from continuing the game past that certain point.
It is useful to classify defects into categories (Defect types) that reveal how the defect was introduced and how it can be found and even hot it can be avoided in the future. The Orthogonal Defect Classification (ODC) system, developed by IBM, was developed for this exact purpose. This system defines multiple categories of classification, depending on the development activity that is taking place. Each defect can be either the result of incorrect implementation or of code that is simply missing.
The first type of defect type is a FUNCTION defect, and this affects the games capability or how the user experiences the game. Function is usually down to human error, where a line of code or more is missing or wrong, causing the game to malfunction in some way. For example, if the player needs to open a certain door to progress further into a game, and the door doesn't open, it means you cannot proceed into the game, and this can be described as a fatal bug. To be able to continue the game, these bugs need to be removed, and this can be done by the game developers releasing a patch to solve it. These function bugs can be found at any stage of the game.
|
The second type of defect type is an ASSIGNMENT defect. This is when values are missing or are incorrectly put by the program meaning the wrong objects will load or objects in general will just load incorrectly. This can cause crashes in the game as a result from it not working properly. This type of bug can be found anywhere in a game, from pre-game (starting menu), in-game (any time you are physically playing the game), to post game (credits). For example, it happens with music in games, sometimes it is coded so it starts at the wrong time, or a completely different piece of music could play. Or if you had a gun that had 10 bullets in, and you used them all up, this would be due to the values of the ammo being incorrect, allowing there to be more ammo than what you have displayed in your HUD.
|
The third type of defect type is a CHECKING defect. This is when loading fails or something isn’t validated (checked) properly before it is used. This error is similar to the assignment bug, as an example of this, is if a character tries to pick up one item, but picks up two instead of one, the game here might’ve only taken into account one of the items you’ve picked up, and therefore only added one into your inventory. This doesn’t usually mean your item is missing that has not appeared; usually you will still be able to use it, despite it not showing up. This bug isn’t a game-breaking one, but will be extremely confusing for the player. These kinds of bug are most commonly found in-game when picking up items.
The fourth type of defect type is a TIMING defect. This is to do with time management and loading of real-time resources. For example, when a sound, animation, or action doesn’t sync correctly to its surroundings, or the player pressing a button. A more detailed example would to be, if you were about to punch another character in the game and the sound of the punch wasn’t correctly synced with the hit, that would be a timing defect, or if the character you were about to punch died or was knocked out before the fist had actually touched them, that would also be a timing defect. These bugs are usually described as cosmetic bugs, as they don’t affect the progression of the game, but they can affect the immersion of the game for the player. These bugs are found most commonly in-game, as it is the result of interaction of some sort from the player.
|
The fifth defect type is the BUILD/PACKAGE/MERGE defect. This is when a build provided such as a new patch for a game, has new mistakes within its code so when it’s attached to the working game, it can potentially break the whole game. For example, in Destiny, Bungee tried to patch several things, several times, like when they tried to patch the heavy ammo fix several times, and each time, it was either not patched, or it was, but broke other parts of the game like some of the guns including one of the highest end guns. There are many bugs like this throughout many games, and they can be found at the pre-game stage, the in-game stage or even the post-game stage, as these patches could include bits of code that could completely prevent the game from loading the title screen, corrupting saved files, freezing mid game at certain sequences, or preventing the credits to roll or for you to actually finish the game.
|
![Picture](/uploads/6/0/8/2/60827849/179567.jpg?441)
The sixth defect type is the ALGORITHM defect. This occurs when calculation problems or decision processing of items and NPCs goes wrong. This error occurs at collision detection. A lot of times within games, there are floors and walls which are positioned incorrectly; this can cause the player to fall out of the map (like in the image of Assassin's Creed Syndicate to the right), which therefore means the collision detection doesn't work properly. Collision detection is very important within a game because it keeps the player where they are in the game. Also, when two or more objects are in the same spot and are colliding with each other at the Z coordinate ( The Z coordinate being the depth coordinate that only exists in 3D) This type of error is called Z-Fighting, it can be shown in-game when two NPCs are standing directly on top of each other and you can see them flicker between one another, or another example of bad Collision detection would be if you were stuck in a room trying to solve a puzzle to get past a door but you can walk straight through the door anyway due to it not being covered in a collide box. Along with Collision detection, another algorithm bug is the LOD (Level of detail) Popping. This is when for example, during a video game, environments and graphics from far away decrease in quality/amount of polygons, but as you get closer, the increase and become more detailed with a higher poly-count. This occurs to avoid putting so much strain on the GPU (Graphics processing unit). LOD Popping itself is when you make the transition happen faster than it would usually do. So an example would be if you were going fast down an open world game and the roads went from low poly to snapping into a straight into detailed poly. This bug is common in any open world in-game environment due to the LOD transition not coded correctly or not wrote in. These bugs are found in the in-game operating region, as it is to do with the actions that take place in the environment of the game.
![Picture](/uploads/6/0/8/2/60827849/6822059.jpg?433)
The seventh defect type is the Documentation defect. This is when information is different to what is in the game. This bug affects text, audio files and graphical assets. This includes any incorrect spelling, grammar, and punctuation, also, incorrect translation of different languages. Text-file bugs also make sure there aren’t any inappropriate words or images for those types of game, to keep it at the age rating that it is aimed for. An example of a text-file bug would be if you talking to a NPC through a text dialogue box and the text just cut off mid-sentence, this would be bad for the player as you don’t know what they had to say, and it could also be very time consuming if they were telling you where to go to continue the game. Text-file bugs are common in all regions again, start up (menu/settings text), in-game (dialogue text) and post-game (credits/quite & save text screen). Audio files can also be a problem with bug documentation because it can cause the audio to not play at all, or the wrong audio to play, or be quiet and muffled. It can also cause the audio to skip or cut and the beginning or end, which makes the game seem very unprofessional. An example of this is if you are fighting in a boss battle and the music kept skipping or cutting out. These bugs are again common in any operating region, as music can be played in the menu screen (start-up), in-game or in the credits (post-game). Lastly within documentation, are graphical assets that can have bugs, such as the rendering of graphics not working correctly or skipping of cut-scenes. An example of graphical assets with bugs could be the intro for the game telling you the introducing story, but it being very glitchy and skipping a lot making it very hard to understand what is going on. This type of bug is common in an in-game cut-scene/open-world environment.
The eighth and final defect type is the interface defect. This is when interaction with the environment and menu systems has problems. For example, interface misuse, like when calling a component, the program calls another component and makes an error in its use of its interface e.g. parameters in the wrong order. Interface misunderstanding, when a calling component embeds assumptions about the behaviour of the called component which are incorrect. For example, if you picked up a type of gun in a game that you previously had 10 bullets in, and the computer assumes this same gun you had before, so it assumes it should have 10 bullets in it, so puts them in, when in reality, it's a gun somebody else has dropped, and it may have only had 3 bullets in. Also, timing errors, the called and the calling component operate at different speeds and out-of-date information is accessed. If the database it's calling from has out of date information, it will call old information which may not be correct any more.
Reference: http://gamesdevelopmentvasilis.blogspot.co.uk/2015/03/game-defect-types-and-triggers.html
www.csee.umbc.edu/courses/undergraduate/.../IntegrationTest.ppt
Reference: http://gamesdevelopmentvasilis.blogspot.co.uk/2015/03/game-defect-types-and-triggers.html
www.csee.umbc.edu/courses/undergraduate/.../IntegrationTest.ppt
Defect Triggers
These triggers describe ways to cause distinct categories of game defects to show up during testing. Together, these triggers account for all of the possible defects that can occur whilst testing.
The Configuration Trigger - This is a trigger that occurs when you're configuring the games settings and the game hasn't been tested for the settings yet, causing errors. This trigger can affect both the graphics and the sound as you can change a graphical setting which results in textures going missing or changing sound settings resulting in audio skipping or being completely missed out. This trigger is rarely found in big AAA games as they have professional testers and most likely lots and lots of them, it is more likely to be found in less popular indie games, that are found and downloaded from the web with less experienced developers.
The Start-up Trigger - This trigger occurs at the game start region when the game is loading. At this point, all of the contents of the game will be loading, e.g environments, saves, models and polygons to play the game, and this can either delay the loading stage of the game, or just freeze it completely, as this is when most of the stress is put on the game. If the game cannot load at this stage, it means that your PC doesn't meet the requirements of the game and there is too much pressure on the hardware and the CPU. This means that you will have to upgrade your hardware if you want to play the game or make it run faster. An example of a start up trigger could be if you were playing Skyrim and didn’t check if your CPU could handle it by seeing the developers’ recommended hardware, This would result in the game either dropping in frames a lot or a crashing out of the game while it's loading.
The Exception Trigger - This type of trigger is most common in games that have been modded. So for developers only of the base game, this trigger doesn't usually occur. An exception trigger occurs when the code of the game, cannot be read by the game. After this happens the game activates the trigger causing the game to crash. An example of an exception trigger could be if you were to mod Grand Theft Auto and the mod replaced or added files that couldn't be read by the game, this would result to the game freezing and eventually crashing out to the desktop. This is most likely to happen in the start up stage when everything is loading up.
The Stress Trigger - This trigger is quite self-explanatory. It occurs in-game when the game or server has a lot of stress put on it due to possibly your PC hardware not being good enough, or there being too many things going on at once on the screen, for example too many items, players or entities spawning on the server. This trigger also responds in a CTD manner if on single player, or CTM (Crash to menu) if on a server due to the game or server freezing because of the amount of stress put on it. An example involving a game of this would be say you are playing a game like Black Ops 2, and 100s of people are trying to join your server at once, it would not only put stress on your pc, but the whole server in general, this would then cause the server to crash, resulting you to be put back into your own lobby.
The Normal Trigger - This trigger is most commonly found in the in-game stage whilst the game is loading. This refers to using the game apart from any stress, configuration, or exception conditions, kind of like the way you would script a demo or show how the game should be played in the user manual. The "normal" code is distinct from the code that handles the exceptions, the code that processes configuration changes and the code that takes over under stressful conditions. The majority of the testing that is done uses normal triggers, as using normal trigger demonstrates the way the game is supposed to work. An example of this is, if when you press "a" in a game, the character is supposed to jump, the normal trigger would be triggered if when you pressed "a" , the character did jump. So it is basically just triggered when the game does what it was supposed to.
The Restart Trigger - This trigger classifies a failure that occurs as a result of quitting, ending the game, turning off the game device, ejecting the game disk, or terminating the games operation in any way. Most games are good and prompt you to save when you get past vital parts of a game before you quit the game, mission or battle, as when ending a game some information needs to be remembered or in some cases forgotten. If the saving sequence is done incorrectly or your console or PC shuts down out of the blue, the player can either get back to were they were before they made a mistake they wanted to alter in the game, or like in most cases, lose lots of precious progress they've put hours into. This happens post-game once your console has shut down or turned off without saving.
These triggers describe ways to cause distinct categories of game defects to show up during testing. Together, these triggers account for all of the possible defects that can occur whilst testing.
The Configuration Trigger - This is a trigger that occurs when you're configuring the games settings and the game hasn't been tested for the settings yet, causing errors. This trigger can affect both the graphics and the sound as you can change a graphical setting which results in textures going missing or changing sound settings resulting in audio skipping or being completely missed out. This trigger is rarely found in big AAA games as they have professional testers and most likely lots and lots of them, it is more likely to be found in less popular indie games, that are found and downloaded from the web with less experienced developers.
The Start-up Trigger - This trigger occurs at the game start region when the game is loading. At this point, all of the contents of the game will be loading, e.g environments, saves, models and polygons to play the game, and this can either delay the loading stage of the game, or just freeze it completely, as this is when most of the stress is put on the game. If the game cannot load at this stage, it means that your PC doesn't meet the requirements of the game and there is too much pressure on the hardware and the CPU. This means that you will have to upgrade your hardware if you want to play the game or make it run faster. An example of a start up trigger could be if you were playing Skyrim and didn’t check if your CPU could handle it by seeing the developers’ recommended hardware, This would result in the game either dropping in frames a lot or a crashing out of the game while it's loading.
The Exception Trigger - This type of trigger is most common in games that have been modded. So for developers only of the base game, this trigger doesn't usually occur. An exception trigger occurs when the code of the game, cannot be read by the game. After this happens the game activates the trigger causing the game to crash. An example of an exception trigger could be if you were to mod Grand Theft Auto and the mod replaced or added files that couldn't be read by the game, this would result to the game freezing and eventually crashing out to the desktop. This is most likely to happen in the start up stage when everything is loading up.
The Stress Trigger - This trigger is quite self-explanatory. It occurs in-game when the game or server has a lot of stress put on it due to possibly your PC hardware not being good enough, or there being too many things going on at once on the screen, for example too many items, players or entities spawning on the server. This trigger also responds in a CTD manner if on single player, or CTM (Crash to menu) if on a server due to the game or server freezing because of the amount of stress put on it. An example involving a game of this would be say you are playing a game like Black Ops 2, and 100s of people are trying to join your server at once, it would not only put stress on your pc, but the whole server in general, this would then cause the server to crash, resulting you to be put back into your own lobby.
The Normal Trigger - This trigger is most commonly found in the in-game stage whilst the game is loading. This refers to using the game apart from any stress, configuration, or exception conditions, kind of like the way you would script a demo or show how the game should be played in the user manual. The "normal" code is distinct from the code that handles the exceptions, the code that processes configuration changes and the code that takes over under stressful conditions. The majority of the testing that is done uses normal triggers, as using normal trigger demonstrates the way the game is supposed to work. An example of this is, if when you press "a" in a game, the character is supposed to jump, the normal trigger would be triggered if when you pressed "a" , the character did jump. So it is basically just triggered when the game does what it was supposed to.
The Restart Trigger - This trigger classifies a failure that occurs as a result of quitting, ending the game, turning off the game device, ejecting the game disk, or terminating the games operation in any way. Most games are good and prompt you to save when you get past vital parts of a game before you quit the game, mission or battle, as when ending a game some information needs to be remembered or in some cases forgotten. If the saving sequence is done incorrectly or your console or PC shuts down out of the blue, the player can either get back to were they were before they made a mistake they wanted to alter in the game, or like in most cases, lose lots of precious progress they've put hours into. This happens post-game once your console has shut down or turned off without saving.
Operating Regions
There are four operating regions that a video game can be during the resting process. These are the Pre-game region, the Start-up Region, the in game region and the Post-game region.
The Pre-game region is the stage before you start a game. During the use of a console the pre-game stage is the time before inserting the game disk where the console is operational but the game is not. At the PC this region is when the system or the device is already powered on before you start the game or app. During this stage players can change the settings of a game such as the quality of graphics, the sounds and the controllers.
The Start Up region is when the player activates some of the game operations until the game is ready to be playe. For example when you are at the menu screen stage of a game, but not yet playing it.
In- game region is the stage that the player actually plays the game. So this is whilst you are playing any part of the game, doing missions, side missions and anything involving cut-scenes.
Post-game region is the last stage of video games, when the player finishes the game and wants to exit from it. There are different options for that depending on the type of game you are playing. The most common options are exit without saving and save and exit. Also, the majority of games include the auto-save option which can be switched on or off very easily at the main menu. The advantage of that is that the progress is saved automatically and as a result it can’t be lost.
There are four operating regions that a video game can be during the resting process. These are the Pre-game region, the Start-up Region, the in game region and the Post-game region.
The Pre-game region is the stage before you start a game. During the use of a console the pre-game stage is the time before inserting the game disk where the console is operational but the game is not. At the PC this region is when the system or the device is already powered on before you start the game or app. During this stage players can change the settings of a game such as the quality of graphics, the sounds and the controllers.
The Start Up region is when the player activates some of the game operations until the game is ready to be playe. For example when you are at the menu screen stage of a game, but not yet playing it.
In- game region is the stage that the player actually plays the game. So this is whilst you are playing any part of the game, doing missions, side missions and anything involving cut-scenes.
Post-game region is the last stage of video games, when the player finishes the game and wants to exit from it. There are different options for that depending on the type of game you are playing. The most common options are exit without saving and save and exit. Also, the majority of games include the auto-save option which can be switched on or off very easily at the main menu. The advantage of that is that the progress is saved automatically and as a result it can’t be lost.
![Picture](/uploads/6/0/8/2/60827849/1870645.jpg?270)
Phase Stages
Pre- Production - Testing begins when the project begins. Not very many people called "Testers"are used at the start to test the game, but as objects are being produced from the start, they will be tested by the designers as they go along the process of creating them, they won't do as much testing as the testers obviously, but the can carry out small tests. Many of the small tests that go on at the start of the project, will set the tone for how well the testing will go on later in the development, because if nothing at all is tested at the start, it is more likely there will be a lot more problems than there needed to be. This stage also involves how good the software stands up to the testing, and how well the tests themselves are organised and executed.
Kickoff - Kick-offs are known to have a positive impact on the software development which leads to, better problem solving, better definitions and a cycle time reduction. In a team in which testing has various levels of testing and game project experience, individual needs are not likely to be addressed at this stage. The different levels of experience can be categorised as Black Box, Grey Box and White Box testers. Black Box testers no absolutely nothing about the internals of the game and are not required to,this is the least time consuming and exhaustive, it is performed by end users, testers and developers, and they do not need to test the algorithms as they are not suited to it. This type of testing can only be done in a trial and error way. Grey Box testers have some sort of knowledge on the internal workings of the game, it is performed by end users, testers and developers, it is somewhat more exhausting as they know more about the internal side and can go into the code themselves and change some small things. These types of tester are still not suited to algorithm testing though. Finally, White Box testers have full knowledge of the internal workings of the application, they test the structural side or the code based side of the game. They know the full ins and outs so can design test data accordingly, it is the most time consuming, as they are suited for algorithm testing, so can go more depth into the coding to sort out problems, where as Black Box and Grey Box testers have to just relay the problems they have found back to the developers so they can alter them.
Alpha - At the Alpha stage full testing can be done. Throughout Alpha testing the game design is fine-tunes. Features are able to be play tested and revised. Systems developed by different programmers are now linked together. The code and art will be checked and then implemented into the game, and then the new code/ build will be checked for new defects. This stage is a critical stage as it brings a form of order to what before seemed like a big mess of levels and objects all over the places beforehand. You know when you game has come to its Alpha stage as all the major game features exist and can be physically tested. The game can be navigated from start to finish by one way or another. The code will pass at least 50% of platform TRC (Technical Requirement Checklist). Basic interface is complete and preliminary documentation is available to QA, and the game is compatible with most specified hardware and software configurations. Other things you should be able to test is that the online multiplayer works, the level script is implemented, first party controllers and memory cards work, final or placeholder art is in for all areas of the game (meaning there is at least some sort of basic or final texture on every object), and placeholder audio is implemented (so if a bomb going off is supposed to make a noise, make sure some sort of noise is there to fill its place or that the actual noise will be there).
Beta - At this stage, the development team has almost stopped creating new code and new artwork and they are now focusing on perfecting what they have already created and identifying and fixing remaining bugs. The term "beta testing" frequently refers to any outside testing, it's only at this stage that final gameplay testing should take place with people outside of the design team. You know your game has hit the Beta stage when all of the major events in the game are there, and can be tested, when the code passes 100% of platform TRC, the game can be navigated on all paths, so for example, if there are many ways to get to the end of the game with different scenarios, they should all be playable. The entire user interface is final, so all the menu systems are how they should look and work for the final game. The game is compatible with most specified hardware and software configurations, So if the game is supposed to be played on a computer on Windows 7 for example, it needs to be able to work on them by now for definite. Also, the game logic and AI needs to be final, which means all the other characters and objects in the game need to work properly, so this includes the dialog and what they look like in general. Basically at this stage, everything needs to be pretty much complete, all controllers should work, final artwork and audio is implemented, all online modes are complete and testable and finally all language version text is implemented and ready for simultaneous release, so this just means each language the game is released in, all the translations should be put in by now so the game is ready to be released at the same time as the original language of the game.
Gold - Once the Beta stage has finished, your game will now be at the Gold stage. This means all of the known critical class 1 bugs should have been fixed (these bugs are the ones that crash the consoles and PCs, and just in general cause something to fail badly, resulting in major loss). Greater than 90% of all known class 2 bugs are fixed (bugs that aren't as serious class 1 bugs, the work around is found, but the performance might be quite slow) Greater than 85% of all known class 3 bugs are fixed (These bugs include database errors, link errors, low response time). Any other known open issues have a workaround that has been communicated to tech support, and the game as a whole is up to a release-level standard. Upon meeting all these things, the game is declared to be at "code lock" (Code lock is where all the code within the game is finished, and playable for the public and there should be no problems).
Release - At this point, the game is released in it's finished form, everything should be complete by this stage, otherwise the company starts to lose more money, and people who have pre-ordered your game and are waiting for it get disappointed and angry. Everything should work, and the developers and everyone involved in making the game, now just have to sit back and wait to see how well it goes down with the public, and if any problems arise or not.
Post-Release - Post release is where if any bugs are reported by the public, the developers have to take second looks at the code and try and patch the problems that have arose, and there will be bugs. Once lots of people have bought the new game and are playing it, it's like testing it on a monumental scale, things like the servers will be put under a lot of stress just because of the sheer amount of people playing the game, especially if it's a AAA game.
Pre- Production - Testing begins when the project begins. Not very many people called "Testers"are used at the start to test the game, but as objects are being produced from the start, they will be tested by the designers as they go along the process of creating them, they won't do as much testing as the testers obviously, but the can carry out small tests. Many of the small tests that go on at the start of the project, will set the tone for how well the testing will go on later in the development, because if nothing at all is tested at the start, it is more likely there will be a lot more problems than there needed to be. This stage also involves how good the software stands up to the testing, and how well the tests themselves are organised and executed.
Kickoff - Kick-offs are known to have a positive impact on the software development which leads to, better problem solving, better definitions and a cycle time reduction. In a team in which testing has various levels of testing and game project experience, individual needs are not likely to be addressed at this stage. The different levels of experience can be categorised as Black Box, Grey Box and White Box testers. Black Box testers no absolutely nothing about the internals of the game and are not required to,this is the least time consuming and exhaustive, it is performed by end users, testers and developers, and they do not need to test the algorithms as they are not suited to it. This type of testing can only be done in a trial and error way. Grey Box testers have some sort of knowledge on the internal workings of the game, it is performed by end users, testers and developers, it is somewhat more exhausting as they know more about the internal side and can go into the code themselves and change some small things. These types of tester are still not suited to algorithm testing though. Finally, White Box testers have full knowledge of the internal workings of the application, they test the structural side or the code based side of the game. They know the full ins and outs so can design test data accordingly, it is the most time consuming, as they are suited for algorithm testing, so can go more depth into the coding to sort out problems, where as Black Box and Grey Box testers have to just relay the problems they have found back to the developers so they can alter them.
Alpha - At the Alpha stage full testing can be done. Throughout Alpha testing the game design is fine-tunes. Features are able to be play tested and revised. Systems developed by different programmers are now linked together. The code and art will be checked and then implemented into the game, and then the new code/ build will be checked for new defects. This stage is a critical stage as it brings a form of order to what before seemed like a big mess of levels and objects all over the places beforehand. You know when you game has come to its Alpha stage as all the major game features exist and can be physically tested. The game can be navigated from start to finish by one way or another. The code will pass at least 50% of platform TRC (Technical Requirement Checklist). Basic interface is complete and preliminary documentation is available to QA, and the game is compatible with most specified hardware and software configurations. Other things you should be able to test is that the online multiplayer works, the level script is implemented, first party controllers and memory cards work, final or placeholder art is in for all areas of the game (meaning there is at least some sort of basic or final texture on every object), and placeholder audio is implemented (so if a bomb going off is supposed to make a noise, make sure some sort of noise is there to fill its place or that the actual noise will be there).
Beta - At this stage, the development team has almost stopped creating new code and new artwork and they are now focusing on perfecting what they have already created and identifying and fixing remaining bugs. The term "beta testing" frequently refers to any outside testing, it's only at this stage that final gameplay testing should take place with people outside of the design team. You know your game has hit the Beta stage when all of the major events in the game are there, and can be tested, when the code passes 100% of platform TRC, the game can be navigated on all paths, so for example, if there are many ways to get to the end of the game with different scenarios, they should all be playable. The entire user interface is final, so all the menu systems are how they should look and work for the final game. The game is compatible with most specified hardware and software configurations, So if the game is supposed to be played on a computer on Windows 7 for example, it needs to be able to work on them by now for definite. Also, the game logic and AI needs to be final, which means all the other characters and objects in the game need to work properly, so this includes the dialog and what they look like in general. Basically at this stage, everything needs to be pretty much complete, all controllers should work, final artwork and audio is implemented, all online modes are complete and testable and finally all language version text is implemented and ready for simultaneous release, so this just means each language the game is released in, all the translations should be put in by now so the game is ready to be released at the same time as the original language of the game.
Gold - Once the Beta stage has finished, your game will now be at the Gold stage. This means all of the known critical class 1 bugs should have been fixed (these bugs are the ones that crash the consoles and PCs, and just in general cause something to fail badly, resulting in major loss). Greater than 90% of all known class 2 bugs are fixed (bugs that aren't as serious class 1 bugs, the work around is found, but the performance might be quite slow) Greater than 85% of all known class 3 bugs are fixed (These bugs include database errors, link errors, low response time). Any other known open issues have a workaround that has been communicated to tech support, and the game as a whole is up to a release-level standard. Upon meeting all these things, the game is declared to be at "code lock" (Code lock is where all the code within the game is finished, and playable for the public and there should be no problems).
Release - At this point, the game is released in it's finished form, everything should be complete by this stage, otherwise the company starts to lose more money, and people who have pre-ordered your game and are waiting for it get disappointed and angry. Everything should work, and the developers and everyone involved in making the game, now just have to sit back and wait to see how well it goes down with the public, and if any problems arise or not.
Post-Release - Post release is where if any bugs are reported by the public, the developers have to take second looks at the code and try and patch the problems that have arose, and there will be bugs. Once lots of people have bought the new game and are playing it, it's like testing it on a monumental scale, things like the servers will be put under a lot of stress just because of the sheer amount of people playing the game, especially if it's a AAA game.
![Picture](/uploads/6/0/8/2/60827849/1723746_orig.jpg)
Life Cycle of a Test Build
Testing is a vital part of software development, and it is important to start it as early as possible, and to make testing a part of the process of deciding requirements.
To get the most useful perspective on your development project, it is worthwhile devoting some thought to the entire life cycle including how feedback from users will influence the future of the application, whether that means completely changing you game or main character, or just one line of code. The 3 different types of tester I mentioned earlier will take part in the cycle (Black, Grey and White) and the steps each type of tester follows will be identical.
Testing is part of a life cycle. The software development life cycle is one you need to make your game to the best of standards you can get it, you will spend ages writing code for the stakeholders and public to turn around and say they don't like it, so you will write more code or change the code to try and make it better, and then you will check to see if it pleases the stakeholders, users, publishers and other people who have an interest in what the software does. If they do, it's great and you can carry on to the next bit, or add in small additions or changes if they don't, so you update or augment your code to their liking, and then the cycle continues.
A basic game testing process consists of:
Plan and Design the test
Most of this is done earlier on during each of the planning phases, planning and designing should be revisited with every build.
What additional test cases have been added?
What new configuration will the game support?
What features will be cut out or added?
These questions should be asked to make sure you are going to be able to cover as much as possible, whilst not going overboard and thinking of every possible test ever, because this will take up valuable time, therefore wasting money. This scope of testing should ensure that no new issues arise in the process of fixing other bugs prior to this release.
Prepare for testing
The code, tests, documents and the test environment are updated by their respective coders and aligned with the work of other coders. So all the code is put together so it can be tested as one. By this time, the development team should have marked the bugs fixed for the new build in the defect database (A database the testers can access and test the specific bug fixes as they go through it and verify that it has actually been fixed and then close off the bug report).
Perform the test
Run the test suites against a new build. If you find a defect, test "around" the bug to ensure that you have all the details necessary to write the most specific and concise bug report as possible. Like in other industries, the more you research you do at this stage, the easier and more useful the bug report will be. You wouldn't want to write a useless bug report with stand alone statements that don't make any sense like "crashes to desktop", you'd write step to step bullet points of how you made it crash to desktop without any unnecessary mumbo-jumbo.
Report the results
Log the completed test suite and then report any defects you find. This is so the developers can see your log and then work through it to find the bugs so they can fix them.
Repair the bug
The test team participants in this step help by being available to discuss the bug with the development team, and they provide any directed testing that may be required to track it down so it can be fixed.
Return to step 1 and re-test
Now the bugs have been fixed you now have to return to step one and plan and design the tests for the new code that has been produced to fix the old one, and you keep repeating the cycle until the code works properly and there is nothing left to test.
Testing is a vital part of software development, and it is important to start it as early as possible, and to make testing a part of the process of deciding requirements.
To get the most useful perspective on your development project, it is worthwhile devoting some thought to the entire life cycle including how feedback from users will influence the future of the application, whether that means completely changing you game or main character, or just one line of code. The 3 different types of tester I mentioned earlier will take part in the cycle (Black, Grey and White) and the steps each type of tester follows will be identical.
Testing is part of a life cycle. The software development life cycle is one you need to make your game to the best of standards you can get it, you will spend ages writing code for the stakeholders and public to turn around and say they don't like it, so you will write more code or change the code to try and make it better, and then you will check to see if it pleases the stakeholders, users, publishers and other people who have an interest in what the software does. If they do, it's great and you can carry on to the next bit, or add in small additions or changes if they don't, so you update or augment your code to their liking, and then the cycle continues.
A basic game testing process consists of:
- Plan and Design the test.
- Prepare for testing.
- Perform the test.
- Report the results.
- Repair the bug.
- Return to Step 1 and re-test.
Plan and Design the test
Most of this is done earlier on during each of the planning phases, planning and designing should be revisited with every build.
What additional test cases have been added?
What new configuration will the game support?
What features will be cut out or added?
These questions should be asked to make sure you are going to be able to cover as much as possible, whilst not going overboard and thinking of every possible test ever, because this will take up valuable time, therefore wasting money. This scope of testing should ensure that no new issues arise in the process of fixing other bugs prior to this release.
Prepare for testing
The code, tests, documents and the test environment are updated by their respective coders and aligned with the work of other coders. So all the code is put together so it can be tested as one. By this time, the development team should have marked the bugs fixed for the new build in the defect database (A database the testers can access and test the specific bug fixes as they go through it and verify that it has actually been fixed and then close off the bug report).
Perform the test
Run the test suites against a new build. If you find a defect, test "around" the bug to ensure that you have all the details necessary to write the most specific and concise bug report as possible. Like in other industries, the more you research you do at this stage, the easier and more useful the bug report will be. You wouldn't want to write a useless bug report with stand alone statements that don't make any sense like "crashes to desktop", you'd write step to step bullet points of how you made it crash to desktop without any unnecessary mumbo-jumbo.
Report the results
Log the completed test suite and then report any defects you find. This is so the developers can see your log and then work through it to find the bugs so they can fix them.
Repair the bug
The test team participants in this step help by being available to discuss the bug with the development team, and they provide any directed testing that may be required to track it down so it can be fixed.
Return to step 1 and re-test
Now the bugs have been fixed you now have to return to step one and plan and design the tests for the new code that has been produced to fix the old one, and you keep repeating the cycle until the code works properly and there is nothing left to test.