Development Testing Tools

Testing XMage

There are two methods to test app:

manual testing from xmage application (server test mode);
automatic tests from IDE (unit tests);

Also you can use additional launcher/java commands:

client: -Dxmage.badconnection - simulate bad/slow network connection with a server (useful to test game's GUI bugs)

Server Test Mode

XMage has a test mode, that helps you with testing implemented cards or other changes. Therefore you have to start the XMage server in the test mode. The test mode will be active if you set the -testMode option to true. You can use it even with release version (no need to install any developers tools or source code).

Test mode features:

adds three fast game buttons in tables list
1. "Quick 2 player": 2 player commander game
2. "Quick 4 player": 4 player commander game with limited range of influence (range = 1)
3. "Quick start MCTS": 2 player duel with the MCTS AI.
add cheat commands button under avatar to setup battlefield and run any cheat commands;
disable deck validation;
simplify registration and login (no password check, e.g. login under any user);
add debug main menu for GUI and rendering testing.

Start server by launcher

Open launcher -> settings -> java -> server options and add additional param like -Dxmage.testMode=true

Start server by IDE

If you run app from a source files then no needs in special argument - server will run in test mode by default. But it can be useful to disable test mode, e.g. for deck validation testing. How to do that:

For IntelliJ IDEA: open run configuration -> application (see IDE setup for details) and insert in program arguments field: -testMode=false
For NetBeans: select the Mage Server project and use: Set Configuration -> Customize... -> Categories: -> Run -> Arguments: _ and insert -testMode=false

Start server by script

Create command file run-server-testmode.cmd in launcher folder and add text to it (for windows). Don't forget to fix version string to actual value:

set SERVER_DIR=.\xmage\mage-server
set JAVA_DIR=.\java\jre1.8.0_201\bin
pushd %CD%

cd /d %JAVA_DIR%
set JAVA_DIR=%CD%

popd
cd /d %SERVER_DIR%
set SERVER_DIR=%CD%

cd /d %SERVER_DIR%

%JAVA_DIR%\java.exe -jar %SERVER_DIR%\lib\mage-server-1.4.50.jar -testMode=true

How to test

Server test mode gives you on the games panel an additional button "Quick Start". With this button you can immediately start a game that uses decks that have to be stored with the name test.dck in the Mage Client project directory. If you haven't that file then xmage will load default deck for each player:

5 [XLN:268] Swamp
5 [XLN:276] Forest
5 [XLN:264] Island
5 [XLN:272] Mountain
5 [XLN:260] Plains

Also you have an additional button (smiley) inside the small player status box to the left. If you push this button during a match -- you will see special dialog to choose test commands like adds cards to the hand, library or battlefield. That commands are defined in the file init.txt that has to be located in the Mage.Server/config (\xmage\mage-server\config) directory. This works also for matches not started with the "Quick Start" button. There already exists an init.txt.example file, that explains the needed format of the file. During a test game you can change the init.txt file, push the smiley button in XMage and get the cards you need in the zone you want to test your changes in.

Init.txt contain commands to run at any time. You can combine commands in groups by [command name]. Game will ask you what group of commands you want to execute. Also you can comment some lines by //. It's useful to have one file for different tests. Look at server logs to see any typo errors in command names. It can use default names: Human and Computer. If you run it for normal game then just replace it to real name too like user 1.

Init.txt file example:

[init]
battlefield:Human:Forest:5
battlefield:Human:Plains:5
battlefield:Human:Mountain:5
battlefield:Human:Swamp:5
battlefield:Human:Island:5
hand:Human:Lightning Bolt:2
library:Human:Doom Blade:2

// special command, see SystemUtil for more special commands list
[@activate opponent ability]

[current test]
graveyard:Human:Bloodghast:1
graveyard:Computer:Bloodghast:1
hand:Human:Bloodghast:1
hand:Computer:Bloodghast:1

[diff set codes example]
battlefield:Human:XLN-Island:1
battlefield:Human:UST-Island:1
battlefield:Human:HOU-Island:1

// @ref command example (all refs will be replaced by commands from ref's group)
[another test]
@init
hand:Human:Lightning Bolt:2

[2x bear to me]
battlefield:Human:Kitesail Corsair:2

[2x bear to comp]
battlefield:Computer:Kitesail Corsair:2
//battlefield:Computer:Grizzly Bears:1

// create any useful commands
[clone]
hand:Human:Clone:3
[force attack]
hand:Human:Pit Fight:3
[exile]
hand:Human:Angelic Edict:3

To be able to use the "Quick Start" mode you also need 2 more deck files in the Mage.Client directory.

test.dck - a deck that will be loaded for both players (Human and Computer)
cheat.dck - can be empty

A good first test is also looking at newly implemented cards in the deck editor. Check if the tooltip text is correct. In many cases this alone does point out problems of the card implementation. And don't forget to delete the Cards.h2 db files in client and server before you restart XMage after changes to get the tooltip text updated to reflect your last changes. Otherwise the old text is taken for tooltip generation in deck editor or for showing text of hand cards. The text in the database for already existing cards is only updated if you delete the cards.h2 db files of the server and client (for sure something that can and should be improved in the future).

Often it's helpful to start the client two times and create a match between two human players to test something. So you have the full control of playing something or attacking and blocking actions. As said before to add cards to the different zones of the players with the init.txt file works also in this kind of test matches.

AI testing

AI making useful logs for AI debugging (computer's decision process with possible commands and game score). Real games and maven/ci builds don't use AI logs. But you can enable it by log4j.properties setup (use log4j.logger.mage.player.ai=info to enable AI logs):

IDE and maven test runner: .\Mage.Tests\src\test\resources\log4j.properties
travis-ci test runner: .travis\log4j.properties
developer server: .\Mage.Server\src\main\resources\log4j.properties
local/real server: .\xmage\mage-server\config\log4j.properties
- server must runs with additional command line -Dlog4j.configuration=file:./config/log4j.properties

Game simulations runs in different threads with termination by timeout. So you can't stop it to debug by default (debug session stops after 10-20 seconds). You must enable COMPUTER_DISABLE_TIMEOUT_IN_GAME_SIMULATIONS - it will disable timeout termination.

AI logs example with description: shot_210926_180643

Unit tests from IDE

Tests can be created in the Mage Tests project, and can be run by building that project. If you make changes to the Mage core, we recommended you run these tests to see if your changes break any logic checked by the test cases in the Mage Tests project. It's also recommended that you create new tests for any errors not yet covered by existing tests. If you've created a new test that points out an error, running this test is an easy way to check if a change you made fixes it. This test also helps make other developers aware if changes they make cause this type of error again.

In case you use Debug mode in your IDE, you may start getting Mage.Client timeouts (mostly happens in IDEA, ok in Eclipse). To manage that use -Ddebug.mage VM argument when running Mage.Client.

Unit tests from command line

You must build full project first (one time only):

Open root folder and run mvn clean install -DskipTests

You must run all tests to create missing database files (one time only):

Open root folder and run mvn install

After that you can edit code and run single or multiple tests by commands like:

mvn install test "-Dsurefire.failIfNoSpecifiedTests=false" -Dtest=EquipAbilityTest#testEquipShroud
mvn install test "-Dsurefire.failIfNoSpecifiedTests=false" -Dtest=VerifyCardDataTest#test_showCardInfo

By default xmage's maven runner uses tree view reporter and shows logs from failed tests only. If you need full logs output then use treeViewRunnerShowAllLogs:

mvn install test "-Dsurefire.failIfNoSpecifiedTests=false" "-Dxmage.build.tests.treeViewRunnerShowAllLogs=true" -Dtest=VerifyCardDataTest#test_showCardInfo

Code coverage and static analyse

It's depends on your developer tools.

Build in support

Code coverage stats by jacoco tools:

build and run tests with coverage data: mvn install -Djacoco.skip=false -Dmaven.test.failure.ignore=true
it will generates two type of files:
- exec-files can be used by IntelliJ IDEA in show coverage data dialog (see below): ./module_name/target/jacoco.exec
- html-report can be used by devs for offline view: ./Mage.Reports/target/site/jacoco-aggregate/index.html

Static code analyse by sonar cloud:

online version available at https://sonarcloud.io/project/overview?id=magefree_mage
core dev can run and upload it to the cloud by mvn -e sonar:sonar -Dsonar.projectKey=magefree_mage -Dsonar.organization=magefree -Dsonar.host.url=https://sonarcloud.io -Dsonar.token=xxx
warning, it's require 10+ Gb memory and 15+ hours to process whole analyse;

IntelliJ IDEA support

Code coverage by IDE tools - just run test or app with "Run ... with Coverage" command:
Code coverage import from jacoco report:

first run unit tests with enabled jacoco (see above);
open search panel and enter "Show Coverage Data" or <CTRL+ALT+F6>;
in dialog box choose exec-files for your module: ./module_name/target/jacoco.exec
open search panel and enter "Coverage" to see code coverage panel;
open code editor to see coverage stats at the left;
it can load coverage data from jacoco files and shows it in code editor: *