Unreal Automated Testing

As a Unreal video game developer, I can guarantee one of the most frustrating things in developing a game is the build you have cooked ended up in the wrong configuration either by you or one of the team members forgot to uncheck a Boolean or inserted a wrong value. It could also be bugs that regressed or just simply new bugs all together. It is definitely not fun to work in such a mess.

That's where Unreal Automated Testing comes in, it can help us perform sanity checks, routine tests to ensure the system is operating in the intended manner that we can trigger with a click of a button and give all of us peace of mind.

---

Unreal Version 4.25

---

Step 1 : Enabling Automation Plugins

Boot up your Unreal and navigate to Edit -> Plugins -> Testing. You can enable all of them but for me the ones below should do the trick.

• Editor Tests

• FbxAutomationTestBuilder

• Functional Testing Editor

• Runtime Tests

• ScreenshotTools

Unreal will prompt you to restart so choose Restart Now.

Step 2: Make a Custom Plugin for Automated Testing

Since we do not want the test files to bloat our shipping builds, we need to make a plugin or a module for it (plugins are way easier to make, you will need to manually setup the configurations for module). Navigate to Plugins, this time we need to click the New Plugin button.

Select Blank, give it sensible name then click Create Plugin.

After compilation, you would need to add your project into the plugin's PublicDependencyModuleNames.

Then, edit %INSERT_PLUGIN_NAME%.uplugin, under Modules, change the "Type" to "DeveloperTool", like so

"Modules": [
   {
      "Name": "INSERT_PLUGIN_NAME",
      "Type": "DeveloperTool",
      "LoadingPhase": "Default"
   }
]

Both of these steps will ensure that your plugin has access to main project files and they will not bloat the final shipping build.

Step 4: Writing A Simple Test aka Unit Test

Finally, it is time to write a test. I will guide you on writing a simple Unit Test that check a value in a Blueprint. In this example, I will be checking the DefaultPlayerName in a GameMode Blueprint.

First off, creating a test script does NOT use the usual pathway where we right-click on the Content Browser and create a script.

We need to navigate to the /Private folder in the plugin and create a .cpp script there. I'll name it GameModeSanityCheck.cpp. After that you will need to include

#include "Tests/AutomationCommon.h" 

to gain access to the commonly used Unreal Automation symbols. Next up, we would need to use the macro

IMPLEMENT_SIMPLE_AUTOMATION_TEST(FTestClass, "FTestName",
   EAutomationTestFlags...)

• FTestClass : The name of the class that contains the test.

• FTestName: The name to address this Test Class (remember to prefix the name with an identifier so easier automation process later down the road).

• EAutomationTestFlags: The flags that specifies the requirement or behaviour of a test. Learn more at https://docs.unrealengine.com/en-US/API/Runtime/Core/Misc/EAutomationTestFlags__Type/index.html

After that we need to override

bool RunTest(const FString& Parameter)

to actually perform the logic for the test. Since we are checking DefaultPlayerName in a GameMode Blueprint this is the logic.

Note that both TestNull and TestTrue won't end the finalize the test, they are just some special comparison functions created to push the log back to Session Frontend console (more on this later). The returns are the finalizers of the test. We are done writing this script. Now we need to compile it.

Since this is a plugin, we cannot compile it using the normal way and we have to compile through the Modules panel that can be found in Windows -> Developer Tools -> Modules.

Just do a simple search of the plugin name, and click Recompile will compile the scripts. Now we can launch Session Frontend that can be found in Windows -> Developer Tools -> Session Frontend to see our newly added test.

However, our newly added test is nowhere to be found. Where is it? This is because test does not support hot-reload out of the box. This has been a long debating issue in Unreal Forums. However, Unreal is just lukewarm on fixing this issue. So this bring us to a side quest of hacking hot-reload into it.

Side Quest: Hacking Hot-Reload into Automated Tests

Navigate to %INSERT_PROJECT_NAME%.cpp and add the following code in.

The piece of code above essentially unregisters of the tests with the identifier mentioned in Step 4's FTestName. So that, Unreal will register the newly updated tests when the module starts. After that, we just compile and see if the test load up in Session Frontend. If it still isn't, click the Refresh Tests button, then it should be up.

Note even though we have hacked hot-reload into the tests but this is not 100% foolproof. Always keep eye on the Output Log to see if the tests are refreshed. Unreal might act funny sometimes and not refreshing the tests. If so just refresh the tests or recompile them.

Final step: Launching the Test

Finally it is time to launch our test. Just check the test you want then press the Start/Stop Tests button. If everything works, you should see something like image below.

---

Congratulations!!! You have just setup Automated Testing in Unreal. The example test here is just a very simple test, this is just the tip of the iceberg. You can go wild automated tests like how Sea of Thieves did. Good luck and happy testing.

Level up your automation by integrating Unreal Automation with Jenkins. Learn more at https://xynanxdb.wixsite.com/portfolio/post/unreal-jenkins-devops-zero-to-hero