This document is about: QUANTUM 3

SWITCH TO

QUANTUM 1 QUANTUM 2 QUANTUM 3

Utility Theory (Beta)

Introduction

Bot SDK's Utility Theory is still in Beta version, which means that it will still get production ready within time, and that the API and performance shall be improved on future versions.

Overview

Utility Theory AI works by providing an architecture for decision making which is strongly based on mathematical modelling. When creating your own UT Agent, you will be constantly creating and adjusting Response Curves, which utilizes Unity's AnimationCurves for it's definition.

Every Response Curve receives an Input value, defined by your own game logic, and it's output is what we call here as the Score. After this process, the UT Agent will then know what are the possible choices that are more useful on that tick (thus, the name Utility Theory) and will start acting accordingly.

By comparison, Utility Theory differs a lot from Finite State Machines and Behaviour Trees, and is more similar to Goal Oriented Action Planning, when talking about how much the AI technique can create immersive behaviours as it does not obligates the user to create a very "rigid" relationship between everything, such as FSM's Transitions and BT's Links.

One of the greatest advantages of the UT AI is that, with Response Curves, it is possible to define a very flexible decision making space. As to exemplify:

Supposing a distance check in a HFSM's Decision: the Agent shall run away if it's distance to an Enemy is smaller than 5. This sort of creates an "invisible wall", a "binary threshold" where the decision making goes from one extreme to the other (from "Don't Run" to "Do Run");
With UT's Response Curves, the decision making can be smoothed through the body of the curve itself. It can be a linear curve where the maximum utility value comes with the value of 5. This means that the alternative of running away becomes linearly more important as the distance value goes towards 5. Or, it can be a curve which grows exponentially, making the decision of running away to be more useful even faster.

Creating a new Utility Theory AI

On the editor's top bar, click on the (+) button, then select the option Utility Theory.

You will then be prompted to save the BT file. Save it wherever you prefer. This will create a data asset used to persist the work that you have done **on the visual editor**.

Note: The name that you choose now will be the name of another data asset generated further when you compile what you have done on the visual editor.
That will be the data asset used to actually drive your Bots from the Quantum simulation, so you can already choose a suggestive name.

When you save the file, the main Bot SDK window will be populated with a single node, which is Consideration Node, for you to begin your work.

The Consideration Node

This is the main node type, which contains all of the data that is important for the UT Agent.

Within Considerations, it is possible to define what are the Response Curves that the specific Consideration analyses. Those curves will then result from 0..n Score values, which will be multiplied together and the result is what we call as "the utility of executing that consideration".

First, all of the Considerations are evaluated, which means that the inputs will be put into the curves to get the resulting Score. Then, the consideration with the greatest Score is chosen as the one to be executed at that particular frame.

So, the idea is that users will create many Consideration nodes which will be frequently evaluated and their execution is what will then drive the agent into action.

Let's analyze a Consideration's fields, one by one.

## Base Values

Base Score (FP): the result of the response curves is summed by the Base Score. Use this to give some fixed utility value for a given Consideration;
Cooldown (FP): when the Consideration is chosen and executed, the Cooldown defines for how many seconds that Consideration will be discarded from the possible choices;
Momentum Amount (Int): if the Consideration is chosen, the Momentum will increase its Rank to the value defined on this field. Use it to define which Considerations should probably keep being executed for more frames, as this will increase their absolute utility value. See the Ranking topic below for more information;
Momentum Decay (Int): if the Consideration is in Momentum (i.e it had it's ranking increased due to momentum), its Rank value will be decreased at every second by the amount defined on this field. Use it to express how fast you want the Momentum to be finished;

Those base values are not mandatory to be used, but they do provide some extra functionality can be very useful.

Actions

This is a node type that allows the user to perform changes in the game state in order to drive their Actor's. Moving an actor from point A to point B, attacking, scanning, etc, can all be done within Actions.

UT's Actions are the same that are used on the HFSM and on the GOAP. Because of that, there is a shared documentation which you can access by clicking here.

In order to edit a Consideration's actions, just double click on the Actions area. Within there, you will be able to link Actions for the following moments:

On Enter: executed when that Consideration is chosen as the most useful and started being executed;
On Update: executed every at every frame while the Consideration is still chosen as the most useful;
On Exit: executed when the Consideration was chosen on the previous frame, but it was not chosen again on the current frame;

Ranking

There are two main ways of choosing the Considerations. Here defined as:

Absolute Utility: the Considerations with the highest Rank value have absolute preference over ones with lower Rank value, meaning that the lower ones are completely ignored when calculating the scores;
Relative Utility: this is the one which was already explained, which is the Score itself, evaluated from the Response Curves.

The Rank value is calculated as an integer. Just to exemplify: supposing 5 Considerations (A, B, C, D and E). Lets suppose Rank value for them:

A = 0;
B = 1;
C = 1;
D = 2;
E = 2.

This means that, at that frame, the Considerations D and E have absolute preference and will have their response curves evaluated and compared to choose between them. Considerations A, B and C will be ignored.

There are two main ways of changing the Rank value, and it is always defined at runtime and can change at every frame. This dynamic Rank value is useful so the user can pre-define some ranking logic which is specific to their game, giving preference to a sub-set of Considerations as needed.

To exemplify: lets suppose a game in which a shooter can either engage in a fight, or run to heal. There might be many considerations which are specific each type of possibility. In order to filter what is more important at some moment, it might be useful to increase the Rank values in order to express that to the UT Agent.

There are only two ways of defining the Rank value:

Defining your own Rank logic: from the quantum solution, it is possible to inherit from the AIFunctionInt class in order to implement some logic which returns an integer value. Here is a code snippet:

namespace Quantum
{
  [System.Serializable]
  public unsafe class SampleRank : AIFunctionInt
  {
    public override int Execute(Frame frame, EntityRef entity = default)
    {
      // Add here any logic to calculate the desired Rank
      return 0;
    }
  }
}

As an example, there could be a IsDangerRank which reads the game state/blackboard values in order to identify if the Agent is currently in danger, meaning that there are enemies close to it, or which has LoS to it. If danger is detected, increase return a Rank value of 10, meaning that the Considerations with such Rank will have now a very high absolute priority. If there is no danger, just return 0.

Once your quantum code is compiled, the Function node will be available on the context menu. To access the Rank node, double click in the Rank area on the Consideration node.

It is also possible to change the Rank value based on the concept of Momentum, explained in the topic below.

Momentum

By using the Base Values from a Consideration, it is possible to specify Momentum, which automatically increases the Rank of a Consideration when it is chosen (similar to an On Enter logic).

The purpose of having Momentum is to prevent the Agent from changing its mind too often. As the Considerations are constantly re-evaluated, it could lead to an Agent starting doing some task (such as chasing a target) and switching to another task (such as returning to protect the base) too quickly, so the Agent could potentially finish nothing.

With Momentum, you can then specify that the Consideration's Rank increases, which creates what we are calling a commitment to the chosen task. The Rank value generated by the Momentum has higher priority than the one calculated dynamically.

Now that we created some Momentum, when does it get reset again? There are two main ways into decreasing it back to zero:

By using the Base Value named Momentum Decay, it is possible to specify a value which will be used to decrease the Momentum's Rank, second by second. So depending on how you setup the Momentum values, it can take just one second of commitment, or many seconds. It is up to you;
It is also possible to cancel the Momentum's Rank with the Commitment checker types, which return a Boolean that you can use to specify when the Momentum should be canceled. To create Commitment checker, inherit from AIFuncionBool and implement it's Execute method. When returning the value FALSE, it means that the Momentum should be canceled.

namespace Quantum
{
  [System.Serializable]
  public unsafe class SampleCommitment : AIFuncionBool
  {
    public override bool Execute(Frame frame, EntityRef entity = default)
    {
      // Implement your checking logic here
      return false;
    }
  }
}

Once your quantum code is compiled, the Function node will be available on the context menu. To access the Commitment nodes, double click in the Commitment area.

The idea is that you might want to sustain a high Rank value, until some condition is complete. For example, if you have an agent which follows another character, you might want it to have a big Rank value once the character starts following it, set the value zero to the Momentum Decay so the Rank sustains a high value, and add a Commitment which checks if the agent can still really reach it's target (it could be a simple distance check). If the target is too far away, then returning True will take the Rank of the Consideration to zero again, increasing the chances that something else will be more useful to the agent.

That said, it is not mandatory to use neither of those techniques, or to use them in exclusivity. You can add Momentum to a Consideration and have both a natural Momentum Decay and a Commitment checker. It is all up to you.

Response Curves

This is the core of this AI technique. The decision making is all based on defining curves, scoring, multiplying them and combining the results to get what is more useful at that frame.

We are re-using Unity's AnimationCurve system in order to define the curves, which are then compiled into its deterministic version which are called FPAnimationCurve.

When creating your own curves, what is mostly important here is to use curves which correctly express how you want some consideration to be evaluated. Is it some behaviour that is only important when it is really really close to a specific value? Does the importance grows linearly? Or perhaps exponentially? It could be zero within a specific range, and then start to increase linearly after some point?

Create your own curves, select the ones from the presets and create new presets.

A very important concept to have in mind is that the Y Axis of the curves (which is the resulting score) should be normalized (i.e between 0..1). This is critical because the curves' results are multiplied, so the proportion needs to be maintained, otherwise the results of the curves will not be really comparable with each other, which would break the principles of the UT.

Here are a few sample images of the response curves saved as presets and used on the Spellcaster Sample:

In order to define more Response Curves for a Consideration, double click in the curves area. It will take you to the curves container.

Use the right mouse button to create a new Response Curve.

Click on the curve to open it's editor window.

The decision of which curve to use completely depends on your game specific needs. It depends on what Input will be inserted on that curve, and how you want that changes to that input value should be reflected on a "utility" value.

To exemplify:

Consider Healing: suppose an Agent which has 10 of Max Health, and starts to desire healing once it's health goes below 5. So, to values which are greater than 5, the result of the utility curve should be zero. Then, for values below zero, we want the utility of healing to increase very fast. This could be a curve which correctly expresses this:
Consider Attacking: suppose an Agent which only desires to Attack if there is at least one enemy on the scenario. It does not matter if it is one, two or ten enemies. This is a "binary threshold" curve, where it goes from zero to one immediately. Even though it removes some of the expressivity that we have with curves, it can still be useful for some types of analysis. The curve would look like this:

Then, it is normal (but not mandatory) that a Consideration will have more than one Response Curve. Just keep creating adding new curves as you need. Just keep an eye on the overhead that is added for reading the Input that is used on the curve (more details regarding that on the next topic).

It is possible to see and edit the response curves from the root view:

Input for Response Curves

Input values are defined by custom user logic, as this is very game specific. It can be a Health value which comes from an entity's component, it can be some data stored on the blackboard, it can be something gathered from some sensors system, etc.

In order to create your own Input types, create a new class which inherits from AIFunctionFP and implement its Execute method.

using Photon.Deterministic;

namespace Quantum
{
  [System.Serializable]
  public unsafe partial class InputEntityHealth : AIFunctionFP
  {
    public override FP Execute(Frame frame, EntityRef entity = default)
    {
      // Read the current health from the component which is in the Agent's entity
      if(frame.TryGet<Health>(entity, out var health) == true)
      {
        return health.Current;
      }
      else
      {
        return 0;
      }
    }
  }
}

Once your quantum code is compiled, the Function node will be available on the context menu. To access the Response Curve nodes, double click in the Response Curves area on the Consideration node.

Linked Considerations

It is possible to link one consideration with others, creating a parent-children relationship.

In this case, the children Considerations are evaluated only if, at a specific frame, the parent Consideration was chosen as the most useful. When it happens, the Children Considerations will only compete with their siblings.

This can be useful mainly in for two reasons:

It can help with performance optimizations. Imagine a parent Consideration which analyzes if "fighting is useful", and the children Considerations are the one which actually evaluates which battle choices to make. These are not computed until it is useful to fight;
Still for performance matters, the curves contained on the parent Consideration doesn't are implicitly calculated for all of its children, so it removes the need of re-computing those curves repeated times;
It helps organizing the Considerations in a "contextual" matter, where the parent Considerations' curves doesn't need to be replicated into it's children;

To link Considerations, simply click on the output slot on the Consideration Node, and link it to the input slot of others':

Compiling your Utility Theory AI

In order to actually use the UT that you created, you have to compile your work.

To compile, you have two options:

The left button is used to compile only the currently opened document;
The right button is used to compile every AI document that you have on your project.

Your UT files will be located at: "Assets/Resources/DB/CircuitExport/UT_Assets".

Setting the AI to be used by your Bots

To finally use the AI created, you just need to reference the compiled assets.
You can do it by loading the asset based on the GUID, or you can just create an AssetRefUTRoot to point to the desired AI asset:

The Utility Reasoner

The Utility Reasoner is the main struct, on the Quantum side, responsible for all of the UT architecture. It holds all of the relevant data for scoring and choosing Considerations.

If you want the Reasoner to be tied to a specific entity, to be considered your Agent, then you can add to that entity a component named UTAgent. It can be done either via quantum code or directly on an Entity Prototype. The UTAgent contains a Utility Reasoner already.

The advantage of doing it directly on a Prototype is that it makes it possible to reference the UTRoot asset directly from Unity, which might be handy.

The Reasoner is a regular Quantum struct. It is used by a component named UTAgent, explained in the next topic, but the Reasoner can also be declared as part of the global data on the DSL. The entire UT API has the EntityRef entity parameter as optional. This all means that it is possible to create a Reasoner (or more) which is not tied to an entity at all, which can be useful for more abstract concepts such as the "imaginary player" on an RTS game which decided where/when/how to spawn the creatures.

To use the Reasoner as part of the global data, simply declare and reference it like this:

// In any DSL file
global
{
    UtilityReasoner UtilityReasoner;
}

// In any other logic file, such as inside a System
f.Global->UtilityReasoner

Initializing and Updating

This is the snippet for initializing a Utility Reasoner, exemplified with the usage of the UTAgent component:

UTManager.Init(f, &utAgent->UtilityReasoner, utAgent->UtilityReasoner.UTRoot, entity);

As for updating a Utility Reasoner:

UTManager.Update(f, &utAgent->UtilityReasoner, entity);

Defining fields values

Find here more information on the alternatives that you have when settings values to fields on Considerations, Inputs, Ranks and Commitments: Defining fields values.

AIParam

Find here more information on how to use the AIParam type, which is useful if you want to have more flexible fields that can be defined in different ways: settings by hand or from Blackboard/Constant/Config Nodes: AIParam.

AIContext

Find here more information on how to pass agent-contextual information as parameter: AIContext.

BotSDKSystem

There is a class which is used to automate some processes such as deallocating Blackboard memory. Find here more information about it: BotSDKSystem.

Visual Editor Comments

Find here more information on how to create comments on the Visual Editor: Visual Editor Comments.

Changing the compilation export folder

By default, assets generated by Bot SDK's compilation will be placed into the folder Assets/Resources/DB/CircuitExport. See here how you can change the export folder: Changing the export folder.

Choosing the saved History Size

It is possible to change the amount of history entries saved on Bot SDK files. Find here more information on this matter: Changing History Save Count.