Utility AI was popularised by Dave Mark, and was used in Guild Wars 2 (or an expansion). For each AI, each of it's actions is scored on how useful it would be to perform at this moment, based on the world state, and custom multipliers/curves. The highest-scoring action is then performed. This is a beautiful system. You can cache the world-state values for multiple actions and AI, and the scoring is just * and +, so it's very cheap. You can switch out multiplying against fixed weights for multiplying against a curve, which gives your AI designers much more elegant control. I found this works well for goals, or for non-complex AI, but falls down when trying to use it for each granular action a player-AI might use. Keeping all the different values (or strata of values) in your head to roughly control what happens, becomes difficult.
Enter GOAP AI. GOAP was popularised by Jeff Orkin in the 2005 videogame F.E.A.R. Simply: It's A* pathfinding, except the destination is your desired world state, other nodes are intermediate world-states, and the edges are actions. This is the AI system that feels most right. When you use it, it feels like what you imagine AI design should feel like. Each world state is defined by a bunch of world properties and their state (bools are simplest but I'd like to see more complex systems like ints with comparisons). E.g. WS.HoldingAtLeast3 = False, WS.OverlappingGoal = False, and WS.Offside = False. Then you define your desired world state (e.g. WS.OverlappingGoal = True), your possible action (e.g. ThrowAtLeast3TowardsGoal), what each action requires (e.g. WS.HoldingAtLeast3 = True) and what it results in (e.g. WS.OverlappingGoal = True). On top of that you give each action a cost (e.g. default to 1, but e.g. PickUpAtLeast3 might have a cost of 3). You don't need to manually chain these together - the GOAP planner will use the actions to find a path from the current world state to the target world state if it can, and use the length of the action chain and the costs to minimise the total cost (e.g. PickUpAtLeast3 => ThrowAtLeast3TowardsGoal).
I implemented the GOAP planner from scratch, based on all the readings and public reference implementations I could find, which greatly helped my understanding of it. I initially tried using other people's implementations but none of them fit exactly. Writing it myself, and just what I needed, made debugging a million times easier and let it fit the requirements better. I implemented A* in the process, but it's not generic. It was easier for me to dedicate it fully to GOAP, instead of implementing an abstract A* machine with A* variable names then overriding for GOAP.
Debug output is the most critical thing to implement when creating game AI systems, and UE4 works well here: it has Visual Logger, which is a time-based log that you can scrub back-and-forth through, only seeing the logs from the tick you have selected. It also differentiates logs per actor, and supports verbosity levels. It's compiled out of shipping games so it has no final performance impact either. This is where I solve all my "why can't the planner find a path here" and "why is the planner picking that action" issues.
(continued in comment)
