I'm trying to teach an AI the combat mechanics of a system similar to Darkest Dungeon.
The goal is for the AI to be able to act well controlling NPCs with random stats and random skills. This means that on each session the AI's character will different values for health, stress, accuracy, dodge, etc... The stats of each skill the character has also has random values - the damage, accuracy, effects.
In my current system-
The inputs for the model are:
- All the stats that the AI's character has.
- All the stats of it's allies.
- A subset of stats of each enemy (in Darkest Dungeon you cannot see all the enemies's stats).
- All the stats of each skill.
The outputs for the model are:
- Which skill to use (out of 4 options)
- Which target (total of 8 options, being any ally (including self) and any enemy)
I'm using an action mask to disable invalid actions (such as using an offensive skill on an ally or targeting a position that has no character on).
The main problem I'm having is that what each action does changes heavily depending on the stats of the skill in that index.
Does anyone have an insight on which kind of learning I'm looking for? So far I have tried using MA-POCA provided by the Unity Ml-Agents package with no success, the model didn't seem to understand that what each action does relies extremely on the associated skill stats.
Searching for papers on the subject only resulted in articles about action-spaces with variable size, which I already solve with by masking invalid actions.
Obs: I'm not limited to training in the Unity environment. The only limitation I have is that the model must be convertible/exported to ONNX format.