I am working on a problem and I want to implement it as a reinforcement learning problem and integrate it into OpenAI gym. My states are in the form of lists of length n which each element is chosen from a discrete interval [0, m]. for example for n=6 and m=3, this is a sample from the observation space:
[0 2 1 3 3 2]
and the possible accessible states from this space is a set of other lists which are achieved by changing a number of k elements the elements in the list with a number from the same [0, m]. for example, for k=1 we can have the following states as two subsequent states of the previous state:
[0 2 2 3 3 2]
or
[0 3 1 3 3 2]
My question is that what is an efficient way to represent the "actions" in the OpenAI gym for such a scenario?
One way that come to my mind is to just use the next state as the action itself, for example, if I write:
action = env.action_space.sample()
the action would be the next state (which also implicitly contains the action) and then in the env.step(action) make the state equal to the next state.
Does anyone know a better way or using the implicit action representation with the next state is the optimal way? Does anyone know a predefined gym environment that also has the same representation? what are the cons of the implicit representation of the actions that I just explained?