What do we need the "metadata" field of the gym environments for?

user_3924118 0
Viewed 2001

I have noticed that the base class Env (from gym) contains a class field called metadata. This field seems to be used to specify how an environment can be rendered. For example, in the case of the FrozenLake environment, metadata is defined as

metadata = {'render.modes': ['human', 'ansi']}

However, this property/field metadata is never used in that file. For example, in this same example, the render method has a parameter where you can specify the render mode (and the render method does not even check that the value passed to this parameter is in the metadata class field), so I am not sure why we would need this metadata field.

So, why do we need the field metadata? Is it used for something else other than specifying the rendering "modes" of an environment? If yes, where?

openai-gym reinforcement-learning
2 Answers

I will try to be as basic as possible. I have run some trials and in my understanding the render has 2 modes, human and ansi which deal with the output on console.(attached is screenshot of my console). In human mode (the default) the output will be SFFF,FHFH with a color tag of current observation (human-friendly)...etc and in ansi mode you will output bytes-like objects where No encoding, decoding, or newline translation is performed e.g \nSFFF\n\x1b[41mF\x1b[0mHFH.

Check out my output below: enter image description here

user_96215520

First of all, I am not a maintainer/contributor to the OpenAir gym repository. Everything below, therefore, comes from my own investigation since I was wondering the same thing. Therefore, the OpenAi Gym team had other reasons to include the metadata property than the ones I wrote down below.

TLDR

The metadata attribute describes some additional information about a gym environment-class that is not needed during training but is useful when performing:

  • Python tests.
  • Using gym utilities.
  • Parallel training utilities.
  • etc.

More in-depth answer

The first important thing to notice is that (in most of the environments) the metadata attribute is defined before the class constructor. This means that it is a class attribute that is shared among all instances of a class. This was done since it is not meant to be used during training but stores some additional information about an environment-class that can be used in other test or gym utilities (this is why it is called META-data).

I think this can be seen by diving a bit into the codebase and checking where the metadata is used. When searching the code, we can see that the the frozen lake environment metadata is used in two places:

The env_test script

Here the render.modes metadata is used in the gym/envs/tests/test_env.py script to test whether all the possible render methods in a given environment are working as expected:

for mode in env.metadata.get('render.modes', []):
    env.render(mode=mode)

The video_recorder script

The second place where the render.modes metadata is used is the gym/wrappers/monitoring/video_recorder.py. Here it is used to check if a render mode is available in the gym environment, which can be recorded with the video recorder.

if 'rgb_array' not in modes:
            if 'ansi' in modes:
                self.ansi_mode = True
            else:
                logger.info('Disabling video recorder because {} neither supports video mode "rgb_array" nor "ansi".'.format(env))
                # Whoops, turns out we shouldn't be enabled after all
                self.enabled = False
                return

Another example of metadata that is used here is the video.frames_per_second metadata.

Another possible use-case

As mentioned above, in most environments, the metadata is defined as a class attribute. By doing this, we can make sure that if the attribute is changed on one of the instances, all the other instances are also aware of this change. This can be handy when you, for example, want to use MPI to train an agent on multiple parallel gym environments at the same time. You could, for example, update the frequency of all the instances at once.

Conclusion

Judging from the above investigation, I think the metadata stores additional information about a gym environment-class, which is not used during training, but that, for example, can be used during:

  • Python tests.
  • Other gym utilities.
  • Parallel training utilities.
  • etc.

You can, of course, also store the same information as separate attributes on the instance, but I think as it is not used during training, storing it as a metadata class attribute is more correct.

user_81356870
Related