Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Release notes for v0.26.0 #3056

Merged
merged 1 commit into from
Sep 6, 2022
Merged

Conversation

pseudo-rnd-thoughts
Copy link
Contributor

@pseudo-rnd-thoughts pseudo-rnd-thoughts commented Sep 2, 2022

Release notes for v0.26.0

This release is aimed to be the last of the major API changes to the core API. All of the previously "turned off" changes of the base API (step termination / truncation, reset info, no seed function, render mode determined by initialization) are now expected by default. We still plan to make breaking changes to Gym itself, but to things that are very easy to upgrade (environments and wrappers), and things that aren't super commonly used (the vector API). Once those aspects are stabilized, we'll do a proper 1.0 release and follow semantic versioning. Additionally, unless something goes terribly wrong with this release and we have to release a patched version, this will be the last release of Gym for a while.

If you've been waiting for a "stable" release of Gym to upgrade your project given all the changes that have been going on, this is the one.

We also just wanted to say that we tremendously appreciate the communities patience with us as we've gone on this journey taking over the maintenance of Gym and making all of these huge changes to the core API. We appreciate your patience and support, but hopefully, all the changes from here on out will be much more minor.

Breaking backward compatibility

These changes are true of all gym's internal wrappers and environments but for environments not updated, we provide the EnvCompatibility wrapper for users to convert old gym v21 / 22 environments to the new core API. This wrapper can be easily applied in gym.make and gym.register through the apply_api_compatibility parameters.

  • Step Termination / truncation - The Env.step function returns 5 values instead of 4 previously (observations, reward, termination, truncation, info). A blog with more details will be released soon to explain this decision. @arjun-kg
  • Reset info - The Env.reset function returns two values (obs and info) with no return_info parameter for gym wrappers and environments. This is important for some environments that provided action masking information for each actions which was not possible for resets. @balisujohn
  • No Seed function - While Env.seed was a helpful function, this was almost solely used for the beginning of the episode and is added to gym.reset(seed=...). In addition, for several environments like Atari that utilise external random number generators, it was not possible to set the seed at any time other than reset. Therefore, seed is no longer expected to function within gym environments and is removed from all gym environments @balisujohn
  • Rendering - It is normal to only use a single render mode and to help open and close the rendering window, we have changed Env.render to not take any arguments and so all render arguments can be part of the environment's constructor i.e., gym.make("CartPole-v1", render_mode="human"). For more detail on the new API, see blog post @younik

Major changes

  • Render modes - In v25, there was a change in the meaning of render modes, i.e. "rgb_array" returned a list of rendered frames with "single_rgb_array" returned a single frame. This has been reverted in this release with "rgb_array" having the same meaning as previously to return a single frame with a new mode "rgb_array_list" returning a list of RGB arrays. The capability to return a list of rendering observations achieved through a wrapper applied during gym.make. Rename "rgb_array" to "rgb_array_list" and "single_rgb_array" to "rgb_array" #3040 @pseudo-rnd-thoughts @younik
  • Added save_video that uses moviepy to render a list of RGB frames and updated RecordVideo to use this function. This removes support for recording ansi outputs. Add save_video util and deprecate RecordVideo in favor of it #3016 @younik
  • RandomNumberGenerator functions: rand, randn, randint, get_state, set_state, hash_seed, create_seed, _bigint_from_bytes and _int_list_from_bigint have been removed. @balisujohn
  • Bump ale-py to 0.8.0 which is compatibility with the new core API
  • Added EnvAPICompatibility wrapper @RedTachyon

Minor changes

@wookayin
Copy link
Contributor

wookayin commented Sep 4, 2022

Given so many breaking changes that seem to have no backward-compatibility options with legacy gym versions, including (but not limited to):

The Env.step function returns 5 values instead of 4

Reset info - The Env.reset function returns two values (obs and info) with no return_info parameter for gym wrappers and environments. @balisujohn (#2962)

If that's the case, why don't we make v1.0.0 release instead of v0.26.0? It will cause lots of really annoying problems once released.

@wookayin
Copy link
Contributor

wookayin commented Sep 4, 2022

Another suggestion: it'd be great if the corresponding PR (or issue number) is added for each item in the release note so people can find the relevant issue more easily.

@blumu
Copy link

blumu commented Sep 20, 2022

It seems to me that something is still missing with the breaking change in 0.26. The compatibility wrapper EnvCompatibility from this PR is helpful for people maintaining RL agents who want to upgrade to the latest version of gym: it allows them to consume older environments written for the older gym contract.
But what about gym environments maintainers, who also want to upgrade to the latest gym version while still being able to run existing gym implementations of RL algorithms? For example right now, if you upgrade your gym environments to 0.26 you can't run stable_baseline agents on it anymore. Is there a plan to add the inverse compatibility layer for that scenario too?

@pseudo-rnd-thoughts
Copy link
Contributor Author

Thanks for the question, currently we are not planning on adding that feature in the future for two reasons. The first is that we are working with a number of the largest rl libraries (stable baselines 3, rl lib, clean rl and tianshou) to upgrade them to v26. This is planned to be done in the next few weeks. If you have a RL training library or environment that needs help / advice on upgrading to v26, please join the gym discord on the readme. The second is the reverse compatibility wrapper would lose number of pieces of information such that the info in reset and termination and truncation in step. Therefore we wouldn't be able to it as a general solution but if you wanted to do it for testing, I would implement yourself. There is already a compatibility function for step that you could use

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants