Gymnasium

Gymnasium is a comprehensive Python library that provides a standardized API for developing and comparing reinforcement learning algorithms, serving as the maintained fork of OpenAI's Gym. It offers a diverse collection of environments including Classic Control (physics-based problems), Box2D (physics simulations), Toy Text (simple discrete environments), MuJoCo (complex multi-joint physics control), and Atari (game emulation environments).

Package Information

• Package Name: gymnasium
• Language: Python
• Installation: pip install gymnasium

Core Imports

import gymnasium as gym

For working with specific components:

from gymnasium import Env, Wrapper, Space
from gymnasium import make, register, spaces, wrappers

Basic Usage

import gymnasium as gym

# Create an environment
env = gym.make('CartPole-v1', render_mode='human')

# Reset environment to get initial observation
observation, info = env.reset(seed=42)

for _ in range(1000):
    # Take a random action
    action = env.action_space.sample()
    
    # Execute the action
    observation, reward, terminated, truncated, info = env.step(action)
    
    # Check if episode is done
    if terminated or truncated:
        observation, info = env.reset()

env.close()

Architecture

Gymnasium's architecture is built around several core concepts:

• Environment (Env): The main interface that defines the reinforcement learning task through step, reset, render, and close methods
• Space: Defines the structure and constraints of action and observation spaces
• Wrapper: Modifies environment behavior without changing the underlying environment
• Registration System: Manages environment discovery and instantiation through string IDs
• Vector Environments: Enables parallel execution of multiple environment instances

This design provides a flexible, extensible framework that serves as the standard interface for reinforcement learning in Python.

Capabilities

Core API

The fundamental classes and interfaces that form the foundation of all Gymnasium environments and wrappers.

class Env:
    def step(self, action): ...
    def reset(self, *, seed=None, options=None): ...
    def render(self): ...
    def close(self): ...
    
class Wrapper(Env):
    def __init__(self, env): ...

Core API

Environment Registration

Functions for discovering, creating, and managing environments through the global registry system.

def make(id: str | EnvSpec, max_episode_steps: int | None = None, **kwargs) -> Env: ...
def register(id: str, entry_point: str, **kwargs) -> None: ...
def spec(id: str) -> EnvSpec: ...

Environment Registration

Spaces

Defines action and observation spaces with various types including continuous, discrete, composite, and structured spaces.

class Box(Space): ...
class Discrete(Space): ...
class Dict(Space): ...
class Tuple(Space): ...

Spaces

Environment Wrappers

Pre-built wrappers for modifying environment behavior including observation transformation, action modification, reward shaping, and rendering enhancements.

class TimeLimit(Wrapper): ...
class FlattenObservation(ObservationWrapper): ...
class ClipAction(ActionWrapper): ...
class NormalizeReward(RewardWrapper): ...

Environment Wrappers

Vector Environments

Batched environment execution for improved performance when training with multiple parallel environments.

class VectorEnv: ...
class SyncVectorEnv(VectorEnv): ...
class AsyncVectorEnv(VectorEnv): ...

Vector Environments