Introduction

OGym is a Rust-native reinforcement learning framework inspired by OpenAI Gym. It provides a flexible, type-safe, and efficient platform for building, training, and interacting with reinforcement learning environments.

Built entirely in Rust, OGym uses ownership and zero-cost abstractions to provide thread safety and strong performance, particularly for parallel environment execution.

Performance

Features

• Type safety first: Strongly typed actions and observations eliminate common runtime shape errors.
• High performance: Compiled environments run with minimal overhead.
• Native vectorization: Batched environments support parallel processing and matrix operations.
• Modular backends: The architecture supports classic control systems and multiple physics engines, including Rapier and MuJoCo.
• Zero-cost abstractions: Idiomatic Rust patterns provide safety without sacrificing performance.

Getting Started

Installation

Add OGym to your Cargo.toml:

[dependencies]
ogym = "0.1.0"

MuJoCo support

MuJoCo environments are opt-in, so the default crate does not require native MuJoCo libraries. Enable the mujoco feature in your Cargo.toml:

[dependencies]
ogym = { version = "0.1.0", features = ["mujoco"] }

Current limitation: The MuJoCo feature relies on OGym’s repository-local patch for mujoco-rust, which crates.io packages cannot pass on to consumers. Use a source checkout for MuJoCo environments until the patched wrapper is published. The default crates.io feature set is unaffected.

You must also install the MuJoCo library separately.

MuJoCo 3.10.0

OGym targets MuJoCo 3.10.0. On Linux x86-64, install the prebuilt archive under the default path expected by OGym:

mkdir -p "$HOME/.local/mujoco"
curl --fail --location \
  https://github.com/google-deepmind/mujoco/releases/download/3.10.0/mujoco-3.10.0-linux-x86_64.tar.gz \
  | tar -xz --strip-components=1 -C "$HOME/.local/mujoco"

Official prebuilt packages are also available for Linux AArch64, macOS universal, and Windows x86-64.

Note: If you install MuJoCo elsewhere, set MUJOCO_DIR to its root directory and ensure its library directory is available to the dynamic linker.

Quick start

The following example runs a Classic Control CartPole environment:

use ogym::{
    env::control::cart_pole::{CartPole, CartPoleConfig},
    env::environment::Environment,
    spaces::Space,
};

fn main() -> Result<(), Box<dyn std::error::Error>> {
    let config = CartPoleConfig::new();
    let mut env = CartPole::new(config)?;

    env.reset(None)?;

    let mut total_reward: f64 = 0.0;

    loop {
        let action = env.space.action.sample()?;
        let experience = env.step(action)?;

        total_reward += experience.reward;

        if experience.terminal.is_done() {
            break;
        }
    }

    println!("Episode finished. Total Reward: {:.2}", total_reward);
    Ok(())
}

Architecture

OGym separates the environment contract, typed spaces, and simulation backends. Native environments implement their transition equations directly. Rapier environments own a PhysicsWorld; feature-gated MuJoCo environments own an MjEnv loaded from embedded or custom XML.

Environment
├── EnvSpace<StateSpace, ActionSpace>
├── reset(seed) -> (state, info)
└── step(action) -> Experience
    ├── current and next state
    ├── reward and action
    ├── info
    └── Terminal

The crate also defines dynamic and const-generic batch interfaces. It does not currently ship a concrete VecEnv or native CartPoleVec implementation.

• Core abstractions describes environment lifecycle and batch contracts.
• Spaces covers validation, sampling, and mixed action modes.
• Backends and dynamics explains native, Rapier, and MuJoCo transitions.
• Implementing an environment is the contributor checklist.

Core abstractions

Environment

Environment uses associated Action, State, and Info types. reset(seed) initializes an episode; step(action) produces an Experience; state() reads the current observation. is_terminal() and is_truncated() remain separate, while is_done() is their disjunction and to_terminal() preserves both flags.

Calling state() or step() before initialization may return NotInitialized; stepping an already completed native environment may return EpisodeDone; invalid kinds, dimensions, or values produce typed errors.

Experience

Experience<S, I, A> contains curr_state, reward: f64, action, next_state, info, terminal, and step. The step field is environment supplied; consumers should not infer it from vector indices.

Terminal

Terminal has Ongoing, Terminate, Truncate, and Both variants. is_terminated(), is_truncated(), and is_done() expose the three useful predicates. from_flags(terminated, truncated) retains simultaneous termination and truncation.

Batch traits

BatchEnvironment uses runtime-sized Vec collections; StaticBatchEnvironment<N> uses arrays and a const batch size. Both define reset, batched stepping, state access, completion flags, and selective reset contracts. They are extension interfaces: OGym currently provides no concrete batch wrapper.

See the generated API documentation for exact signatures.

Spaces

Space associates an Item type with four operations: sample, contains, shape, and bounds. EnvSpace<S, A> pairs the state and action spaces exposed by an environment.

Classic Control and LunarLander use Mixed so action mode is encoded in both the configured space and the action value. Passing a continuous item to a discrete instance, or the reverse, is an error. MuJoCo environments use dynamically sized DVector<f64> actions because custom XML can change nu.

Backends and dynamics

Native Rust

Classic Control integrates explicit ordinary differential or difference equations. Toy Text environments implement finite-state transition functions, with seeded randomness where transitions or card draws are stochastic.

Rapier2D

PhysicsWorld owns Rapier rigid bodies, colliders, joints, collision channels, and the query pipeline. Each step advances with an environment-selected \(\Delta t\). At a high level the constrained rigid-body system is

\[ M(q)\ddot q + C(q,\dot q) + g(q) = \tau + J(q)^T\lambda, \]

with joint/contact impulses \(\lambda\) resolved by Rapier. Environment pages specify the forces, motor controls, state projection, reward, and episode conditions layered over that solve.

MuJoCo

The optional mujoco feature exposes MjEnv, which loads XML, owns MuJoCo model/state objects, validates control length, and advances frame_skip internal steps. The agent timestep is

\[ \Delta t_{env} = \text{frame_skip},\Delta t_{model}. \]

MuJoCo solves multibody dynamics with actuation and constraints, conventionally written

\[ M(q)\dot v + c(q,v) = \tau(u) + J(q)^T f, \qquad q_{t+1}=\operatorname{integrate}(q_t,v_{t+1}). \]

MjEnv exposes qpos, qvel, controls, body kinematics, inertias, contact forces, tendon data, and actuator/constraint forces used to construct observations and rewards. Custom XML can change dimensions, ranges, timestep, and body names; environment-specific XML must retain every body/site name that its Rust implementation looks up.

References: Rapier rigid bodies, MuJoCo computation, and MuJoCo XML reference.

Implementing an environment

Place a new environment under the appropriate backend with config.rs, env.rs, and mod.rs; keep model assets and backend helpers beside it.

1. Define a configuration with documented defaults and meaningful builder methods where fields are private.
2. Define exact State, Action, and Info types and an EnvSpace when the dimensions are static.
3. Make seeded resets reproducible, clear the step counter, and initialize every backend state used by state().
4. Validate action kind, dimension, and bounds before applying it.
5. Return an Experience containing the pre-step and post-step observations and a correct Terminal value.
6. Keep task termination separate from the configured time limit.
7. Test valid transitions, invalid actions, boundaries, seeded resets, rewards, and simultaneous terminal/truncation behavior.
8. Export the environment and config from mod.rs and add a specification page using the common environment template.

For MuJoCo, gate exports behind the mujoco feature, embed the default XML, derive dimensions from the loaded model, and document required body/site names for custom XML.

Environments

OGym provides 21 environments behind one typed Environment contract. The state returned by an environment is the agent-visible observation; it is not necessarily the backend’s complete physical state.

Every environment page specifies the observation and action spaces, transition or backend dynamics, reward, termination and truncation conditions, configuration, and returned information. See notation and conventions before comparing equations across families.

Notation and conventions

OGym calls the agent-visible observation a state in its Rust API. This documentation writes it as \(s_t\), while \(x_t\) denotes a backend’s latent state when the distinction matters. An interaction is

\[ (s_{t+1}, r_t, d_t, i_t) = \operatorname{step}(s_t, a_t), \]

where \(a_t\) is the action, \(r_t\) the reward, \(d_t\) the Terminal status, and \(i_t\) the environment-specific information value.

• Terminate means an environment-defined terminal event occurred.
• Truncate means an external horizon was reached.
• Both records both events on the same transition.
• Ongoing records neither event.

Intervals use \([l,h]\) for inclusive space bounds and \((l,h)\) when the health test uses strict inequalities. clip(x,l,h) limits a scalar to that interval. \(\lVert x\rVert_2\) is the Euclidean norm and \(\lVert x\rVert_2^2=\sum_i x_i^2\).

For MuJoCo, \(q\), \(\dot q\), and \(u\) denote generalized positions, velocities, and actuator controls. A default-model dimension is not a promise for custom XML: the action length is always nu, and observation length is assembled from the loaded model.

Classic Control

Classic Control environments integrate compact analytic models in native Rust. All four can select discrete or continuous control at construction through their configuration.

Angles are radians, angular velocities are radians per step unit, and each page states its integrator and clipping rules.

Info is () for Acrobot, CartPole, and Pendulum. MountainCar uses Option<()> and currently returns None.

Acrobot

Acrobot is a two-link underactuated pendulum. Only the second joint is actuated.

State space

The exposed SVector<f64, 6> is

\[ s=[\cos\theta_1,\sin\theta_1,\cos\theta_2,\sin\theta_2,\dot\theta_1,\dot\theta_2]. \]

The first four bounds are \([-1,1]\); velocity bounds default to \(\pm4\pi\) and \(\pm9\pi\). Reset samples the latent \([\theta_1,\theta_2,\dot\theta_1,\dot\theta_2]\) uniformly from \([-0.1,0.1]^4\).

Action space

• Discrete: 0, 1, 2 map to torque \(-1,0,+1\).
• Continuous: one value in \([-1,1]\), clipped to the interval.

Uniform noise in \((-n,n)\) is added when torque_noise_max = n > 0.

Dynamics and reward

Let \(m_i,l_1,c_i,I,g\) come from the config and define

\[ d_1=m_1c_1^2+m_2(l_1^2+c_2^2+2l_1c_2\cos\theta_2)+2I, \quad d_2=m_2(c_2^2+l_1c_2\cos\theta_2)+I. \]

\[ \phi_2=m_2c_2g\cos(\theta_1+\theta_2-\pi/2), \]

\[ \phi_1=-m_2l_1c_2\dot\theta_2^2\sin\theta_2-2m_2l_1c_2\dot\theta_1\dot\theta_2\sin\theta_2+(m_1c_1+m_2l_1)g\cos(\theta_1-\pi/2)+\phi_2. \]

Book dynamics use

\[ \ddot\theta_2=\frac{\tau+(d_2/d_1)\phi_1-m_2l_1c_2\dot\theta_1^2\sin\theta_2-\phi_2}{m_2c_2^2+I-d_2^2/d_1}, \]

while Nips omits the \(m_2l_1c_2\dot\theta_1^2\sin\theta_2\) term. In both cases \(\ddot\theta_1=-(d_2\ddot\theta_2+\phi_1)/d_1\). RK4 integrates with dt; angles wrap to \([-\pi,\pi)\) and velocities are clipped. Reward is \(-1\) until the terminal transition, which receives \(0\).

Episode end

Termination occurs when

\[ \cos\theta_1+\cos(\theta_1+\theta_2)<-1. \]

Truncation occurs at max_t steps (default 500).

Configuration

The config provides corresponding with_* builders plus use_book_dynamics() and use_nips_dynamics().

Reference: Gymnasium Acrobot.

CartPole

CartPole balances a pole hinged to a cart moving on a horizontal track.

State and action spaces

The SVector<f64, 4> observation is \([x,\dot x,\theta,\dot\theta]\). Bounds are \(x\in[-2x_{max},2x_{max}]\), \(\theta\in[-2\theta_{max},2\theta_{max}]\), with velocity components represented by \(\pm\texttt{f64::MAX}\).

• Discrete: 0 applies \(-f\), 1 applies \(+f\).
• Continuous: \(a\in[-1,1]\) applies \(F=af\); out-of-range values are rejected.

Reset samples every state component uniformly from \([-0.05,0.05]\).

Dynamics and reward

Let \(m=m_c+m_p\), \(p=m_pl\), and

\[ T=\frac{F+p\dot\theta^2\sin\theta}{m}. \]

Then

\[ \ddot\theta=\frac{g\sin\theta-\cos\theta,T}{l(4/3-m_p\cos^2\theta/m)}, \qquad \ddot x=T-\frac{p\ddot\theta\cos\theta}{m}. \]

Euler updates positions before velocities; semi-implicit Euler updates velocities first. Both use timestep tau. Every accepted step rewards \(+1\), including a terminal step.

Episode end

Termination occurs when \(|x|>x_{max}\) or \(|\theta|>\theta_{max}\). Truncation occurs at t_max steps.

Configuration

Builders configure each physical parameter, thresholds, integrator, and action mode.

Reference: Gymnasium CartPole.

MountainCar

MountainCar drives an underpowered car out of a valley by building momentum.

State and action spaces

The state is SVector<f64, 2>: \([x,v]\), bounded by min_x..max_x and \([-max_v,max_v]\). Reset samples \(x\sim U(-0.6,-0.4)\) and sets \(v=0\).

• Discrete: 0, 1, 2 map to direction \(-1,0,+1\).
• Continuous: one command \(a\in[-1,1]\); out-of-range values are rejected.

Dynamics and reward

For mapped force command \(u\),

\[ v_{t+1}=\operatorname{clip}(v_t+f u-g\cos(3x_t),-v_{max},v_{max}), \]

\[ x_{t+1}=\operatorname{clip}(x_t+v_{t+1},x_{min},x_{max}). \]

At the left boundary, negative velocity is reset to zero. Constant reward is always \(-1\). ActionPenalty is \(-0.1u^2\) plus \(+100\) on the terminal transition.

Episode end

Termination requires both \(x\ge x_{goal}\) and \(v\ge v_{goal}\). Truncation occurs at max_t steps.

Configuration

with_continuous_action() applies the continuous-task preset: f=0.0015, goal_x=0.45, max_t=999, and ActionPenalty. with_discrete_action() restores the discrete preset. Apply mode selection before later overrides because it replaces those values.

Reference: Gymnasium MountainCar.

Pendulum

Pendulum applies torque to swing and hold a single link upright.

State and action spaces

The SVector<f64, 3> observation is \([\cos\theta,\sin\theta,\dot\theta]\), bounded by \([-1,1]\), \([-1,1]\), and \([-v_{max},v_{max}]\).

• Continuous (default): torque \(u\in[-\tau_{max},\tau_{max}]\), with clipping.
• Discrete: n evenly spaced torques, \(u(a)=2a\tau_{max}/(n-1)-\tau_{max}\).

Reset samples \(\theta\in[-x_0,x_0]\) and \(\dot\theta\in[-y_0,y_0]\); a non-positive range fixes that component to zero.

Dynamics and reward

With \(\bar\theta\) normalized to \([-\pi,\pi)\),

\[ c=\bar\theta^2+0.1\dot\theta^2+0.001u^2, \]

\[ \dot\theta_{t+1}=\operatorname{clip}\left(\dot\theta_t+\Delta t\left(\frac{3g}{2l}\sin\theta_t+\frac{3u}{ml^2}\right),-v_{max},v_{max}\right), \]

\[ \theta_{t+1}=\theta_t+\Delta t\dot\theta_{t+1},\qquad r_t=-c. \]

Episode end

The task never terminates from state. It truncates at max_t steps.

Configuration

Reference: Gymnasium Pendulum.

Toy Text

Toy Text environments are finite Markov decision processes with u32 discrete actions. Grid states are row-major integers; Blackjack uses a MultiDiscrete observation.

Blackjack

State and action spaces

The Vec<u32> state is [player_sum, dealer_showing, usable_ace] in MultiDiscrete([32,11,2]). Actions are 0 = stick and 1 = hit. Cards are drawn uniformly from the infinite-deck table [A,2,...,9,10,J,Q,K], where all face cards count as 10.

Dynamics and reward

A usable ace contributes 11 when doing so keeps the hand at or below 21. On hit, draw \(c\sim\text{Deck}\), append it, and return reward \(-1\) if the player busts or \(0\) otherwise. On stick, the dealer draws until its sum is at least 17, then

\[ r=\operatorname{sign}(\operatorname{score}_{player}-\operatorname{score}_{dealer}). \]

With sab, a player natural against a non-natural dealer is an outright \(+1\). Otherwise, when natural is enabled, a natural that already wins pays \(+1.5\).

Episode end

The episode terminates when a hit busts the player or immediately after stick. It never truncates.

Configuration

Info is (). Reference: Gymnasium Blackjack.

CliffWalking

State and action spaces

For an nrow × ncol grid, state \(s=r,n_{col}+c\) belongs to Discrete(nrow*ncol). Actions are 0 = up, 1 = right, 2 = down, and 3 = left; boundary moves stay in place. Start is bottom-left and goal is bottom-right. The bottom-row cells between them are the cliff.

Dynamics and reward

Let \(\delta(a)\) be the selected unit grid displacement. The candidate cell is

\[ (r’,c’)=\operatorname{clip}_{grid}((r,c)+\delta(a)). \]

Entering the cliff returns to start with reward \(-100\) and does not terminate. Every other move rewards \(-1\).

Episode end

Reaching the goal terminates. The step that reaches max_episode_steps truncates, so it may produce Both if it also reaches the goal.

Configuration

Info is (). Reference: Gymnasium CliffWalking.

FrozenLake

State and action spaces

State \(s=r,n_{col}+c\) is row-major. Actions are 0 = left, 1 = down, 2 = right, and 3 = up. The map uses S (start), F (frozen), H (hole), and G (goal).

Dynamics and reward

Without slipperiness, the requested direction is applied with boundary clipping. With is_slippery, the actual direction is uniformly selected from

\[ {(a-1)\bmod4,\ a,\ (a+1)\bmod4}, \qquad P=1/3\text{ each}. \]

The reward is \(1\) on G and \(0\) everywhere else.

Episode end

Entering H or G terminates. Reaching max_episode_steps truncates and can coincide with termination.

Configuration

The state-space size and start location are derived from map; provide a non-empty rectangular map containing S. Info is ().

Reference: Gymnasium FrozenLake.

Taxi

State and action spaces

The 500 states encode taxi row \(r\), column \(c\), passenger index \(p\in{R,G,Y,B,\text{taxi}}\), and destination \(d\in{R,G,Y,B}\):

\[ s=4\left(5(5r+c)+p\right)+d. \]

Actions are 0 = south, 1 = north, 2 = east, 3 = west, 4 = pickup, and 5 = dropoff. Grid boundaries and the fixed interior walls block movement.

Dynamics and reward

Movement and a legal pickup reward \(-1\). Illegal pickup/dropoff rewards \(-10\). Dropping the carried passenger at the destination rewards \(+20\) and terminates. Dropping at another named location leaves the passenger there with the ordinary \(-1\) reward.

Reset samples taxi position, destination, and a distinct waiting location from a seeded RNG.

Episode end

Successful delivery terminates. Reaching max_episode_steps truncates and may coincide with delivery.

Configuration

Info is (). Reference: Gymnasium Taxi.

Rapier environments

These tasks use Rapier2D rigid bodies, colliders, revolute joints, collision events, and ray queries. The shared equations and timestep model are described in Backends and dynamics.

BipedalWalker

State space

The SVector<f32, 24> observation contains, in order: hull angle; scaled hull angular, horizontal, and vertical velocity; left hip angle/speed; left knee angle-plus-one/speed; left foot contact; the corresponding five right-leg values; and 10 normalized lidar hit fractions. If lidar_count differs from 10, values are copied into the fixed 24-vector up to its capacity and remaining elements are zero.

Action space

The continuous SVector<f32, 4> controls left hip, left knee, right hip, and right knee. Values are clipped to \([-1,1]\) when applied. With control_speed=false, sign chooses target direction and magnitude scales maximum motor torque. With control_speed=true, the value scales target speed while maximum torque stays fixed. There is no discrete mode.

Dynamics and reward

Rapier advances one step with \(\Delta t=1/\text{fps}\). Joint motors and contact constraints enter the shared rigid-body equation from the backend chapter. Define

\[ H_t=130x_t/\text{scale}-5|\theta_t|. \]

The ordinary reward is

\[ r_t=H_t-H_{t-1}-0.00035,\tau_{motor}\sum_i\operatorname{clip}(|a_i|,0,1). \]

Hull contact or movement behind \(x=0\) overrides it with \(-100\).

Episode end

Termination occurs on hull contact, \(x<0\), or reaching the end of the generated track. Truncation occurs at max_episode_steps.

Configuration

All fields are public; builders cover the common mode, rate, horizon, scale, and hull changes. Info is ().

Reference: Gymnasium BipedalWalker.

LunarLander

State space

The SVector<f32, 8> observation is normalized horizontal and vertical displacement from the helipad, scaled horizontal and vertical velocity, angle, scaled angular velocity, and left/right leg-contact flags. Its declared bounds are \([-2.5,2.5]^2\), \([-10,10]^2\), \([-2\pi,2\pi]\), \([-10,10]\), and \([0,1]^2\).

Action space

• Discrete: 0 idle, 1 left orientation engine, 2 main engine, 3 right orientation engine.
• Continuous: \([a_m,a_s]\in[-1,1]^2\). The main engine activates for \(a_m>0\) with power \(0.5\operatorname{clip}(a_m,0,1)+0.5\). The side engine activates for \(|a_s|>0.5\), with direction sign(a_s) and power \(\operatorname{clip}(|a_s|,0.5,1)\).

Dynamics and reward

Engine impulses are applied at offset points, creating both translation and torque; gravity and optional wind/turbulence act before a Rapier step of \(1/\text{fps}\). For observation \(s\), shaping is

\[ H(s)=-100\sqrt{s_0^2+s_1^2}-100\sqrt{s_2^2+s_3^2}-100|s_4|+10s_6+10s_7. \]

\[ r_t=H(s_{t+1})-H(s_t)-0.30p_{main}-0.03p_{side}. \]

A successful landing overrides reward to \(+100\); a crash or leaving the horizontal screen overrides it to \(-100\). In the current collision handler, any started collision whose colliders both have parent bodies sets the crash flag; this is broader than hull-only collision and is an implementation difference to keep in mind when comparing with Gymnasium.

Episode end

Termination occurs when the current crash flag is set, when the lander leaves the horizontal screen, or on a settled two-leg landing with near-zero linear and angular velocity. Truncation occurs at max_steps.

Configuration

All fields are public and common values have builders. Info is ().

Reference: Gymnasium LunarLander.

MuJoCo environments

MuJoCo environments require the mujoco feature and MuJoCo 3.10.0. Construct them with new(None) for defaults or new(Some(config)). Actions and observations are DVector<f64>; control length is the loaded model’s nu, and controls use its actuator ranges.

Each action is held for frame_skip MuJoCo steps. Thus \(\Delta t=\text{frame_skip}\times\text{model timestep}\), and velocity rewards use displacement divided by this agent timestep. See backend dynamics.

Custom XML may change nq, nv, nu, body count, ranges, timestep, and therefore observation/action dimensions. It must preserve body and site names used by task-specific lookups.

Ant

State and action spaces

With defaults, the 105-value observation concatenates qpos[2..] (13), all qvel (14), and the six external contact-force components for each non-world body (78), clipped to \([-1,1]\). Including root x/y adds two values; disabling contact forces removes 78. The default action has 8 actuator controls in \([-1,1]\).

Dynamics, reward, and info

For torso displacement over agent timestep \(\Delta t\), \(v_x=(x_{t+1}-x_t)/\Delta t\). With clipped full-model contact-force vector \(f_c\),

\[ r=w_fv_x+r_{healthy}-w_u\lVert u\rVert_2^2-w_c\lVert f_c\rVert_2^2. \]

Healthy means every simulator-state value is finite and root height \(z\in[z_{min},z_{max}]\). Info reports x/y position and velocity, distance from origin, and forward, control, contact, and survival reward components.

Episode end

When enabled, leaving the healthy range terminates. max_episode_steps truncates.

Configuration

AntConfig uses builders because its fields are private to the crate.

Reference: Gymnasium Ant.

HalfCheetah

State and action spaces

The default 17-value observation is qpos[1..] followed by all qvel; retaining root x produces 18 values. The default action is 6 controls in \([-1,1]\).

Dynamics, reward, and info

\[ v_x=\frac{x_{t+1}-x_t}{\Delta t},\qquad r=w_fv_x-w_u\lVert u\rVert_2^2. \]

Info contains x_position, x_velocity, reward_forward, and reward_ctrl. Reset adds uniform position noise and Gaussian velocity noise of configured scale.

Episode end

There is no state termination. The episode truncates at max_episode_steps.

Configuration

Reference: Gymnasium HalfCheetah.

Hopper

State and action spaces

The default 11-value observation is qpos[1..] followed by all velocities clipped to \([-10,10]\); retaining root x adds one value. The default action is 3 controls in \([-1,1]\).

Dynamics and reward

\[ r=w_fv_x+r_{healthy}-w_u\lVert u\rVert_2^2. \]

Health requires every simulator-state component after the first two positions to lie strictly in healthy_state_range, root height in healthy_z_range, and torso angle in healthy_angle_range. Info reports x position/velocity, z displacement from reset, and reward components.

Episode end

Unhealthy state terminates when terminate_when_unhealthy is true. The horizon truncates.

Configuration

Reference: Gymnasium Hopper.

Humanoid

State and action spaces

The default 348-value observation concatenates qpos[2..], all qvel, non-world body inertias (cinert), non-world body spatial velocities (cvel), actuator generalized forces after the six root DOFs, and non-world external contact forces. Configuration can retain root x/y or omit any auxiliary group. The default action is 17 controls in \([-0.4,0.4]\).

Dynamics, reward, and info

Horizontal velocity is computed from the mass-weighted body center. With simulator control \(u\) and external forces \(f_c\),

\[ r=w_fv_x+r_{healthy}-w_u\lVert u\rVert_2^2- \operatorname{clip}(w_c\lVert f_c\rVert_2^2,c_{min},c_{max}). \]

Health is strict root height \(z\in(z_{min},z_{max})\). Info exposes xyz position, xy velocity, summed tendon length/velocity, and reward components.

Episode end

Unhealthy state terminates when configured; the horizon truncates.

Configuration

HumanoidConfig exposes builders for:

Reference: Gymnasium Humanoid.

HumanoidStandup

State and action spaces

The default 348-value observation has the same groups and switches as Humanoid: qpos[2..], qvel, non-world cinert, non-world cvel, actuator generalized forces after the root DOFs, and non-world external forces. The default action is 17 controls in \([-0.4,0.4]\).

Dynamics, reward, and info

Let \(z\) be root height, \(h\) the MuJoCo model timestep, \(u\) the simulator controls, and \(f_c\) external contact forces:

\[ r=w_h\frac{z}{h}-w_u\lVert u\rVert_2^2- \operatorname{clip}(w_i\lVert f_c\rVert_2^2,i_{min},i_{max})+1. \]

Info includes x/y position, distance from origin, summed tendon length/velocity, and upright/control/impact reward components.

Episode end

There is no state termination. The horizon truncates.

Configuration

Reference: Gymnasium HumanoidStandup.

InvertedDoublePendulum

State and action spaces

The fixed 9-value observation is

\[ [x,\sin\theta_1,\sin\theta_2,\cos\theta_1,\cos\theta_2, \operatorname{clip}(\dot q_0,-10,10), \operatorname{clip}(\dot q_1,-10,10), \operatorname{clip}(\dot q_2,-10,10), \operatorname{clip}(f_{constraint,0},-10,10)]. \]

The default action is one cart control in \([-1,1]\).

Dynamics and reward

Let the free tip site position be \((x_{tip},z_{tip})\). Then

\[ p_d=0.01x_{tip}^2+(z_{tip}-2)^2, \qquad p_v=0.001\dot q_1^2+0.005\dot q_2^2, \]

\[ r=r_{healthy}\mathbf{1}[z_{tip}>1]-p_d-p_v. \]

Info contains cart x position and velocity.

Episode end

The task terminates when \(z_{tip}\le1\) and truncates at max_steps.

Configuration

Reference: Gymnasium InvertedDoublePendulum.

InvertedPendulum

State and action spaces

The 4-value observation is all qpos followed by all qvel: cart position, pole angle, cart velocity, and pole angular velocity. The default action is one cart control in \([-3,3]\).

Dynamics and reward

MuJoCo advances the cart-pole for two model steps by default. For observation \(s\),

\[ \text{terminated}=\neg\operatorname{finite}(s)\ \lor\ |s_1|>0.2, \qquad r=\begin{cases}0&\text{terminated}\1&\text{otherwise.}\end{cases} \]

Info["reward_survive"] contains this reward.

Episode end

Non-finite state or pole angle beyond \(0.2\) radians terminates. The horizon truncates.

Configuration

Reference: Gymnasium InvertedPendulum.

Pusher

State and action spaces

The 23-value observation is the seven arm joint positions, seven arm velocities, fingertip COM (3), object COM (3), and goal COM (3). Object/goal slide-joint coordinates are deliberately omitted. The default action is seven controls in \([-2,2]\).

Dynamics and reward

For fingertip \(p_f\), object \(p_o\), goal \(p_g\), and control \(u\),

\[ r=-w_n\lVert p_o-p_f\rVert_2-w_d\lVert p_g-p_o\rVert_2-w_u\lVert u\rVert_2^2. \]

The three signed components are returned as reward_near, reward_dist, and reward_ctrl. Reset randomizes the object in the configured task region while the goal remains fixed.

Episode end

There is no success termination. The task truncates at max_episode_steps.

Configuration

Custom XML must contain bodies named object, goal, and tips_arm and preserve the assumed seven arm joints.

Reference: Gymnasium Pusher.

Reacher

State and action spaces

The 10-value observation is

\[ [\cos\theta_0,\cos\theta_1,\sin\theta_0,\sin\theta_1, q_{target,x},q_{target,y},\dot\theta_0,\dot\theta_1, p_{finger,x}-p_{target,x},p_{finger,y}-p_{target,y}]. \]

The default action is two controls in \([-1,1]\).

Dynamics and reward

\[ r=-w_d\lVert p_{finger}-p_{target}\rVert_2-w_u\lVert u\rVert_2^2. \]

The two terms are returned in Info. Reset samples a target uniformly by rejection inside a radius-0.2 disc and adds small arm position/velocity noise.

Episode end

There is no distance-based termination. The episode truncates at max_episode_steps.

Configuration

Custom XML must contain target and fingertip bodies and preserve the assumed joint layout.

Reference: Gymnasium Reacher.

Swimmer

State and action spaces

The default 8-value observation concatenates qpos[2..] with all qvel, omitting root x/y. Keeping those positions produces 10 values. The default action is two controls in \([-1,1]\).

Dynamics and reward

\[ v_x=\frac{x_{t+1}-x_t}{\Delta t},\qquad r=w_fv_x-w_u\lVert u\rVert_2^2. \]

Info contains x position and the forward/control reward components.

Episode end

There is no state termination. The horizon truncates.

Configuration

Reference: Gymnasium Swimmer.

Walker2d

State and action spaces

The default 17-value observation is qpos[1..] followed by all velocities clipped to \([-10,10]\); retaining root x produces 18 values. The default action is six controls in \([-1,1]\).

Dynamics and reward

\[ r=w_fv_x+r_{healthy}-w_u\lVert u\rVert_2^2. \]

Health requires root height strictly inside healthy_z_range and torso angle strictly inside healthy_angle_range. Info contains x position/velocity, z displacement from reset, and forward/control/survival components.

Episode end

Unhealthy state terminates when configured; the horizon truncates.

Configuration

Reference: Gymnasium Walker2d.

API Reference

Open the generated Rust API documentation.

Contributing

Contributions from the community are welcome.

• Bug reports and feature requests: Open an issue on GitHub.
• Pull requests: Submit pull requests for bug fixes or new environments. For major architecture changes, open an issue first so the approach can be discussed.

OGym is licensed under the MIT License. See the LICENSE file for details.

Roadmap

• GPU acceleration for vectorized environments.
• Expanded MuJoCo asset-loading support.
• Additional Gym-standard environments, including Atari.
• Python bindings using PyO3.